2024-09-11T12:34:13.118 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T12:34:13.140 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T12:34:13.226 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299 branch: wip-athakkar-testing-2024-09-11-1208 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: null first_in_suite: false job_id: '7901299' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-athakkar-testing-2024-09-11-1208 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-athakkar-testing-2024-09-11-1208 sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d owner: scheduled_khiremat@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1861 sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d sleep_before_teardown: 0 subset: 3/4 suite: fs:volumes suite_branch: wip-athakkar-testing-2024-09-11-1208 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLe619R/WLj/vQo/TEasqG2SAbToCDob5Ohq1FM03xHwg4y+/VnmZkAltOj8rcIURv24Am/0kLDVT2jn6J7RYMk= smithi157.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJUIcygRQXg0AsTZEu9r7TJgQZ0MZvzIfX7HnUSofZBdy4Gcad4HZu75KOOrfxQ4wTeJzETIrjLRkL+9jbwqLo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-11_12:26:28 tube: smithi user: khiremat verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T12:34:13.227 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/qa; will attempt to use it 2024-09-11T12:34:13.227 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/qa/tasks 2024-09-11T12:34:13.227 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T12:34:13.229 INFO:teuthology.task.internal:Checking packages... 2024-09-11T12:34:13.252 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d' 2024-09-11T12:34:13.252 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T12:34:13.252 INFO:teuthology.packaging:ref: None 2024-09-11T12:34:13.252 INFO:teuthology.packaging:tag: None 2024-09-11T12:34:13.252 INFO:teuthology.packaging:branch: wip-athakkar-testing-2024-09-11-1208 2024-09-11T12:34:13.252 INFO:teuthology.packaging:sha1: 13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d 2024-09-11T12:34:13.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-athakkar-testing-2024-09-11-1208 2024-09-11T12:34:13.399 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:34:13.401 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T12:34:13.409 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T12:34:13.410 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T12:34:13.446 INFO:teuthology.task.internal:Saving configuration 2024-09-11T12:34:13.465 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T12:34:13.478 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T12:34:13.503 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 12:27:59.191528', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC2FLpdjsUmwGc3Vl2qgAJyv++uCiTFjSei8r5585Wc8'} 2024-09-11T12:34:13.526 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi157.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 12:27:59.193233', 'locked_by': 'scheduled_khiremat@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIhBrVLk3yRHpLk9Qj6OuN/QQgUzVM834YyxqDd4QGvq'} 2024-09-11T12:34:13.526 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T12:34:13.538 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-11T12:34:13.538 INFO:teuthology.task.internal:roles: ubuntu@smithi157.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-11T12:34:13.538 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T12:34:13.631 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc8d2da7ac0>, signals=[15]) 2024-09-11T12:34:13.631 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T12:34:13.642 INFO:teuthology.task.internal:Opening connections... 2024-09-11T12:34:13.642 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:34:13.644 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:34:13.739 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:34:13.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:34:13.836 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T12:34:13.843 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-09-11T12:34:13.848 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-09-11T12:34:13.849 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-09-11T12:34:13.895 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T12:34:13.895 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Ubuntu" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="22.04" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_CODENAME=jammy 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:ID=ubuntu 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE=debian 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T12:34:13.896 INFO:teuthology.orchestra.run.smithi078.stdout:UBUNTU_CODENAME=jammy 2024-09-11T12:34:13.897 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-09-11T12:34:13.922 DEBUG:teuthology.orchestra.run.smithi157:> uname -m 2024-09-11T12:34:13.928 INFO:teuthology.orchestra.run.smithi157.stdout:x86_64 2024-09-11T12:34:13.928 DEBUG:teuthology.orchestra.run.smithi157:> cat /etc/os-release 2024-09-11T12:34:13.976 INFO:teuthology.orchestra.run.smithi157.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T12:34:13.976 INFO:teuthology.orchestra.run.smithi157.stdout:NAME="Ubuntu" 2024-09-11T12:34:13.976 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_ID="22.04" 2024-09-11T12:34:13.976 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T12:34:13.976 INFO:teuthology.orchestra.run.smithi157.stdout:VERSION_CODENAME=jammy 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:ID=ubuntu 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:ID_LIKE=debian 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T12:34:13.977 INFO:teuthology.orchestra.run.smithi157.stdout:UBUNTU_CODENAME=jammy 2024-09-11T12:34:13.977 INFO:teuthology.lock.ops:Updating smithi157.front.sepia.ceph.com on lock server 2024-09-11T12:34:13.997 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T12:34:14.009 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T12:34:14.039 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T12:34:14.039 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-09-11T12:34:14.042 DEBUG:teuthology.orchestra.run.smithi157:> test '!' -e /home/ubuntu/cephtest 2024-09-11T12:34:14.046 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T12:34:14.061 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T12:34:14.061 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-09-11T12:34:14.093 DEBUG:teuthology.orchestra.run.smithi157:> test -z $(ls -A /var/lib/ceph) 2024-09-11T12:34:14.121 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T12:34:14.181 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T12:34:14.201 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-11T12:34:14.201 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-11T12:34:14.201 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-11T12:34:14.201 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T12:34:14.202 DEBUG:teuthology.orchestra.run.smithi157:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T12:34:14.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:34:14.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:34:14.207 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-11T12:34:14.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2024-09-11T12:34:14.387 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:14.387 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-11T12:34:14.387 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-11T12:34:14.392 INFO:teuthology.orchestra.run.smithi078.stdout:5.15.0-56-generic 2024-09-11T12:34:14.393 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:14.393 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-11T12:34:14.393 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2024-09-11T12:34:14.393 DEBUG:teuthology.orchestra.run.smithi078:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-11T12:34:14.399 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:14.400 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-11T12:34:14.400 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-11T12:34:14.405 INFO:teuthology.orchestra.run.smithi157.stdout:5.15.0-56-generic 2024-09-11T12:34:14.406 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:14.406 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-11T12:34:14.406 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2024-09-11T12:34:14.406 DEBUG:teuthology.orchestra.run.smithi157:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-11T12:34:14.458 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-11T12:34:14.464 INFO:teuthology.orchestra.run.smithi157.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: git config --global init.defaultBranch 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: 2024-09-11T12:34:14.607 INFO:teuthology.orchestra.run.smithi078.stderr:hint: git branch -m 2024-09-11T12:34:14.608 INFO:teuthology.orchestra.run.smithi078.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-11T12:34:14.610 DEBUG:teuthology.orchestra.run.smithi078:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: git config --global init.defaultBranch 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: 2024-09-11T12:34:14.613 INFO:teuthology.orchestra.run.smithi157.stderr:hint: git branch -m 2024-09-11T12:34:14.614 INFO:teuthology.orchestra.run.smithi157.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-11T12:34:14.617 DEBUG:teuthology.orchestra.run.smithi157:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-11T12:34:14.679 DEBUG:teuthology.orchestra.run.smithi078:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-11T12:34:14.683 DEBUG:teuthology.orchestra.run.smithi157:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-11T12:34:14.733 DEBUG:teuthology.orchestra.run.smithi078:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-11T12:34:14.740 DEBUG:teuthology.orchestra.run.smithi157:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-11T12:34:46.102 INFO:teuthology.orchestra.run.smithi157.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-11T12:34:46.103 INFO:teuthology.orchestra.run.smithi157.stderr: * [new branch] main -> origin/main 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190312 -> 20190312 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190416 -> 20190416 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190514 -> 20190514 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190618 -> 20190618 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190717 -> 20190717 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190815 -> 20190815 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20190923 -> 20190923 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20191022 -> 20191022 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20191215 -> 20191215 2024-09-11T12:34:46.133 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200122 -> 20200122 2024-09-11T12:34:46.134 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200316 -> 20200316 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200421 -> 20200421 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200519 -> 20200519 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200619 -> 20200619 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200721 -> 20200721 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200817 -> 20200817 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20200918 -> 20200918 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201022 -> 20201022 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201118 -> 20201118 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20201218 -> 20201218 2024-09-11T12:34:46.136 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210208 -> 20210208 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210315 -> 20210315 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210511 -> 20210511 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210716 -> 20210716 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210818 -> 20210818 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20210919 -> 20210919 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20211027 -> 20211027 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20211216 -> 20211216 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220209 -> 20220209 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220310 -> 20220310 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220411 -> 20220411 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220509 -> 20220509 2024-09-11T12:34:46.137 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220610 -> 20220610 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220708 -> 20220708 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220815 -> 20220815 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20220913 -> 20220913 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221012 -> 20221012 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221109 -> 20221109 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20221214 -> 20221214 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230117 -> 20230117 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230210 -> 20230210 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230310 -> 20230310 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230404 -> 20230404 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230515 -> 20230515 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230625 -> 20230625 2024-09-11T12:34:46.141 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230804 -> 20230804 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20230919 -> 20230919 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231030 -> 20231030 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231111 -> 20231111 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20231211 -> 20231211 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240115 -> 20240115 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240220 -> 20240220 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240312 -> 20240312 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240410 -> 20240410 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240513 -> 20240513 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240610 -> 20240610 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240709 -> 20240709 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240811 -> 20240811 2024-09-11T12:34:46.142 INFO:teuthology.orchestra.run.smithi157.stderr: * [new tag] 20240909 -> 20240909 2024-09-11T12:34:48.605 INFO:teuthology.orchestra.run.smithi078.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-11T12:34:48.605 INFO:teuthology.orchestra.run.smithi078.stderr: * [new branch] main -> origin/main 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190312 -> 20190312 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190416 -> 20190416 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190514 -> 20190514 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190618 -> 20190618 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190717 -> 20190717 2024-09-11T12:34:48.643 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190815 -> 20190815 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20190923 -> 20190923 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20191022 -> 20191022 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20191215 -> 20191215 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200122 -> 20200122 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200316 -> 20200316 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200421 -> 20200421 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200519 -> 20200519 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200619 -> 20200619 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200721 -> 20200721 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200817 -> 20200817 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20200918 -> 20200918 2024-09-11T12:34:48.644 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201022 -> 20201022 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201118 -> 20201118 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20201218 -> 20201218 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210208 -> 20210208 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210315 -> 20210315 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210511 -> 20210511 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210716 -> 20210716 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210818 -> 20210818 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20210919 -> 20210919 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20211027 -> 20211027 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20211216 -> 20211216 2024-09-11T12:34:48.648 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220209 -> 20220209 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220310 -> 20220310 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220411 -> 20220411 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220509 -> 20220509 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220610 -> 20220610 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220708 -> 20220708 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220815 -> 20220815 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20220913 -> 20220913 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221012 -> 20221012 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221109 -> 20221109 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20221214 -> 20221214 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230117 -> 20230117 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230210 -> 20230210 2024-09-11T12:34:48.649 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230310 -> 20230310 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230404 -> 20230404 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230515 -> 20230515 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230625 -> 20230625 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230804 -> 20230804 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20230919 -> 20230919 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20231030 -> 20231030 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20231111 -> 20231111 2024-09-11T12:34:48.650 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20231211 -> 20231211 2024-09-11T12:34:48.651 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240115 -> 20240115 2024-09-11T12:34:48.651 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240220 -> 20240220 2024-09-11T12:34:48.651 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240312 -> 20240312 2024-09-11T12:34:48.651 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240410 -> 20240410 2024-09-11T12:34:48.652 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240513 -> 20240513 2024-09-11T12:34:48.652 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240610 -> 20240610 2024-09-11T12:34:48.652 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240709 -> 20240709 2024-09-11T12:34:48.652 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240811 -> 20240811 2024-09-11T12:34:48.652 INFO:teuthology.orchestra.run.smithi078.stderr: * [new tag] 20240909 -> 20240909 2024-09-11T12:34:50.655 INFO:teuthology.orchestra.run.smithi157.stderr:Updating files: 38% (1342/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2063/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2427/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-11T12:34:50.660 INFO:teuthology.orchestra.run.smithi157.stdout:HEAD is now at 4a0367b3 Merge branch 'rtl8852b' into 'main' 2024-09-11T12:34:50.682 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-11T12:34:50.682 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:50.830 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:34:50.831 DEBUG:teuthology.orchestra.run.smithi157:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-11T12:34:50.833 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-09-11T12:34:52.393 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11 12:34:52 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-11T12:34:52.393 INFO:teuthology.orchestra.run.smithi157.stderr:FINISHED --2024-09-11 12:34:52-- 2024-09-11T12:34:52.394 INFO:teuthology.orchestra.run.smithi157.stderr:Total wall clock time: 1.5s 2024-09-11T12:34:52.394 INFO:teuthology.orchestra.run.smithi157.stderr:Downloaded: 1 files, 43M in 1.4s (30.6 MB/s) 2024-09-11T12:34:52.398 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-11T12:34:52.398 DEBUG:teuthology.orchestra.run.smithi157:> sudo dpkg -i /tmp/linux-image.deb 2024-09-11T12:34:52.620 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-11T12:34:53.155 INFO:teuthology.orchestra.run.smithi078.stderr:Updating files: 44% (1548/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 59% (2066/3501) Updating files: 59% (2069/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 69% (2428/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2891/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-11T12:34:53.160 INFO:teuthology.orchestra.run.smithi078.stdout:HEAD is now at 4a0367b3 Merge branch 'rtl8852b' into 'main' 2024-09-11T12:34:53.182 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-11T12:34:53.182 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-11T12:34:53.321 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:34:53.321 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-11T12:34:53.324 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi078.front.sepia.ceph.com to complete... 2024-09-11T12:34:54.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11 12:34:54 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-11T12:34:54.377 INFO:teuthology.orchestra.run.smithi078.stderr:FINISHED --2024-09-11 12:34:54-- 2024-09-11T12:34:54.377 INFO:teuthology.orchestra.run.smithi078.stderr:Total wall clock time: 1.0s 2024-09-11T12:34:54.377 INFO:teuthology.orchestra.run.smithi078.stderr:Downloaded: 1 files, 43M in 0.9s (47.4 MB/s) 2024-09-11T12:34:54.382 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-11T12:34:54.382 DEBUG:teuthology.orchestra.run.smithi078:> sudo dpkg -i /tmp/linux-image.deb 2024-09-11T12:34:54.621 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-11T12:34:54.647 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-11T12:34:54.652 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-11T12:34:54.736 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-11T12:34:56.713 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-11T12:34:56.717 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-11T12:34:56.845 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-11T12:34:58.694 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-11T12:34:58.802 INFO:teuthology.orchestra.run.smithi157.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:00.912 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-11T12:35:01.021 INFO:teuthology.orchestra.run.smithi078.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:20.843 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2024-09-11T12:35:20.858 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T12:35:20.891 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-11T12:35:21.192 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:21.212 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:21.465 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T12:35:21.471 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T12:35:21.534 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T12:35:21.540 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T12:35:21.689 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T12:35:21.690 INFO:teuthology.orchestra.run.smithi157.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T12:35:21.690 INFO:teuthology.orchestra.run.smithi157.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T12:35:21.727 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-11T12:35:21.910 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-deb -c /tmp/linux-image.deb 2024-09-11T12:35:21.918 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-11T12:35:21.918 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-11T12:35:21.918 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:21.936 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:21.937 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:21.958 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-11T12:35:21.959 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-11T12:35:21.960 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-11T12:35:21.961 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-11T12:35:21.962 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-11T12:35:22.064 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-11T12:35:22.065 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-11T12:35:22.066 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-11T12:35:22.067 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-11T12:35:22.068 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-11T12:35:22.069 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-11T12:35:22.070 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-11T12:35:22.071 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-11T12:35:22.072 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-11T12:35:22.073 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-11T12:35:22.074 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-11T12:35:22.075 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-11T12:35:22.076 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-11T12:35:22.077 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-11T12:35:22.078 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-11T12:35:22.079 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-11T12:35:22.080 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-11T12:35:22.081 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-11T12:35:22.082 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-11T12:35:22.083 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-11T12:35:22.084 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-11T12:35:22.085 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-11T12:35:22.086 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-11T12:35:22.087 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-11T12:35:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-11T12:35:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-11T12:35:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-11T12:35:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-11T12:35:22.088 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-11T12:35:22.089 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-11T12:35:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-11T12:35:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-11T12:35:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-11T12:35:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-11T12:35:22.090 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-11T12:35:22.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-11T12:35:22.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-11T12:35:22.091 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-11T12:35:22.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-11T12:35:22.091 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-11T12:35:22.092 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-11T12:35:22.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-11T12:35:22.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-11T12:35:22.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-11T12:35:22.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-11T12:35:22.093 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-11T12:35:22.094 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-11T12:35:22.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-11T12:35:22.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-11T12:35:22.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-11T12:35:22.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-11T12:35:22.095 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-11T12:35:22.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-11T12:35:22.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-11T12:35:22.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-11T12:35:22.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-11T12:35:22.096 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-11T12:35:22.097 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-11T12:35:22.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-11T12:35:22.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-11T12:35:22.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-11T12:35:22.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-11T12:35:22.098 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-11T12:35:22.099 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-11T12:35:22.100 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-11T12:35:22.100 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-11T12:35:22.100 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-11T12:35:22.100 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-11T12:35:22.100 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-11T12:35:22.101 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-11T12:35:22.101 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-11T12:35:22.101 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-11T12:35:22.101 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-11T12:35:22.101 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-11T12:35:22.102 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-11T12:35:22.103 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-11T12:35:22.104 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-11T12:35:22.105 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-11T12:35:22.106 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-11T12:35:22.107 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-11T12:35:22.108 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-11T12:35:22.109 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-11T12:35:22.109 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-11T12:35:22.109 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-11T12:35:22.109 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-11T12:35:22.109 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-11T12:35:22.110 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-11T12:35:22.111 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-11T12:35:22.112 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-11T12:35:22.113 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-11T12:35:22.113 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-11T12:35:22.113 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-11T12:35:22.113 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-11T12:35:22.113 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-11T12:35:22.114 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-11T12:35:22.115 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-11T12:35:22.116 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-11T12:35:22.116 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-11T12:35:22.116 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-11T12:35:22.116 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-11T12:35:22.116 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-11T12:35:22.117 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-11T12:35:22.118 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-11T12:35:22.119 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-11T12:35:22.120 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-11T12:35:22.120 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-11T12:35:22.120 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-11T12:35:22.120 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-11T12:35:22.120 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-11T12:35:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-11T12:35:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-11T12:35:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-11T12:35:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-11T12:35:22.121 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-11T12:35:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-11T12:35:22.123 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-11T12:35:22.124 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-11T12:35:22.124 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-11T12:35:22.124 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-11T12:35:22.124 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-11T12:35:22.124 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-11T12:35:22.125 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-11T12:35:22.126 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-11T12:35:22.126 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-11T12:35:22.126 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-11T12:35:22.126 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-11T12:35:22.126 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-11T12:35:22.127 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-11T12:35:22.128 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-11T12:35:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-11T12:35:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-11T12:35:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-11T12:35:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-11T12:35:22.129 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-11T12:35:22.130 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-11T12:35:22.131 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-11T12:35:22.132 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-11T12:35:22.133 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-11T12:35:22.136 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-11T12:35:22.137 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-11T12:35:22.138 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-11T12:35:22.139 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-11T12:35:22.140 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-11T12:35:22.141 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-11T12:35:22.142 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-11T12:35:22.143 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-11T12:35:22.144 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-11T12:35:22.145 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-11T12:35:22.146 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-11T12:35:22.147 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-11T12:35:22.148 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-11T12:35:22.149 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-11T12:35:22.150 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-11T12:35:22.151 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-11T12:35:22.152 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-11T12:35:22.153 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-11T12:35:22.154 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-11T12:35:22.155 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-11T12:35:22.156 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-11T12:35:22.157 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-11T12:35:22.158 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-11T12:35:22.159 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-11T12:35:22.160 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-11T12:35:22.161 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-11T12:35:22.162 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-11T12:35:22.163 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-11T12:35:22.164 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-11T12:35:22.165 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-11T12:35:22.166 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-11T12:35:22.167 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-11T12:35:22.168 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-11T12:35:22.169 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-11T12:35:22.170 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-11T12:35:22.171 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-11T12:35:22.172 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-11T12:35:22.173 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-11T12:35:22.174 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-11T12:35:22.279 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-11T12:35:22.280 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-11T12:35:22.281 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-11T12:35:22.282 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-11T12:35:22.283 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-11T12:35:22.284 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-11T12:35:22.285 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-11T12:35:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-11T12:35:22.287 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-11T12:35:22.288 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-11T12:35:22.289 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-11T12:35:22.290 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-11T12:35:22.291 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-11T12:35:22.292 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-11T12:35:22.293 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-11T12:35:22.294 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-11T12:35:22.295 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-11T12:35:22.296 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-11T12:35:22.297 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-11T12:35:22.298 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-11T12:35:22.299 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-11T12:35:22.300 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-11T12:35:22.301 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-11T12:35:22.302 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-11T12:35:22.303 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-11T12:35:22.304 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-11T12:35:22.305 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-11T12:35:22.306 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-11T12:35:22.307 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-11T12:35:22.308 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-11T12:35:22.309 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-11T12:35:22.310 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-11T12:35:22.311 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-11T12:35:22.312 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-11T12:35:22.313 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-11T12:35:22.314 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-11T12:35:22.315 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-11T12:35:22.316 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-11T12:35:22.317 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-11T12:35:22.318 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-11T12:35:22.319 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-11T12:35:22.320 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-11T12:35:22.321 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-11T12:35:22.322 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-11T12:35:22.323 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-11T12:35:22.324 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-11T12:35:22.325 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-11T12:35:22.326 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-11T12:35:22.327 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-11T12:35:22.328 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-11T12:35:22.329 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-11T12:35:22.330 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-11T12:35:22.331 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-11T12:35:22.332 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-11T12:35:22.333 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-11T12:35:22.334 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-11T12:35:22.335 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-11T12:35:22.336 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-11T12:35:22.337 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-11T12:35:22.338 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-11T12:35:22.339 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-11T12:35:22.340 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-11T12:35:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-11T12:35:22.342 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-11T12:35:22.343 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-11T12:35:22.344 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-11T12:35:22.345 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-11T12:35:22.346 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-11T12:35:22.347 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-11T12:35:22.348 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-11T12:35:22.349 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-11T12:35:22.350 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-11T12:35:22.351 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-11T12:35:22.352 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-11T12:35:22.353 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-11T12:35:22.354 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-11T12:35:22.355 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-11T12:35:22.356 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-11T12:35:22.357 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-11T12:35:22.358 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-11T12:35:22.359 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-11T12:35:22.360 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-11T12:35:22.361 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-11T12:35:22.362 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-11T12:35:22.363 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-11T12:35:22.364 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-11T12:35:22.365 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-11T12:35:22.366 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-11T12:35:22.367 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-11T12:35:22.368 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-11T12:35:22.369 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-11T12:35:22.370 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-11T12:35:22.371 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-11T12:35:22.372 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-11T12:35:22.373 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-11T12:35:22.374 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-11T12:35:22.375 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-11T12:35:22.376 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-11T12:35:22.377 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-11T12:35:22.378 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-11T12:35:22.379 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-11T12:35:22.380 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-11T12:35:22.381 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-11T12:35:22.382 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-11T12:35:22.383 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-11T12:35:22.384 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-11T12:35:22.385 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-11T12:35:22.386 INFO:teuthology.orchestra.run.smithi157.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-11T12:35:22.391 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:22.391 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:22.392 DEBUG:teuthology.orchestra.run.smithi157:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-11T12:35:22.398 INFO:teuthology.orchestra.run.smithi157.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:22.398 INFO:teuthology.orchestra.run.smithi157.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:22.399 INFO:teuthology.orchestra.run.smithi157.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:22.399 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-11T12:35:22.399 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:22.399 DEBUG:teuthology.orchestra.run.smithi157:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-11T12:35:22.441 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi157.front.sepia.ceph.com to complete... 2024-09-11T12:35:22.860 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub' 2024-09-11T12:35:22.861 INFO:teuthology.orchestra.run.smithi157.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T12:35:22.884 INFO:teuthology.orchestra.run.smithi157.stderr:Generating grub configuration file ... 2024-09-11T12:35:23.045 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:23.051 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:23.266 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T12:35:23.271 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T12:35:23.310 INFO:teuthology.orchestra.run.smithi157.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T12:35:23.315 INFO:teuthology.orchestra.run.smithi157.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T12:35:23.428 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T12:35:23.428 INFO:teuthology.orchestra.run.smithi157.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T12:35:23.428 INFO:teuthology.orchestra.run.smithi157.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T12:35:23.450 INFO:teuthology.orchestra.run.smithi157.stderr:done 2024-09-11T12:35:23.940 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub' 2024-09-11T12:35:23.963 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T12:35:23.992 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-09-11T12:35:24.298 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:24.312 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:24.597 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T12:35:24.605 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T12:35:24.670 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T12:35:24.675 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T12:35:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T12:35:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T12:35:24.836 INFO:teuthology.orchestra.run.smithi078.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T12:35:24.871 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-09-11T12:35:25.039 DEBUG:teuthology.orchestra.run.smithi078:> dpkg-deb -c /tmp/linux-image.deb 2024-09-11T12:35:25.047 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-11T12:35:25.047 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-11T12:35:25.047 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.062 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.062 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-11T12:35:25.084 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-11T12:35:25.085 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-11T12:35:25.086 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-11T12:35:25.087 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-11T12:35:25.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-11T12:35:25.088 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-11T12:35:25.164 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-11T12:35:25.165 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-11T12:35:25.166 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-11T12:35:25.167 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-11T12:35:25.168 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-11T12:35:25.169 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-11T12:35:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-11T12:35:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-11T12:35:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-11T12:35:25.173 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-11T12:35:25.174 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-11T12:35:25.175 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-11T12:35:25.176 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-11T12:35:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-11T12:35:25.178 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-11T12:35:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-11T12:35:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-11T12:35:25.181 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-11T12:35:25.182 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-11T12:35:25.183 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-11T12:35:25.184 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-11T12:35:25.185 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-11T12:35:25.186 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-11T12:35:25.187 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-11T12:35:25.188 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-11T12:35:25.189 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-11T12:35:25.190 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-11T12:35:25.191 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-11T12:35:25.192 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-11T12:35:25.193 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-11T12:35:25.194 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-11T12:35:25.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-11T12:35:25.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-11T12:35:25.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-11T12:35:25.195 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-11T12:35:25.196 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-11T12:35:25.197 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-11T12:35:25.198 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-11T12:35:25.199 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-11T12:35:25.200 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-11T12:35:25.201 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-11T12:35:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-11T12:35:25.203 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-11T12:35:25.204 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-11T12:35:25.205 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-11T12:35:25.206 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-11T12:35:25.207 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-11T12:35:25.208 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-11T12:35:25.209 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-11T12:35:25.210 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-11T12:35:25.211 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-11T12:35:25.212 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-11T12:35:25.213 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-11T12:35:25.216 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-11T12:35:25.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-11T12:35:25.216 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-11T12:35:25.217 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-11T12:35:25.218 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-11T12:35:25.219 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-11T12:35:25.220 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-11T12:35:25.221 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-11T12:35:25.222 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-11T12:35:25.223 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-11T12:35:25.224 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-11T12:35:25.225 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-11T12:35:25.226 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-11T12:35:25.227 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-11T12:35:25.228 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-11T12:35:25.229 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-11T12:35:25.230 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-11T12:35:25.231 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-11T12:35:25.232 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-11T12:35:25.233 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-11T12:35:25.296 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-11T12:35:25.297 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-11T12:35:25.298 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-11T12:35:25.299 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-11T12:35:25.300 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-11T12:35:25.301 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-11T12:35:25.302 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-11T12:35:25.303 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-11T12:35:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-11T12:35:25.305 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-11T12:35:25.306 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-11T12:35:25.307 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-11T12:35:25.308 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-11T12:35:25.309 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-11T12:35:25.310 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-11T12:35:25.311 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-11T12:35:25.312 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-11T12:35:25.313 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-11T12:35:25.314 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-11T12:35:25.315 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-11T12:35:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-11T12:35:25.317 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-11T12:35:25.318 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-11T12:35:25.319 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-11T12:35:25.320 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-11T12:35:25.321 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-11T12:35:25.322 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-11T12:35:25.323 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-11T12:35:25.324 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-11T12:35:25.325 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-11T12:35:25.326 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-11T12:35:25.327 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-11T12:35:25.328 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-11T12:35:25.329 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-11T12:35:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-11T12:35:25.331 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-11T12:35:25.332 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-11T12:35:25.333 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-11T12:35:25.334 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-11T12:35:25.335 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-11T12:35:25.336 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-11T12:35:25.337 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-11T12:35:25.338 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-11T12:35:25.339 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-11T12:35:25.340 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-11T12:35:25.341 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-11T12:35:25.342 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-11T12:35:25.343 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-11T12:35:25.344 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-11T12:35:25.345 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-11T12:35:25.346 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-11T12:35:25.347 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-11T12:35:25.348 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-11T12:35:25.349 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-11T12:35:25.350 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-11T12:35:25.351 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-11T12:35:25.352 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-11T12:35:25.353 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-11T12:35:25.354 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-11T12:35:25.355 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-11T12:35:25.356 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-11T12:35:25.357 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-11T12:35:25.358 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-11T12:35:25.359 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-11T12:35:25.360 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-11T12:35:25.361 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-11T12:35:25.362 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-11T12:35:25.363 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-11T12:35:25.364 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-11T12:35:25.364 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-11T12:35:25.364 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-11T12:35:25.403 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-11T12:35:25.404 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-11T12:35:25.405 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-11T12:35:25.406 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-11T12:35:25.407 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-11T12:35:25.408 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-11T12:35:25.409 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-11T12:35:25.410 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-11T12:35:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-11T12:35:25.412 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-11T12:35:25.413 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-11T12:35:25.414 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-11T12:35:25.415 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-11T12:35:25.416 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-11T12:35:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-11T12:35:25.418 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-11T12:35:25.419 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-11T12:35:25.420 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-11T12:35:25.421 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-11T12:35:25.421 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-11T12:35:25.421 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-11T12:35:25.425 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-11T12:35:25.426 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-11T12:35:25.427 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-11T12:35:25.428 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-11T12:35:25.429 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-11T12:35:25.430 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-11T12:35:25.431 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-11T12:35:25.432 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-11T12:35:25.433 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-11T12:35:25.434 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-11T12:35:25.435 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-11T12:35:25.436 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-11T12:35:25.437 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-11T12:35:25.438 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-11T12:35:25.439 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-11T12:35:25.440 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-11T12:35:25.441 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-11T12:35:25.442 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-11T12:35:25.443 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-11T12:35:25.444 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-11T12:35:25.445 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-11T12:35:25.446 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-11T12:35:25.447 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-11T12:35:25.448 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-11T12:35:25.449 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-11T12:35:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-11T12:35:25.451 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-11T12:35:25.452 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-11T12:35:25.453 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-11T12:35:25.454 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-11T12:35:25.459 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.459 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.459 DEBUG:teuthology.orchestra.run.smithi078:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-11T12:35:25.466 INFO:teuthology.orchestra.run.smithi078.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-11T12:35:25.466 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-11T12:35:25.466 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:25.467 DEBUG:teuthology.orchestra.run.smithi078:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-11T12:35:25.509 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi078.front.sepia.ceph.com to complete... 2024-09-11T12:35:25.980 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub' 2024-09-11T12:35:25.982 INFO:teuthology.orchestra.run.smithi078.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T12:35:26.005 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-09-11T12:35:26.209 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:26.215 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:35:26.495 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T12:35:26.501 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T12:35:26.539 INFO:teuthology.orchestra.run.smithi078.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T12:35:26.545 INFO:teuthology.orchestra.run.smithi078.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T12:35:26.676 INFO:teuthology.orchestra.run.smithi078.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T12:35:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T12:35:26.677 INFO:teuthology.orchestra.run.smithi078.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T12:35:26.697 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-09-11T12:35:54.589 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-11T12:35:54.590 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:35:54.591 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:35:57.804 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-11T12:35:57.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:35:57.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:12.989 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-11T12:36:16.317 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T12:36:21.993 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:36:21.993 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:25.053 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-11T12:36:25.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:36:25.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:28.385 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T12:36:37.061 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:36:37.062 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:40.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-11T12:36:40.386 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:36:40.387 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:55.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:36:55.142 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:36:58.813 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T12:37:13.665 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.157 2024-09-11T12:37:13.814 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:37:13.815 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:37:13.818 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-09-11T12:37:31.681 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:37:31.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:37:31.819 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:37:31.819 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:37:32.155 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-11T12:37:32.159 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-11T12:37:32.847 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:37:32.847 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-11T12:37:32.847 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-09-11T12:37:32.898 INFO:teuthology.orchestra.run.smithi078.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:37:32.898 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:37:32.898 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T12:37:32.898 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T12:37:33.153 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:37:33.154 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-11T12:37:33.154 DEBUG:teuthology.orchestra.run.smithi157:> uname -r 2024-09-11T12:37:33.202 INFO:teuthology.orchestra.run.smithi157.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:37:33.202 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-11T12:37:33.202 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T12:37:33.203 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T12:37:33.902 DEBUG:teuthology.parallel:result is None 2024-09-11T12:37:34.205 DEBUG:teuthology.parallel:result is None 2024-09-11T12:37:34.206 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T12:37:34.213 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T12:37:34.213 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T12:37:34.217 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T12:37:34.225 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T12:37:34.250 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T12:37:34.297 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T12:37:34.297 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T12:37:34.300 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T12:37:34.325 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T12:37:34.347 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T12:37:34.347 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T12:37:34.365 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T12:37:34.450 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T12:37:34.468 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T12:37:34.486 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T12:37:34.507 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T12:37:34.510 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T12:37:34.517 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T12:37:34.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T12:37:34.521 DEBUG:teuthology.orchestra.run.smithi157:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T12:37:34.540 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T12:37:34.570 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T12:37:34.571 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T12:37:34.585 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T12:37:34.594 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T12:37:34.649 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T12:37:34.704 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:37:34.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T12:37:34.773 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T12:37:34.782 DEBUG:teuthology.orchestra.run.smithi157:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T12:37:34.831 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:37:34.831 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T12:37:34.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-09-11T12:37:34.920 DEBUG:teuthology.orchestra.run.smithi157:> sudo service rsyslog restart 2024-09-11T12:37:35.118 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T12:37:35.125 INFO:teuthology.task.internal:Starting timer... 2024-09-11T12:37:35.126 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T12:37:35.168 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T12:37:35.218 DEBUG:teuthology.task.selinux:Excluding smithi078: OS 'ubuntu' does not support SELinux 2024-09-11T12:37:35.218 DEBUG:teuthology.task.selinux:Excluding smithi157: OS 'ubuntu' does not support SELinux 2024-09-11T12:37:35.218 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T12:37:35.218 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-11T12:37:35.218 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T12:37:35.218 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T12:37:35.271 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T12:37:35.275 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T12:37:35.395 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T12:37:35.411 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T12:37:35.413 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi078.front.sepia.ceph.com,smithi157.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T12:45:43.656 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi157.front.sepia.ceph.com')] 2024-09-11T12:45:43.658 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:45:43.659 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:45:43.743 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-09-11T12:45:43.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-09-11T12:45:43.892 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:45:43.892 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi157.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T12:45:43.971 DEBUG:teuthology.orchestra.run.smithi157:> true 2024-09-11T12:45:44.087 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi157.front.sepia.ceph.com' 2024-09-11T12:45:44.088 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T12:45:44.099 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T12:45:44.099 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T12:45:44.100 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T12:45:44.103 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T12:45:44.103 DEBUG:teuthology.orchestra.run.smithi157:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T12:45:44.138 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Command line: ntpd -gq 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: ---------------------------------------------------- 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: corporation. Support and training for ntp-4 are 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: available at https://www.nwtime.org/support 2024-09-11T12:45:44.139 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: ---------------------------------------------------- 2024-09-11T12:45:44.140 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: proto: precision = 0.041 usec (-24) 2024-09-11T12:45:44.140 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: basedate set to 2022-02-04 2024-09-11T12:45:44.140 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: gps base set to 2022-02-06 (week 2196) 2024-09-11T12:45:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T12:45:44.140 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T12:45:44.141 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T12:45:44.141 INFO:teuthology.orchestra.run.smithi078.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen normally on 3 enp3s0f1 172.21.15.78:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen normally on 4 lo [::1]:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d4dd%5]:123 2024-09-11T12:45:44.142 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:44 ntpd[18442]: Listening on routing socket on fd #22 for interface updates 2024-09-11T12:45:44.169 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T12:45:44.169 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Command line: ntpd -gq 2024-09-11T12:45:44.169 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: ---------------------------------------------------- 2024-09-11T12:45:44.169 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: corporation. Support and training for ntp-4 are 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: available at https://www.nwtime.org/support 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: ---------------------------------------------------- 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: proto: precision = 0.042 usec (-24) 2024-09-11T12:45:44.170 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: basedate set to 2022-02-04 2024-09-11T12:45:44.171 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: gps base set to 2022-02-06 (week 2196) 2024-09-11T12:45:44.171 INFO:teuthology.orchestra.run.smithi157.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T12:45:44.171 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T12:45:44.172 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T12:45:44.172 INFO:teuthology.orchestra.run.smithi157.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen normally on 3 enp3s0f1 172.21.15.157:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen normally on 4 lo [::1]:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7ea1%5]:123 2024-09-11T12:45:44.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:44 ntpd[18422]: Listening on routing socket on fd #22 for interface updates 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:45.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:45 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:45.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:45.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:45 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:47.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:47.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:47 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:47.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:47 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:48.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:48.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:48 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:48.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:48 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:49.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:49 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:49.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:49 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:50.139 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:50 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:50.170 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:50.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:50.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:50.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:50.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:50.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:50 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:51.140 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stdout:11 Sep 12:45:51 ntpd[18442]: ntpd: time slew -0.002446 s 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stdout:ntpd: time slew -0.002446s 2024-09-11T12:45:51.141 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T12:45:51.142 INFO:teuthology.orchestra.run.smithi078.stderr:11 Sep 12:45:51 ntpd[18442]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T12:45:51.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:51.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:51.171 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T12:45:51.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T12:45:51.172 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T12:45:51.173 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T12:45:51.173 INFO:teuthology.orchestra.run.smithi157.stdout:11 Sep 12:45:51 ntpd[18422]: ntpd: time slew -0.001713 s 2024-09-11T12:45:51.173 INFO:teuthology.orchestra.run.smithi157.stdout:ntpd: time slew -0.001713s 2024-09-11T12:45:51.173 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T12:45:51.174 INFO:teuthology.orchestra.run.smithi157.stderr:11 Sep 12:45:51 ntpd[18422]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================== 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.265 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T12:45:51.266 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-09-11T12:45:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.266 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T12:45:51.267 INFO:teuthology.run_tasks:Running task install... 2024-09-11T12:45:51.276 DEBUG:teuthology.task.install:project ceph 2024-09-11T12:45:51.276 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'}} 2024-09-11T12:45:51.277 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'} 2024-09-11T12:45:51.277 INFO:teuthology.task.install:Using flavor: default 2024-09-11T12:45:51.283 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T12:45:51.283 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-11T12:45:51.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo apt-key list | grep Ceph 2024-09-11T12:45:51.284 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-key list | grep Ceph 2024-09-11T12:45:51.396 INFO:teuthology.orchestra.run.smithi078.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T12:45:51.416 INFO:teuthology.orchestra.run.smithi157.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T12:45:51.446 INFO:teuthology.orchestra.run.smithi078.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T12:45:51.446 INFO:teuthology.orchestra.run.smithi078.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T12:45:51.447 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-11T12:45:51.447 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-11T12:45:51.448 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d 2024-09-11T12:45:51.467 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T12:45:51.467 INFO:teuthology.orchestra.run.smithi157.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T12:45:51.468 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-11T12:45:51.468 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-11T12:45:51.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d 2024-09-11T12:45:51.592 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default/ 2024-09-11T12:45:51.592 INFO:teuthology.task.install.deb:Package version is 19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:45:51.605 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default/ 2024-09-11T12:45:51.605 INFO:teuthology.task.install.deb:Package version is 19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:45:51.713 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:45:51.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T12:45:51.722 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:45:51.722 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T12:45:51.733 DEBUG:teuthology.orchestra.run.smithi078:> sudo apt-get update 2024-09-11T12:45:51.741 DEBUG:teuthology.orchestra.run.smithi157:> sudo apt-get update 2024-09-11T12:45:51.966 INFO:teuthology.orchestra.run.smithi078.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T12:45:51.973 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T12:45:51.987 INFO:teuthology.orchestra.run.smithi078.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-11T12:45:51.995 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy Release [7,005 B] 2024-09-11T12:45:52.009 INFO:teuthology.orchestra.run.smithi078.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T12:45:52.017 INFO:teuthology.orchestra.run.smithi157.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T12:45:52.030 INFO:teuthology.orchestra.run.smithi078.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-11T12:45:52.038 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-11T12:45:52.051 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T12:45:52.054 INFO:teuthology.orchestra.run.smithi078.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T12:45:52.060 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:5 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T12:45:52.060 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T12:45:52.073 INFO:teuthology.orchestra.run.smithi078.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2024-09-11T12:45:52.081 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,419 B] 2024-09-11T12:45:52.117 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T12:45:52.125 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T12:45:52.471 INFO:teuthology.orchestra.run.smithi078.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T12:45:52.477 INFO:teuthology.orchestra.run.smithi157.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T12:45:52.664 INFO:teuthology.orchestra.run.smithi078.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T12:45:52.667 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T12:45:56.727 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 312 kB in 1s (366 kB/s) 2024-09-11T12:45:56.889 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 312 kB in 1s (368 kB/s) 2024-09-11T12:45:57.763 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2024-09-11T12:45:57.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4207-g13fc0377-1jammy cephadm=19.3.0-4207-g13fc0377-1jammy ceph-mds=19.3.0-4207-g13fc0377-1jammy ceph-mgr=19.3.0-4207-g13fc0377-1jammy ceph-common=19.3.0-4207-g13fc0377-1jammy ceph-fuse=19.3.0-4207-g13fc0377-1jammy ceph-test=19.3.0-4207-g13fc0377-1jammy ceph-volume=19.3.0-4207-g13fc0377-1jammy radosgw=19.3.0-4207-g13fc0377-1jammy python3-rados=19.3.0-4207-g13fc0377-1jammy python3-rgw=19.3.0-4207-g13fc0377-1jammy python3-cephfs=19.3.0-4207-g13fc0377-1jammy python3-rbd=19.3.0-4207-g13fc0377-1jammy libcephfs2=19.3.0-4207-g13fc0377-1jammy libcephfs-dev=19.3.0-4207-g13fc0377-1jammy librados2=19.3.0-4207-g13fc0377-1jammy librbd1=19.3.0-4207-g13fc0377-1jammy rbd-fuse=19.3.0-4207-g13fc0377-1jammy python3-cephfs=19.3.0-4207-g13fc0377-1jammy cephfs-shell=19.3.0-4207-g13fc0377-1jammy cephfs-top=19.3.0-4207-g13fc0377-1jammy cephfs-mirror=19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:45:57.855 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2024-09-11T12:45:57.923 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-11T12:45:57.938 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4207-g13fc0377-1jammy cephadm=19.3.0-4207-g13fc0377-1jammy ceph-mds=19.3.0-4207-g13fc0377-1jammy ceph-mgr=19.3.0-4207-g13fc0377-1jammy ceph-common=19.3.0-4207-g13fc0377-1jammy ceph-fuse=19.3.0-4207-g13fc0377-1jammy ceph-test=19.3.0-4207-g13fc0377-1jammy ceph-volume=19.3.0-4207-g13fc0377-1jammy radosgw=19.3.0-4207-g13fc0377-1jammy python3-rados=19.3.0-4207-g13fc0377-1jammy python3-rgw=19.3.0-4207-g13fc0377-1jammy python3-cephfs=19.3.0-4207-g13fc0377-1jammy python3-rbd=19.3.0-4207-g13fc0377-1jammy libcephfs2=19.3.0-4207-g13fc0377-1jammy libcephfs-dev=19.3.0-4207-g13fc0377-1jammy librados2=19.3.0-4207-g13fc0377-1jammy librbd1=19.3.0-4207-g13fc0377-1jammy rbd-fuse=19.3.0-4207-g13fc0377-1jammy python3-cephfs=19.3.0-4207-g13fc0377-1jammy cephfs-shell=19.3.0-4207-g13fc0377-1jammy cephfs-top=19.3.0-4207-g13fc0377-1jammy cephfs-mirror=19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:45:58.015 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-11T12:45:58.038 INFO:teuthology.orchestra.run.smithi078.stdout:Building dependency tree... 2024-09-11T12:45:58.038 INFO:teuthology.orchestra.run.smithi078.stdout:Reading state information... 2024-09-11T12:45:58.198 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-09-11T12:45:58.199 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-09-11T12:45:58.224 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T12:45:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T12:45:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T12:45:58.224 INFO:teuthology.orchestra.run.smithi078.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T12:45:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T12:45:58.225 INFO:teuthology.orchestra.run.smithi078.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T12:45:58.225 INFO:teuthology.orchestra.run.smithi078.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T12:45:58.227 INFO:teuthology.orchestra.run.smithi078.stdout:The following additional packages will be installed: 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-11T12:45:58.228 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-11T12:45:58.229 INFO:teuthology.orchestra.run.smithi078.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout:Suggested packages: 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout:Recommended packages: 2024-09-11T12:45:58.230 INFO:teuthology.orchestra.run.smithi078.stdout: btrfs-tools 2024-09-11T12:45:58.390 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T12:45:58.390 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T12:45:58.391 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T12:45:58.391 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T12:45:58.391 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout:The following NEW packages will be installed: 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T12:45:58.392 INFO:teuthology.orchestra.run.smithi078.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T12:45:58.393 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-11T12:45:58.394 INFO:teuthology.orchestra.run.smithi078.stdout: socat unzip xmlstarlet zip 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-09-11T12:45:58.397 INFO:teuthology.orchestra.run.smithi157.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-11T12:45:58.398 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T12:45:58.399 INFO:teuthology.orchestra.run.smithi157.stdout:Recommended packages: 2024-09-11T12:45:58.400 INFO:teuthology.orchestra.run.smithi157.stdout: btrfs-tools 2024-09-11T12:45:58.540 INFO:teuthology.orchestra.run.smithi078.stdout:0 upgraded, 124 newly installed, 0 to remove and 334 not upgraded. 2024-09-11T12:45:58.540 INFO:teuthology.orchestra.run.smithi078.stdout:Need to get 251 MB of archives. 2024-09-11T12:45:58.540 INFO:teuthology.orchestra.run.smithi078.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-11T12:45:58.541 INFO:teuthology.orchestra.run.smithi078.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4207-g13fc0377-1jammy [3,633 kB] 2024-09-11T12:45:58.549 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-09-11T12:45:58.549 INFO:teuthology.orchestra.run.smithi157.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T12:45:58.549 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T12:45:58.549 INFO:teuthology.orchestra.run.smithi157.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-11T12:45:58.549 INFO:teuthology.orchestra.run.smithi157.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-11T12:45:58.551 INFO:teuthology.orchestra.run.smithi157.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T12:45:58.552 INFO:teuthology.orchestra.run.smithi157.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-11T12:45:58.553 INFO:teuthology.orchestra.run.smithi157.stdout: socat unzip xmlstarlet zip 2024-09-11T12:45:58.587 INFO:teuthology.orchestra.run.smithi078.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-11T12:45:58.619 INFO:teuthology.orchestra.run.smithi157.stdout:0 upgraded, 124 newly installed, 0 to remove and 334 not upgraded. 2024-09-11T12:45:58.620 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 251 MB of archives. 2024-09-11T12:45:58.620 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 933 MB of additional disk space will be used. 2024-09-11T12:45:58.620 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-11T12:45:58.638 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T12:45:58.648 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-11T12:45:58.669 INFO:teuthology.orchestra.run.smithi078.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T12:45:58.688 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T12:45:58.700 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4207-g13fc0377-1jammy [3,633 kB] 2024-09-11T12:45:58.710 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T12:45:58.712 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T12:45:58.715 INFO:teuthology.orchestra.run.smithi078.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4207-g13fc0377-1jammy [3,313 kB] 2024-09-11T12:45:58.715 INFO:teuthology.orchestra.run.smithi078.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-11T12:45:58.723 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T12:45:58.762 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T12:45:58.763 INFO:teuthology.orchestra.run.smithi078.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4207-g13fc0377-1jammy [720 kB] 2024-09-11T12:45:58.764 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T12:45:58.771 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T12:45:58.774 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T12:45:58.774 INFO:teuthology.orchestra.run.smithi078.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4207-g13fc0377-1jammy [356 kB] 2024-09-11T12:45:58.775 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T12:45:58.780 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T12:45:58.782 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4207-g13fc0377-1jammy [32.8 kB] 2024-09-11T12:45:58.782 INFO:teuthology.orchestra.run.smithi078.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4207-g13fc0377-1jammy [180 kB] 2024-09-11T12:45:58.783 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T12:45:58.784 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T12:45:58.785 INFO:teuthology.orchestra.run.smithi078.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4207-g13fc0377-1jammy [70.4 kB] 2024-09-11T12:45:58.785 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T12:45:58.786 INFO:teuthology.orchestra.run.smithi078.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4207-g13fc0377-1jammy [338 kB] 2024-09-11T12:45:58.786 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T12:45:58.792 INFO:teuthology.orchestra.run.smithi078.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4207-g13fc0377-1jammy [8,017 kB] 2024-09-11T12:45:58.800 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T12:45:58.800 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T12:45:58.802 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T12:45:58.802 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T12:45:58.815 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T12:45:58.816 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T12:45:58.817 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T12:45:58.817 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T12:45:58.818 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T12:45:58.818 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T12:45:58.833 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T12:45:58.833 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T12:45:58.835 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T12:45:58.836 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T12:45:58.850 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T12:45:58.850 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T12:45:58.851 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T12:45:58.852 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T12:45:58.852 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T12:45:58.854 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T12:45:58.867 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T12:45:58.867 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T12:45:58.872 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T12:45:58.872 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T12:45:58.884 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T12:45:58.885 INFO:teuthology.orchestra.run.smithi157.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T12:45:58.886 INFO:teuthology.orchestra.run.smithi157.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T12:45:58.887 INFO:teuthology.orchestra.run.smithi157.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T12:45:58.889 INFO:teuthology.orchestra.run.smithi157.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T12:45:58.890 INFO:teuthology.orchestra.run.smithi157.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T12:45:58.901 INFO:teuthology.orchestra.run.smithi157.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T12:45:58.903 INFO:teuthology.orchestra.run.smithi157.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T12:45:58.904 INFO:teuthology.orchestra.run.smithi157.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T12:45:58.905 INFO:teuthology.orchestra.run.smithi157.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T12:45:58.912 INFO:teuthology.orchestra.run.smithi078.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T12:45:58.918 INFO:teuthology.orchestra.run.smithi157.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T12:45:58.932 INFO:teuthology.orchestra.run.smithi078.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4207-g13fc0377-1jammy [112 kB] 2024-09-11T12:45:58.933 INFO:teuthology.orchestra.run.smithi078.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4207-g13fc0377-1jammy [473 kB] 2024-09-11T12:45:58.941 INFO:teuthology.orchestra.run.smithi078.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4207-g13fc0377-1jammy [28.3 MB] 2024-09-11T12:45:58.963 INFO:teuthology.orchestra.run.smithi157.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T12:45:58.964 INFO:teuthology.orchestra.run.smithi157.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T12:45:58.965 INFO:teuthology.orchestra.run.smithi157.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T12:45:58.965 INFO:teuthology.orchestra.run.smithi157.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T12:45:58.967 INFO:teuthology.orchestra.run.smithi157.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T12:45:58.985 INFO:teuthology.orchestra.run.smithi157.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T12:45:58.987 INFO:teuthology.orchestra.run.smithi157.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T12:45:58.988 INFO:teuthology.orchestra.run.smithi157.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T12:45:59.003 INFO:teuthology.orchestra.run.smithi078.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T12:45:59.004 INFO:teuthology.orchestra.run.smithi157.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T12:45:59.004 INFO:teuthology.orchestra.run.smithi157.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T12:45:59.005 INFO:teuthology.orchestra.run.smithi157.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T12:45:59.008 INFO:teuthology.orchestra.run.smithi157.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T12:45:59.008 INFO:teuthology.orchestra.run.smithi157.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T12:45:59.008 INFO:teuthology.orchestra.run.smithi157.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T12:45:59.016 INFO:teuthology.orchestra.run.smithi078.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T12:45:59.019 INFO:teuthology.orchestra.run.smithi157.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T12:45:59.019 INFO:teuthology.orchestra.run.smithi157.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T12:45:59.023 INFO:teuthology.orchestra.run.smithi157.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T12:45:59.036 INFO:teuthology.orchestra.run.smithi157.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T12:45:59.038 INFO:teuthology.orchestra.run.smithi157.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T12:45:59.039 INFO:teuthology.orchestra.run.smithi157.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T12:45:59.042 INFO:teuthology.orchestra.run.smithi157.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T12:45:59.045 INFO:teuthology.orchestra.run.smithi157.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-11T12:45:59.045 INFO:teuthology.orchestra.run.smithi157.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T12:45:59.047 INFO:teuthology.orchestra.run.smithi157.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T12:45:59.053 INFO:teuthology.orchestra.run.smithi157.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T12:45:59.057 INFO:teuthology.orchestra.run.smithi157.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T12:45:59.058 INFO:teuthology.orchestra.run.smithi157.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T12:45:59.066 INFO:teuthology.orchestra.run.smithi078.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T12:45:59.070 INFO:teuthology.orchestra.run.smithi157.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T12:45:59.071 INFO:teuthology.orchestra.run.smithi157.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T12:45:59.072 INFO:teuthology.orchestra.run.smithi157.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T12:45:59.072 INFO:teuthology.orchestra.run.smithi157.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T12:45:59.074 INFO:teuthology.orchestra.run.smithi157.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T12:45:59.075 INFO:teuthology.orchestra.run.smithi157.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T12:45:59.082 INFO:teuthology.orchestra.run.smithi157.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T12:45:59.087 INFO:teuthology.orchestra.run.smithi157.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T12:45:59.088 INFO:teuthology.orchestra.run.smithi157.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T12:45:59.090 INFO:teuthology.orchestra.run.smithi157.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T12:45:59.105 INFO:teuthology.orchestra.run.smithi157.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T12:45:59.239 INFO:teuthology.orchestra.run.smithi078.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T12:45:59.247 INFO:teuthology.orchestra.run.smithi078.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T12:45:59.275 INFO:teuthology.orchestra.run.smithi078.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T12:45:59.283 INFO:teuthology.orchestra.run.smithi078.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T12:45:59.284 INFO:teuthology.orchestra.run.smithi078.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T12:45:59.303 INFO:teuthology.orchestra.run.smithi078.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T12:45:59.306 INFO:teuthology.orchestra.run.smithi078.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T12:45:59.311 INFO:teuthology.orchestra.run.smithi078.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T12:45:59.314 INFO:teuthology.orchestra.run.smithi078.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T12:45:59.321 INFO:teuthology.orchestra.run.smithi078.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T12:45:59.396 INFO:teuthology.orchestra.run.smithi078.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T12:45:59.397 INFO:teuthology.orchestra.run.smithi078.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T12:45:59.399 INFO:teuthology.orchestra.run.smithi078.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T12:45:59.400 INFO:teuthology.orchestra.run.smithi078.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T12:45:59.400 INFO:teuthology.orchestra.run.smithi078.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T12:45:59.403 INFO:teuthology.orchestra.run.smithi078.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T12:45:59.404 INFO:teuthology.orchestra.run.smithi078.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T12:45:59.404 INFO:teuthology.orchestra.run.smithi078.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T12:45:59.404 INFO:teuthology.orchestra.run.smithi078.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T12:45:59.439 INFO:teuthology.orchestra.run.smithi078.stdout:Get:39 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4207-g13fc0377-1jammy [5,175 kB] 2024-09-11T12:45:59.484 INFO:teuthology.orchestra.run.smithi078.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T12:45:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T12:45:59.485 INFO:teuthology.orchestra.run.smithi078.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T12:45:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T12:45:59.487 INFO:teuthology.orchestra.run.smithi078.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T12:45:59.523 INFO:teuthology.orchestra.run.smithi078.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4207-g13fc0377-1jammy [247 kB] 2024-09-11T12:45:59.526 INFO:teuthology.orchestra.run.smithi078.stdout:Get:46 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4207-g13fc0377-1jammy [124 kB] 2024-09-11T12:45:59.527 INFO:teuthology.orchestra.run.smithi078.stdout:Get:47 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4207-g13fc0377-1jammy [1,560 kB] 2024-09-11T12:45:59.554 INFO:teuthology.orchestra.run.smithi078.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4207-g13fc0377-1jammy [6,293 kB] 2024-09-11T12:45:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T12:45:59.565 INFO:teuthology.orchestra.run.smithi078.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T12:45:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T12:45:59.566 INFO:teuthology.orchestra.run.smithi078.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T12:45:59.567 INFO:teuthology.orchestra.run.smithi078.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T12:45:59.646 INFO:teuthology.orchestra.run.smithi078.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T12:45:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T12:45:59.647 INFO:teuthology.orchestra.run.smithi078.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T12:45:59.652 INFO:teuthology.orchestra.run.smithi078.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T12:45:59.653 INFO:teuthology.orchestra.run.smithi078.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T12:45:59.669 INFO:teuthology.orchestra.run.smithi078.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4207-g13fc0377-1jammy [23.0 MB] 2024-09-11T12:45:59.727 INFO:teuthology.orchestra.run.smithi078.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T12:45:59.728 INFO:teuthology.orchestra.run.smithi078.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T12:45:59.729 INFO:teuthology.orchestra.run.smithi078.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T12:45:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T12:45:59.731 INFO:teuthology.orchestra.run.smithi078.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T12:45:59.800 INFO:teuthology.orchestra.run.smithi157.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4207-g13fc0377-1jammy [3,313 kB] 2024-09-11T12:45:59.808 INFO:teuthology.orchestra.run.smithi078.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T12:45:59.809 INFO:teuthology.orchestra.run.smithi078.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T12:45:59.811 INFO:teuthology.orchestra.run.smithi078.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T12:45:59.812 INFO:teuthology.orchestra.run.smithi078.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T12:45:59.812 INFO:teuthology.orchestra.run.smithi078.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T12:45:59.890 INFO:teuthology.orchestra.run.smithi078.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T12:45:59.998 INFO:teuthology.orchestra.run.smithi078.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T12:45:59.999 INFO:teuthology.orchestra.run.smithi078.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T12:46:00.001 INFO:teuthology.orchestra.run.smithi078.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T12:46:00.001 INFO:teuthology.orchestra.run.smithi078.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T12:46:00.002 INFO:teuthology.orchestra.run.smithi078.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T12:46:00.074 INFO:teuthology.orchestra.run.smithi078.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T12:46:00.076 INFO:teuthology.orchestra.run.smithi078.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T12:46:00.076 INFO:teuthology.orchestra.run.smithi078.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T12:46:00.084 INFO:teuthology.orchestra.run.smithi078.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4207-g13fc0377-1jammy [14.1 kB] 2024-09-11T12:46:00.084 INFO:teuthology.orchestra.run.smithi078.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4207-g13fc0377-1jammy [880 kB] 2024-09-11T12:46:00.092 INFO:teuthology.orchestra.run.smithi078.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4207-g13fc0377-1jammy [2,434 kB] 2024-09-11T12:46:00.139 INFO:teuthology.orchestra.run.smithi078.stdout:Get:82 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4207-g13fc0377-1jammy [801 kB] 2024-09-11T12:46:00.145 INFO:teuthology.orchestra.run.smithi078.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T12:46:00.146 INFO:teuthology.orchestra.run.smithi078.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T12:46:00.150 INFO:teuthology.orchestra.run.smithi078.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4207-g13fc0377-1jammy [156 kB] 2024-09-11T12:46:00.152 INFO:teuthology.orchestra.run.smithi078.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4207-g13fc0377-1jammy [61.3 MB] 2024-09-11T12:46:00.163 INFO:teuthology.orchestra.run.smithi078.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T12:46:00.164 INFO:teuthology.orchestra.run.smithi078.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T12:46:00.164 INFO:teuthology.orchestra.run.smithi078.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T12:46:00.165 INFO:teuthology.orchestra.run.smithi078.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T12:46:00.165 INFO:teuthology.orchestra.run.smithi078.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T12:46:00.166 INFO:teuthology.orchestra.run.smithi078.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T12:46:00.170 INFO:teuthology.orchestra.run.smithi078.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T12:46:00.244 INFO:teuthology.orchestra.run.smithi078.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T12:46:00.246 INFO:teuthology.orchestra.run.smithi078.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T12:46:00.247 INFO:teuthology.orchestra.run.smithi078.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T12:46:00.321 INFO:teuthology.orchestra.run.smithi157.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4207-g13fc0377-1jammy [720 kB] 2024-09-11T12:46:00.326 INFO:teuthology.orchestra.run.smithi078.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T12:46:00.328 INFO:teuthology.orchestra.run.smithi078.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-11T12:46:00.329 INFO:teuthology.orchestra.run.smithi078.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T12:46:00.330 INFO:teuthology.orchestra.run.smithi078.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T12:46:00.331 INFO:teuthology.orchestra.run.smithi078.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T12:46:00.335 INFO:teuthology.orchestra.run.smithi078.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T12:46:00.336 INFO:teuthology.orchestra.run.smithi078.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T12:46:00.379 INFO:teuthology.orchestra.run.smithi157.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4207-g13fc0377-1jammy [356 kB] 2024-09-11T12:46:00.407 INFO:teuthology.orchestra.run.smithi078.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T12:46:00.407 INFO:teuthology.orchestra.run.smithi078.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T12:46:00.408 INFO:teuthology.orchestra.run.smithi078.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T12:46:00.431 INFO:teuthology.orchestra.run.smithi157.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4207-g13fc0377-1jammy [32.8 kB] 2024-09-11T12:46:00.432 INFO:teuthology.orchestra.run.smithi157.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4207-g13fc0377-1jammy [180 kB] 2024-09-11T12:46:00.478 INFO:teuthology.orchestra.run.smithi157.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4207-g13fc0377-1jammy [70.4 kB] 2024-09-11T12:46:00.483 INFO:teuthology.orchestra.run.smithi157.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4207-g13fc0377-1jammy [338 kB] 2024-09-11T12:46:00.488 INFO:teuthology.orchestra.run.smithi078.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T12:46:00.490 INFO:teuthology.orchestra.run.smithi078.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T12:46:00.491 INFO:teuthology.orchestra.run.smithi078.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T12:46:00.498 INFO:teuthology.orchestra.run.smithi078.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T12:46:00.499 INFO:teuthology.orchestra.run.smithi078.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T12:46:00.499 INFO:teuthology.orchestra.run.smithi078.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T12:46:00.501 INFO:teuthology.orchestra.run.smithi078.stdout:Get:113 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T12:46:00.530 INFO:teuthology.orchestra.run.smithi157.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4207-g13fc0377-1jammy [8,017 kB] 2024-09-11T12:46:00.579 INFO:teuthology.orchestra.run.smithi078.stdout:Get:114 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T12:46:01.180 INFO:teuthology.orchestra.run.smithi157.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4207-g13fc0377-1jammy [112 kB] 2024-09-11T12:46:01.223 INFO:teuthology.orchestra.run.smithi157.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4207-g13fc0377-1jammy [473 kB] 2024-09-11T12:46:01.233 INFO:teuthology.orchestra.run.smithi157.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4207-g13fc0377-1jammy [28.3 MB] 2024-09-11T12:46:01.405 INFO:teuthology.orchestra.run.smithi078.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4207-g13fc0377-1jammy [8,625 kB] 2024-09-11T12:46:01.585 INFO:teuthology.orchestra.run.smithi078.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4207-g13fc0377-1jammy [14.2 kB] 2024-09-11T12:46:01.585 INFO:teuthology.orchestra.run.smithi078.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4207-g13fc0377-1jammy [55.8 MB] 2024-09-11T12:46:02.531 INFO:teuthology.orchestra.run.smithi157.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4207-g13fc0377-1jammy [5,175 kB] 2024-09-11T12:46:02.689 INFO:teuthology.orchestra.run.smithi157.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4207-g13fc0377-1jammy [247 kB] 2024-09-11T12:46:02.691 INFO:teuthology.orchestra.run.smithi157.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4207-g13fc0377-1jammy [124 kB] 2024-09-11T12:46:02.693 INFO:teuthology.orchestra.run.smithi157.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4207-g13fc0377-1jammy [1,560 kB] 2024-09-11T12:46:02.743 INFO:teuthology.orchestra.run.smithi157.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4207-g13fc0377-1jammy [6,293 kB] 2024-09-11T12:46:02.944 INFO:teuthology.orchestra.run.smithi157.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4207-g13fc0377-1jammy [23.0 MB] 2024-09-11T12:46:03.240 INFO:teuthology.orchestra.run.smithi078.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4207-g13fc0377-1jammy [125 kB] 2024-09-11T12:46:03.241 INFO:teuthology.orchestra.run.smithi078.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4207-g13fc0377-1jammy [232 kB] 2024-09-11T12:46:03.243 INFO:teuthology.orchestra.run.smithi078.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4207-g13fc0377-1jammy [33.9 kB] 2024-09-11T12:46:03.244 INFO:teuthology.orchestra.run.smithi078.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4207-g13fc0377-1jammy [23.4 kB] 2024-09-11T12:46:03.244 INFO:teuthology.orchestra.run.smithi078.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4207-g13fc0377-1jammy [38.9 kB] 2024-09-11T12:46:03.245 INFO:teuthology.orchestra.run.smithi078.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4207-g13fc0377-1jammy [16.0 MB] 2024-09-11T12:46:03.699 INFO:teuthology.orchestra.run.smithi157.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4207-g13fc0377-1jammy [14.1 kB] 2024-09-11T12:46:03.699 INFO:teuthology.orchestra.run.smithi157.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4207-g13fc0377-1jammy [880 kB] 2024-09-11T12:46:03.735 INFO:teuthology.orchestra.run.smithi157.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4207-g13fc0377-1jammy [2,434 kB] 2024-09-11T12:46:03.768 INFO:teuthology.orchestra.run.smithi078.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4207-g13fc0377-1jammy [91.3 kB] 2024-09-11T12:46:03.790 INFO:teuthology.orchestra.run.smithi157.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4207-g13fc0377-1jammy [801 kB] 2024-09-11T12:46:03.803 INFO:teuthology.orchestra.run.smithi157.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4207-g13fc0377-1jammy [156 kB] 2024-09-11T12:46:03.805 INFO:teuthology.orchestra.run.smithi157.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4207-g13fc0377-1jammy [61.3 MB] 2024-09-11T12:46:04.427 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 251 MB in 5s (46.8 MB/s) 2024-09-11T12:46:04.573 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-11T12:46:04.636 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-11T12:46:04.639 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-11T12:46:04.680 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-11T12:46:04.830 INFO:teuthology.orchestra.run.smithi157.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4207-g13fc0377-1jammy [8,625 kB] 2024-09-11T12:46:04.956 INFO:teuthology.orchestra.run.smithi157.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4207-g13fc0377-1jammy [14.2 kB] 2024-09-11T12:46:04.957 INFO:teuthology.orchestra.run.smithi157.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4207-g13fc0377-1jammy [55.8 MB] 2024-09-11T12:46:04.983 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T12:46:04.996 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T12:46:05.033 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T12:46:05.336 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cmd2. 2024-09-11T12:46:05.349 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-11T12:46:05.394 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-11T12:46:05.885 INFO:teuthology.orchestra.run.smithi157.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4207-g13fc0377-1jammy [125 kB] 2024-09-11T12:46:05.886 INFO:teuthology.orchestra.run.smithi157.stdout:Get:119 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4207-g13fc0377-1jammy [232 kB] 2024-09-11T12:46:05.888 INFO:teuthology.orchestra.run.smithi157.stdout:Get:120 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4207-g13fc0377-1jammy [33.9 kB] 2024-09-11T12:46:05.889 INFO:teuthology.orchestra.run.smithi157.stdout:Get:121 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4207-g13fc0377-1jammy [23.4 kB] 2024-09-11T12:46:05.890 INFO:teuthology.orchestra.run.smithi157.stdout:Get:122 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4207-g13fc0377-1jammy [38.9 kB] 2024-09-11T12:46:05.890 INFO:teuthology.orchestra.run.smithi157.stdout:Get:123 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4207-g13fc0377-1jammy [16.0 MB] 2024-09-11T12:46:05.930 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T12:46:05.944 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T12:46:06.031 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T12:46:06.142 INFO:teuthology.orchestra.run.smithi157.stdout:Get:124 https://3.chacra.ceph.com/r/ceph/wip-athakkar-testing-2024-09-11-1208/13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4207-g13fc0377-1jammy [91.3 kB] 2024-09-11T12:46:06.351 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T12:46:06.364 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T12:46:06.409 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T12:46:06.687 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T12:46:06.701 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T12:46:06.756 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T12:46:06.879 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 251 MB in 8s (33.1 MB/s) 2024-09-11T12:46:07.142 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-11T12:46:07.165 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T12:46:07.180 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:07.208 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-11T12:46:07.211 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-11T12:46:07.232 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:07.249 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-11T12:46:07.535 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T12:46:07.548 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T12:46:07.593 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T12:46:07.627 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T12:46:07.642 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:07.686 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:07.938 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cmd2. 2024-09-11T12:46:07.952 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-11T12:46:07.996 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-11T12:46:08.022 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T12:46:08.037 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:08.080 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:08.341 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T12:46:08.355 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T12:46:08.417 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T12:46:08.431 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T12:46:08.441 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T12:46:08.467 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T12:46:08.728 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T12:46:08.742 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T12:46:08.770 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librados2. 2024-09-11T12:46:08.785 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../010-librados2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:08.786 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T12:46:08.828 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librados2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:09.147 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T12:46:09.162 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T12:46:09.205 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T12:46:09.256 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librbd1. 2024-09-11T12:46:09.271 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../011-librbd1_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:09.314 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librbd1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:09.676 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T12:46:09.691 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:09.709 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T12:46:09.723 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../012-libcephfs2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:09.742 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:09.767 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libcephfs2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:10.045 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rados. 2024-09-11T12:46:10.060 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../013-python3-rados_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:10.095 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rados (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:10.112 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T12:46:10.127 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:10.170 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:10.406 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T12:46:10.420 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:10.464 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-ceph-argparse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:10.532 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T12:46:10.547 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T12:46:10.599 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:46:10.717 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T12:46:10.731 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../015-python3-cephfs_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:10.775 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cephfs (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:10.960 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T12:46:10.975 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T12:46:11.010 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T12:46:11.279 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librados2. 2024-09-11T12:46:11.294 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../010-librados2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:11.312 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T12:46:11.326 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../016-python3-ceph-common_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:11.338 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librados2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:11.361 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:11.681 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T12:46:11.696 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T12:46:11.731 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T12:46:11.816 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librbd1. 2024-09-11T12:46:11.830 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../011-librbd1_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:11.957 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librbd1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:11.984 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T12:46:11.998 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../018-python3-rbd_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:12.042 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rbd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:12.387 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T12:46:12.401 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T12:46:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T12:46:12.602 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T12:46:12.616 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../012-libcephfs2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:12.660 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:12.706 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T12:46:12.721 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T12:46:12.764 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T12:46:12.971 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rados. 2024-09-11T12:46:12.985 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../013-python3-rados_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:13.017 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T12:46:13.029 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rados (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:13.032 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T12:46:13.067 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T12:46:13.332 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T12:46:13.346 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../014-python3-ceph-argparse_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:13.399 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-argparse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:13.412 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package lua5.1. 2024-09-11T12:46:13.427 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T12:46:13.471 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T12:46:13.685 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T12:46:13.699 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../015-python3-cephfs_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:13.734 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cephfs (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:13.782 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package lua-any. 2024-09-11T12:46:13.797 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-11T12:46:13.840 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T12:46:13.996 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T12:46:14.010 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../016-python3-ceph-common_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:14.053 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:14.076 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package zip. 2024-09-11T12:46:14.091 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-11T12:46:14.126 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T12:46:14.407 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T12:46:14.422 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../017-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T12:46:14.438 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package unzip. 2024-09-11T12:46:14.453 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T12:46:14.465 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T12:46:14.496 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T12:46:14.726 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T12:46:14.741 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../018-python3-rbd_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:14.793 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rbd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:14.908 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package luarocks. 2024-09-11T12:46:14.923 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T12:46:14.983 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T12:46:15.154 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T12:46:15.169 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../019-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T12:46:15.212 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T12:46:15.383 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package librgw2. 2024-09-11T12:46:15.396 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../027-librgw2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:15.441 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking librgw2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:15.466 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T12:46:15.481 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../020-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T12:46:15.523 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T12:46:15.801 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T12:46:15.816 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../021-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T12:46:15.859 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T12:46:16.127 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T12:46:16.142 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../028-python3-rgw_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:16.177 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:16.188 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua5.1. 2024-09-11T12:46:16.203 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../022-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T12:46:16.246 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T12:46:16.514 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T12:46:16.528 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T12:46:16.557 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-any. 2024-09-11T12:46:16.572 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T12:46:16.572 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../023-lua-any_27ubuntu1_all.deb ... 2024-09-11T12:46:16.609 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T12:46:16.883 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T12:46:16.887 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package zip. 2024-09-11T12:46:16.898 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../030-libradosstriper1_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:16.902 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../024-zip_3.0-12build2_amd64.deb ... 2024-09-11T12:46:16.941 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libradosstriper1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:16.945 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T12:46:17.256 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package unzip. 2024-09-11T12:46:17.271 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../025-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T12:46:17.295 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-common. 2024-09-11T12:46:17.309 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../031-ceph-common_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:17.314 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T12:46:17.345 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:17.861 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package luarocks. 2024-09-11T12:46:17.875 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../026-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T12:46:17.910 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T12:46:18.343 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package librgw2. 2024-09-11T12:46:18.357 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../027-librgw2_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:18.401 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking librgw2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:18.509 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-base. 2024-09-11T12:46:18.523 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../032-ceph-base_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:18.576 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-base (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:18.937 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T12:46:18.951 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../028-python3-rgw_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:18.995 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:19.105 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T12:46:19.119 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T12:46:19.163 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T12:46:19.357 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T12:46:19.371 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../029-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T12:46:19.415 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T12:46:19.466 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T12:46:19.481 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T12:46:19.524 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T12:46:19.709 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T12:46:19.724 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../030-libradosstriper1_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:19.768 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libradosstriper1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:19.861 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T12:46:19.875 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T12:46:19.919 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T12:46:20.196 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-common. 2024-09-11T12:46:20.211 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../031-ceph-common_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:20.230 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T12:46:20.245 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T12:46:20.254 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:20.288 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T12:46:20.591 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T12:46:20.606 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T12:46:20.650 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T12:46:20.927 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T12:46:20.942 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T12:46:20.977 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T12:46:21.281 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-portend. 2024-09-11T12:46:21.295 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-11T12:46:21.339 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T12:46:21.459 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-base. 2024-09-11T12:46:21.475 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../032-ceph-base_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:21.527 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-base (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:21.642 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T12:46:21.656 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T12:46:21.700 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T12:46:22.003 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T12:46:22.018 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T12:46:22.061 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T12:46:22.122 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T12:46:22.136 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../033-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T12:46:22.180 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T12:46:22.450 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T12:46:22.457 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T12:46:22.464 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../034-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T12:46:22.471 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T12:46:22.508 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T12:46:22.523 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T12:46:22.918 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T12:46:22.933 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T12:46:22.985 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T12:46:23.045 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T12:46:23.059 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../035-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T12:46:23.103 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T12:46:23.304 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T12:46:23.319 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T12:46:23.363 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T12:46:23.439 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T12:46:23.455 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../036-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T12:46:23.497 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T12:46:23.707 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-mako. 2024-09-11T12:46:23.722 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T12:46:23.774 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T12:46:23.792 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T12:46:23.808 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../037-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T12:46:23.867 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T12:46:24.170 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T12:46:24.177 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T12:46:24.185 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../038-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T12:46:24.192 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T12:46:24.229 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T12:46:24.244 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T12:46:24.547 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-portend. 2024-09-11T12:46:24.562 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../039-python3-portend_3.0.0-1_all.deb ... 2024-09-11T12:46:24.597 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T12:46:24.622 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T12:46:24.637 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T12:46:24.689 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T12:46:24.875 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T12:46:24.890 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../040-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T12:46:24.933 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T12:46:25.000 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-webob. 2024-09-11T12:46:25.014 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T12:46:25.058 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T12:46:25.270 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T12:46:25.285 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../041-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T12:46:25.328 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T12:46:25.464 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T12:46:25.476 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T12:46:25.531 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T12:46:25.648 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T12:46:25.662 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../042-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T12:46:25.698 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T12:46:25.839 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T12:46:25.854 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T12:46:25.898 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T12:46:26.018 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T12:46:26.033 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../043-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T12:46:26.076 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T12:46:26.379 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T12:46:26.393 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../044-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T12:46:26.434 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-paste. 2024-09-11T12:46:26.449 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T12:46:26.462 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T12:46:26.484 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T12:46:26.765 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-mako. 2024-09-11T12:46:26.779 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../045-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T12:46:26.804 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T12:46:26.818 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T12:46:26.823 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T12:46:26.870 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T12:46:27.168 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T12:46:27.183 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../046-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T12:46:27.227 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T12:46:27.257 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T12:46:27.272 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T12:46:27.323 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T12:46:27.479 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T12:46:27.494 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../047-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T12:46:27.537 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T12:46:27.635 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T12:46:27.650 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T12:46:27.720 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T12:46:27.874 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webob. 2024-09-11T12:46:27.889 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../048-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T12:46:27.932 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T12:46:28.088 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T12:46:28.103 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T12:46:28.155 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T12:46:28.268 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T12:46:28.282 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../049-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T12:46:28.355 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T12:46:28.534 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T12:46:28.549 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T12:46:28.593 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T12:46:28.663 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T12:46:28.678 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../050-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T12:46:28.721 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T12:46:28.904 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-idna. 2024-09-11T12:46:28.919 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T12:46:28.962 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T12:46:29.024 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-paste. 2024-09-11T12:46:29.039 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../051-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T12:46:29.074 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T12:46:29.307 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T12:46:29.322 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T12:46:29.357 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T12:46:29.394 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T12:46:29.408 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../052-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T12:46:29.452 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T12:46:29.635 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-requests. 2024-09-11T12:46:29.650 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T12:46:29.685 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T12:46:29.738 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T12:46:29.770 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../053-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T12:46:29.788 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T12:46:29.989 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T12:46:30.003 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T12:46:30.046 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T12:46:30.083 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T12:46:30.097 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../054-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T12:46:30.140 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T12:46:30.333 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T12:46:30.348 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:30.391 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:30.485 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T12:46:30.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../055-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T12:46:30.544 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T12:46:30.771 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T12:46:30.786 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:30.838 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:30.932 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T12:46:30.946 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../056-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T12:46:30.990 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T12:46:31.166 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T12:46:31.181 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T12:46:31.224 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T12:46:31.293 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-idna. 2024-09-11T12:46:31.307 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../057-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T12:46:31.351 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T12:46:31.527 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T12:46:31.541 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T12:46:31.585 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T12:46:31.679 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T12:46:31.694 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../058-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T12:46:31.737 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T12:46:31.922 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T12:46:31.936 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../065-ceph-mgr_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:31.988 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:32.057 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests. 2024-09-11T12:46:32.072 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../059-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T12:46:32.115 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T12:46:33.208 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T12:46:33.223 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../066-ceph-mon_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:33.258 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mon (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:33.452 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T12:46:33.467 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../060-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T12:46:33.510 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T12:46:33.770 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T12:46:33.785 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../067-ceph-osd_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:33.805 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T12:46:33.819 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-osd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:33.820 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../061-ceph-mgr-modules-core_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:33.863 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:34.226 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T12:46:34.240 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../062-libsqlite3-mod-ceph_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:34.284 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:34.590 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph. 2024-09-11T12:46:34.605 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../068-ceph_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:34.612 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T12:46:34.626 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../063-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T12:46:34.648 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:34.670 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T12:46:34.967 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T12:46:34.982 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../069-ceph-fuse_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:34.993 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T12:46:35.007 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../064-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T12:46:35.025 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:35.056 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T12:46:35.376 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T12:46:35.379 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T12:46:35.391 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../065-ceph-mgr_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:35.393 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../070-ceph-mds_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:35.428 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mds (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:35.434 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:35.790 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephadm. 2024-09-11T12:46:35.805 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../071-cephadm_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:35.837 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T12:46:35.848 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:35.853 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../066-ceph-mon_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:35.887 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mon (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:36.185 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T12:46:36.200 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T12:46:36.243 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T12:46:36.466 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T12:46:36.479 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../067-ceph-osd_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:36.525 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-osd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:36.537 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T12:46:36.552 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T12:46:36.595 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T12:46:37.294 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph. 2024-09-11T12:46:37.308 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../068-ceph_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:37.352 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:37.592 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-babel. 2024-09-11T12:46:37.606 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T12:46:37.683 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T12:46:37.698 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T12:46:37.713 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../069-ceph-fuse_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:37.754 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:38.028 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T12:46:38.042 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T12:46:38.074 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T12:46:38.086 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T12:46:38.089 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../070-ceph-mds_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:38.124 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mds (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:38.415 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T12:46:38.430 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:38.465 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:38.661 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephadm. 2024-09-11T12:46:38.676 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../071-cephadm_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:38.719 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:38.802 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T12:46:38.817 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T12:46:38.860 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T12:46:39.055 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T12:46:39.070 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../072-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T12:46:39.113 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T12:46:39.129 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-routes. 2024-09-11T12:46:39.144 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T12:46:39.187 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T12:46:39.432 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T12:46:39.447 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../073-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T12:46:39.515 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T12:46:39.543 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T12:46:39.558 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:39.601 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:40.545 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-babel. 2024-09-11T12:46:40.559 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../074-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T12:46:40.604 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T12:46:40.940 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T12:46:40.954 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../075-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T12:46:40.998 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T12:46:41.334 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T12:46:41.348 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../076-ceph-mgr-cephadm_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:41.392 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:41.729 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T12:46:41.743 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../077-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T12:46:41.788 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T12:46:41.932 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T12:46:41.947 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T12:46:41.998 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T12:46:42.115 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-routes. 2024-09-11T12:46:42.129 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../078-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T12:46:42.173 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T12:46:42.485 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T12:46:42.499 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T12:46:42.510 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T12:46:42.524 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../079-ceph-mgr-dashboard_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:42.543 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T12:46:42.568 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:42.938 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T12:46:42.953 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T12:46:43.074 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T12:46:43.361 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T12:46:43.375 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T12:46:43.410 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T12:46:44.045 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T12:46:44.059 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:44.103 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:44.998 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T12:46:45.014 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../080-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T12:46:45.015 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T12:46:45.030 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T12:46:45.065 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T12:46:45.073 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T12:46:45.376 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T12:46:45.391 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T12:46:45.434 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T12:46:45.593 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T12:46:45.607 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../081-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T12:46:45.651 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T12:46:45.771 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T12:46:45.787 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T12:46:45.830 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T12:46:46.013 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T12:46:46.028 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../082-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T12:46:46.140 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T12:46:46.166 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T12:46:46.181 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-11T12:46:46.224 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T12:46:46.468 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T12:46:46.482 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../083-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T12:46:46.494 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T12:46:46.510 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T12:46:46.526 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T12:46:46.552 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T12:46:47.039 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T12:46:47.055 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T12:46:47.097 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T12:46:47.169 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T12:46:47.183 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../084-ceph-mgr-diskprediction-local_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:47.227 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:47.400 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T12:46:47.416 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T12:46:47.484 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T12:46:47.804 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T12:46:47.819 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T12:46:47.853 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T12:46:48.164 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T12:46:48.180 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../085-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T12:46:48.222 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T12:46:48.626 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T12:46:48.641 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:48.675 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T12:46:48.684 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:48.691 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../086-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T12:46:48.755 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T12:46:49.004 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T12:46:49.018 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T12:46:49.087 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T12:46:49.087 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T12:46:49.102 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../087-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T12:46:49.145 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T12:46:49.382 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T12:46:49.396 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T12:46:49.432 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T12:46:49.465 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T12:46:49.486 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../088-python3-rsa_4.8-1_all.deb ... 2024-09-11T12:46:49.672 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T12:46:49.693 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package jq. 2024-09-11T12:46:49.707 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T12:46:49.751 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T12:46:49.951 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T12:46:49.967 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../089-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T12:46:49.987 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package socat. 2024-09-11T12:46:50.002 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T12:46:50.027 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T12:46:50.045 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T12:46:50.340 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T12:46:50.346 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T12:46:50.355 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T12:46:50.362 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../090-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T12:46:50.398 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T12:46:50.404 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T12:46:50.724 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T12:46:50.739 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../091-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T12:46:50.761 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-test. 2024-09-11T12:46:50.776 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../099-ceph-test_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:50.782 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T12:46:50.820 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-test (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:51.093 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T12:46:51.109 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../092-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T12:46:51.151 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T12:46:51.857 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T12:46:51.872 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../093-ceph-mgr-k8sevents_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:51.915 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:52.227 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T12:46:52.241 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../094-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T12:46:52.284 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T12:46:52.646 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T12:46:52.661 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../095-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T12:46:52.704 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T12:46:52.874 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T12:46:52.890 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../100-ceph-volume_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:52.932 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ceph-volume (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:52.965 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package jq. 2024-09-11T12:46:52.980 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../096-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T12:46:53.024 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T12:46:53.285 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package socat. 2024-09-11T12:46:53.299 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../097-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T12:46:53.312 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-11T12:46:53.327 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../101-cephfs-mirror_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:53.334 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T12:46:53.370 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-mirror (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:53.621 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T12:46:53.636 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../098-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T12:46:53.656 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-colorama. 2024-09-11T12:46:53.672 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-11T12:46:53.679 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T12:46:53.706 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-11T12:46:53.942 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-shell. 2024-09-11T12:46:53.958 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../103-cephfs-shell_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:54.000 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-shell (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:54.033 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-test. 2024-09-11T12:46:54.048 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../099-ceph-test_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:54.083 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-test (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:54.270 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package cephfs-top. 2024-09-11T12:46:54.285 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../104-cephfs-top_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:54.327 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking cephfs-top (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:54.581 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T12:46:54.596 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../105-libcephfs-dev_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:54.638 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libcephfs-dev (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:54.933 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T12:46:54.949 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T12:46:54.991 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T12:46:55.296 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T12:46:55.312 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T12:46:55.354 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T12:46:55.725 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T12:46:55.739 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T12:46:55.799 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T12:46:56.114 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T12:46:56.128 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../100-ceph-volume_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:56.172 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ceph-volume (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:56.205 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package pkg-config. 2024-09-11T12:46:56.220 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T12:46:56.263 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T12:46:56.516 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T12:46:56.531 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T12:46:56.551 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-11T12:46:56.563 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../101-cephfs-mirror_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:56.566 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T12:46:56.626 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-mirror (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:56.929 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-colorama. 2024-09-11T12:46:56.942 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../102-python3-colorama_0.4.4-1_all.deb ... 2024-09-11T12:46:56.944 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T12:46:56.959 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T12:46:56.971 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-11T12:46:57.002 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T12:46:57.240 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-shell. 2024-09-11T12:46:57.253 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../103-cephfs-shell_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:57.290 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-shell (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:57.305 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T12:46:57.319 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T12:46:57.388 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T12:46:57.526 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package cephfs-top. 2024-09-11T12:46:57.537 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../104-cephfs-top_19.3.0-4207-g13fc0377-1jammy_all.deb ... 2024-09-11T12:46:57.576 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking cephfs-top (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:57.708 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T12:46:57.723 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T12:46:57.767 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T12:46:57.828 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T12:46:57.839 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../105-libcephfs-dev_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:46:57.895 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libcephfs-dev (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:46:58.095 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T12:46:58.110 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T12:46:58.153 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T12:46:58.223 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T12:46:58.236 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../106-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T12:46:58.273 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T12:46:58.481 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-py. 2024-09-11T12:46:58.496 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-11T12:46:58.539 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T12:46:58.619 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T12:46:58.632 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../107-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T12:46:58.677 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T12:46:58.868 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T12:46:58.883 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T12:46:58.926 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T12:46:59.014 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T12:46:59.026 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../108-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T12:46:59.064 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T12:46:59.363 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T12:46:59.378 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T12:46:59.422 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T12:46:59.511 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pkg-config. 2024-09-11T12:46:59.523 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../109-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T12:46:59.560 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T12:46:59.733 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-toml. 2024-09-11T12:46:59.747 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-11T12:46:59.791 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T12:46:59.872 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T12:46:59.885 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../110-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T12:46:59.922 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T12:47:00.077 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T12:47:00.092 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T12:47:00.135 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T12:47:00.350 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T12:47:00.362 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../111-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T12:47:00.400 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T12:47:00.464 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T12:47:00.479 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T12:47:00.522 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T12:47:00.711 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T12:47:00.723 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../112-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T12:47:00.761 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T12:47:00.825 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T12:47:00.840 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T12:47:00.883 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T12:47:01.039 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T12:47:01.051 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../113-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T12:47:01.114 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T12:47:01.409 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T12:47:01.421 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../114-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T12:47:01.458 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T12:47:01.538 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package radosgw. 2024-09-11T12:47:01.552 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../122-radosgw_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:47:01.596 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking radosgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:01.819 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-py. 2024-09-11T12:47:01.831 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../115-python3-py_1.10.0-1_all.deb ... 2024-09-11T12:47:01.869 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T12:47:02.190 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T12:47:02.201 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../116-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T12:47:02.240 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T12:47:02.524 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T12:47:02.541 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../123-rbd-fuse_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:47:02.582 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking rbd-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:02.710 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T12:47:02.722 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../117-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T12:47:02.760 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T12:47:02.885 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T12:47:03.087 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-toml. 2024-09-11T12:47:03.099 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../118-python3-toml_0.10.2-1_all.deb ... 2024-09-11T12:47:03.137 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T12:47:03.154 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T12:47:03.280 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T12:47:03.440 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T12:47:03.453 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../119-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T12:47:03.490 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T12:47:03.519 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T12:47:03.869 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T12:47:03.881 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../120-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T12:47:03.918 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T12:47:03.942 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T12:47:04.188 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T12:47:04.200 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../121-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T12:47:04.238 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T12:47:04.547 INFO:teuthology.orchestra.run.smithi078.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T12:47:04.588 INFO:teuthology.orchestra.run.smithi078.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-11T12:47:04.632 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:05.000 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package radosgw. 2024-09-11T12:47:05.013 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../122-radosgw_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:47:05.075 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking radosgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:05.142 INFO:teuthology.orchestra.run.smithi078.stdout:Adding system user cephadm....done 2024-09-11T12:47:05.272 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T12:47:05.557 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T12:47:05.770 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T12:47:05.779 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../123-rbd-fuse_19.3.0-4207-g13fc0377-1jammy_amd64.deb ... 2024-09-11T12:47:05.816 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T12:47:05.819 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking rbd-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:05.942 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T12:47:06.096 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T12:47:06.202 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T12:47:06.367 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T12:47:06.469 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T12:47:06.492 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T12:47:06.596 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T12:47:06.751 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T12:47:06.905 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T12:47:07.156 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T12:47:07.274 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T12:47:07.542 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-11T12:47:07.791 INFO:teuthology.orchestra.run.smithi157.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T12:47:07.802 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T12:47:07.804 INFO:teuthology.orchestra.run.smithi157.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-11T12:47:07.861 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:08.087 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T12:47:08.363 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T12:47:08.521 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user cephadm....done 2024-09-11T12:47:08.614 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T12:47:08.683 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T12:47:08.968 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T12:47:08.974 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T12:47:09.236 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T12:47:09.241 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-ceph-argparse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:09.362 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T12:47:09.518 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T12:47:09.613 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T12:47:09.644 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T12:47:09.770 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-11T12:47:09.881 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T12:47:10.007 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T12:47:10.046 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T12:47:10.172 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T12:47:10.317 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T12:47:10.430 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T12:47:10.686 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T12:47:10.852 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T12:47:10.887 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T12:47:10.945 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-11T12:47:10.951 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T12:47:11.052 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T12:47:11.205 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T12:47:11.412 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T12:47:11.490 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T12:47:11.555 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T12:47:11.791 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T12:47:11.822 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T12:47:12.099 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T12:47:12.159 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T12:47:12.408 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T12:47:12.427 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T12:47:12.534 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T12:47:12.694 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-argparse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:12.987 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T12:47:13.105 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T12:47:13.220 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T12:47:13.231 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-11T12:47:13.488 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T12:47:13.490 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T12:47:13.616 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T12:47:13.632 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T12:47:13.765 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T12:47:13.875 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T12:47:14.099 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T12:47:14.316 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T12:47:14.364 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T12:47:14.429 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T12:47:14.501 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T12:47:14.555 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T12:47:14.760 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T12:47:14.923 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T12:47:15.049 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T12:47:15.145 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T12:47:15.472 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-11T12:47:15.657 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T12:47:15.790 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T12:47:15.967 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T12:47:16.141 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T12:47:16.277 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T12:47:16.395 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T12:47:16.400 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T12:47:16.527 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:16.837 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T12:47:17.064 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T12:47:17.187 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T12:47:17.331 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T12:47:17.457 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T12:47:17.583 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T12:47:17.909 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T12:47:18.319 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T12:47:18.402 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T12:47:18.578 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T12:47:18.662 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:18.797 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T12:47:18.963 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T12:47:19.073 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T12:47:19.199 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T12:47:19.264 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-11T12:47:19.325 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T12:47:19.582 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T12:47:19.635 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T12:47:19.886 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T12:47:19.950 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T12:47:20.146 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:20.209 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T12:47:20.263 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T12:47:20.335 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:20.556 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T12:47:20.670 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T12:47:20.824 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T12:47:21.030 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T12:47:21.100 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T12:47:21.368 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T12:47:21.644 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T12:47:21.920 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T12:47:22.038 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T12:47:22.286 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T12:47:22.306 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T12:47:22.432 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T12:47:22.546 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:22.663 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T12:47:22.708 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T12:47:22.948 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T12:47:23.020 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T12:47:23.099 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T12:47:23.225 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T12:47:23.279 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T12:47:23.388 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T12:47:23.498 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T12:47:23.526 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T12:47:23.603 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T12:47:23.819 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T12:47:23.841 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T12:47:23.967 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T12:47:24.070 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:24.179 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T12:47:24.360 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T12:47:24.498 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T12:47:24.627 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T12:47:24.762 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T12:47:24.765 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T12:47:25.042 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T12:47:25.046 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:25.172 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T12:47:25.309 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T12:47:25.482 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T12:47:25.585 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T12:47:25.783 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T12:47:25.853 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T12:47:25.954 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T12:47:26.067 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T12:47:26.238 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T12:47:26.295 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T12:47:26.364 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T12:47:26.571 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T12:47:26.666 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T12:47:26.847 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T12:47:26.983 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T12:47:27.132 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T12:47:27.251 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T12:47:27.377 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T12:47:27.517 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T12:47:27.528 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T12:47:27.608 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T12:47:27.810 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T12:47:27.864 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T12:47:27.989 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T12:47:28.162 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T12:47:28.288 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librados2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:28.383 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T12:47:28.414 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:28.523 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T12:47:28.658 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T12:47:28.801 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T12:47:28.851 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libcephfs2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:28.977 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libradosstriper1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:29.085 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T12:47:29.112 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T12:47:29.186 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T12:47:29.397 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librbd1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:29.504 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T12:47:29.514 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-modules-core (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:29.665 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:29.814 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T12:47:29.875 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T12:47:29.875 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T12:47:30.098 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T12:47:30.301 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T12:47:30.516 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libcephfs-dev (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:30.560 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T12:47:30.642 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rados (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:30.768 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up librgw2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:30.812 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T12:47:30.894 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-top (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:31.028 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T12:47:31.080 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T12:47:31.474 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T12:47:31.775 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T12:47:32.127 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T12:47:32.245 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librados2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.298 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rbd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.363 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.416 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up rbd-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.489 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T12:47:32.525 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-rgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.651 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up python3-cephfs (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.850 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:32.976 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libradosstriper1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.069 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-shell (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.102 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T12:47:33.203 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.387 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librbd1 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.513 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-modules-core (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.639 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:33.747 INFO:teuthology.orchestra.run.smithi078.stdout:Adding group ceph....done 2024-09-11T12:47:33.848 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T12:47:33.848 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T12:47:34.148 INFO:teuthology.orchestra.run.smithi078.stdout:Adding system user ceph....done 2024-09-11T12:47:34.326 INFO:teuthology.orchestra.run.smithi078.stdout:Setting system user ceph properties....done 2024-09-11T12:47:34.341 INFO:teuthology.orchestra.run.smithi078.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T12:47:34.474 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libcephfs-dev (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:34.481 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T12:47:34.625 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rados (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:34.751 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up librgw2 (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:34.868 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-top (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:34.882 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T12:47:34.969 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T12:47:35.554 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up cephfs-mirror (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:35.761 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-11T12:47:35.761 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-11T12:47:36.240 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rbd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.369 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up rbd-fuse (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.380 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-test (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.506 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up radosgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.516 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-rgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.643 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up python3-cephfs (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.760 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-shell (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:36.861 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-common (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:37.011 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T12:47:37.011 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T12:47:37.440 INFO:teuthology.orchestra.run.smithi157.stdout:Adding group ceph....done 2024-09-11T12:47:37.574 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-base (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:37.865 INFO:teuthology.orchestra.run.smithi157.stdout:Adding system user ceph....done 2024-09-11T12:47:37.933 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T12:47:38.010 INFO:teuthology.orchestra.run.smithi157.stdout:Setting system user ceph properties....done 2024-09-11T12:47:38.024 INFO:teuthology.orchestra.run.smithi157.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T12:47:38.166 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T12:47:38.581 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T12:47:38.593 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mds (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:38.823 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T12:47:38.823 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T12:47:39.279 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up cephfs-mirror (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:39.478 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:39.488 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-11T12:47:39.488 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-11T12:47:39.690 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T12:47:39.690 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T12:47:40.147 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-test (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:40.298 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up radosgw (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:40.447 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-osd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:40.774 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T12:47:40.774 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T12:47:40.859 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T12:47:40.859 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T12:47:41.414 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:41.459 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-base (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:41.540 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:41.674 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mon (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:41.826 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T12:47:41.867 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T12:47:41.868 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T12:47:42.494 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mds (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:42.507 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:42.651 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:42.701 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T12:47:42.701 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T12:47:42.777 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-mgr-dashboard (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:42.929 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ceph-volume (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:43.289 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T12:47:43.330 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:43.406 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T12:47:43.533 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T12:47:43.544 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T12:47:43.544 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T12:47:44.181 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-osd (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:44.510 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T12:47:44.510 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T12:47:44.702 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T12:47:45.157 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:45.283 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T12:47:45.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-11T12:47:45.468 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mon (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:45.499 INFO:teuthology.orchestra.run.smithi078.stdout:Reading package lists... 2024-09-11T12:47:45.660 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T12:47:45.660 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T12:47:45.682 INFO:teuthology.orchestra.run.smithi078.stdout:Building dependency tree... 2024-09-11T12:47:45.683 INFO:teuthology.orchestra.run.smithi078.stdout:Reading state information... 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T12:47:45.866 INFO:teuthology.orchestra.run.smithi078.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T12:47:45.867 INFO:teuthology.orchestra.run.smithi078.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T12:47:45.867 INFO:teuthology.orchestra.run.smithi078.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T12:47:45.867 INFO:teuthology.orchestra.run.smithi078.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T12:47:45.868 INFO:teuthology.orchestra.run.smithi078.stdout:The following additional packages will be installed: 2024-09-11T12:47:45.868 INFO:teuthology.orchestra.run.smithi078.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-11T12:47:45.868 INFO:teuthology.orchestra.run.smithi078.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-11T12:47:45.869 INFO:teuthology.orchestra.run.smithi078.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-11T12:47:45.869 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-11T12:47:45.869 INFO:teuthology.orchestra.run.smithi078.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-11T12:47:45.870 INFO:teuthology.orchestra.run.smithi078.stdout:Suggested packages: 2024-09-11T12:47:45.870 INFO:teuthology.orchestra.run.smithi078.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-11T12:47:45.870 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-11T12:47:45.870 INFO:teuthology.orchestra.run.smithi078.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-11T12:47:45.927 INFO:teuthology.orchestra.run.smithi078.stdout:The following NEW packages will be installed: 2024-09-11T12:47:45.927 INFO:teuthology.orchestra.run.smithi078.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-11T12:47:45.928 INFO:teuthology.orchestra.run.smithi078.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-11T12:47:45.928 INFO:teuthology.orchestra.run.smithi078.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-11T12:47:45.928 INFO:teuthology.orchestra.run.smithi078.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-11T12:47:45.928 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-11T12:47:45.929 INFO:teuthology.orchestra.run.smithi078.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-11T12:47:45.929 INFO:teuthology.orchestra.run.smithi078.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-11T12:47:45.930 INFO:teuthology.orchestra.run.smithi078.stdout:The following packages will be upgraded: 2024-09-11T12:47:45.930 INFO:teuthology.orchestra.run.smithi078.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-11T12:47:46.116 INFO:teuthology.orchestra.run.smithi078.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-11T12:47:46.116 INFO:teuthology.orchestra.run.smithi078.stdout:Need to get 55.8 MB of archives. 2024-09-11T12:47:46.116 INFO:teuthology.orchestra.run.smithi078.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-11T12:47:46.116 INFO:teuthology.orchestra.run.smithi078.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-11T12:47:46.525 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-cephadm (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:46.739 INFO:teuthology.orchestra.run.smithi078.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-11T12:47:46.754 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:46.755 INFO:teuthology.orchestra.run.smithi078.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-11T12:47:46.880 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-mgr-dashboard (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:46.881 INFO:teuthology.orchestra.run.smithi078.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-11T12:47:46.881 INFO:teuthology.orchestra.run.smithi078.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-11T12:47:46.882 INFO:teuthology.orchestra.run.smithi078.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-11T12:47:46.911 INFO:teuthology.orchestra.run.smithi078.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-11T12:47:46.911 INFO:teuthology.orchestra.run.smithi078.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-11T12:47:46.916 INFO:teuthology.orchestra.run.smithi078.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-11T12:47:46.917 INFO:teuthology.orchestra.run.smithi078.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-11T12:47:46.918 INFO:teuthology.orchestra.run.smithi078.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-11T12:47:46.921 INFO:teuthology.orchestra.run.smithi078.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-11T12:47:46.984 INFO:teuthology.orchestra.run.smithi078.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-11T12:47:46.985 INFO:teuthology.orchestra.run.smithi078.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-11T12:47:46.985 INFO:teuthology.orchestra.run.smithi078.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-11T12:47:47.023 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ceph-volume (19.3.0-4207-g13fc0377-1jammy) ... 2024-09-11T12:47:47.066 INFO:teuthology.orchestra.run.smithi078.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-11T12:47:47.392 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T12:47:47.501 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T12:47:47.628 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T12:47:47.745 INFO:teuthology.orchestra.run.smithi078.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-11T12:47:47.747 INFO:teuthology.orchestra.run.smithi078.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-11T12:47:47.748 INFO:teuthology.orchestra.run.smithi078.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-11T12:47:47.751 INFO:teuthology.orchestra.run.smithi078.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-11T12:47:47.751 INFO:teuthology.orchestra.run.smithi078.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-11T12:47:47.755 INFO:teuthology.orchestra.run.smithi078.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-11T12:47:47.757 INFO:teuthology.orchestra.run.smithi078.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-11T12:47:47.807 INFO:teuthology.orchestra.run.smithi078.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-11T12:47:47.808 INFO:teuthology.orchestra.run.smithi078.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-11T12:47:47.842 INFO:teuthology.orchestra.run.smithi078.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-11T12:47:47.893 INFO:teuthology.orchestra.run.smithi078.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-11T12:47:47.905 INFO:teuthology.orchestra.run.smithi078.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-11T12:47:47.981 INFO:teuthology.orchestra.run.smithi078.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-11T12:47:47.984 INFO:teuthology.orchestra.run.smithi078.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-11T12:47:47.984 INFO:teuthology.orchestra.run.smithi078.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-11T12:47:47.985 INFO:teuthology.orchestra.run.smithi078.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-11T12:47:47.986 INFO:teuthology.orchestra.run.smithi078.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-11T12:47:47.999 INFO:teuthology.orchestra.run.smithi078.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-11T12:47:48.001 INFO:teuthology.orchestra.run.smithi078.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-11T12:47:48.488 INFO:teuthology.orchestra.run.smithi078.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-11T12:47:48.489 INFO:teuthology.orchestra.run.smithi078.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-11T12:47:48.489 INFO:teuthology.orchestra.run.smithi078.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-11T12:47:48.489 INFO:teuthology.orchestra.run.smithi078.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-11T12:47:48.489 INFO:teuthology.orchestra.run.smithi078.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-11T12:47:48.490 INFO:teuthology.orchestra.run.smithi078.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-11T12:47:48.503 INFO:teuthology.orchestra.run.smithi078.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-11T12:47:48.503 INFO:teuthology.orchestra.run.smithi078.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-11T12:47:48.790 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T12:47:48.932 INFO:teuthology.orchestra.run.smithi078.stdout:Preconfiguring packages ... 2024-09-11T12:47:49.093 INFO:teuthology.orchestra.run.smithi078.stdout:Fetched 55.8 MB in 3s (21.3 MB/s) 2024-09-11T12:47:49.344 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143280 files and directories currently installed.) 2024-09-11T12:47:49.348 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:49.482 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:49.527 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T12:47:49.531 DEBUG:teuthology.orchestra.run.smithi157:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-11T12:47:49.607 INFO:teuthology.orchestra.run.smithi157.stdout:Reading package lists... 2024-09-11T12:47:49.791 INFO:teuthology.orchestra.run.smithi157.stdout:Building dependency tree... 2024-09-11T12:47:49.792 INFO:teuthology.orchestra.run.smithi157.stdout:Reading state information... 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T12:47:49.975 INFO:teuthology.orchestra.run.smithi157.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T12:47:49.976 INFO:teuthology.orchestra.run.smithi157.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T12:47:49.976 INFO:teuthology.orchestra.run.smithi157.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T12:47:49.976 INFO:teuthology.orchestra.run.smithi157.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T12:47:49.977 INFO:teuthology.orchestra.run.smithi157.stdout:The following additional packages will be installed: 2024-09-11T12:47:49.977 INFO:teuthology.orchestra.run.smithi157.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-11T12:47:49.977 INFO:teuthology.orchestra.run.smithi157.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-11T12:47:49.977 INFO:teuthology.orchestra.run.smithi157.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-11T12:47:49.978 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-11T12:47:49.978 INFO:teuthology.orchestra.run.smithi157.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-11T12:47:49.978 INFO:teuthology.orchestra.run.smithi157.stdout:Suggested packages: 2024-09-11T12:47:49.979 INFO:teuthology.orchestra.run.smithi157.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-11T12:47:49.979 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-11T12:47:49.979 INFO:teuthology.orchestra.run.smithi157.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-11T12:47:50.035 INFO:teuthology.orchestra.run.smithi157.stdout:The following NEW packages will be installed: 2024-09-11T12:47:50.036 INFO:teuthology.orchestra.run.smithi157.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-11T12:47:50.036 INFO:teuthology.orchestra.run.smithi157.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-11T12:47:50.036 INFO:teuthology.orchestra.run.smithi157.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-11T12:47:50.037 INFO:teuthology.orchestra.run.smithi157.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-11T12:47:50.037 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-11T12:47:50.037 INFO:teuthology.orchestra.run.smithi157.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-11T12:47:50.037 INFO:teuthology.orchestra.run.smithi157.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-11T12:47:50.038 INFO:teuthology.orchestra.run.smithi157.stdout:The following packages will be upgraded: 2024-09-11T12:47:50.038 INFO:teuthology.orchestra.run.smithi157.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-11T12:47:50.113 INFO:teuthology.orchestra.run.smithi157.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-11T12:47:50.113 INFO:teuthology.orchestra.run.smithi157.stdout:Need to get 55.8 MB of archives. 2024-09-11T12:47:50.113 INFO:teuthology.orchestra.run.smithi157.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-11T12:47:50.113 INFO:teuthology.orchestra.run.smithi157.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-11T12:47:50.241 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-11T12:47:50.248 INFO:teuthology.orchestra.run.smithi157.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-11T12:47:50.252 INFO:teuthology.orchestra.run.smithi157.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-11T12:47:50.280 INFO:teuthology.orchestra.run.smithi157.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-11T12:47:50.280 INFO:teuthology.orchestra.run.smithi157.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-11T12:47:50.280 INFO:teuthology.orchestra.run.smithi157.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-11T12:47:50.287 INFO:teuthology.orchestra.run.smithi157.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-11T12:47:50.288 INFO:teuthology.orchestra.run.smithi157.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-11T12:47:50.289 INFO:teuthology.orchestra.run.smithi157.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-11T12:47:50.290 INFO:teuthology.orchestra.run.smithi157.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-11T12:47:50.291 INFO:teuthology.orchestra.run.smithi157.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-11T12:47:50.292 INFO:teuthology.orchestra.run.smithi157.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-11T12:47:50.301 INFO:teuthology.orchestra.run.smithi157.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-11T12:47:50.301 INFO:teuthology.orchestra.run.smithi157.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-11T12:47:50.302 INFO:teuthology.orchestra.run.smithi157.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-11T12:47:50.318 INFO:teuthology.orchestra.run.smithi157.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-11T12:47:50.530 INFO:teuthology.orchestra.run.smithi157.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-11T12:47:50.532 INFO:teuthology.orchestra.run.smithi157.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-11T12:47:50.532 INFO:teuthology.orchestra.run.smithi157.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-11T12:47:50.535 INFO:teuthology.orchestra.run.smithi157.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-11T12:47:50.535 INFO:teuthology.orchestra.run.smithi157.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-11T12:47:50.540 INFO:teuthology.orchestra.run.smithi157.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-11T12:47:50.541 INFO:teuthology.orchestra.run.smithi157.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-11T12:47:50.548 INFO:teuthology.orchestra.run.smithi157.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-11T12:47:50.548 INFO:teuthology.orchestra.run.smithi157.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-11T12:47:50.569 INFO:teuthology.orchestra.run.smithi157.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-11T12:47:50.570 INFO:teuthology.orchestra.run.smithi157.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-11T12:47:50.580 INFO:teuthology.orchestra.run.smithi157.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-11T12:47:50.599 INFO:teuthology.orchestra.run.smithi157.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-11T12:47:50.603 INFO:teuthology.orchestra.run.smithi157.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-11T12:47:50.604 INFO:teuthology.orchestra.run.smithi157.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-11T12:47:50.605 INFO:teuthology.orchestra.run.smithi157.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-11T12:47:50.606 INFO:teuthology.orchestra.run.smithi157.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-11T12:47:50.617 INFO:teuthology.orchestra.run.smithi157.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-11T12:47:50.619 INFO:teuthology.orchestra.run.smithi157.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-11T12:47:50.764 INFO:teuthology.orchestra.run.smithi157.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-11T12:47:50.764 INFO:teuthology.orchestra.run.smithi157.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-11T12:47:50.764 INFO:teuthology.orchestra.run.smithi157.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-11T12:47:50.764 INFO:teuthology.orchestra.run.smithi157.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-11T12:47:50.765 INFO:teuthology.orchestra.run.smithi157.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-11T12:47:50.765 INFO:teuthology.orchestra.run.smithi157.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-11T12:47:50.778 INFO:teuthology.orchestra.run.smithi157.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-11T12:47:50.778 INFO:teuthology.orchestra.run.smithi157.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-11T12:47:51.162 INFO:teuthology.orchestra.run.smithi157.stdout:Preconfiguring packages ... 2024-09-11T12:47:51.315 INFO:teuthology.orchestra.run.smithi157.stdout:Fetched 55.8 MB in 1s (76.2 MB/s) 2024-09-11T12:47:51.374 INFO:teuthology.orchestra.run.smithi078.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-11T12:47:51.601 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143280 files and directories currently installed.) 2024-09-11T12:47:51.605 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:51.916 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:52.006 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package flex. 2024-09-11T12:47:52.081 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143280 files and directories currently installed.) 2024-09-11T12:47:52.085 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-11T12:47:52.130 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-11T12:47:52.524 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-11T12:47:52.566 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-11T12:47:52.718 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-11T12:47:53.126 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:47:53.600 INFO:teuthology.orchestra.run.smithi157.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-11T12:47:53.663 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143353 files and directories currently installed.) 2024-09-11T12:47:53.666 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:53.817 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:53.896 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package flex. 2024-09-11T12:47:53.967 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143280 files and directories currently installed.) 2024-09-11T12:47:53.971 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-11T12:47:54.011 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-11T12:47:54.205 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:54.340 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:54.655 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-11T12:47:54.908 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-11T12:47:55.099 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package bison. 2024-09-11T12:47:55.175 INFO:teuthology.orchestra.run.smithi078.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143353 files and directories currently installed.) 2024-09-11T12:47:55.177 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-11T12:47:55.218 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-11T12:47:55.279 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-11T12:47:55.635 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-11T12:47:55.650 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-11T12:47:55.694 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-11T12:47:56.005 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package dump. 2024-09-11T12:47:56.020 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-11T12:47:56.051 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:47:56.055 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking dump (0.4b47-3) ... 2024-09-11T12:47:56.317 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package indent. 2024-09-11T12:47:56.332 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:47:56.375 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-11T12:47:56.636 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-11T12:47:56.651 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-11T12:47:56.674 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143353 files and directories currently installed.) 2024-09-11T12:47:56.678 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:56.694 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-11T12:47:56.906 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:57.065 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-11T12:47:57.081 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-11T12:47:57.115 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-11T12:47:57.306 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-11T12:47:57.326 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-11T12:47:57.342 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-11T12:47:57.384 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-11T12:47:57.442 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-11T12:47:57.729 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-11T12:47:57.745 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-11T12:47:57.788 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-11T12:47:57.807 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-11T12:47:58.040 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-11T12:47:58.056 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-11T12:47:58.098 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-11T12:47:58.218 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package bison. 2024-09-11T12:47:58.288 INFO:teuthology.orchestra.run.smithi157.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143353 files and directories currently installed.) 2024-09-11T12:47:58.292 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-11T12:47:58.337 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-11T12:47:58.351 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libjson-perl. 2024-09-11T12:47:58.367 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-11T12:47:58.400 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-11T12:47:58.670 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-11T12:47:58.686 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-11T12:47:58.696 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-11T12:47:58.711 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-11T12:47:58.729 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-11T12:47:58.754 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-11T12:47:59.116 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package dump. 2024-09-11T12:47:59.130 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-11T12:47:59.174 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking dump (0.4b47-3) ... 2024-09-11T12:47:59.444 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package indent. 2024-09-11T12:47:59.459 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:47:59.511 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-11T12:47:59.738 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-11T12:47:59.753 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-11T12:47:59.797 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-11T12:48:00.074 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-11T12:48:00.090 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-11T12:48:00.124 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:00.226 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-11T12:48:00.241 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-11T12:48:00.293 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-11T12:48:00.402 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-11T12:48:00.418 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-11T12:48:00.460 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:00.604 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-11T12:48:00.619 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-11T12:48:00.670 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-11T12:48:00.788 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-11T12:48:00.804 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-11T12:48:00.846 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:01.124 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-11T12:48:01.140 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-11T12:48:01.166 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-11T12:48:01.181 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-11T12:48:01.191 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:01.224 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-11T12:48:01.502 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-11T12:48:01.517 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-11T12:48:01.518 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-11T12:48:01.534 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-11T12:48:01.560 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:01.576 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-11T12:48:01.863 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libjson-perl. 2024-09-11T12:48:01.879 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-11T12:48:01.946 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-11T12:48:01.962 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:01.980 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-11T12:48:02.004 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:02.318 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-11T12:48:02.324 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-11T12:48:02.334 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-11T12:48:02.340 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:02.374 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:02.376 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-11T12:48:02.635 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-11T12:48:02.651 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:02.693 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:02.946 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-11T12:48:02.961 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-11T12:48:03.012 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:48:03.532 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-11T12:48:03.547 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-11T12:48:03.591 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-11T12:48:03.804 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-11T12:48:03.820 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-11T12:48:03.871 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:03.935 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package modemmanager. 2024-09-11T12:48:03.951 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-11T12:48:03.993 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:04.207 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-11T12:48:04.224 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-11T12:48:04.274 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:04.558 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package network-manager. 2024-09-11T12:48:04.574 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-11T12:48:04.616 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-11T12:48:04.652 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-11T12:48:04.668 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-11T12:48:04.710 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:05.113 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-11T12:48:05.129 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-11T12:48:05.179 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-11T12:48:05.616 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-11T12:48:05.632 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-11T12:48:05.683 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-11T12:48:05.792 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package ppp. 2024-09-11T12:48:05.808 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-11T12:48:06.086 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-11T12:48:06.102 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:06.144 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:06.179 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-11T12:48:06.488 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package pptp-linux. 2024-09-11T12:48:06.503 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-11T12:48:06.547 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-11T12:48:06.614 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-11T12:48:06.630 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:06.680 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:06.766 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-11T12:48:06.780 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-11T12:48:06.815 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-11T12:48:07.068 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-11T12:48:07.083 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-11T12:48:07.127 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client-common (238) ... 2024-09-11T12:48:07.244 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-11T12:48:07.261 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:07.310 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:07.439 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-11T12:48:07.453 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:07.497 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:07.529 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-11T12:48:07.545 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-11T12:48:07.587 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:48:08.035 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-common. 2024-09-11T12:48:08.049 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-11T12:48:08.108 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-11T12:48:08.110 INFO:teuthology.orchestra.run.smithi078.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-11T12:48:08.123 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-11T12:48:08.133 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-common (238) ... 2024-09-11T12:48:08.166 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-11T12:48:08.504 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package modemmanager. 2024-09-11T12:48:08.518 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-11T12:48:08.553 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:08.592 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-14. 2024-09-11T12:48:08.606 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:08.652 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:09.260 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager. 2024-09-11T12:48:09.276 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-11T12:48:09.334 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-11T12:48:09.705 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql. 2024-09-11T12:48:09.720 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-11T12:48:09.763 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql (14+238) ... 2024-09-11T12:48:10.015 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-client. 2024-09-11T12:48:10.030 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-11T12:48:10.034 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package ppp. 2024-09-11T12:48:10.050 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-11T12:48:10.073 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-client (14+238) ... 2024-09-11T12:48:10.317 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-11T12:48:10.332 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-11T12:48:10.375 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-11T12:48:10.407 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-11T12:48:10.711 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package tcl8.6. 2024-09-11T12:48:10.726 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-11T12:48:10.730 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package pptp-linux. 2024-09-11T12:48:10.745 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-11T12:48:10.769 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-11T12:48:10.797 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-11T12:48:11.022 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package tcl. 2024-09-11T12:48:11.038 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-11T12:48:11.077 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-11T12:48:11.125 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-11T12:48:11.139 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-11T12:48:11.183 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-11T12:48:11.450 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package wpasupplicant. 2024-09-11T12:48:11.466 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-11T12:48:11.486 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-11T12:48:11.501 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-11T12:48:11.508 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-11T12:48:11.544 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-common (238) ... 2024-09-11T12:48:11.838 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-11T12:48:11.854 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-11T12:48:11.856 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-11T12:48:11.871 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:11.895 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-11T12:48:11.914 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:12.282 INFO:teuthology.orchestra.run.smithi078.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-11T12:48:12.298 INFO:teuthology.orchestra.run.smithi078.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-11T12:48:12.340 INFO:teuthology.orchestra.run.smithi078.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-11T12:48:12.528 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-common. 2024-09-11T12:48:12.543 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-11T12:48:12.604 INFO:teuthology.orchestra.run.smithi157.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-11T12:48:12.617 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client-common (238) ... 2024-09-11T12:48:12.635 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-common (238) ... 2024-09-11T12:48:12.830 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-11T12:48:12.939 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-11T12:48:13.082 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:13.102 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-14. 2024-09-11T12:48:13.117 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-11T12:48:13.154 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:13.208 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-11T12:48:13.334 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-11T12:48:13.460 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:14.347 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql. 2024-09-11T12:48:14.362 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-11T12:48:14.405 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql (14+238) ... 2024-09-11T12:48:14.416 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-11T12:48:14.515 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-11T12:48:14.660 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-client. 2024-09-11T12:48:14.672 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-11T12:48:14.716 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-client (14+238) ... 2024-09-11T12:48:14.927 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-11T12:48:14.927 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-11T12:48:14.984 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-11T12:48:14.999 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-11T12:48:15.034 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-11T12:48:15.304 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl8.6. 2024-09-11T12:48:15.319 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-11T12:48:15.362 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-11T12:48:15.665 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package tcl. 2024-09-11T12:48:15.681 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-11T12:48:15.727 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-11T12:48:15.877 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-11T12:48:16.003 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-11T12:48:16.076 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package wpasupplicant. 2024-09-11T12:48:16.092 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-11T12:48:16.129 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:48:16.159 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-11T12:48:16.280 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-11T12:48:16.397 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-11T12:48:16.455 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-11T12:48:16.471 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-11T12:48:16.498 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:16.505 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-11T12:48:16.633 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-11T12:48:16.758 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-11T12:48:16.917 INFO:teuthology.orchestra.run.smithi157.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-11T12:48:16.933 INFO:teuthology.orchestra.run.smithi157.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-11T12:48:16.983 INFO:teuthology.orchestra.run.smithi157.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-11T12:48:17.268 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-common (238) ... 2024-09-11T12:48:17.414 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-11T12:48:17.523 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up dump (0.4b47-3) ... 2024-09-11T12:48:17.601 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-11T12:48:17.667 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:17.792 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-11T12:48:17.939 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-11T12:48:17.943 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-11T12:48:18.048 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-11T12:48:18.069 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-11T12:48:18.187 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-11T12:48:18.208 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:18.268 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-11T12:48:18.309 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-11T12:48:18.330 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-11T12:48:18.427 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-11T12:48:18.456 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-11T12:48:18.553 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:18.607 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-11T12:48:18.733 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:18.862 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-11T12:48:19.028 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-client (14+238) ... 2024-09-11T12:48:19.153 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-11T12:48:19.508 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-11T12:48:19.592 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:which means they are currently configured by ifupdown: 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:- enp3s0f1 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout:remove their configuration from /etc/network/interfaces. 2024-09-11T12:48:19.723 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:19.855 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-11T12:48:19.954 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-11T12:48:19.954 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-11T12:48:20.263 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-11T12:48:20.678 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-11T12:48:20.686 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-11T12:48:20.804 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-11T12:48:20.930 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-11T12:48:21.048 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-11T12:48:21.157 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-11T12:48:21.308 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:21.517 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-11T12:48:21.581 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:21.644 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-11T12:48:21.849 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:21.975 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-11T12:48:22.101 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-common (238) ... 2024-09-11T12:48:22.341 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-11T12:48:22.458 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up dump (0.4b47-3) ... 2024-09-11T12:48:22.545 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-11T12:48:22.643 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:23.047 INFO:teuthology.orchestra.run.smithi078.stdout:Adding user postgres to group ssl-cert 2024-09-11T12:48:23.153 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-11T12:48:23.298 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:23.298 INFO:teuthology.orchestra.run.smithi078.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-11T12:48:23.304 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-11T12:48:23.430 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-11T12:48:23.556 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-11T12:48:23.565 INFO:teuthology.orchestra.run.smithi078.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-11T12:48:23.565 INFO:teuthology.orchestra.run.smithi078.stdout:Removing obsolete dictionary files: 2024-09-11T12:48:23.646 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-11T12:48:23.724 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-11T12:48:23.858 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-11T12:48:24.010 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-11T12:48:24.136 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:24.262 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-11T12:48:24.305 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-11T12:48:24.430 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-client (14+238) ... 2024-09-11T12:48:24.589 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-11T12:48:24.777 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:25.116 INFO:teuthology.orchestra.run.smithi078.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-11T12:48:25.131 INFO:teuthology.orchestra.run.smithi078.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:This user must also own the server process. 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:The database cluster will be initialized with locale "en_US". 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:The default text search configuration will be set to "english". 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:Data page checksums are disabled. 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:25.177 INFO:teuthology.orchestra.run.smithi078.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout:which means they are currently configured by ifupdown: 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout:- enp3s0f1 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout:remove their configuration from /etc/network/interfaces. 2024-09-11T12:48:25.178 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:creating subdirectories ... ok 2024-09-11T12:48:25.179 INFO:teuthology.orchestra.run.smithi078.stdout:selecting dynamic shared memory implementation ... posix 2024-09-11T12:48:25.202 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default max_connections ... 100 2024-09-11T12:48:25.234 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default shared_buffers ... 128MB 2024-09-11T12:48:25.248 INFO:teuthology.orchestra.run.smithi078.stdout:selecting default time zone ... Etc/UTC 2024-09-11T12:48:25.249 INFO:teuthology.orchestra.run.smithi078.stdout:creating configuration files ... ok 2024-09-11T12:48:25.311 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-11T12:48:25.411 INFO:teuthology.orchestra.run.smithi078.stdout:running bootstrap script ... ok 2024-09-11T12:48:25.702 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-11T12:48:26.029 INFO:teuthology.orchestra.run.smithi078.stdout:performing post-bootstrap initialization ... ok 2024-09-11T12:48:26.098 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-11T12:48:26.575 INFO:teuthology.orchestra.run.smithi078.stdout:syncing data to disk ... ok 2024-09-11T12:48:26.835 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-11T12:48:26.970 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-11T12:48:27.080 INFO:teuthology.orchestra.run.smithi078.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-11T12:48:27.138 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-11T12:48:27.263 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-common (238) ... 2024-09-11T12:48:28.247 INFO:teuthology.orchestra.run.smithi157.stdout:Adding user postgres to group ssl-cert 2024-09-11T12:48:28.521 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:28.521 INFO:teuthology.orchestra.run.smithi157.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-11T12:48:28.796 INFO:teuthology.orchestra.run.smithi157.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-11T12:48:28.797 INFO:teuthology.orchestra.run.smithi157.stdout:Removing obsolete dictionary files: 2024-09-11T12:48:29.518 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-11T12:48:30.006 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-11T12:48:30.051 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:30.177 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-11T12:48:30.303 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up postgresql (14+238) ... 2024-09-11T12:48:30.429 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:30.538 INFO:teuthology.orchestra.run.smithi078.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:30.540 INFO:teuthology.orchestra.run.smithi157.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-11T12:48:30.555 INFO:teuthology.orchestra.run.smithi157.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-11T12:48:30.619 INFO:teuthology.orchestra.run.smithi157.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-11T12:48:30.619 INFO:teuthology.orchestra.run.smithi157.stdout:This user must also own the server process. 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout:The database cluster will be initialized with locale "en_US". 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout:The default text search configuration will be set to "english". 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout:Data page checksums are disabled. 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:30.620 INFO:teuthology.orchestra.run.smithi157.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-11T12:48:30.621 INFO:teuthology.orchestra.run.smithi157.stdout:creating subdirectories ... ok 2024-09-11T12:48:30.621 INFO:teuthology.orchestra.run.smithi157.stdout:selecting dynamic shared memory implementation ... posix 2024-09-11T12:48:30.644 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default max_connections ... 100 2024-09-11T12:48:30.678 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default shared_buffers ... 128MB 2024-09-11T12:48:30.692 INFO:teuthology.orchestra.run.smithi157.stdout:selecting default time zone ... Etc/UTC 2024-09-11T12:48:30.693 INFO:teuthology.orchestra.run.smithi157.stdout:creating configuration files ... ok 2024-09-11T12:48:30.822 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-11T12:48:30.822 INFO:teuthology.orchestra.run.smithi078.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-11T12:48:30.853 INFO:teuthology.orchestra.run.smithi157.stdout:running bootstrap script ... ok 2024-09-11T12:48:31.476 INFO:teuthology.orchestra.run.smithi157.stdout:performing post-bootstrap initialization ... ok 2024-09-11T12:48:31.591 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T12:48:31.692 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-11T12:48:32.010 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T12:48:32.088 INFO:teuthology.orchestra.run.smithi157.stdout:syncing data to disk ... ok 2024-09-11T12:48:32.153 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T12:48:32.610 INFO:teuthology.orchestra.run.smithi157.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-11T12:48:33.556 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-11T12:48:33.656 INFO:teuthology.orchestra.run.smithi078.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T12:48:34.244 INFO:teuthology.orchestra.run.smithi078.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T12:48:34.251 DEBUG:teuthology.parallel:result is None 2024-09-11T12:48:35.612 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:35.738 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-11T12:48:35.864 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up postgresql (14+238) ... 2024-09-11T12:48:35.990 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-11T12:48:36.141 INFO:teuthology.orchestra.run.smithi157.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-11T12:48:36.412 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-11T12:48:36.412 INFO:teuthology.orchestra.run.smithi157.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-11T12:48:37.193 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T12:48:37.294 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-11T12:48:37.621 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T12:48:37.773 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T12:48:39.150 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-11T12:48:39.275 INFO:teuthology.orchestra.run.smithi157.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T12:48:39.830 INFO:teuthology.orchestra.run.smithi157.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T12:48:39.836 DEBUG:teuthology.parallel:result is None 2024-09-11T12:48:39.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d 2024-09-11T12:48:39.983 DEBUG:teuthology.orchestra.run.smithi078:> dpkg-query -W -f '${Version}' ceph 2024-09-11T12:48:40.006 INFO:teuthology.orchestra.run.smithi078.stdout:19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:48:40.007 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:48:40.007 INFO:teuthology.task.install:The correct ceph version 19.3.0-4207-g13fc0377-1jammy is installed. 2024-09-11T12:48:40.009 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d 2024-09-11T12:48:40.145 DEBUG:teuthology.orchestra.run.smithi157:> dpkg-query -W -f '${Version}' ceph 2024-09-11T12:48:40.166 INFO:teuthology.orchestra.run.smithi157.stdout:19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:48:40.166 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4207-g13fc0377-1jammy 2024-09-11T12:48:40.166 INFO:teuthology.task.install:The correct ceph version 19.3.0-4207-g13fc0377-1jammy is installed. 2024-09-11T12:48:40.168 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T12:48:40.168 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:40.169 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T12:48:40.190 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:40.190 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T12:48:40.226 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T12:48:40.226 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:40.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T12:48:40.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T12:48:40.316 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:40.317 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T12:48:40.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T12:48:40.398 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T12:48:40.398 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:40.398 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T12:48:40.418 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T12:48:40.480 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:40.481 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T12:48:40.499 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T12:48:40.561 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T12:48:40.562 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:40.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T12:48:40.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T12:48:40.645 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:40.645 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T12:48:40.663 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T12:48:40.725 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T12:48:40.806 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T12:48:40.806 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /var/log/ceph 2024-09-11T12:48:40.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 777 /var/log/ceph 2024-09-11T12:48:40.828 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T12:48:40.828 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T12:48:40.872 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T12:48:40.890 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T12:48:40.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T12:48:40.932 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T12:48:40.957 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-11T12:48:40.957 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-11T12:48:40.957 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-11T12:48:40.957 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:40.957 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-11T12:48:41.001 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-11T12:48:41.092 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:41.092 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-11T12:48:41.101 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-11T12:48:41.193 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T12:48:41.193 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-11T12:48:41.194 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299', 'branch': 'wip-athakkar-testing-2024-09-11-1208', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': None, 'first_in_suite': False, 'job_id': '7901299', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-athakkar-testing-2024-09-11-1208'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-athakkar-testing-2024-09-11-1208', 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'}}, 'owner': 'scheduled_khiremat@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1861, 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d', 'sleep_before_teardown': 0, 'subset': '3/4', 'suite': 'fs:volumes', 'suite_branch': 'wip-athakkar-testing-2024-09-11-1208', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi078.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLe619R/WLj/vQo/TEasqG2SAbToCDob5Ohq1FM03xHwg4y+/VnmZkAltOj8rcIURv24Am/0kLDVT2jn6J7RYMk=', 'smithi157.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHJUIcygRQXg0AsTZEu9r7TJgQZ0MZvzIfX7HnUSofZBdy4Gcad4HZu75KOOrfxQ4wTeJzETIrjLRkL+9jbwqLo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-11_12:26:28', 'tube': 'smithi', 'user': 'khiremat', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T12:48:41.194 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T12:48:41.198 DEBUG:teuthology.orchestra.run.smithi157:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T12:48:41.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T12:48:41.248 DEBUG:teuthology.orchestra.run.smithi157:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T12:48:41.306 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:41.306 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T12:48:41.317 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T12:48:41.317 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 897 Links: 1 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 12:41:59.521990240 +0000 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 12:41:59.237992778 +0000 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 12:41:59.237992778 +0000 2024-09-11T12:48:41.370 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 12:41:59.237992778 +0000 2024-09-11T12:48:41.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T12:48:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:48:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:48:41.431 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000406182 s, 1.3 MB/s 2024-09-11T12:48:41.433 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T12:48:41.491 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-09-11T12:48:41.541 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 916 Links: 1 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 12:42:00.073985309 +0000 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 12:41:59.821987561 +0000 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 12:41:59.821987561 +0000 2024-09-11T12:48:41.542 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 12:41:59.821987561 +0000 2024-09-11T12:48:41.542 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T12:48:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:48:41.607 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:48:41.608 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000383353 s, 1.3 MB/s 2024-09-11T12:48:41.609 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T12:48:41.663 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-09-11T12:48:41.713 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 931 Links: 1 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 12:42:00.693979771 +0000 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 12:42:00.405982344 +0000 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 12:42:00.405982344 +0000 2024-09-11T12:48:41.714 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 12:42:00.405982344 +0000 2024-09-11T12:48:41.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T12:48:41.775 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:48:41.775 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:48:41.775 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000376279 s, 1.4 MB/s 2024-09-11T12:48:41.777 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T12:48:41.832 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-09-11T12:48:41.885 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 947 Links: 1 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-09-11 12:42:01.305974306 +0000 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-09-11 12:42:01.013976915 +0000 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-09-11 12:42:01.013976915 +0000 2024-09-11T12:48:41.886 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-09-11 12:42:01.013976915 +0000 2024-09-11T12:48:41.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T12:48:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:48:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:48:41.947 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000390522 s, 1.3 MB/s 2024-09-11T12:48:41.949 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T12:48:42.008 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:42.008 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:42.008 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T12:48:42.017 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T12:48:42.017 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_1 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 899 Links: 1 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-11 12:41:59.173994071 +0000 2024-09-11T12:48:42.068 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-11 12:41:58.901996500 +0000 2024-09-11T12:48:42.069 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-11 12:41:58.901996500 +0000 2024-09-11T12:48:42.069 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-11 12:41:58.901996500 +0000 2024-09-11T12:48:42.069 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T12:48:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-11T12:48:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-11T12:48:42.130 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000275163 s, 1.9 MB/s 2024-09-11T12:48:42.132 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T12:48:42.186 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_2 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 913 Links: 1 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-11 12:41:59.701989354 +0000 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-11 12:41:59.469991428 +0000 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-11 12:41:59.469991428 +0000 2024-09-11T12:48:42.240 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-11 12:41:59.469991428 +0000 2024-09-11T12:48:42.241 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T12:48:42.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-11T12:48:42.302 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-11T12:48:42.302 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000272628 s, 1.9 MB/s 2024-09-11T12:48:42.304 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T12:48:42.358 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_3 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 930 Links: 1 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-11 12:42:00.025986461 +0000 2024-09-11T12:48:42.408 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-11 12:42:00.021986497 +0000 2024-09-11T12:48:42.409 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-11 12:42:00.021986497 +0000 2024-09-11T12:48:42.409 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-11 12:42:00.021986497 +0000 2024-09-11T12:48:42.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T12:48:42.469 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-11T12:48:42.470 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-11T12:48:42.470 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.000273294 s, 1.9 MB/s 2024-09-11T12:48:42.471 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T12:48:42.525 DEBUG:teuthology.orchestra.run.smithi157:> stat /dev/vg_nvme/lv_4 2024-09-11T12:48:42.576 INFO:teuthology.orchestra.run.smithi157.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T12:48:42.577 INFO:teuthology.orchestra.run.smithi157.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T12:48:42.577 INFO:teuthology.orchestra.run.smithi157.stdout:Device: 6h/6d Inode: 947 Links: 1 2024-09-11T12:48:42.577 INFO:teuthology.orchestra.run.smithi157.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T12:48:42.577 INFO:teuthology.orchestra.run.smithi157.stdout:Access: 2024-09-11 12:42:00.865978958 +0000 2024-09-11T12:48:42.578 INFO:teuthology.orchestra.run.smithi157.stdout:Modify: 2024-09-11 12:42:00.613981210 +0000 2024-09-11T12:48:42.578 INFO:teuthology.orchestra.run.smithi157.stdout:Change: 2024-09-11 12:42:00.613981210 +0000 2024-09-11T12:48:42.578 INFO:teuthology.orchestra.run.smithi157.stdout: Birth: 2024-09-11 12:42:00.613981210 +0000 2024-09-11T12:48:42.578 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T12:48:42.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records in 2024-09-11T12:48:42.638 INFO:teuthology.orchestra.run.smithi157.stderr:1+0 records out 2024-09-11T12:48:42.638 INFO:teuthology.orchestra.run.smithi157.stderr:512 bytes copied, 0.00026965 s, 1.9 MB/s 2024-09-11T12:48:42.640 DEBUG:teuthology.orchestra.run.smithi157:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T12:48:42.693 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:42.693 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-11T12:48:42.694 INFO:tasks.ceph:Generating config... 2024-09-11T12:48:42.695 INFO:tasks.ceph:[client] pid file = 2024-09-11T12:48:42.695 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-11T12:48:42.695 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-11T12:48:42.695 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-11T12:48:42.695 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T12:48:42.695 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T12:48:42.696 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T12:48:42.696 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T12:48:42.696 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T12:48:42.696 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-11T12:48:42.696 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T12:48:42.697 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T12:48:42.697 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T12:48:42.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T12:48:42.741 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T12:48:42.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T12:48:42.841 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T12:48:42.860 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.78'), ('mon.b', '172.21.15.157'), ('mon.c', '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]')] 2024-09-11T12:48:42.861 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.78,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-11T12:48:42.862 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:42.862 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T12:48:42.914 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.78 --add b 172.21.15.157 --addv c '[v2:172.21.15.157:3301,v1:172.21.15.157:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: generated fsid 163b1675-69a3-4b68-a3e4-8c2fa35bac4f 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:setting min_mon_release = quincy 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:epoch 0 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:fsid 163b1675-69a3-4b68-a3e4-8c2fa35bac4f 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:last_changed 2024-09-11T12:48:43.003873+0000 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:created 2024-09-11T12:48:43.003873+0000 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:min_mon_release 17 (quincy) 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:election_strategy: 1 2024-09-11T12:48:43.004 INFO:teuthology.orchestra.run.smithi078.stdout:0: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.a 2024-09-11T12:48:43.005 INFO:teuthology.orchestra.run.smithi078.stdout:1: [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] mon.b 2024-09-11T12:48:43.005 INFO:teuthology.orchestra.run.smithi078.stdout:2: [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] mon.c 2024-09-11T12:48:43.005 INFO:teuthology.orchestra.run.smithi078.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T12:48:43.007 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T12:48:43.015 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 163b1675-69a3-4b68-a3e4-8c2fa35bac4f... 2024-09-11T12:48:43.015 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T12:48:43.060 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T12:48:43.098 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-09-11T12:48:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2024-09-11T12:48:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T12:48:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2024-09-11T12:48:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2024-09-11T12:48:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.78,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on skipped message = False 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 163b1675-69a3-4b68-a3e4-8c2fa35bac4f 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2024-09-11T12:48:43.102 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug trim objects = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2024-09-11T12:48:43.103 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bdev async discard = True 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bdev enable discard = True 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore allocator = bitmap 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-09-11T12:48:43.104 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2024-09-11T12:48:43.105 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug client = 20 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug finisher = 20 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2024-09-11T12:48:43.106 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T12:48:43.107 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: mon down mkfs grace = 300 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2024-09-11T12:48:43.108 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = "" 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2024-09-11T12:48:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout:[global] 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: chdir = "" 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: auth supported = cephx 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: filestore xattr use omap = true 2024-09-11T12:48:43.110 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon clock drift allowed = 1.000 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: osd crush chooseleaf type = 0 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: auth debug = true 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on old message = true 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on bug = true 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: mon pg warn max object skew = 0 2024-09-11T12:48:43.111 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default size = 2 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.112 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow primary affinity = true 2024-09-11T12:48:43.113 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd allow pg remap = true 2024-09-11T12:48:43.113 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on legacy crush tunables = false 2024-09-11T12:48:43.113 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on crush straw calc version zero = false 2024-09-11T12:48:43.115 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on no sortbitwise = false 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on osd down out interval zero = false 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon warn on too few osds = false 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon_allow_pool_size_one = true 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: osd default data pool replay window = 5 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon allow pool delete = true 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.116 INFO:teuthology.orchestra.run.smithi157.stdout: mon cluster log file level = debug 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: debug asserts on shutdown = true 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: mon health detail to clog = false 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: mon host = "172.21.15.78,172.21.15.157,[v2:172.21.15.157:3301,v1:172.21.15.157:6790]" 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: ms die on skipped message = False 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: fsid = 163b1675-69a3-4b68-a3e4-8c2fa35bac4f 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout:[osd] 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: osd journal size = 100 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.117 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub load threshold = 5.0 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd scrub max interval = 600 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd mclock profile = high_recovery_ops 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd recover clone overlap = true 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd recovery max chunk = 1048576 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug shutdown = true 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug op order = true 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug verify stray on activate = true 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug trim objects = true 2024-09-11T12:48:43.118 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: osd open classes on start = true 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug pg log writeout = true 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: osd deep scrub update digest min age = 30 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: osd map max advance = 10 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: journal zero on create = true 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: filestore ondisk finisher threads = 3 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: filestore apply finisher threads = 3 2024-09-11T12:48:43.119 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.120 INFO:teuthology.orchestra.run.smithi157.stdout: bdev debug aio = true 2024-09-11T12:48:43.120 INFO:teuthology.orchestra.run.smithi157.stdout: osd debug misdirected ops = true 2024-09-11T12:48:43.120 INFO:teuthology.orchestra.run.smithi157.stdout: bdev async discard = True 2024-09-11T12:48:43.120 INFO:teuthology.orchestra.run.smithi157.stdout: bdev enable discard = True 2024-09-11T12:48:43.120 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore allocator = bitmap 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore block size = 96636764160 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: bluestore fsck on mount = True 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluefs = 1/20 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: debug bluestore = 1/20 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-11T12:48:43.121 INFO:teuthology.orchestra.run.smithi157.stdout: debug osd = 20 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: debug rocksdb = 4/10 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd full ratio = 0.9 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd nearfull ratio = 0.8 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: osd failsafe full ratio = 0.95 2024-09-11T12:48:43.122 INFO:teuthology.orchestra.run.smithi157.stdout: osd objectstore = bluestore 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr] 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug mgr = 20 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: mgr/telemetry/nag = false 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug client = 20 2024-09-11T12:48:43.123 INFO:teuthology.orchestra.run.smithi157.stdout: debug finisher = 20 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout:[mon] 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: debug ms = 1 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: debug mon = 20 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: debug paxos = 20 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: debug auth = 20 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon data avail warn = 5 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon mgr mkfs grace = 240 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min pgs per osd = 4 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd reporter subtree level = osd 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon osd prime pg temp = true 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: mon reweight min bytes per osd = 10 2024-09-11T12:48:43.124 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T12:48:43.125 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.126 INFO:teuthology.orchestra.run.smithi157.stdout: # 1m isn't quite enough 2024-09-11T12:48:43.126 INFO:teuthology.orchestra.run.smithi157.stdout: mon_down_mkfs_grace = 2m 2024-09-11T12:48:43.126 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.126 INFO:teuthology.orchestra.run.smithi157.stdout: mon_warn_on_filestore_osds = false 2024-09-11T12:48:43.126 INFO:teuthology.orchestra.run.smithi157.stdout: mon down mkfs grace = 300 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout: 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout:[client] 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout: rgw cache enabled = true 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable ops log = true 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout: rgw enable usage log = true 2024-09-11T12:48:43.127 INFO:teuthology.orchestra.run.smithi157.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout: pid file = "" 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.a] 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.b] 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mon.c] 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-11T12:48:43.128 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-11T12:48:43.129 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-11T12:48:43.130 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T12:48:43.130 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T12:48:43.229 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T12:48:43.229 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:43.229 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T12:48:43.239 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:43.239 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T12:48:43.289 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:48:43.289 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:43.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T12:48:43.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T12:48:43.359 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:48:43.359 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:43.409 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:48:43.409 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:43.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T12:48:43.409 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T12:48:43.442 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:48:43.442 DEBUG:teuthology.orchestra.run.smithi157:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:43.492 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T12:48:43.493 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T12:48:43.493 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-11T12:48:43.548 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-11T12:48:43.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T12:48:43.611 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T12:48:43.631 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T12:48:43.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-11T12:48:43.686 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-11T12:48:43.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-11T12:48:43.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-11T12:48:43.805 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-11T12:48:43.810 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-11T12:48:43.829 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-11T12:48:43.886 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-11T12:48:43.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-11T12:48:43.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-11T12:48:44.005 INFO:teuthology.orchestra.run.smithi157.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-11T12:48:44.010 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-11T12:48:44.029 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T12:48:44.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-11T12:48:44.072 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-11T12:48:44.088 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-11T12:48:44.170 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-11T12:48:44.186 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-11T12:48:44.228 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-11T12:48:44.245 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-11T12:48:44.327 INFO:teuthology.orchestra.run.smithi157.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-11T12:48:44.344 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T12:48:44.345 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi157.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-11T12:48:44.345 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T12:48:44.364 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:44.364 INFO:tasks.ceph:role: osd.0 2024-09-11T12:48:44.364 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:48:44.365 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:48:44.431 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:48:44.432 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:48:44.432 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:48:44.432 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:48:44.432 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:48:44.847 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-09-11T12:48:44.851 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-09-11T12:48:44.851 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T12:48:44.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T12:48:45.002 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:48:45.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:48:45.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T12:48:45.068 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:45.068 INFO:tasks.ceph:role: osd.1 2024-09-11T12:48:45.068 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:48:45.068 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T12:48:45.136 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:48:45.136 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:48:45.137 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:48:45.509 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-09-11T12:48:45.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-09-11T12:48:45.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T12:48:45.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T12:48:45.649 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:48:45.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:48:45.650 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T12:48:45.712 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:45.713 INFO:tasks.ceph:role: osd.2 2024-09-11T12:48:45.713 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:48:45.713 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:48:45.780 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:48:46.175 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-09-11T12:48:46.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-09-11T12:48:46.181 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T12:48:46.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T12:48:46.287 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:48:46.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:48:46.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-11T12:48:46.349 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-11T12:48:46.350 INFO:tasks.ceph:role: osd.3 2024-09-11T12:48:46.350 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:48:46.350 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:48:46.422 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:48:46.821 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-09-11T12:48:46.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-09-11T12:48:46.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-11T12:48:46.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-11T12:48:46.953 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:48:46.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:48:46.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:47.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:47.079+0000 7fd0bae2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T12:48:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:47.083+0000 7fd0bae2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T12:48:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:47.083+0000 7fd0bae2b7c0 -1 bdev(0x55d2e3477800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T12:48:47.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:47.083+0000 7fd0bae2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T12:48:50.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T12:48:50.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:50.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:50.455+0000 7f5d534047c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T12:48:50.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:50.455+0000 7f5d534047c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T12:48:50.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:50.455+0000 7f5d534047c0 -1 bdev(0x5614102ad800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T12:48:50.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:50.455+0000 7f5d534047c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T12:48:53.671 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T12:48:53.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:53.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:53.779+0000 7fcceb33c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T12:48:53.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:53.779+0000 7fcceb33c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T12:48:53.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:53.779+0000 7fcceb33c7c0 -1 bdev(0x559685c2f800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T12:48:53.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:53.779+0000 7fcceb33c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T12:48:57.056 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T12:48:57.078 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:48:57.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:57.171+0000 7f5e94f8f7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-11T12:48:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:57.171+0000 7f5e94f8f7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-11T12:48:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:57.171+0000 7f5e94f8f7c0 -1 bdev(0x557960e0d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-11T12:48:57.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:48:57.171+0000 7f5e94f8f7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-11T12:49:00.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-11T12:49:00.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-11T12:49:00.469 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T12:49:00.469 INFO:tasks.ceph:role: osd.4 2024-09-11T12:49:00.469 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:49:00.469 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T12:49:00.531 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:49:00.531 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:49:00.531 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:49:00.531 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:49:00.532 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:49:00.556 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-09-11T12:49:00.559 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-11T12:49:00.559 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-11T12:49:00.632 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-11T12:49:00.690 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:49:00.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:49:00.691 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-11T12:49:00.754 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T12:49:00.755 INFO:tasks.ceph:role: osd.5 2024-09-11T12:49:00.755 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:49:00.755 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:49:00.814 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:49:00.838 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-09-11T12:49:00.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-11T12:49:00.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-11T12:49:00.907 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-11T12:49:00.962 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:49:00.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:49:00.963 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-11T12:49:01.021 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T12:49:01.021 INFO:tasks.ceph:role: osd.6 2024-09-11T12:49:01.021 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:49:01.022 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T12:49:01.081 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:49:01.081 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:49:01.082 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:49:01.106 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-09-11T12:49:01.109 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-11T12:49:01.109 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-11T12:49:01.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-11T12:49:01.230 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:49:01.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:49:01.231 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-11T12:49:01.293 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-11T12:49:01.294 INFO:tasks.ceph:role: osd.7 2024-09-11T12:49:01.294 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:49:01.294 DEBUG:teuthology.orchestra.run.smithi157:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-11T12:49:01.353 INFO:teuthology.orchestra.run.smithi157.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T12:49:01.353 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout: = sunit=0 swidth=0 blks 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T12:49:01.354 INFO:teuthology.orchestra.run.smithi157.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T12:49:01.378 INFO:teuthology.orchestra.run.smithi157.stdout:Discarding blocks...Done. 2024-09-11T12:49:01.381 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi157.front.sepia.ceph.com -o noatime 2024-09-11T12:49:01.381 DEBUG:teuthology.orchestra.run.smithi157:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-11T12:49:01.447 DEBUG:teuthology.orchestra.run.smithi157:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-11T12:49:01.502 INFO:teuthology.orchestra.run.smithi157.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T12:49:01.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:49:01.503 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:01.620 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:01.619+0000 7f29f950b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-11T12:49:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:01.619+0000 7f29f950b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-11T12:49:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:01.619+0000 7f29f950b7c0 -1 bdev(0x556afec11800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-11T12:49:01.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:01.619+0000 7f29f950b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-11T12:49:04.870 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-11T12:49:04.892 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:04.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:04.987+0000 7f070feac7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-11T12:49:04.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:04.987+0000 7f070feac7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-11T12:49:04.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:04.987+0000 7f070feac7c0 -1 bdev(0x558cccb47800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-11T12:49:04.991 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:04.987+0000 7f070feac7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-11T12:49:08.227 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-11T12:49:08.249 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:08.339+0000 7f02004467c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-11T12:49:08.343 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:08.339+0000 7f02004467c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-11T12:49:08.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:08.343+0000 7f02004467c0 -1 bdev(0x55b68476f800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-11T12:49:08.344 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:08.343+0000 7f02004467c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-11T12:49:11.195 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:49:11.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:49:11.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-11T12:49:11.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:11.699 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:11.699+0000 7f8b9aa2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-11T12:49:11.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:11.699+0000 7f8b9aa2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-11T12:49:11.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:11.699+0000 7f8b9aa2b7c0 -1 bdev(0x564bab17b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-11T12:49:11.700 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:11.699+0000 7f8b9aa2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-11T12:49:14.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-11T12:49:14.957 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T12:49:14.957 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:14.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-11T12:49:14.979 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:14.979 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-11T12:49:15.041 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.041 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-11T12:49:15.109 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T12:49:15.168 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.169 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T12:49:15.233 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T12:49:15.293 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.293 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-11T12:49:15.352 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.352 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T12:49:15.373 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.373 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-11T12:49:15.439 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.439 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-11T12:49:15.503 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.504 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-11T12:49:15.566 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.566 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-11T12:49:15.626 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.626 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-11T12:49:15.685 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.686 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-11T12:49:15.750 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.750 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-11T12:49:15.759 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:15.759 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-11T12:49:15.813 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.813 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-11T12:49:15.822 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:15.822 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-11T12:49:15.870 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T12:49:15.871 DEBUG:teuthology.orchestra.run.smithi078:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T12:49:15.874 DEBUG:teuthology.orchestra.run.smithi157:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T12:49:15.914 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.y] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg== 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.a] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A== 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.c] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g== 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.0] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ== 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.1] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA== 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.2] 2024-09-11T12:49:15.915 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.3] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.x] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.b] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[mds.d] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.4] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.5] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q== 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.6] 2024-09-11T12:49:15.916 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw== 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.7] 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw== 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout:[client.2] 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw== 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA== 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout:[client.3] 2024-09-11T12:49:15.917 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.y] 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.a] 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.c] 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.0] 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.1] 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA== 2024-09-11T12:49:15.928 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.2] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.3] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[mgr.x] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.b] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[mds.d] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.4] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.5] 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q== 2024-09-11T12:49:15.929 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.6] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw== 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout:[osd.7] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw== 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout:[client.0] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout:[client.2] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw== 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout:[client.1] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA== 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout:[client.3] 2024-09-11T12:49:15.930 INFO:teuthology.orchestra.run.smithi157.stdout: key = AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA== 2024-09-11T12:49:15.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T12:49:15.967 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T12:49:16.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.093 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.148 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.209 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.266 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T12:49:16.379 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T12:49:16.432 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.436 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.494 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-11T12:49:16.550 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.553 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.606 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.609 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.665 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.721 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T12:49:16.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.778 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.837 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.893 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:16.957 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-11T12:49:17.014 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T12:49:17.014 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T12:49:17.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T12:49:17.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T12:49:17.159 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T12:49:17.179 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T12:49:17.289 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T12:49:17.309 DEBUG:teuthology.orchestra.run.smithi157:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T12:49:17.374 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T12:49:17.485 DEBUG:teuthology.orchestra.run.smithi157:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T12:49:17.505 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:17.508 DEBUG:teuthology.orchestra.run.smithi157:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T12:49:17.560 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T12:49:17.561 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T12:49:17.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T12:49:17.564 INFO:tasks.ceph.mon.a:Started 2024-09-11T12:49:17.564 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T12:49:17.564 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T12:49:17.606 INFO:tasks.ceph.mon.b:Started 2024-09-11T12:49:17.606 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T12:49:17.606 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T12:49:17.609 INFO:tasks.ceph.mon.c:Started 2024-09-11T12:49:17.609 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T12:49:17.609 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-11T12:49:17.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-11T12:49:17.612 INFO:tasks.ceph.mgr.y:Started 2024-09-11T12:49:17.612 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T12:49:17.613 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T12:49:17.615 INFO:tasks.ceph.mgr.x:Started 2024-09-11T12:49:17.616 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-11T12:49:17.616 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-11T12:49:17.620 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-11T12:49:17.620 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-11T12:49:17.671 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-11T12:49:17.671 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-11T12:49:17.720 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-11T12:49:17.720 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-11T12:49:17.768 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-11T12:49:17.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-11T12:49:17.819 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-11T12:49:17.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-11T12:49:17.863 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-11T12:49:17.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-11T12:49:17.912 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-11T12:49:17.912 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-11T12:49:17.968 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-11T12:49:17.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-11T12:49:18.021 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-11T12:49:18.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-11T12:49:18.071 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-11T12:49:18.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-11T12:49:18.117 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-11T12:49:18.117 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-11T12:49:18.147 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-11T12:49:18.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-11T12:49:18.185 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-11T12:49:18.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-11T12:49:18.240 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-11T12:49:18.240 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-11T12:49:18.280 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-11T12:49:18.280 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-11T12:49:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 -- 172.21.15.78:0/2398291974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9480159ff0 msgr2=0x7f948013e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 --2- 172.21.15.78:0/2398291974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9480159ff0 0x7f948013e5d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f94801602e0 tx=0x7f947800eec0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 -- 172.21.15.78:0/2398291974 shutdown_connections 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 --2- 172.21.15.78:0/2398291974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9480159ff0 0x7f948013e5d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 --2- 172.21.15.78:0/2398291974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94801632c0 0x7f9480159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 --2- 172.21.15.78:0/2398291974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9464003a80 0x7f9480162d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.275+0000 7f9474c00640 1 -- 172.21.15.78:0/2398291974 >> 172.21.15.78:0/2398291974 conn(0x7f9480074490 msgr2=0x7f948013f690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.279+0000 7f9474c00640 1 -- 172.21.15.78:0/2398291974 shutdown_connections 2024-09-11T12:49:18.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.279+0000 7f9474c00640 1 -- 172.21.15.78:0/2398291974 wait complete. 2024-09-11T12:49:18.285 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-11T12:49:18.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-11T12:49:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 -- 172.21.15.78:0/4277823113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca8c141ae0 msgr2=0x7fca8c150540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 --2- 172.21.15.78:0/4277823113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca8c141ae0 0x7fca8c150540 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fca7c00da90 tx=0x7fca7c00df50 comp rx=0 tx=0).stop 2024-09-11T12:49:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 -- 172.21.15.78:0/4277823113 shutdown_connections 2024-09-11T12:49:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 --2- 172.21.15.78:0/4277823113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca8c15c160 0x7fca8c0cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 --2- 172.21.15.78:0/4277823113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca8c141ec0 0x7fca8c14f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 --2- 172.21.15.78:0/4277823113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca8c141ae0 0x7fca8c150540 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7fca92600640 1 -- 172.21.15.78:0/4277823113 >> 172.21.15.78:0/4277823113 conn(0x7fca8c06e860 msgr2=0x7fca8c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.294 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-11T12:49:18.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-11T12:49:18.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.291+0000 7fca92600640 1 -- 172.21.15.78:0/4277823113 shutdown_connections 2024-09-11T12:49:18.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.287+0000 7eff12200640 1 -- 172.21.15.78:0/2781266479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 msgr2=0x7eff04053160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.291+0000 7eff12200640 1 --2- 172.21.15.78:0/2781266479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff04053160 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff00009870 tx=0x7eff0001c870 comp rx=0 tx=0).stop 2024-09-11T12:49:18.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.295+0000 7eff12200640 1 -- 172.21.15.78:0/2781266479 shutdown_connections 2024-09-11T12:49:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.295+0000 7eff12200640 1 --2- 172.21.15.78:0/2781266479 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff04053160 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.295+0000 7eff12200640 1 -- 172.21.15.78:0/2781266479 >> 172.21.15.78:0/2781266479 conn(0x7eff0401d780 msgr2=0x7eff04054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.299+0000 7fca92600640 1 -- 172.21.15.78:0/4277823113 wait complete. 2024-09-11T12:49:18.306 DEBUG:tasks.ceph:set 18 configs 2024-09-11T12:49:18.306 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.307 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.307 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.299+0000 7eff12200640 1 -- 172.21.15.78:0/2781266479 shutdown_connections 2024-09-11T12:49:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.299+0000 7eff12200640 1 -- 172.21.15.78:0/2781266479 wait complete. 2024-09-11T12:49:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.303+0000 7eff12200640 1 Processor -- start 2024-09-11T12:49:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7eff12200640 1 -- start start 2024-09-11T12:49:18.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7f9790e00640 1 -- 172.21.15.78:0/1335657214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9770004810 msgr2=0x7f9770004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7f9790e00640 1 --2- 172.21.15.78:0/1335657214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9770004810 0x7f9770004c90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f97700055b0 tx=0x7f977c01d400 comp rx=0 tx=0).stop 2024-09-11T12:49:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7f9790e00640 1 -- 172.21.15.78:0/1335657214 shutdown_connections 2024-09-11T12:49:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7f9790e00640 1 --2- 172.21.15.78:0/1335657214 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9770004810 0x7f9770004c90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.307+0000 7f9790e00640 1 -- 172.21.15.78:0/1335657214 >> 172.21.15.78:0/1335657214 conn(0x7f978c06c8b0 msgr2=0x7f978c06cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.315+0000 7f9790e00640 1 -- 172.21.15.78:0/1335657214 shutdown_connections 2024-09-11T12:49:18.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff040dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 0x7eff040ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff040e2ee0 0x7eff040dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff040edf80 con 0x7eff040e2ee0 2024-09-11T12:49:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff040ede00 con 0x7eff04036620 2024-09-11T12:49:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff040ee100 con 0x7eff04034340 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 0x7eff040ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 0x7eff040ccd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33260/0 (socket says 172.21.15.78:33260) 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0aa00640 1 -- 172.21.15.78:0/1424602155 learned_addr learned my addr 172.21.15.78:0/1424602155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff040dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 -- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 msgr2=0x7eff040ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 0x7eff040ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 -- 172.21.15.78:0/1424602155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff040e2ee0 msgr2=0x7eff040dbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff040e2ee0 0x7eff040dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 -- 172.21.15.78:0/1424602155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff040dadf0 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff0b400640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff040dfb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff0c06bd80 tx=0x7eff0c06a0c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c06cba0 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff0c06cd40 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff040db0d0 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c071430 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7eff040db690 con 0x7eff04034340 2024-09-11T12:49:18.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7eff0c093400 con 0x7eff04034340 2024-09-11T12:49:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff0c09c730 con 0x7eff04034340 2024-09-11T12:49:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7f9790e00640 1 -- 172.21.15.78:0/1335657214 wait complete. 2024-09-11T12:49:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7f9790e00640 1 Processor -- start 2024-09-11T12:49:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.319+0000 7f9790e00640 1 -- start start 2024-09-11T12:49:18.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff04035f10 con 0x7eff04034340 2024-09-11T12:49:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f978c152700 0x7f978c204810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 0x7f978c1f6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f978c0ba210 con 0x7f9770004810 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f978c0ba090 con 0x7f978c152700 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9790e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f978c0ba390 con 0x7f978c1556a0 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36102/0 (socket says 172.21.15.78:36102) 2024-09-11T12:49:18.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 -- 172.21.15.78:0/2132362173 learned_addr learned my addr 172.21.15.78:0/2132362173 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f978a800640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 0x7f978c1f6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 -- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 msgr2=0x7f978c1f6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 0x7f978c1f6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 -- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f978c152700 msgr2=0x7f978c204810 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f978c152700 0x7f978c204810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f9789e00640 1 -- 172.21.15.78:0/2132362173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c152470 con 0x7f9770004810 2024-09-11T12:49:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.323+0000 7f978a800640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 0x7f978c1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:18.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.327+0000 7f9789e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f977800b3f0 tx=0x7f9778009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.331+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7eff0c0a1020 con 0x7eff04034340 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778015020 con 0x7f9770004810 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9778011920 con 0x7f9770004810 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778011d50 con 0x7f9770004810 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c2050f0 con 0x7f9770004810 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c2056b0 con 0x7f9770004810 2024-09-11T12:49:18.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9778018020 con 0x7f9770004810 2024-09-11T12:49:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.335+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9778019840 con 0x7f9770004810 2024-09-11T12:49:18.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.343+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97640052a0 con 0x7f9770004810 2024-09-11T12:49:18.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.347+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f977801f020 con 0x7f9770004810 2024-09-11T12:49:18.388 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.388 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.388 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.388 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.423+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fc18c01dc80 con 0x7fc18c054c80 2024-09-11T12:49:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 -- 172.21.15.78:0/2058174340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9c10 msgr2=0x7fd2500722a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 --2- 172.21.15.78:0/2058174340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9c10 0x7fd2500722a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd240009840 tx=0x7fd24001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 -- 172.21.15.78:0/2058174340 shutdown_connections 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 --2- 172.21.15.78:0/2058174340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9c10 0x7fd2500722a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 --2- 172.21.15.78:0/2058174340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9830 0x7fd250071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 -- 172.21.15.78:0/2058174340 >> 172.21.15.78:0/2058174340 conn(0x7fd25006c8b0 msgr2=0x7fd25006cc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 -- 172.21.15.78:0/2058174340 shutdown_connections 2024-09-11T12:49:18.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.427+0000 7fd254e00640 1 -- 172.21.15.78:0/2058174340 wait complete. 2024-09-11T12:49:18.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.431+0000 7fd254e00640 1 Processor -- start 2024-09-11T12:49:18.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 -- start start 2024-09-11T12:49:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9c10 0x7fd25014bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 0x7fd25014a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd250162c00 con 0x7fd2500b9830 2024-09-11T12:49:18.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd250162a80 con 0x7fd2500d3a70 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd254e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd250162d80 con 0x7fd2500b9c10 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36116/0 (socket says 172.21.15.78:36116) 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 -- 172.21.15.78:0/2094161700 learned_addr learned my addr 172.21.15.78:0/2094161700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24d400640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9c10 0x7fd25014bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24e800640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 0x7fd25014a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 -- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9c10 msgr2=0x7fd25014bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9c10 0x7fd25014bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 -- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 msgr2=0x7fd25014a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 0x7fd25014a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 -- 172.21.15.78:0/2094161700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25014cd80 con 0x7fd2500b9830 2024-09-11T12:49:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24e800640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 0x7fd25014a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.435+0000 7fd24de00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd24800ba80 tx=0x7fd24800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.439+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd248019070 con 0x7fd2500b9830 2024-09-11T12:49:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.439+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24800cd10 con 0x7fd2500b9830 2024-09-11T12:49:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.439+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2480146a0 con 0x7fd2500b9830 2024-09-11T12:49:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.443+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25014b000 con 0x7fd2500b9830 2024-09-11T12:49:18.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.443+0000 7fc187400640 1 -- 172.21.15.78:0/2213931925 <== mon.1 v2:172.21.15.157:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7fc1800164d0 con 0x7fc18c054c80 2024-09-11T12:49:18.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.443+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd25014b5c0 con 0x7fd2500b9830 2024-09-11T12:49:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.447+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd24801c070 con 0x7fd2500b9830 2024-09-11T12:49:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.447+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd24801edc0 con 0x7fd2500b9830 2024-09-11T12:49:18.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.447+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd25014d0c0 con 0x7fd2500b9830 2024-09-11T12:49:18.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.451+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd248023030 con 0x7fd2500b9830 2024-09-11T12:49:18.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c054c80 msgr2=0x7fc18c047c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 --2- 172.21.15.78:0/2213931925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c054c80 0x7fc18c047c20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc18001af00 tx=0x7fc1800a5680 comp rx=0 tx=0).stop 2024-09-11T12:49:18.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 shutdown_connections 2024-09-11T12:49:18.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 --2- 172.21.15.78:0/2213931925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc18c054c80 0x7fc18c047c20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 --2- 172.21.15.78:0/2213931925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc18c006ff0 0x7fc18c009e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 --2- 172.21.15.78:0/2213931925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc18c006120 0x7fc18c0df1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.463+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 >> 172.21.15.78:0/2213931925 conn(0x7fc18c01d780 msgr2=0x7fc18c0d8e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.471+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 shutdown_connections 2024-09-11T12:49:18.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.471+0000 7fc198c00640 1 -- 172.21.15.78:0/2213931925 wait complete. 2024-09-11T12:49:18.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.499+0000 7fda63e00640 1 -- 172.21.15.78:0/3561583342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda640aecc0 msgr2=0x7fda640d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.499+0000 7fda63e00640 1 --2- 172.21.15.78:0/3561583342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda640aecc0 0x7fda640d0760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fda54009840 tx=0x7fda5401c300 comp rx=0 tx=0).stop 2024-09-11T12:49:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.507+0000 7fda63e00640 1 -- 172.21.15.78:0/3561583342 shutdown_connections 2024-09-11T12:49:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.507+0000 7fda63e00640 1 --2- 172.21.15.78:0/3561583342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda640aecc0 0x7fda640d0760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.507+0000 7fda63e00640 1 --2- 172.21.15.78:0/3561583342 >> v2:172.21.15.78:3300/0 conn(0x7fda6415da20 0x7fda6415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.507+0000 7fda63e00640 1 -- 172.21.15.78:0/3561583342 >> 172.21.15.78:0/3561583342 conn(0x7fda640731c0 msgr2=0x7fda640735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.507+0000 7fda63e00640 1 -- 172.21.15.78:0/3561583342 shutdown_connections 2024-09-11T12:49:18.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.511+0000 7fda63e00640 1 -- 172.21.15.78:0/3561583342 wait complete. 2024-09-11T12:49:18.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.511+0000 7fda63e00640 1 Processor -- start 2024-09-11T12:49:18.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.511+0000 7fda63e00640 1 -- start start 2024-09-11T12:49:18.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 0x7fda641fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda6415da20 0x7fda640b1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda6416b890 con 0x7fda640d2b30 2024-09-11T12:49:18.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda6416b710 con 0x7fda640aecc0 2024-09-11T12:49:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda6416ba10 con 0x7fda6415da20 2024-09-11T12:49:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33286/0 (socket says 172.21.15.78:33286) 2024-09-11T12:49:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 -- 172.21.15.78:0/2839603431 learned_addr learned my addr 172.21.15.78:0/2839603431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda63400640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda6415da20 0x7fda640b1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62000640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 0x7fda641fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 -- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda6415da20 msgr2=0x7fda640b1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda6415da20 0x7fda640b1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 -- 172.21.15.78:0/2839603431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 msgr2=0x7fda641fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 0x7fda641fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62a00640 1 -- 172.21.15.78:0/2839603431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6414f9e0 con 0x7fda640aecc0 2024-09-11T12:49:18.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.515+0000 7fda62000640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 0x7fda641fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:18.528 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda62a00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00ba50 tx=0x7fda5c00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c019070 con 0x7fda640aecc0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda5c00ccf0 con 0x7fda640aecc0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0146a0 con 0x7fda640aecc0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6414fcc0 con 0x7fda640aecc0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda641fc3f0 con 0x7fda640aecc0 2024-09-11T12:49:18.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.523+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fda5c01c070 con 0x7fda640aecc0 2024-09-11T12:49:18.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.527+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fda5c01e440 con 0x7fda640aecc0 2024-09-11T12:49:18.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.535+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda640d0740 con 0x7fda640aecc0 2024-09-11T12:49:18.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.539+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fda5c01d3b0 con 0x7fda640aecc0 2024-09-11T12:49:18.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7eff040d98f0 con 0x7eff04034340 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 -- 172.21.15.78:0/1593541378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be41584a0 msgr2=0x7f9be406e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 --2- 172.21.15.78:0/1593541378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be41584a0 0x7f9be406e720 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9be40cf830 tx=0x7f9bdc01cae0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 -- 172.21.15.78:0/1593541378 shutdown_connections 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 --2- 172.21.15.78:0/1593541378 >> v2:172.21.15.78:3300/0 conn(0x7f9be40cc230 0x7f9be415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 --2- 172.21.15.78:0/1593541378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be415c540 0x7f9be40b2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 --2- 172.21.15.78:0/1593541378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be41584a0 0x7f9be406e720 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 -- 172.21.15.78:0/1593541378 >> 172.21.15.78:0/1593541378 conn(0x7f9be406cb50 msgr2=0x7f9be40b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 -- 172.21.15.78:0/1593541378 shutdown_connections 2024-09-11T12:49:18.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.547+0000 7f9bea400640 1 -- 172.21.15.78:0/1593541378 wait complete. 2024-09-11T12:49:18.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.555+0000 7f9bea400640 1 Processor -- start 2024-09-11T12:49:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.555+0000 7f9bea400640 1 -- start start 2024-09-11T12:49:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.555+0000 7f9bea400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.555+0000 7f9be9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.555+0000 7f9be9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33312/0 (socket says 172.21.15.78:33312) 2024-09-11T12:49:18.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 -- 172.21.15.78:0/1811609886 learned_addr learned my addr 172.21.15.78:0/1811609886 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 0x7f9be41390b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7efeff400640 1 -- 172.21.15.78:0/1424602155 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7eff0c08f030 con 0x7eff04034340 2024-09-11T12:49:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be3e00640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 0x7f9be41390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9be415c540 0x7f9be415c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9be40703b0 con 0x7f9be415c540 2024-09-11T12:49:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9be4070230 con 0x7f9be40cc230 2024-09-11T12:49:18.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9be4070530 con 0x7f9be41584a0 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 -- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 msgr2=0x7f9be41390b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 0x7f9be41390b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 -- 172.21.15.78:0/1811609886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9be415c540 msgr2=0x7f9be415c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9a00640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9be415c540 0x7f9be415c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9be415c540 0x7f9be415c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be9000640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9be41470b0 con 0x7f9be40cc230 2024-09-11T12:49:18.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.559+0000 7f9be3e00640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 0x7f9be41390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.563+0000 7f9be9000640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b10 tx=0x7f9bdc0a5880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.563+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0ba020 con 0x7f9be40cc230 2024-09-11T12:49:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.563+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bdc004d40 con 0x7f9be40cc230 2024-09-11T12:49:18.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.563+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc0ba020 con 0x7f9be40cc230 2024-09-11T12:49:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.563+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9be414a2a0 con 0x7f9be40cc230 2024-09-11T12:49:18.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.567+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9be414a820 con 0x7f9be40cc230 2024-09-11T12:49:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.567+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9bdc0bd070 con 0x7f9be40cc230 2024-09-11T12:49:18.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.567+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9bdc005360 con 0x7f9be40cc230 2024-09-11T12:49:18.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9be4070a70 con 0x7f9be40cc230 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 msgr2=0x7eff040dfb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff040dfb80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7eff0c06bd80 tx=0x7eff0c06a0c0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 shutdown_connections 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff040e2ee0 0x7eff040dbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff04036620 0x7eff040ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 --2- 172.21.15.78:0/1424602155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff04034340 0x7eff040dfb80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 >> 172.21.15.78:0/1424602155 conn(0x7eff0401d780 msgr2=0x7eff040d8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 shutdown_connections 2024-09-11T12:49:18.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7eff12200640 1 -- 172.21.15.78:0/1424602155 wait complete. 2024-09-11T12:49:18.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7f1d1e000640 1 -- 172.21.15.78:0/3593623964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10055070 msgr2=0x7f1d10035ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.571+0000 7f1d1e000640 1 --2- 172.21.15.78:0/3593623964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10055070 0x7f1d10035ff0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c009840 tx=0x7f1d0c01c6a0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.575+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9bdc0c4070 con 0x7f9be40cc230 2024-09-11T12:49:18.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.587+0000 7f1d1e000640 1 -- 172.21.15.78:0/3593623964 shutdown_connections 2024-09-11T12:49:18.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.587+0000 7f1d1e000640 1 --2- 172.21.15.78:0/3593623964 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10055070 0x7f1d10035ff0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.587+0000 7f1d1e000640 1 --2- 172.21.15.78:0/3593623964 >> v2:172.21.15.157:3300/0 conn(0x7f1d10033740 0x7f1d100e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.587+0000 7f1d1e000640 1 -- 172.21.15.78:0/3593623964 >> 172.21.15.78:0/3593623964 conn(0x7f1d1001d780 msgr2=0x7f1d10054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.591+0000 7f1d1e000640 1 -- 172.21.15.78:0/3593623964 shutdown_connections 2024-09-11T12:49:18.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.591+0000 7f1d1e000640 1 -- 172.21.15.78:0/3593623964 wait complete. 2024-09-11T12:49:18.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.591+0000 7f1d1e000640 1 Processor -- start 2024-09-11T12:49:18.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.591+0000 7f1d1e000640 1 -- start start 2024-09-11T12:49:18.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10036530 0x7f1d100d7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36144/0 (socket says 172.21.15.78:36144) 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1cc00640 1 -- 172.21.15.78:0/2147799210 learned_addr learned my addr 172.21.15.78:0/2147799210 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 0x7f1d100e5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d100393a0 con 0x7f1d10033740 2024-09-11T12:49:18.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d1003eb50 con 0x7f1d10055070 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d10039520 con 0x7f1d10036530 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d1d600640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 0x7f1d100e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.595+0000 7f1d17e00640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10036530 0x7f1d100d7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 -- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10036530 msgr2=0x7f1d100d7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10036530 0x7f1d100d7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 -- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 msgr2=0x7f1d100e5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 0x7f1d100e5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d100e5980 con 0x7f1d10033740 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1d600640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 0x7f1d100e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1cc00640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009540 tx=0x7f1d04009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0401f9f0 con 0x7f1d10033740 2024-09-11T12:49:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d10048260 con 0x7f1d10033740 2024-09-11T12:49:18.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d100d8190 con 0x7f1d10033740 2024-09-11T12:49:18.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d0401fc80 con 0x7f1d10033740 2024-09-11T12:49:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d040216a0 con 0x7f1d10033740 2024-09-11T12:49:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.599+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cf40052a0 con 0x7f1d10033740 2024-09-11T12:49:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.603+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1d04021840 con 0x7f1d10033740 2024-09-11T12:49:18.611 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.615+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d04029b50 con 0x7f1d10033740 2024-09-11T12:49:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.615+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1d04021d60 con 0x7f1d10033740 2024-09-11T12:49:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.619+0000 7f455be00640 1 -- 172.21.15.78:0/2019481233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45540e2ee0 msgr2=0x7f4554034340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.619+0000 7f455be00640 1 --2- 172.21.15.78:0/2019481233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45540e2ee0 0x7f4554034340 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f455c0750b0 tx=0x7f455c097620 comp rx=0 tx=0).stop 2024-09-11T12:49:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.627+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f9764005860 con 0x7f9770004810 2024-09-11T12:49:18.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.627+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd2501603a0 con 0x7fd2500b9830 2024-09-11T12:49:18.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.631+0000 7f455be00640 1 -- 172.21.15.78:0/2019481233 shutdown_connections 2024-09-11T12:49:18.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.631+0000 7f455be00640 1 --2- 172.21.15.78:0/2019481233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45540e2ee0 0x7f4554034340 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.631+0000 7f455be00640 1 -- 172.21.15.78:0/2019481233 >> 172.21.15.78:0/2019481233 conn(0x7f455401d780 msgr2=0x7f4554054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.631+0000 7f455be00640 1 -- 172.21.15.78:0/2019481233 shutdown_connections 2024-09-11T12:49:18.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.631+0000 7f455be00640 1 -- 172.21.15.78:0/2019481233 wait complete. 2024-09-11T12:49:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 -- 172.21.15.78:0/397525300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180737a0 msgr2=0x7fab1815bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 --2- 172.21.15.78:0/397525300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180737a0 0x7fab1815bca0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fab04013f30 tx=0x7fab04024d70 comp rx=0 tx=0).stop 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 -- 172.21.15.78:0/344892797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd210005420 msgr2=0x7fd21000ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 --2- 172.21.15.78:0/344892797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd210005420 0x7fd21000ba50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd2180017b0 tx=0x7fd21801c7a0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 -- 172.21.15.78:0/344892797 shutdown_connections 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 --2- 172.21.15.78:0/344892797 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd210005420 0x7fd21000ba50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 -- 172.21.15.78:0/344892797 >> 172.21.15.78:0/344892797 conn(0x7fd22401d780 msgr2=0x7fd224054570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 -- 172.21.15.78:0/344892797 shutdown_connections 2024-09-11T12:49:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fd22be00640 1 -- 172.21.15.78:0/344892797 wait complete. 2024-09-11T12:49:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 -- 172.21.15.78:0/2792109973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce401c370 msgr2=0x7f5ce401c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 --2- 172.21.15.78:0/2792109973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce401c370 0x7f5ce401c800 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8009840 tx=0x7f5cd801c7b0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f455be00640 1 Processor -- start 2024-09-11T12:49:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f455be00640 1 -- start start 2024-09-11T12:49:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 -- 172.21.15.78:0/397525300 shutdown_connections 2024-09-11T12:49:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f9782000640 1 -- 172.21.15.78:0/2132362173 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7f977801f020 con 0x7f9770004810 2024-09-11T12:49:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 --2- 172.21.15.78:0/397525300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab180737a0 0x7fab1815bca0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 --2- 172.21.15.78:0/397525300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab1815b650 0x7fab18073260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7fab1d200640 1 -- 172.21.15.78:0/397525300 >> 172.21.15.78:0/397525300 conn(0x7fab18075610 msgr2=0x7fab18075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 -- 172.21.15.78:0/2792109973 shutdown_connections 2024-09-11T12:49:18.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 --2- 172.21.15.78:0/2792109973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce401c370 0x7f5ce401c800 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fab1d200640 1 -- 172.21.15.78:0/397525300 shutdown_connections 2024-09-11T12:49:18.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fab1d200640 1 -- 172.21.15.78:0/397525300 wait complete. 2024-09-11T12:49:18.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 --2- 172.21.15.78:0/2792109973 >> v2:172.21.15.157:3300/0 conn(0x7f5ce4005a70 0x7f5ce4006860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.635+0000 7f5cf0e00640 1 -- 172.21.15.78:0/2792109973 >> 172.21.15.78:0/2792109973 conn(0x7f5ce4054570 msgr2=0x7f5ce4054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f5cf0e00640 1 -- 172.21.15.78:0/2792109973 shutdown_connections 2024-09-11T12:49:18.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f5cf0e00640 1 -- 172.21.15.78:0/2792109973 wait complete. 2024-09-11T12:49:18.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36170/0 (socket says 172.21.15.78:36170) 2024-09-11T12:49:18.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22be00640 1 Processor -- start 2024-09-11T12:49:18.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455aa00640 1 -- 172.21.15.78:0/2468131871 learned_addr learned my addr 172.21.15.78:0/2468131871 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22be00640 1 -- start start 2024-09-11T12:49:18.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4554052a50 0x7f45541778d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fab1d200640 1 Processor -- start 2024-09-11T12:49:18.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 0x7f455400ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 0x7fd2240dc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4554055460 con 0x7f4554035f10 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:42196/0 (socket says 172.21.15.78:42196) 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22aa00640 1 -- 172.21.15.78:0/380237764 learned_addr learned my addr 172.21.15.78:0/380237764 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45540552e0 con 0x7f45540e2ee0 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4554033900 con 0x7f4554052a50 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 0x7f455400ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455a000640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4554052a50 0x7f45541778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7fd22be00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 0x7fd2240dc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 -- start start 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240e7450 con 0x7fd224031d00 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2240e72d0 con 0x7fd224032180 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22b400640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 0x7fd2240dc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22a000640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 0x7fd2240dc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2240e2200 con 0x7fd210005420 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 -- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4554052a50 msgr2=0x7f45541778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4554052a50 0x7f45541778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 -- 172.21.15.78:0/2468131871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 msgr2=0x7f455404b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.639+0000 7f455b400640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4554181130 con 0x7f45540e2ee0 2024-09-11T12:49:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 msgr2=0x7f978c151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f455aa00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f977800b3f0 tx=0x7f9778009f90 comp rx=0 tx=0).stop 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 -- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 msgr2=0x7fd2240dc880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 0x7fd2240dc880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 -- 172.21.15.78:0/380237764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 msgr2=0x7fd2240dc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 0x7fd2240dc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22404aa50 con 0x7fd210005420 2024-09-11T12:49:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22b400640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 0x7fd2240dc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22aa00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd22c06bd80 tx=0x7fd22c06ab80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c068ac0 con 0x7fd210005420 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd22c0513f0 con 0x7fd210005420 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 shutdown_connections 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f978c1556a0 0x7f978c1f6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f978c152700 0x7f978c204810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 --2- 172.21.15.78:0/2132362173 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9770004810 0x7f978c151f30 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 >> 172.21.15.78:0/2132362173 conn(0x7f978c06c8b0 msgr2=0x7f978c0b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f455b400640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 0x7f455400ab90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c60 tx=0x7f454c00c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab180737a0 0x7fab18158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab181516c0 0x7fab18148310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab18072160 con 0x7fab180737a0 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab18071fe0 con 0x7fab1815b650 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab1d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab180722e0 con 0x7fab181516c0 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33402/0 (socket says 172.21.15.78:33402) 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 -- 172.21.15.78:0/2357808178 learned_addr learned my addr 172.21.15.78:0/2357808178 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab16a00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab181516c0 0x7fab18148310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 Processor -- start 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 -- start start 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c068ac0 con 0x7fd210005420 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22404ad30 con 0x7fd210005420 2024-09-11T12:49:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd224051780 con 0x7fd210005420 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ce401c370 0x7f5ce40521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce40defe0 0x7f5ce4053ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce401d0d0 con 0x7f5ce401c370 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce401cf50 con 0x7f5ce4005a70 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5cf0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce401d250 con 0x7f5ce40defe0 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33414/0 (socket says 172.21.15.78:33414) 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 -- 172.21.15.78:0/3142613122 learned_addr learned my addr 172.21.15.78:0/3142613122 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 -- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce40defe0 msgr2=0x7f5ce4053ac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce40defe0 0x7f5ce4053ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 -- 172.21.15.78:0/3142613122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ce401c370 msgr2=0x7f5ce40521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ce401c370 0x7f5ce40521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f5ce9e00640 1 -- 172.21.15.78:0/3142613122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce4052a20 con 0x7f5ce4005a70 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce9e00640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c480 tx=0x7f5cdc00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0052d0 con 0x7f5ce4005a70 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cdc005490 con 0x7f5ce4005a70 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fd22a000640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 0x7fd2240dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:18.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd22c0b4950 con 0x7fd210005420 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 shutdown_connections 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f9790e00640 1 -- 172.21.15.78:0/2132362173 wait complete. 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c018030 con 0x7f45540e2ee0 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f454c00cd50 con 0x7f45540e2ee0 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c004280 con 0x7f45540e2ee0 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4554181410 con 0x7f45540e2ee0 2024-09-11T12:49:18.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541819d0 con 0x7f45540e2ee0 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 -- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab181516c0 msgr2=0x7fab18148310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab181516c0 0x7fab18148310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 -- 172.21.15.78:0/2357808178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab180737a0 msgr2=0x7fab18158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab180737a0 0x7fab18158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.643+0000 7fab17e00640 1 -- 172.21.15.78:0/2357808178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab18155a10 con 0x7fab1815b650 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab17e00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab040b8040 tx=0x7fab040ada90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040c2020 con 0x7fab1815b650 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab040aee50 con 0x7fab1815b650 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab040ba040 con 0x7fab1815b650 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab1d200640 1 -- 172.21.15.78:0/2357808178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab18155cf0 con 0x7fab1815b650 2024-09-11T12:49:18.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fab1d200640 1 -- 172.21.15.78:0/2357808178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab18148850 con 0x7fab1815b650 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd22c0933f0 con 0x7fd210005420 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f454c01b070 con 0x7f45540e2ee0 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f454c01d910 con 0x7f45540e2ee0 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45340052a0 con 0x7f45540e2ee0 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5cf0e00640 1 -- 172.21.15.78:0/3142613122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce4052d00 con 0x7f5ce4005a70 2024-09-11T12:49:18.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5cf0e00640 1 -- 172.21.15.78:0/3142613122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce40532c0 con 0x7f5ce4005a70 2024-09-11T12:49:18.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc012870 con 0x7f5ce4005a70 2024-09-11T12:49:18.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5cdc012a10 con 0x7f5ce4005a70 2024-09-11T12:49:18.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5cdc01b8a0 con 0x7f5ce4005a70 2024-09-11T12:49:18.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2240078c0 con 0x7fd210005420 2024-09-11T12:49:18.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.647+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cc00052a0 con 0x7f5ce4005a70 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd246000640 1 -- 172.21.15.78:0/2094161700 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fd24801e4a0 con 0x7fd2500b9830 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1815bca0 con 0x7fab1815b650 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fab040c5070 con 0x7fab1815b650 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fab04003da0 con 0x7fab1815b650 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 msgr2=0x7fd25014c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd24800ba80 tx=0x7fd24800bf40 comp rx=0 tx=0).stop 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f454c022020 con 0x7f45540e2ee0 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 shutdown_connections 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2500d3a70 0x7fd25014a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2500b9c10 0x7fd25014bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 --2- 172.21.15.78:0/2094161700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2500b9830 0x7fd25014c4a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 >> 172.21.15.78:0/2094161700 conn(0x7fd25006c8b0 msgr2=0x7fd250161260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 shutdown_connections 2024-09-11T12:49:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.651+0000 7fd254e00640 1 -- 172.21.15.78:0/2094161700 wait complete. 2024-09-11T12:49:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd22c098030 con 0x7fd210005420 2024-09-11T12:49:18.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 -- 172.21.15.78:0/1580679537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 msgr2=0x7f9cd415aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1580679537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd415aa00 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9cd406ebb0 tx=0x7f9cc001c7d0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 -- 172.21.15.78:0/1580679537 shutdown_connections 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1580679537 >> v2:172.21.15.157:3300/0 conn(0x7f9cd415af40 0x7f9cd40b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1580679537 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd415aa00 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1580679537 >> v2:172.21.15.78:3300/0 conn(0x7f9cd4150b80 0x7f9cd415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f9cdb200640 1 -- 172.21.15.78:0/1580679537 >> 172.21.15.78:0/1580679537 conn(0x7f9cd4071dd0 msgr2=0x7f9cd40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fab0401ea80 con 0x7fab1815b650 2024-09-11T12:49:18.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.655+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5cdc01baf0 con 0x7f5ce4005a70 2024-09-11T12:49:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.667+0000 7f9cdb200640 1 -- 172.21.15.78:0/1580679537 shutdown_connections 2024-09-11T12:49:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.667+0000 7f9cdb200640 1 -- 172.21.15.78:0/1580679537 wait complete. 2024-09-11T12:49:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.667+0000 7f9cdb200640 1 Processor -- start 2024-09-11T12:49:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.679+0000 7f9cdb200640 1 -- start start 2024-09-11T12:49:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.679+0000 7f9cdb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd4205350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cd414d4f0 0x7f9cd414d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:18.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40b31b0 con 0x7f9cd414d4f0 2024-09-11T12:49:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cd40b3030 con 0x7f9cd4150b80 2024-09-11T12:49:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9cd40b3330 con 0x7f9cd4150f60 2024-09-11T12:49:18.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33434/0 (socket says 172.21.15.78:33434) 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3e00640 1 -- 172.21.15.78:0/1187008042 learned_addr learned my addr 172.21.15.78:0/1187008042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd4205350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 -- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 msgr2=0x7f9cd414c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 -- 172.21.15.78:0/1187008042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cd414d4f0 msgr2=0x7f9cd414d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cd414d4f0 0x7f9cd414d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 -- 172.21.15.78:0/1187008042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cd415d890 con 0x7f9cd4150f60 2024-09-11T12:49:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3e00640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cd3400640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd4205350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ac000 tx=0x7f9cc00a56c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc00b2040 con 0x7f9cd4150f60 2024-09-11T12:49:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cc0005480 con 0x7f9cd4150f60 2024-09-11T12:49:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cd4205890 con 0x7f9cd4150f60 2024-09-11T12:49:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0004290 con 0x7f9cd4150f60 2024-09-11T12:49:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.687+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cd414ee00 con 0x7f9cd4150f60 2024-09-11T12:49:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.691+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7fda64069440 con 0x7fda640aecc0 2024-09-11T12:49:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.691+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9cc00a5ac0 con 0x7f9cd4150f60 2024-09-11T12:49:18.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.691+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9cc0004ca0 con 0x7f9cd4150f60 2024-09-11T12:49:18.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.691+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cd415aa00 con 0x7f9cd4150f60 2024-09-11T12:49:18.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.699+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9cc00c4020 con 0x7f9cd4150f60 2024-09-11T12:49:18.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.703+0000 7fda5aa00640 1 -- 172.21.15.78:0/2839603431 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7fda5c014840 con 0x7fda640aecc0 2024-09-11T12:49:18.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 msgr2=0x7fda641507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fda5c00ba50 tx=0x7fda5c00bf10 comp rx=0 tx=0).stop 2024-09-11T12:49:18.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 shutdown_connections 2024-09-11T12:49:18.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda6415da20 0x7fda640b1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.715 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.715 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda640d2b30 0x7fda641fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 --2- 172.21.15.78:0/2839603431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda640aecc0 0x7fda641507a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 >> 172.21.15.78:0/2839603431 conn(0x7fda640731c0 msgr2=0x7fda640ae590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 shutdown_connections 2024-09-11T12:49:18.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.711+0000 7fda63e00640 1 -- 172.21.15.78:0/2839603431 wait complete. 2024-09-11T12:49:18.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.735+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f9be40b3930 con 0x7f9be40cc230 2024-09-11T12:49:18.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.743+0000 7f9be0c00640 1 -- 172.21.15.78:0/1811609886 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f9be41470b0 con 0x7f9be40cc230 2024-09-11T12:49:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 msgr2=0x7f9be4163070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc009b10 tx=0x7f9bdc0a5880 comp rx=0 tx=0).stop 2024-09-11T12:49:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 shutdown_connections 2024-09-11T12:49:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9be415c540 0x7f9be415c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9be41584a0 0x7f9be41390b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 --2- 172.21.15.78:0/1811609886 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9be40cc230 0x7f9be4163070 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 >> 172.21.15.78:0/1811609886 conn(0x7f9be406cb50 msgr2=0x7f9be4159e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 shutdown_connections 2024-09-11T12:49:18.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.747+0000 7f9bea400640 1 -- 172.21.15.78:0/1811609886 wait complete. 2024-09-11T12:49:18.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.771+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f1cf4005590 con 0x7f1d10033740 2024-09-11T12:49:18.775 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.775 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.771+0000 7f1d14c00640 1 -- 172.21.15.78:0/2147799210 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7f1d04029d50 con 0x7f1d10033740 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 msgr2=0x7f1d100d9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d04009540 tx=0x7f1d04009a00 comp rx=0 tx=0).stop 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 shutdown_connections 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d10055070 0x7f1d100e5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d10036530 0x7f1d100d7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 --2- 172.21.15.78:0/2147799210 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d10033740 0x7f1d100d9260 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 >> 172.21.15.78:0/2147799210 conn(0x7f1d1001d780 msgr2=0x7f1d100099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 shutdown_connections 2024-09-11T12:49:18.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f1d1e000640 1 -- 172.21.15.78:0/2147799210 wait complete. 2024-09-11T12:49:18.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.775+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f4534005590 con 0x7f45540e2ee0 2024-09-11T12:49:18.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.779+0000 7f4552a00640 1 -- 172.21.15.78:0/2468131871 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7f454c01db60 con 0x7f45540e2ee0 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 msgr2=0x7f455400ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 0x7f455400ab90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f454c009c60 tx=0x7f454c00c430 comp rx=0 tx=0).stop 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 shutdown_connections 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45540e2ee0 0x7f455400ab90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4554052a50 0x7f45541778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 --2- 172.21.15.78:0/2468131871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4554035f10 0x7f455404b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 >> 172.21.15.78:0/2468131871 conn(0x7f455401d780 msgr2=0x7f45540de230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 shutdown_connections 2024-09-11T12:49:18.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.787+0000 7f455be00640 1 -- 172.21.15.78:0/2468131871 wait complete. 2024-09-11T12:49:18.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.791+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f5cc0005590 con 0x7f5ce4005a70 2024-09-11T12:49:18.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.791+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fd224035bb0 con 0x7fd210005420 2024-09-11T12:49:18.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5ce2000640 1 -- 172.21.15.78:0/3142613122 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v16)=0 v16) ==== 121+0+0 (secure 0 0 0) 0x7f5cdc01bcd0 con 0x7f5ce4005a70 2024-09-11T12:49:18.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 msgr2=0x7f5ce4180250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc00c480 tx=0x7f5cdc00c940 comp rx=0 tx=0).stop 2024-09-11T12:49:18.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 shutdown_connections 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5ce40defe0 0x7f5ce4053ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5ce401c370 0x7f5ce40521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 --2- 172.21.15.78:0/3142613122 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5ce4005a70 0x7f5ce4180250 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 >> 172.21.15.78:0/3142613122 conn(0x7f5ce4054570 msgr2=0x7f5ce40de2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.795+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 shutdown_connections 2024-09-11T12:49:18.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7f5cd7400640 1 -- 172.21.15.78:0/3142613122 wait complete. 2024-09-11T12:49:18.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd223400640 1 -- 172.21.15.78:0/380237764 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7fd22c08f030 con 0x7fd210005420 2024-09-11T12:49:18.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 msgr2=0x7fd224048220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd22c06bd80 tx=0x7fd22c06ab80 comp rx=0 tx=0).stop 2024-09-11T12:49:18.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 shutdown_connections 2024-09-11T12:49:18.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd224032180 0x7fd2240dc880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd224031d00 0x7fd2240dc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 --2- 172.21.15.78:0/380237764 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd210005420 0x7fd224048220 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.799+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 >> 172.21.15.78:0/380237764 conn(0x7fd22401d780 msgr2=0x7fd22400a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.803+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7fab1814d7e0 con 0x7fab1815b650 2024-09-11T12:49:18.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.803+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 shutdown_connections 2024-09-11T12:49:18.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.807+0000 7fd22be00640 1 -- 172.21.15.78:0/380237764 wait complete. 2024-09-11T12:49:18.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0f400640 1 -- 172.21.15.78:0/2357808178 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7fab040055a0 con 0x7fab1815b650 2024-09-11T12:49:18.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 msgr2=0x7fab181f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fab040b8040 tx=0x7fab040ada90 comp rx=0 tx=0).stop 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 shutdown_connections 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab1815b650 0x7fab181f6cc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab181516c0 0x7fab18148310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 --2- 172.21.15.78:0/2357808178 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab180737a0 0x7fab18158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 >> 172.21.15.78:0/2357808178 conn(0x7fab18075610 msgr2=0x7fab1814db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 shutdown_connections 2024-09-11T12:49:18.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.811+0000 7fab0cc00640 1 -- 172.21.15.78:0/2357808178 wait complete. 2024-09-11T12:49:18.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.827+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f9cd4069420 con 0x7f9cd4150f60 2024-09-11T12:49:18.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.831+0000 7f9ccbe00640 1 -- 172.21.15.78:0/1187008042 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7f9cd4205890 con 0x7f9cd4150f60 2024-09-11T12:49:18.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 msgr2=0x7f9cd4205350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:18.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd4205350 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00ac000 tx=0x7f9cc00a56c0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 shutdown_connections 2024-09-11T12:49:18.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cd414d4f0 0x7f9cd414d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cd4150f60 0x7f9cd4205350 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 --2- 172.21.15.78:0/1187008042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cd4150b80 0x7f9cd414c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 >> 172.21.15.78:0/1187008042 conn(0x7f9cd4071dd0 msgr2=0x7f9cd4073480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 shutdown_connections 2024-09-11T12:49:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:18.835+0000 7f9cdb200640 1 -- 172.21.15.78:0/1187008042 wait complete. 2024-09-11T12:49:18.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.858 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi078') 2024-09-11T12:49:18.858 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T12:49:18.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T12:49:19.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- 172.21.15.78:0/2315728957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b81408c0 msgr2=0x7f91b80cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 --2- 172.21.15.78:0/2315728957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b81408c0 0x7f91b80cd760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f91a8009870 tx=0x7f91a801c830 comp rx=0 tx=0).stop 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- 172.21.15.78:0/2315728957 shutdown_connections 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 --2- 172.21.15.78:0/2315728957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b81408c0 0x7f91b80cd760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- 172.21.15.78:0/2315728957 >> 172.21.15.78:0/2315728957 conn(0x7f91b80757e0 msgr2=0x7f91b8072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- 172.21.15.78:0/2315728957 shutdown_connections 2024-09-11T12:49:19.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- 172.21.15.78:0/2315728957 wait complete. 2024-09-11T12:49:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 Processor -- start 2024-09-11T12:49:19.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- start start 2024-09-11T12:49:19.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 0x7f91b8163ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 0x7f91b80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b81408c0 0x7f91b8153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8169130 con 0x7f91b80d03e0 2024-09-11T12:49:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91b8168fb0 con 0x7f91b81408c0 2024-09-11T12:49:19.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91be200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f91b81692b0 con 0x7f91b80b31a0 2024-09-11T12:49:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91aea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 0x7f91b80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 0x7f91b8163ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91aea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 0x7f91b80cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36224/0 (socket says 172.21.15.78:36224) 2024-09-11T12:49:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91aea00640 1 -- 172.21.15.78:0/722141669 learned_addr learned my addr 172.21.15.78:0/722141669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:19.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7e00640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b81408c0 0x7f91b8153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 -- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b81408c0 msgr2=0x7f91b8153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b81408c0 0x7f91b8153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 -- 172.21.15.78:0/722141669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 msgr2=0x7f91b80cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 0x7f91b80cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 -- 172.21.15.78:0/722141669 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b81fbd80 con 0x7f91b80b31a0 2024-09-11T12:49:19.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.039+0000 7f91b7400640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 0x7f91b8163ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cdce0 tx=0x7f91a8005db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:19.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a80bb070 con 0x7f91b80b31a0 2024-09-11T12:49:19.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a8004320 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a80b2040 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b813fa00 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b81fb3e0 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f91a80a57d0 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f91a80ad6f0 con 0x7f91b80b31a0 2024-09-11T12:49:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.043+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b8072430 con 0x7f91b80b31a0 2024-09-11T12:49:19.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.047+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f91a80c4020 con 0x7f91b80b31a0 2024-09-11T12:49:19.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.107+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f91b8072150 con 0x7f91b80b31a0 2024-09-11T12:49:19.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91afe00640 1 -- 172.21.15.78:0/722141669 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f91a8016450 con 0x7f91b80b31a0 2024-09-11T12:49:19.117 INFO:teuthology.orchestra.run.smithi078.stderr:adjusted tunables profile to default 2024-09-11T12:49:19.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 msgr2=0x7f91b8163ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 0x7f91b8163ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91b80cdce0 tx=0x7f91a8005db0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 shutdown_connections 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91b81408c0 0x7f91b8153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91b80d03e0 0x7f91b80cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 --2- 172.21.15.78:0/722141669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91b80b31a0 0x7f91b8163ad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 >> 172.21.15.78:0/722141669 conn(0x7f91b80757e0 msgr2=0x7f91b80702d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 shutdown_connections 2024-09-11T12:49:19.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:19.115+0000 7f91be200640 1 -- 172.21.15.78:0/722141669 wait complete. 2024-09-11T12:49:19.138 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T12:49:19.138 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T12:49:19.139 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:19.139 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T12:49:19.159 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:19.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T12:49:19.221 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:19.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T12:49:19.288 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:49:19.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-11T12:49:19.349 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:19.349 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-11T12:49:19.371 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:19.371 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-11T12:49:19.431 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:19.431 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-11T12:49:19.495 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:49:19.495 DEBUG:teuthology.orchestra.run.smithi157:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-11T12:49:19.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 93361912-1069-4275-84d3-de4f9b60f305 0 2024-09-11T12:49:19.736 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2542991271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8071dd0 msgr2=0x7fc5d80a9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2542991271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8071dd0 0x7fc5d80a9f90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009900 tx=0x7fc5c801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2542991271 shutdown_connections 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2542991271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d815fce0 0x7fc5d815be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2542991271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d8071dd0 0x7fc5d80a9f90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2542991271 >> 172.21.15.157:0/2542991271 conn(0x7fc5d80757e0 msgr2=0x7fc5d8072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2542991271 shutdown_connections 2024-09-11T12:49:19.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2542991271 wait complete. 2024-09-11T12:49:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 Processor -- start 2024-09-11T12:49:19.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- start start 2024-09-11T12:49:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d80721b0 0x7fc5d8155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5d815fce0 0x7fc5d8155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d8169260 con 0x7fc5d815fce0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5d81690e0 con 0x7fc5d80721b0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5ddc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5d81693e0 con 0x7fc5d8071dd0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d80721b0 0x7fc5d8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36028/0 (socket says 172.21.15.157:36028) 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 -- 172.21.15.157:0/2854074973 learned_addr learned my addr 172.21.15.157:0/2854074973 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d7400640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5d815fce0 0x7fc5d8155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 -- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d80721b0 msgr2=0x7fc5d8155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d80721b0 0x7fc5d8155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 -- 172.21.15.157:0/2854074973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5d815fce0 msgr2=0x7fc5d8155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5d815fce0 0x7fc5d8155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.735+0000 7fc5d6a00640 1 -- 172.21.15.157:0/2854074973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5d81503c0 con 0x7fc5d8071dd0 2024-09-11T12:49:19.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5d6a00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009850 tx=0x7fc5c80a57c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80b2040 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5c8004400 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8005290 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5d81506a0 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5d8150c60 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc5c80046d0 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7fc5c80b6460 con 0x7fc5d8071dd0 2024-09-11T12:49:19.742 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5d80a9f90 con 0x7fc5d8071dd0 2024-09-11T12:49:19.743 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.739+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fc5c80164d0 con 0x7fc5d8071dd0 2024-09-11T12:49:19.812 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.811+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "93361912-1069-4275-84d3-de4f9b60f305", "id": 0} v 0) -- 0x7fc5d80b0750 con 0x7fc5d8071dd0 2024-09-11T12:49:19.816 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5cea00640 1 -- 172.21.15.157:0/2854074973 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "93361912-1069-4275-84d3-de4f9b60f305", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7fc5c8003600 con 0x7fc5d8071dd0 2024-09-11T12:49:19.816 INFO:teuthology.orchestra.run.smithi157.stdout:0 2024-09-11T12:49:19.817 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 msgr2=0x7fc5d814fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009850 tx=0x7fc5c80a57c0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 shutdown_connections 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5d815fce0 0x7fc5d8155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5d80721b0 0x7fc5d8155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 --2- 172.21.15.157:0/2854074973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5d8071dd0 0x7fc5d814fae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 >> 172.21.15.157:0/2854074973 conn(0x7fc5d80757e0 msgr2=0x7fc5d8152330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 shutdown_connections 2024-09-11T12:49:19.818 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.815+0000 7fc5ddc00640 1 -- 172.21.15.157:0/2854074973 wait complete. 2024-09-11T12:49:19.835 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 1d52a8ee-cb50-431f-8014-ce5223a7b1ae 1 2024-09-11T12:49:19.978 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 -- 172.21.15.157:0/3677122638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 msgr2=0x7f076015be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 --2- 172.21.15.157:0/3677122638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f076015be90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0760072e60 tx=0x7f075801cd80 comp rx=0 tx=0).stop 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 -- 172.21.15.157:0/3677122638 shutdown_connections 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 --2- 172.21.15.157:0/3677122638 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f076015be90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 --2- 172.21.15.157:0/3677122638 >> v2:172.21.15.78:3300/0 conn(0x7f0760071dd0 0x7f07600a9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 -- 172.21.15.157:0/3677122638 >> 172.21.15.157:0/3677122638 conn(0x7f07600757e0 msgr2=0x7f0760072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 -- 172.21.15.157:0/3677122638 shutdown_connections 2024-09-11T12:49:19.979 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.975+0000 7f0766c00640 1 -- 172.21.15.157:0/3677122638 wait complete. 2024-09-11T12:49:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 Processor -- start 2024-09-11T12:49:19.980 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- start start 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07600721b0 0x7f0760152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f07601525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0760168fa0 con 0x7f07600721b0 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0760168e20 con 0x7f076015fce0 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0760169120 con 0x7f0760071dd0 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.981 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36030/0 (socket says 172.21.15.157:36030) 2024-09-11T12:49:19.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0764c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f07601525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 -- 172.21.15.157:0/794192218 learned_addr learned my addr 172.21.15.157:0/794192218 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:19.982 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0757400640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07600721b0 0x7f0760152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 -- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 msgr2=0x7f07601525d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f07601525d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 -- 172.21.15.157:0/794192218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07600721b0 msgr2=0x7f0760152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07600721b0 0x7f0760152090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 -- 172.21.15.157:0/794192218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07601f79e0 con 0x7f0760071dd0 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0764c00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f07601525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075fe00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f075000e940 tx=0x7f075000ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:19.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075000cd80 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0750005020 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0750005450 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07601f7ca0 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f07601f8260 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0750004b40 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f07500187e0 con 0x7f0760071dd0 2024-09-11T12:49:19.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.979+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07600a9f90 con 0x7f0760071dd0 2024-09-11T12:49:19.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:19.983+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0750026020 con 0x7f0760071dd0 2024-09-11T12:49:20.055 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.051+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1d52a8ee-cb50-431f-8014-ce5223a7b1ae", "id": 1} v 0) -- 0x7f07600b0750 con 0x7f0760071dd0 2024-09-11T12:49:20.058 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.055+0000 7f075cc00640 1 -- 172.21.15.157:0/794192218 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1d52a8ee-cb50-431f-8014-ce5223a7b1ae", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f07500055f0 con 0x7f0760071dd0 2024-09-11T12:49:20.059 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 msgr2=0x7f07601435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f075000e940 tx=0x7f075000ee00 comp rx=0 tx=0).stop 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 shutdown_connections 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f076015fce0 0x7f07601525d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07600721b0 0x7f0760152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.060 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 --2- 172.21.15.157:0/794192218 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0760071dd0 0x7f07601435d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 >> 172.21.15.157:0/794192218 conn(0x7f07600757e0 msgr2=0x7f076015d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 shutdown_connections 2024-09-11T12:49:20.061 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.059+0000 7f0766c00640 1 -- 172.21.15.157:0/794192218 wait complete. 2024-09-11T12:49:20.078 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new b2bd4467-e229-4a13-9273-59945dd19f90 2 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 -- 172.21.15.157:0/4017009973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b80aac50 msgr2=0x7fe6b815f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 --2- 172.21.15.157:0/4017009973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b80aac50 0x7fe6b815f880 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6b813f110 tx=0x7fe6a001c890 comp rx=0 tx=0).stop 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 -- 172.21.15.157:0/4017009973 shutdown_connections 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 --2- 172.21.15.157:0/4017009973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b80aac50 0x7fe6b815f880 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 --2- 172.21.15.157:0/4017009973 >> v2:172.21.15.78:3300/0 conn(0x7fe6b813e880 0x7fe6b80aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 -- 172.21.15.157:0/4017009973 >> 172.21.15.157:0/4017009973 conn(0x7fe6b8075610 msgr2=0x7fe6b8075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 -- 172.21.15.157:0/4017009973 shutdown_connections 2024-09-11T12:49:20.220 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.215+0000 7fe6be800640 1 -- 172.21.15.157:0/4017009973 wait complete. 2024-09-11T12:49:20.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 Processor -- start 2024-09-11T12:49:20.221 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 -- start start 2024-09-11T12:49:20.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6b80aac50 0x7fe6b8150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b815fdc0 0x7fe6b8150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b81689f0 con 0x7fe6b80aac50 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b813ee70 con 0x7fe6b813e880 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe6b8168b70 con 0x7fe6b815fdc0 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60912/0 (socket says 172.21.15.157:60912) 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 -- 172.21.15.157:0/3955799873 learned_addr learned my addr 172.21.15.157:0/3955799873 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6b7e00640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b815fdc0 0x7fe6b8150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 -- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b815fdc0 msgr2=0x7fe6b8150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b815fdc0 0x7fe6b8150860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.223 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 -- 172.21.15.157:0/3955799873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6b80aac50 msgr2=0x7fe6b8150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.224 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6b80aac50 0x7fe6b8150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.224 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 -- 172.21.15.157:0/3955799873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b8151600 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6afe00640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00b0040 tx=0x7fe6a00a56f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a00b2040 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6a00043a0 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b81518e0 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.219+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0005280 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.223+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b8151ea0 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.223+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe6a00046d0 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.223+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7fe6a00ad440 con 0x7fe6b813e880 2024-09-11T12:49:20.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.223+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b80d0890 con 0x7fe6b813e880 2024-09-11T12:49:20.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.223+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe6a00164b0 con 0x7fe6b813e880 2024-09-11T12:49:20.295 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.291+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b2bd4467-e229-4a13-9273-59945dd19f90", "id": 2} v 0) -- 0x7fe6b815efd0 con 0x7fe6b813e880 2024-09-11T12:49:20.299 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.295+0000 7fe6af400640 1 -- 172.21.15.157:0/3955799873 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b2bd4467-e229-4a13-9273-59945dd19f90", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7fe6a0003600 con 0x7fe6b813e880 2024-09-11T12:49:20.300 INFO:teuthology.orchestra.run.smithi157.stdout:2 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 msgr2=0x7fe6b81f6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe6a00b0040 tx=0x7fe6a00a56f0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 shutdown_connections 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6b815fdc0 0x7fe6b8150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6b813e880 0x7fe6b81f6170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 --2- 172.21.15.157:0/3955799873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6b80aac50 0x7fe6b8150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 >> 172.21.15.157:0/3955799873 conn(0x7fe6b8075610 msgr2=0x7fe6b813a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 shutdown_connections 2024-09-11T12:49:20.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.299+0000 7fe6be800640 1 -- 172.21.15.157:0/3955799873 wait complete. 2024-09-11T12:49:20.318 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 7331094e-bbbc-41ce-bbc9-cc6746ef8aca 3 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.455+0000 7fb44dc00640 1 -- 172.21.15.157:0/4212176558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44815ff90 msgr2=0x7fb4480d0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.455+0000 7fb44dc00640 1 --2- 172.21.15.157:0/4212176558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44815ff90 0x7fb4480d0ab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb440009c80 tx=0x7fb44001cda0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- 172.21.15.157:0/4212176558 shutdown_connections 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 --2- 172.21.15.157:0/4212176558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44815ff90 0x7fb4480d0ab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 --2- 172.21.15.157:0/4212176558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb44813e880 0x7fb4480aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- 172.21.15.157:0/4212176558 >> 172.21.15.157:0/4212176558 conn(0x7fb448075610 msgr2=0x7fb448075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- 172.21.15.157:0/4212176558 shutdown_connections 2024-09-11T12:49:20.460 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- 172.21.15.157:0/4212176558 wait complete. 2024-09-11T12:49:20.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 Processor -- start 2024-09-11T12:49:20.461 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- start start 2024-09-11T12:49:20.462 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 0x7fb448148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb44815ff90 0x7fb44815d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb448168b70 con 0x7fb44815ff90 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb4481689f0 con 0x7fb44813e880 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb44dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb448168cf0 con 0x7fb4480aac50 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb43e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 0x7fb448148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36072/0 (socket says 172.21.15.157:36072) 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 -- 172.21.15.157:0/344179168 learned_addr learned my addr 172.21.15.157:0/344179168 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:20.463 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb447400640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb44815ff90 0x7fb44815d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 -- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 msgr2=0x7fb448148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 0x7fb448148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 -- 172.21.15.157:0/344179168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb44815ff90 msgr2=0x7fb44815d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb44815ff90 0x7fb44815d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 -- 172.21.15.157:0/344179168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4480cf150 con 0x7fb4480aac50 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb43e000640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 0x7fb448148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.459+0000 7fb446a00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9f0 tx=0x7fb43800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb43800cd80 con 0x7fb4480aac50 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb438010430 con 0x7fb4480aac50 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb438010860 con 0x7fb4480aac50 2024-09-11T12:49:20.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4480cf430 con 0x7fb4480aac50 2024-09-11T12:49:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb44814a5e0 con 0x7fb4480aac50 2024-09-11T12:49:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb438010a00 con 0x7fb4480aac50 2024-09-11T12:49:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fb438010cf0 con 0x7fb4480aac50 2024-09-11T12:49:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4480aa710 con 0x7fb4480aac50 2024-09-11T12:49:20.467 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.463+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb438018420 con 0x7fb4480aac50 2024-09-11T12:49:20.535 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.531+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7331094e-bbbc-41ce-bbc9-cc6746ef8aca", "id": 3} v 0) -- 0x7fb4480d0890 con 0x7fb4480aac50 2024-09-11T12:49:20.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb43f400640 1 -- 172.21.15.157:0/344179168 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7331094e-bbbc-41ce-bbc9-cc6746ef8aca", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fb438018680 con 0x7fb4480aac50 2024-09-11T12:49:20.541 INFO:teuthology.orchestra.run.smithi157.stdout:3 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 msgr2=0x7fb4480ce040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb43800e9f0 tx=0x7fb43800eeb0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 shutdown_connections 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb44815ff90 0x7fb44815d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb44813e880 0x7fb448148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 --2- 172.21.15.157:0/344179168 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb4480aac50 0x7fb4480ce040 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 >> 172.21.15.157:0/344179168 conn(0x7fb448075610 msgr2=0x7fb44813c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 shutdown_connections 2024-09-11T12:49:20.543 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.539+0000 7fb44dc00640 1 -- 172.21.15.157:0/344179168 wait complete. 2024-09-11T12:49:20.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new d1c94064-370e-4ace-b2d9-e2b5692e0cae 4 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- 172.21.15.157:0/1329194728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee406ce20 msgr2=0x7faee4073b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 --2- 172.21.15.157:0/1329194728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee406ce20 0x7faee4073b20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7faee41581d0 tx=0x7faed801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- 172.21.15.157:0/1329194728 shutdown_connections 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 --2- 172.21.15.157:0/1329194728 >> v2:172.21.15.78:3300/0 conn(0x7faee40cda60 0x7faee415e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 --2- 172.21.15.157:0/1329194728 >> v2:172.21.15.157:3300/0 conn(0x7faee40ce110 0x7faee40ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 --2- 172.21.15.157:0/1329194728 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee406ce20 0x7faee4073b20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- 172.21.15.157:0/1329194728 >> 172.21.15.157:0/1329194728 conn(0x7faee40ccd10 msgr2=0x7faee40cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.734 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- 172.21.15.157:0/1329194728 shutdown_connections 2024-09-11T12:49:20.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- 172.21.15.157:0/1329194728 wait complete. 2024-09-11T12:49:20.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 Processor -- start 2024-09-11T12:49:20.735 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.731+0000 7faeea400640 1 -- start start 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faee406ce20 0x7faee406fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faee415ab90 con 0x7faee406ce20 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faee415aa10 con 0x7faee40cda60 2024-09-11T12:49:20.737 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faee415ad10 con 0x7faee40ce110 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36080/0 (socket says 172.21.15.157:36080) 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60944/0 (socket says 172.21.15.157:60944) 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee9a00640 1 -- 172.21.15.157:0/1225806397 learned_addr learned my addr 172.21.15.157:0/1225806397 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee9000640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faee406ce20 0x7faee406fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 -- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 msgr2=0x7faee41fa800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 -- 172.21.15.157:0/1225806397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faee406ce20 msgr2=0x7faee406fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faee406ce20 0x7faee406fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 -- 172.21.15.157:0/1225806397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faee414b560 con 0x7faee40cda60 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee9a00640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:20.738 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee3e00640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faed000d690 tx=0x7faed000db50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:20.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0017070 con 0x7faee40cda60 2024-09-11T12:49:20.739 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faed0005310 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faee414b060 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faed0009820 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faee41fb370 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faed00099c0 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7faed0004cc0 con 0x7faee40cda60 2024-09-11T12:49:20.740 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.735+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faee415e470 con 0x7faee40cda60 2024-09-11T12:49:20.741 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.739+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7faed0009cb0 con 0x7faee40cda60 2024-09-11T12:49:20.815 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.811+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1c94064-370e-4ace-b2d9-e2b5692e0cae", "id": 4} v 0) -- 0x7faee414fdf0 con 0x7faee40cda60 2024-09-11T12:49:20.821 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faee0c00640 1 -- 172.21.15.157:0/1225806397 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1c94064-370e-4ace-b2d9-e2b5692e0cae", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7faed00047b0 con 0x7faee40cda60 2024-09-11T12:49:20.821 INFO:teuthology.orchestra.run.smithi157.stdout:4 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 msgr2=0x7faee414ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faed000d690 tx=0x7faed000db50 comp rx=0 tx=0).stop 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 shutdown_connections 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faee40ce110 0x7faee41fa800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faee40cda60 0x7faee414ab20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 --2- 172.21.15.157:0/1225806397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faee406ce20 0x7faee406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 >> 172.21.15.157:0/1225806397 conn(0x7faee40ccd10 msgr2=0x7faee41597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 shutdown_connections 2024-09-11T12:49:20.823 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.819+0000 7faeea400640 1 -- 172.21.15.157:0/1225806397 wait complete. 2024-09-11T12:49:20.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 792b544e-17db-4154-8674-225c7fd0b12f 5 2024-09-11T12:49:20.967 INFO:tasks.ceph.mgr.x.smithi157.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T12:49:20.968 INFO:tasks.ceph.mgr.x.smithi157.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T12:49:20.968 INFO:tasks.ceph.mgr.x.smithi157.stderr: from numpy import show_config as show_numpy_config 2024-09-11T12:49:20.983 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.979+0000 7fd087600640 1 -- 172.21.15.157:0/4235563794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 msgr2=0x7fd064003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.979+0000 7fd087600640 1 --2- 172.21.15.157:0/4235563794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd064003e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd07000c310 tx=0x7fd07001cef0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.979+0000 7fd087600640 1 -- 172.21.15.157:0/4235563794 shutdown_connections 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.979+0000 7fd087600640 1 --2- 172.21.15.157:0/4235563794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd064003e80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.979+0000 7fd087600640 1 -- 172.21.15.157:0/4235563794 >> 172.21.15.157:0/4235563794 conn(0x7fd080075610 msgr2=0x7fd080075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- 172.21.15.157:0/4235563794 shutdown_connections 2024-09-11T12:49:20.984 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- 172.21.15.157:0/4235563794 wait complete. 2024-09-11T12:49:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 Processor -- start 2024-09-11T12:49:20.985 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- start start 2024-09-11T12:49:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd080149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 0x7fd0801605b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:20.987 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0800708c0 con 0x7fd064003a60 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd080070740 con 0x7fd08014ea30 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd08015c910 con 0x7fd08014a620 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd085600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 0x7fd0801605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36090/0 (socket says 172.21.15.157:36090) 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 -- 172.21.15.157:0/4243795732 learned_addr learned my addr 172.21.15.157:0/4243795732 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd085600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 0x7fd0801605b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:60974/0 (socket says 172.21.15.157:60974) 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd084c00640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd080149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 -- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 msgr2=0x7fd0801605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 0x7fd0801605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 -- 172.21.15.157:0/4243795732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 msgr2=0x7fd080149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:20.988 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd080149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 -- 172.21.15.157:0/4243795732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0801f8170 con 0x7fd08014a620 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07fe00640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd06c003e70 tx=0x7fd06c001680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c015070 con 0x7fd08014a620 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd06c011840 con 0x7fd08014a620 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0801f8450 con 0x7fd08014a620 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.983+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd06c011c70 con 0x7fd08014a620 2024-09-11T12:49:20.989 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.987+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd08014e230 con 0x7fd08014a620 2024-09-11T12:49:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.987+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd06c018070 con 0x7fd08014a620 2024-09-11T12:49:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.987+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fd06c010cd0 con 0x7fd08014a620 2024-09-11T12:49:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.987+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0800aa5a0 con 0x7fd08014a620 2024-09-11T12:49:20.990 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:20.987+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd06c0103c0 con 0x7fd08014a620 2024-09-11T12:49:21.059 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.055+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "792b544e-17db-4154-8674-225c7fd0b12f", "id": 5} v 0) -- 0x7fd08015ffc0 con 0x7fd08014a620 2024-09-11T12:49:21.065 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd07cc00640 1 -- 172.21.15.157:0/4243795732 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "792b544e-17db-4154-8674-225c7fd0b12f", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fd06c0105a0 con 0x7fd08014a620 2024-09-11T12:49:21.065 INFO:teuthology.orchestra.run.smithi157.stdout:5 2024-09-11T12:49:21.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 msgr2=0x7fd0801f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd06c003e70 tx=0x7fd06c001680 comp rx=0 tx=0).stop 2024-09-11T12:49:21.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 shutdown_connections 2024-09-11T12:49:21.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd08014ea30 0x7fd0801605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.066 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd08014a620 0x7fd0801f78b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 --2- 172.21.15.157:0/4243795732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd064003a60 0x7fd080149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 >> 172.21.15.157:0/4243795732 conn(0x7fd080075610 msgr2=0x7fd08015f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 shutdown_connections 2024-09-11T12:49:21.067 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.063+0000 7fd087600640 1 -- 172.21.15.157:0/4243795732 wait complete. 2024-09-11T12:49:21.084 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new e2bb1238-c03a-4869-b5f7-767ae0794da7 6 2024-09-11T12:49:21.225 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- 172.21.15.157:0/4082120547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68075610 msgr2=0x7f3c6815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 --2- 172.21.15.157:0/4082120547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68075610 0x7f3c6815cab0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5801c850 comp rx=0 tx=0).stop 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- 172.21.15.157:0/4082120547 shutdown_connections 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 --2- 172.21.15.157:0/4082120547 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c680733e0 0x7f3c6815ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 --2- 172.21.15.157:0/4082120547 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68075610 0x7f3c6815cab0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- 172.21.15.157:0/4082120547 >> 172.21.15.157:0/4082120547 conn(0x7f3c680b2380 msgr2=0x7f3c680b2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- 172.21.15.157:0/4082120547 shutdown_connections 2024-09-11T12:49:21.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- 172.21.15.157:0/4082120547 wait complete. 2024-09-11T12:49:21.227 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 Processor -- start 2024-09-11T12:49:21.227 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.223+0000 7f3c70800640 1 -- start start 2024-09-11T12:49:21.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c680733e0 0x7f3c68153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68168e30 con 0x7f3c680733e0 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c68168cb0 con 0x7f3c68073000 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c68168fb0 con 0x7f3c68075610 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.157:36096/0 (socket says 172.21.15.157:36096) 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6e800640 1 -- 172.21.15.157:0/1762698486 learned_addr learned my addr 172.21.15.157:0/1762698486 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:32768/0 (socket says 172.21.15.157:32768) 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6d400640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c680733e0 0x7f3c68153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 -- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 msgr2=0x7f3c6813e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 -- 172.21.15.157:0/1762698486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c680733e0 msgr2=0x7f3c68153d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c680733e0 0x7f3c68153d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.229 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 -- 172.21.15.157:0/1762698486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c68154250 con 0x7f3c68073000 2024-09-11T12:49:21.230 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6e800640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c6de00640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c390 tx=0x7f3c5c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c017070 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5c0053c0 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c003840 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c681544d0 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c68154a90 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3c5c01a070 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f3c5c011c20 con 0x7f3c68073000 2024-09-11T12:49:21.231 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.227+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6815ff80 con 0x7f3c68073000 2024-09-11T12:49:21.232 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.231+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3c5c021020 con 0x7f3c68073000 2024-09-11T12:49:21.301 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.299+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e2bb1238-c03a-4869-b5f7-767ae0794da7", "id": 6} v 0) -- 0x7f3c680afcf0 con 0x7f3c68073000 2024-09-11T12:49:21.307 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.303+0000 7f3c62000640 1 -- 172.21.15.157:0/1762698486 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e2bb1238-c03a-4869-b5f7-767ae0794da7", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3c68154250 con 0x7f3c68073000 2024-09-11T12:49:21.307 INFO:teuthology.orchestra.run.smithi157.stdout:6 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 msgr2=0x7f3c680b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00c390 tx=0x7f3c5c00c850 comp rx=0 tx=0).stop 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 shutdown_connections 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c68075610 0x7f3c6813e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c680733e0 0x7f3c68153d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.308 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 --2- 172.21.15.157:0/1762698486 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c68073000 0x7f3c680b0ab0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 >> 172.21.15.157:0/1762698486 conn(0x7f3c680b2380 msgr2=0x7f3c68155110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 shutdown_connections 2024-09-11T12:49:21.309 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.307+0000 7f3c70800640 1 -- 172.21.15.157:0/1762698486 wait complete. 2024-09-11T12:49:21.326 DEBUG:teuthology.orchestra.run.smithi157:> sudo ceph --cluster ceph osd new 07f8cdba-fb9a-4130-8cec-61c127a027d8 7 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- 172.21.15.157:0/3417097696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35fc003c40 msgr2=0x7f35fc004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 --2- 172.21.15.157:0/3417097696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35fc003c40 0x7f35fc004040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3608006f90 tx=0x7f360801cf20 comp rx=0 tx=0).stop 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- 172.21.15.157:0/3417097696 shutdown_connections 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 --2- 172.21.15.157:0/3417097696 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35fc003c40 0x7f35fc004040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- 172.21.15.157:0/3417097696 >> 172.21.15.157:0/3417097696 conn(0x7f3618075610 msgr2=0x7f3618075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- 172.21.15.157:0/3417097696 shutdown_connections 2024-09-11T12:49:21.469 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- 172.21.15.157:0/3417097696 wait complete. 2024-09-11T12:49:21.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 Processor -- start 2024-09-11T12:49:21.471 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.467+0000 7f361d800640 1 -- start start 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 0x7f3618155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 0x7f361806c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f361815fa00 con 0x7f35fc003c40 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f361815f880 con 0x7f361806d900 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f361815fb80 con 0x7f361815b070 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 0x7f361806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.157:32784/0 (socket says 172.21.15.157:32784) 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360fe00640 1 -- 172.21.15.157:0/528983555 learned_addr learned my addr 172.21.15.157:0/528983555 (peer_addr_for_me v2:172.21.15.157:0/0) 2024-09-11T12:49:21.475 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3616a00640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 0x7f3618155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 -- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 msgr2=0x7f361815ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 -- 172.21.15.157:0/528983555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 msgr2=0x7f3618155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 0x7f3618155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 -- 172.21.15.157:0/528983555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3618154ec0 con 0x7f361815b070 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360fe00640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3616a00640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 0x7f3618155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f3617400640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 0x7f361806c650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f36180abbe0 tx=0x7f3610011f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361000fcc0 con 0x7f361815b070 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f361000fe60 con 0x7f361815b070 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36181551a0 con 0x7f361815b070 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3610004d70 con 0x7f361815b070 2024-09-11T12:49:21.476 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3618155760 con 0x7f361815b070 2024-09-11T12:49:21.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3610003980 con 0x7f361815b070 2024-09-11T12:49:21.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f3610003c70 con 0x7f361815b070 2024-09-11T12:49:21.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.471+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36180aa760 con 0x7f361815b070 2024-09-11T12:49:21.477 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.475+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f3610004030 con 0x7f361815b070 2024-09-11T12:49:21.505 INFO:tasks.ceph.mgr.y.smithi078.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T12:49:21.505 INFO:tasks.ceph.mgr.y.smithi078.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T12:49:21.505 INFO:tasks.ceph.mgr.y.smithi078.stderr: from numpy import show_config as show_numpy_config 2024-09-11T12:49:21.547 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.543+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd new", "uuid": "07f8cdba-fb9a-4130-8cec-61c127a027d8", "id": 7} v 0) -- 0x7f36180d0890 con 0x7f361815b070 2024-09-11T12:49:21.553 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f360ea00640 1 -- 172.21.15.157:0/528983555 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "07f8cdba-fb9a-4130-8cec-61c127a027d8", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f3610004210 con 0x7f361815b070 2024-09-11T12:49:21.553 INFO:teuthology.orchestra.run.smithi157.stdout:7 2024-09-11T12:49:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 msgr2=0x7f361806c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 0x7f361806c650 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f36180abbe0 tx=0x7f3610011f10 comp rx=0 tx=0).stop 2024-09-11T12:49:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 shutdown_connections 2024-09-11T12:49:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f361815b070 0x7f361806c650 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.554 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f361806d900 0x7f361815ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 --2- 172.21.15.157:0/528983555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35fc003c40 0x7f3618155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 >> 172.21.15.157:0/528983555 conn(0x7f3618075610 msgr2=0x7f361813f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 shutdown_connections 2024-09-11T12:49:21.555 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:49:21.551+0000 7f361d800640 1 -- 172.21.15.157:0/528983555 wait complete. 2024-09-11T12:49:21.572 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T12:49:21.572 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T12:49:21.575 INFO:tasks.ceph.osd.0:Started 2024-09-11T12:49:21.575 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T12:49:21.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T12:49:21.578 INFO:tasks.ceph.osd.1:Started 2024-09-11T12:49:21.578 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T12:49:21.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T12:49:21.581 INFO:tasks.ceph.osd.2:Started 2024-09-11T12:49:21.582 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-11T12:49:21.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-11T12:49:21.586 INFO:tasks.ceph.osd.3:Started 2024-09-11T12:49:21.586 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-11T12:49:21.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-11T12:49:21.589 INFO:tasks.ceph.osd.4:Started 2024-09-11T12:49:21.589 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-11T12:49:21.589 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-11T12:49:21.592 INFO:tasks.ceph.osd.5:Started 2024-09-11T12:49:21.592 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-11T12:49:21.593 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-11T12:49:21.595 INFO:tasks.ceph.osd.6:Started 2024-09-11T12:49:21.596 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-11T12:49:21.596 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-11T12:49:21.600 INFO:tasks.ceph.osd.7:Started 2024-09-11T12:49:21.600 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:21.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- 172.21.15.78:0/910417133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97780b4520 msgr2=0x7f9778154710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 --2- 172.21.15.78:0/910417133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97780b4520 0x7f9778154710 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9764009840 tx=0x7f976401c820 comp rx=0 tx=0).stop 2024-09-11T12:49:21.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- 172.21.15.78:0/910417133 shutdown_connections 2024-09-11T12:49:21.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 --2- 172.21.15.78:0/910417133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97780b4520 0x7f9778154710 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 --2- 172.21.15.78:0/910417133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f977806fd00 0x7f977815ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- 172.21.15.78:0/910417133 >> 172.21.15.78:0/910417133 conn(0x7f977806c650 msgr2=0x7f97780d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- 172.21.15.78:0/910417133 shutdown_connections 2024-09-11T12:49:21.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- 172.21.15.78:0/910417133 wait complete. 2024-09-11T12:49:21.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 Processor -- start 2024-09-11T12:49:21.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.767+0000 7f977f400640 1 -- start start 2024-09-11T12:49:21.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 0x7f97780b29a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 0x7f977814f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 0x7f9778151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:21.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f977816b260 con 0x7f97780b4520 2024-09-11T12:49:21.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f977816b0e0 con 0x7f977806fd00 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f977816b3e0 con 0x7f97780700e0 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 0x7f97780b29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 0x7f977814f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 0x7f97780b29a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33500/0 (socket says 172.21.15.78:33500) 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977ca00640 1 -- 172.21.15.78:0/1815756384 learned_addr learned my addr 172.21.15.78:0/1815756384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977d400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 0x7f9778151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 -- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 msgr2=0x7f97780b29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 0x7f97780b29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 -- 172.21.15.78:0/1815756384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 msgr2=0x7f9778151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 0x7f9778151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 -- 172.21.15.78:0/1815756384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97780b7ed0 con 0x7f97780700e0 2024-09-11T12:49:21.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977d400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 0x7f9778151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9777e00640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 0x7f977814f670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97640ac000 tx=0x7f9764005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97640b2040 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9764004330 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977814eb90 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764003860 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f97780b2ee0 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f97640bd060 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9764003d90 con 0x7f97780700e0 2024-09-11T12:49:21.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.771+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97780b4e50 con 0x7f97780700e0 2024-09-11T12:49:21.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.775+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9764016410 con 0x7f97780700e0 2024-09-11T12:49:21.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.831+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9778069070 con 0x7f97780700e0 2024-09-11T12:49:21.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f9774c00640 1 -- 172.21.15.78:0/1815756384 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f97640c4090 con 0x7f97780700e0 2024-09-11T12:49:21.838 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:21.838 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":10,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:21.548249+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 msgr2=0x7f977814f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 0x7f977814f670 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f97640ac000 tx=0x7f9764005d20 comp rx=0 tx=0).stop 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 shutdown_connections 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97780b4520 0x7f9778151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97780700e0 0x7f977814f670 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 --2- 172.21.15.78:0/1815756384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f977806fd00 0x7f97780b29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:21.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 >> 172.21.15.78:0/1815756384 conn(0x7f977806c650 msgr2=0x7f977815cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 shutdown_connections 2024-09-11T12:49:21.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:21.835+0000 7f977f400640 1 -- 172.21.15.78:0/1815756384 wait complete. 2024-09-11T12:49:21.857 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T12:49:21.857 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T12:49:22.664 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:22.799 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- 172.21.15.78:0/4233770250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b2d10 msgr2=0x7f0a540b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 --2- 172.21.15.78:0/4233770250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b2d10 0x7f0a540b1880 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c009c80 tx=0x7f0a4c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- 172.21.15.78:0/4233770250 shutdown_connections 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 --2- 172.21.15.78:0/4233770250 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b2d10 0x7f0a540b1880 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 --2- 172.21.15.78:0/4233770250 >> v2:172.21.15.157:3300/0 conn(0x7f0a540b3d50 0x7f0a540b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- 172.21.15.78:0/4233770250 >> 172.21.15.78:0/4233770250 conn(0x7f0a540d1a60 msgr2=0x7f0a540d1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- 172.21.15.78:0/4233770250 shutdown_connections 2024-09-11T12:49:22.800 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- 172.21.15.78:0/4233770250 wait complete. 2024-09-11T12:49:22.801 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 Processor -- start 2024-09-11T12:49:22.801 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.795+0000 7f0a5a800640 1 -- start start 2024-09-11T12:49:22.802 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 0x7f0a540b49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b3d50 0x7f0a54070df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54168ea0 con 0x7f0a5415ba20 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a54168d20 con 0x7f0a540b2d10 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a54169020 con 0x7f0a540b3d50 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 0x7f0a540b49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a52a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b3d50 0x7f0a54070df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 0x7f0a540b49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33514/0 (socket says 172.21.15.78:33514) 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36248/0 (socket says 172.21.15.78:36248) 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53400640 1 -- 172.21.15.78:0/376243546 learned_addr learned my addr 172.21.15.78:0/376243546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 -- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b3d50 msgr2=0x7f0a54070df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:22.803 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b3d50 0x7f0a54070df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.804 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 -- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 msgr2=0x7f0a540b49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:22.804 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 0x7f0a540b49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.804 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 -- 172.21.15.78:0/376243546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a540ae990 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a53e00640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a5415f5f0 tx=0x7f0a4c009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0bb070 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a4c004c50 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a540b5240 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0b2040 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a540b5800 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0a4c0be070 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f0a4c0047b0 con 0x7f0a5415ba20 2024-09-11T12:49:22.805 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.799+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a540d15f0 con 0x7f0a5415ba20 2024-09-11T12:49:22.806 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.803+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0a4c016990 con 0x7f0a5415ba20 2024-09-11T12:49:22.866 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0a54159b60 con 0x7f0a5415ba20 2024-09-11T12:49:22.867 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a4b400640 1 -- 172.21.15.78:0/376243546 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f0a4c0c6020 con 0x7f0a5415ba20 2024-09-11T12:49:22.867 INFO:teuthology.misc.health.smithi078.stdout: 2024-09-11T12:49:22.867 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":10,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:21.548249+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:22.868 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 msgr2=0x7f0a540b5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:22.868 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a5415f5f0 tx=0x7f0a4c009330 comp rx=0 tx=0).stop 2024-09-11T12:49:22.868 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 shutdown_connections 2024-09-11T12:49:22.868 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a5415ba20 0x7f0a540b5b20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.869 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a540b3d50 0x7f0a54070df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.869 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 --2- 172.21.15.78:0/376243546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a540b2d10 0x7f0a540b49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:22.869 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 >> 172.21.15.78:0/376243546 conn(0x7f0a540d1a60 msgr2=0x7f0a540cceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:22.869 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 shutdown_connections 2024-09-11T12:49:22.869 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:22.863+0000 7f0a5a800640 1 -- 172.21.15.78:0/376243546 wait complete. 2024-09-11T12:49:22.885 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-11T12:49:23.050 INFO:tasks.ceph.osd.0.smithi078.stderr:2024-09-11T12:49:23.047+0000 7f17364ac7c0 -1 Falling back to public interface 2024-09-11T12:49:23.261 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-11T12:49:23.259+0000 7f1ef2afd7c0 -1 Falling back to public interface 2024-09-11T12:49:23.270 INFO:tasks.ceph.osd.3.smithi078.stderr:2024-09-11T12:49:23.267+0000 7fb43fee27c0 -1 Falling back to public interface 2024-09-11T12:49:23.313 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-11T12:49:23.311+0000 7fbffc02b7c0 -1 Falling back to public interface 2024-09-11T12:49:23.373 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-11T12:49:23.371+0000 7f75131887c0 -1 Falling back to public interface 2024-09-11T12:49:23.494 INFO:tasks.ceph.osd.2.smithi078.stderr:2024-09-11T12:49:23.491+0000 7f9bdaa487c0 -1 Falling back to public interface 2024-09-11T12:49:23.550 INFO:tasks.ceph.osd.1.smithi078.stderr:2024-09-11T12:49:23.547+0000 7f1eb936e7c0 -1 Falling back to public interface 2024-09-11T12:49:23.613 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-11T12:49:23.611+0000 7f7ef50517c0 -1 Falling back to public interface 2024-09-11T12:49:24.110 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:49:24.107+0000 7fa5b5e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T12:49:24.110 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:49:24.107+0000 7fa5b5e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T12:49:27.329 INFO:tasks.ceph.osd.0.smithi078.stderr:2024-09-11T12:49:27.323+0000 7f17364ac7c0 -1 osd.0 0 log_to_monitors true 2024-09-11T12:49:27.465 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-11T12:49:27.462+0000 7f1ef2afd7c0 -1 osd.5 0 log_to_monitors true 2024-09-11T12:49:27.534 INFO:tasks.ceph.osd.3.smithi078.stderr:2024-09-11T12:49:27.531+0000 7fb43fee27c0 -1 osd.3 0 log_to_monitors true 2024-09-11T12:49:27.554 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-11T12:49:27.550+0000 7fbffc02b7c0 -1 osd.4 0 log_to_monitors true 2024-09-11T12:49:27.629 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-11T12:49:27.626+0000 7f75131887c0 -1 osd.7 0 log_to_monitors true 2024-09-11T12:49:27.654 INFO:tasks.ceph.osd.2.smithi078.stderr:2024-09-11T12:49:27.651+0000 7f9bdaa487c0 -1 osd.2 0 log_to_monitors true 2024-09-11T12:49:27.826 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-11T12:49:27.822+0000 7f7ef50517c0 -1 osd.6 0 log_to_monitors true 2024-09-11T12:49:28.263 INFO:tasks.ceph.osd.1.smithi078.stderr:2024-09-11T12:49:28.259+0000 7f1eb936e7c0 -1 osd.1 0 log_to_monitors true 2024-09-11T12:49:29.483 INFO:tasks.ceph.osd.0.smithi078.stderr:2024-09-11T12:49:29.479+0000 7f1731400640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T12:49:29.677 INFO:tasks.ceph.osd.0.smithi078.stderr:2024-09-11T12:49:29.671+0000 7f1727400640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:29.692 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- 172.21.15.78:0/1392242086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62140b4220 msgr2=0x7f62140b46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 --2- 172.21.15.78:0/1392242086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62140b4220 0x7f62140b46b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62140742c0 tx=0x7f620c01cb80 comp rx=0 tx=0).stop 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- 172.21.15.78:0/1392242086 shutdown_connections 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 --2- 172.21.15.78:0/1392242086 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62140b4220 0x7f62140b46b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 --2- 172.21.15.78:0/1392242086 >> v2:172.21.15.157:3300/0 conn(0x7f621415aa80 0x7f621415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- 172.21.15.78:0/1392242086 >> 172.21.15.78:0/1392242086 conn(0x7f6214068a80 msgr2=0x7f621406d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:29.850 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- 172.21.15.78:0/1392242086 shutdown_connections 2024-09-11T12:49:29.851 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- 172.21.15.78:0/1392242086 wait complete. 2024-09-11T12:49:29.852 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 Processor -- start 2024-09-11T12:49:29.852 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.847+0000 7f621aa00640 1 -- start start 2024-09-11T12:49:29.854 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 0x7f62141fa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:29.854 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 0x7f62141fe3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:29.855 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621415aa80 0x7f62141fe930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:29.855 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f621406cc60 con 0x7f6214154b90 2024-09-11T12:49:29.855 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f621406cae0 con 0x7f621415aa80 2024-09-11T12:49:29.855 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f621406cde0 con 0x7f62140b4220 2024-09-11T12:49:29.855 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6219600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 0x7f62141fa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 0x7f62141fe3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6219600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 0x7f62141fa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:48192/0 (socket says 172.21.15.78:48192) 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6219600640 1 -- 172.21.15.78:0/3599779393 learned_addr learned my addr 172.21.15.78:0/3599779393 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621a000640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621415aa80 0x7f62141fe930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 -- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 msgr2=0x7f62141fa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 0x7f62141fa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:29.856 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 -- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621415aa80 msgr2=0x7f62141fe930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621415aa80 0x7f62141fe930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 -- 172.21.15.78:0/3599779393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6214155250 con 0x7f6214154b90 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6218c00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 0x7f62141fe3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f620000e6c0 tx=0x7f620000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6200004220 con 0x7f6214154b90 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f620000c8b0 con 0x7f6214154b90 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62141ff0c0 con 0x7f6214154b90 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62140b1cc0 con 0x7f6214154b90 2024-09-11T12:49:29.857 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.851+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620000cd00 con 0x7f6214154b90 2024-09-11T12:49:29.858 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.855+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f00052a0 con 0x7f6214154b90 2024-09-11T12:49:29.858 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.855+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 4) ==== 94876+0+0 (secure 0 0 0) 0x7f6200003600 con 0x7f6214154b90 2024-09-11T12:49:29.860 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.855+0000 7f6211600640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 0x7f61f4073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:29.860 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.855+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2231+0+0 (secure 0 0 0) 0x7f6200091320 con 0x7f6214154b90 2024-09-11T12:49:29.864 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.859+0000 7f6219600640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 0x7f61f4073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:29.864 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.859+0000 7f6219600640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 0x7f61f4073ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6208000f70 tx=0x7f6208009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:29.875 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:29.871+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f620005d910 con 0x7f6214154b90 2024-09-11T12:49:30.108 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.103+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f61f0005590 con 0x7f6214154b90 2024-09-11T12:49:30.109 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.103+0000 7f6211600640 1 -- 172.21.15.78:0/3599779393 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7058 (secure 0 0 0) 0x7f6200061e60 con 0x7f6214154b90 2024-09-11T12:49:30.109 INFO:teuthology.misc.health.smithi078.stdout: 2024-09-11T12:49:30.109 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":13,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:29.387904+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:30.111 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 msgr2=0x7f61f4073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 0x7f61f4073ab0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6208000f70 tx=0x7f6208009040 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 msgr2=0x7f62141fe3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 0x7f62141fe3f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f620000e6c0 tx=0x7f620000eb80 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 shutdown_connections 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61f40715f0 0x7f61f4073ab0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f621415aa80 0x7f62141fe930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6214154b90 0x7f62141fe3f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 --2- 172.21.15.78:0/3599779393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62140b4220 0x7f62141fa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 >> 172.21.15.78:0/3599779393 conn(0x7f6214068a80 msgr2=0x7f6214159df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 shutdown_connections 2024-09-11T12:49:30.112 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:30.107+0000 7f621aa00640 1 -- 172.21.15.78:0/3599779393 wait complete. 2024-09-11T12:49:30.133 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-11T12:49:31.071 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-11T12:49:31.066+0000 7f7ef0a00640 -1 osd.6 0 waiting for initial osdmap 2024-09-11T12:49:31.095 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-11T12:49:31.090+0000 7f1eee400640 -1 osd.5 0 waiting for initial osdmap 2024-09-11T12:49:31.099 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-11T12:49:31.094+0000 7f750ea00640 -1 osd.7 0 waiting for initial osdmap 2024-09-11T12:49:31.100 INFO:tasks.ceph.osd.6.smithi157.stderr:2024-09-11T12:49:31.098+0000 7f7ee6000640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.104 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-11T12:49:31.102+0000 7fbff6400640 -1 osd.4 0 waiting for initial osdmap 2024-09-11T12:49:31.129 INFO:tasks.ceph.osd.5.smithi157.stderr:2024-09-11T12:49:31.126+0000 7f1ee3a00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.137 INFO:tasks.ceph.osd.7.smithi157.stderr:2024-09-11T12:49:31.134+0000 7f7504000640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.157 INFO:tasks.ceph.osd.4.smithi157.stderr:2024-09-11T12:49:31.154+0000 7fbfec400640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.188 INFO:tasks.ceph.osd.3.smithi078.stderr:2024-09-11T12:49:31.183+0000 7fb43b800640 -1 osd.3 0 waiting for initial osdmap 2024-09-11T12:49:31.218 INFO:tasks.ceph.osd.2.smithi078.stderr:2024-09-11T12:49:31.215+0000 7f9bd5a00640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T12:49:31.242 INFO:tasks.ceph.osd.3.smithi078.stderr:2024-09-11T12:49:31.239+0000 7fb430e00640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.250 INFO:tasks.ceph.osd.2.smithi078.stderr:2024-09-11T12:49:31.247+0000 7f9bcba00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:31.276 INFO:tasks.ceph.osd.1.smithi078.stderr:2024-09-11T12:49:31.271+0000 7f1eb4c00640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T12:49:31.298 INFO:tasks.ceph.osd.1.smithi078.stderr:2024-09-11T12:49:31.295+0000 7f1eaa200640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T12:49:36.940 DEBUG:teuthology.orchestra.run.smithi078:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:37.078 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 -- 172.21.15.78:0/3995385604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 msgr2=0x7fa31406e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.078 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 --2- 172.21.15.78:0/3995385604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa31406e0f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009900 tx=0x7fa2fc01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.078 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 -- 172.21.15.78:0/3995385604 shutdown_connections 2024-09-11T12:49:37.078 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 --2- 172.21.15.78:0/3995385604 >> v2:172.21.15.157:3300/0 conn(0x7fa31406e630 0x7fa3140ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.078 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 --2- 172.21.15.78:0/3995385604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa31406e0f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.079 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 -- 172.21.15.78:0/3995385604 >> 172.21.15.78:0/3995385604 conn(0x7fa31406c8d0 msgr2=0x7fa31406cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:37.079 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 -- 172.21.15.78:0/3995385604 shutdown_connections 2024-09-11T12:49:37.079 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.070+0000 7fa319600640 1 -- 172.21.15.78:0/3995385604 wait complete. 2024-09-11T12:49:37.084 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 Processor -- start 2024-09-11T12:49:37.085 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- start start 2024-09-11T12:49:37.085 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa31406e630 0x7fa3140b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.085 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.085 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa31415aff0 0x7fa3141531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.085 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140cf4a0 con 0x7fa31415aff0 2024-09-11T12:49:37.086 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140cf320 con 0x7fa31406e630 2024-09-11T12:49:37.086 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa3140b27a0 con 0x7fa314072790 2024-09-11T12:49:37.086 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa313400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa31415aff0 0x7fa3141531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.086 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.086 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa312a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa31406e630 0x7fa3140b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.087 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:50304/0 (socket says 172.21.15.78:50304) 2024-09-11T12:49:37.087 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 -- 172.21.15.78:0/2871188794 learned_addr learned my addr 172.21.15.78:0/2871188794 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:37.087 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 -- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa31406e630 msgr2=0x7fa3140b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.087 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa31406e630 0x7fa3140b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.087 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 -- 172.21.15.78:0/2871188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa31415aff0 msgr2=0x7fa3141531d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.088 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa31415aff0 0x7fa3141531d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.088 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 -- 172.21.15.78:0/2871188794 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa31413d370 con 0x7fa314072790 2024-09-11T12:49:37.088 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30be00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009850 tx=0x7fa2fc0a5800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:37.088 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0b2040 con 0x7fa314072790 2024-09-11T12:49:37.088 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2fc004340 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3140aeb10 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0052d0 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.074+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3140ae7d0 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3140ce6d0 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 4) ==== 94876+0+0 (secure 0 0 0) 0x7fa2fc005470 con 0x7fa314072790 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa30aa00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 0x7fa2f0073ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.089 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa2fc12b770 con 0x7fa314072790 2024-09-11T12:49:37.090 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa312a00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 0x7fa2f0073ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.090 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.078+0000 7fa312a00640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 0x7fa2f0073ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa304007fd0 tx=0x7fa304009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:37.090 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.082+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa3140aeb10 con 0x7fa314072790 2024-09-11T12:49:37.143 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.138+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fa2fc0f3610 con 0x7fa314072790 2024-09-11T12:49:37.264 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.258+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3140d05a0 con 0x7fa314072790 2024-09-11T12:49:37.266 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa30aa00640 1 -- 172.21.15.78:0/2871188794 <== mon.2 v2:172.21.15.157:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7fa2fc0164d0 con 0x7fa314072790 2024-09-11T12:49:37.266 INFO:teuthology.misc.health.smithi078.stdout: 2024-09-11T12:49:37.266 INFO:teuthology.misc.health.smithi078.stdout:{"epoch":19,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:36.122853+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:37.268 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 msgr2=0x7fa2f0073ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.268 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 0x7fa2f0073ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa304007fd0 tx=0x7fa304009040 comp rx=0 tx=0).stop 2024-09-11T12:49:37.268 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 msgr2=0x7fa3140ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.268 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc009850 tx=0x7fa2fc0a5800 comp rx=0 tx=0).stop 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 shutdown_connections 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2f00715f0 0x7fa2f0073ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa31415aff0 0x7fa3141531d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa314072790 0x7fa3140ae290 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 --2- 172.21.15.78:0/2871188794 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa31406e630 0x7fa3140b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 >> 172.21.15.78:0/2871188794 conn(0x7fa31406c8d0 msgr2=0x7fa31415b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 shutdown_connections 2024-09-11T12:49:37.269 INFO:teuthology.misc.health.smithi078.stderr:2024-09-11T12:49:37.262+0000 7fa319600640 1 -- 172.21.15.78:0/2871188794 wait complete. 2024-09-11T12:49:37.286 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-11T12:49:37.286 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T12:49:37.286 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:49:37.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:49:37.290 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:49:37.290 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:49:37.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:49:37.293 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:49:37.293 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:49:37.293 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:49:37.297 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:49:37.297 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:49:37.297 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:49:37.301 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:49:37.301 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-11T12:49:37.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:37.383 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:49:37.384 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:49:37.398 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:49:37.398 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:49:37.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 -- 172.21.15.78:0/2510181047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c41439c0 msgr2=0x7f55c40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 --2- 172.21.15.78:0/2510181047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c41439c0 0x7f55c40ad870 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f55b4009bc0 tx=0x7f55b401cba0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 -- 172.21.15.78:0/2510181047 shutdown_connections 2024-09-11T12:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 --2- 172.21.15.78:0/2510181047 >> v2:172.21.15.157:3300/0 conn(0x7f55c4073b20 0x7f55c415ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 --2- 172.21.15.78:0/2510181047 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c41439c0 0x7f55c40ad870 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 -- 172.21.15.78:0/2510181047 >> 172.21.15.78:0/2510181047 conn(0x7f55c406e800 msgr2=0x7f55c4071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:37.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 -- 172.21.15.78:0/2510181047 shutdown_connections 2024-09-11T12:49:37.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.446+0000 7f55cae00640 1 -- 172.21.15.78:0/2510181047 wait complete. 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 Processor -- start 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- start start 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 0x7f55c40d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 0x7f55c413c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55c416ac90 con 0x7f55c41439c0 2024-09-11T12:49:37.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55c416ab10 con 0x7f55c415e8f0 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55c416ae10 con 0x7f55c4073b20 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49324/0 (socket says 172.21.15.78:49324) 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 -- 172.21.15.78:0/700868513 learned_addr learned my addr 172.21.15.78:0/700868513 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55ca400640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 0x7f55c413c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9a00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 0x7f55c40d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 -- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 msgr2=0x7f55c40d2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 0x7f55c40d2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 -- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 msgr2=0x7f55c413c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 0x7f55c413c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 -- 172.21.15.78:0/700868513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55c41fabe0 con 0x7f55c41439c0 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55ca400640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 0x7f55c413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:37.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9a00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 0x7f55c40d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c9000640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c400 tx=0x7f55b000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0011790 con 0x7f55c41439c0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55b0017070 con 0x7f55c41439c0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55c41faec0 con 0x7f55c41439c0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b0011cc0 con 0x7f55c41439c0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55c41fb480 con 0x7f55c41439c0 2024-09-11T12:49:37.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.450+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55c40d3150 con 0x7f55c41439c0 2024-09-11T12:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.454+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f55b001c460 con 0x7f55c41439c0 2024-09-11T12:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.454+0000 7f55c1600640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 0x7f55a4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.454+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f55b0091920 con 0x7f55c41439c0 2024-09-11T12:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.454+0000 7f55c9a00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 0x7f55a4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:37.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.454+0000 7f55c9a00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 0x7f55a4073b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55b40af000 tx=0x7f55b401b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:37.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.458+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f55b005da40 con 0x7f55c41439c0 2024-09-11T12:49:37.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.654+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f55c415bfc0 con 0x7f55c41439c0 2024-09-11T12:49:37.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.654+0000 7f55c1600640 1 -- 172.21.15.78:0/700868513 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f55b005d340 con 0x7f55c41439c0 2024-09-11T12:49:37.661 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:37.661 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":2,"btime":"2024-09-11T12:49:37:391827+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T12:49:37.661 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 2 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 msgr2=0x7f55a4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 0x7f55a4073b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f55b40af000 tx=0x7f55b401b3e0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 msgr2=0x7f55c413bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f55b000c400 tx=0x7f55b000c8c0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 shutdown_connections 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55a4071680 0x7f55a4073b40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55c415e8f0 0x7f55c413c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55c41439c0 0x7f55c413bea0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 --2- 172.21.15.78:0/700868513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55c4073b20 0x7f55c40d2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 >> 172.21.15.78:0/700868513 conn(0x7f55c406e800 msgr2=0x7f55c413d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 shutdown_connections 2024-09-11T12:49:37.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:37.658+0000 7f55cae00640 1 -- 172.21.15.78:0/700868513 wait complete. 2024-09-11T12:49:39.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 -- 172.21.15.78:0/2216866722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 msgr2=0x7f47a015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 --2- 172.21.15.78:0/2216866722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a015ca20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4788009840 tx=0x7f478801c760 comp rx=0 tx=0).stop 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 -- 172.21.15.78:0/2216866722 shutdown_connections 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 --2- 172.21.15.78:0/2216866722 >> v2:172.21.15.157:3300/0 conn(0x7f47a00b3070 0x7f47a00b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 --2- 172.21.15.78:0/2216866722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a015ca20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 -- 172.21.15.78:0/2216866722 >> 172.21.15.78:0/2216866722 conn(0x7f47a00b1c50 msgr2=0x7f47a0070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 -- 172.21.15.78:0/2216866722 shutdown_connections 2024-09-11T12:49:39.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.826+0000 7f47a6200640 1 -- 172.21.15.78:0/2216866722 wait complete. 2024-09-11T12:49:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 Processor -- start 2024-09-11T12:49:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- start start 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a01527f0 0x7f47a0151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a01faa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47a0168860 con 0x7f47a00b3070 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47a01686e0 con 0x7f47a01527f0 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47a01689e0 con 0x7f47a0152bd0 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49326/0 (socket says 172.21.15.78:49326) 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 -- 172.21.15.78:0/2639357294 learned_addr learned my addr 172.21.15.78:0/2639357294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f4796a00640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a01527f0 0x7f47a0151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479fe00640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a01faa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 -- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 msgr2=0x7f47a01faa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a01faa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 -- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a01527f0 msgr2=0x7f47a0151760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a01527f0 0x7f47a0151760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 -- 172.21.15.78:0/2639357294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a014fe30 con 0x7f47a00b3070 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f479f400640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f479000c490 tx=0x7f479000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790016020 con 0x7f47a00b3070 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4790012940 con 0x7f47a00b3070 2024-09-11T12:49:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790012d90 con 0x7f47a00b3070 2024-09-11T12:49:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a0070740 con 0x7f47a00b3070 2024-09-11T12:49:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.830+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a0070d00 con 0x7f47a00b3070 2024-09-11T12:49:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47a00b34e0 con 0x7f47a00b3070 2024-09-11T12:49:39.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4790019020 con 0x7f47a00b3070 2024-09-11T12:49:39.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f4797e00640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 0x7f477c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:39.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f4790091940 con 0x7f47a00b3070 2024-09-11T12:49:39.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f4796a00640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 0x7f477c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:39.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.834+0000 7f4796a00640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 0x7f477c073b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f47880af000 tx=0x7f478801afa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:39.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:39.838+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4790020020 con 0x7f47a00b3070 2024-09-11T12:49:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47a00695e0 con 0x7f47a00b3070 2024-09-11T12:49:40.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f4797e00640 1 -- 172.21.15.78:0/2639357294 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3912 (secure 0 0 0) 0x7f479005d560 con 0x7f47a00b3070 2024-09-11T12:49:40.043 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:40.044 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":3,"btime":"2024-09-11T12:49:38:393333+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4390,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-11T12:49:40.044 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 3 2024-09-11T12:49:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 msgr2=0x7f477c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 0x7f477c073b40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f47880af000 tx=0x7f478801afa0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 msgr2=0x7f47a00700b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f479000c490 tx=0x7f479000c950 comp rx=0 tx=0).stop 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 shutdown_connections 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f477c071680 0x7f477c073b40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47a0152bd0 0x7f47a01faa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47a01527f0 0x7f47a0151760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 --2- 172.21.15.78:0/2639357294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47a00b3070 0x7f47a00700b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.038+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 >> 172.21.15.78:0/2639357294 conn(0x7f47a00b1c50 msgr2=0x7f47a015da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.042+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 shutdown_connections 2024-09-11T12:49:40.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.042+0000 7f47a6200640 1 -- 172.21.15.78:0/2639357294 wait complete. 2024-09-11T12:49:40.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:49:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- 172.21.15.78:0/3811428658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 msgr2=0x7faff40b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- 172.21.15.78:0/3811428658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40b5220 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fafec009bc0 tx=0x7fafec01cb40 comp rx=0 tx=0).stop 2024-09-11T12:49:40.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- 172.21.15.78:0/3811428658 shutdown_connections 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- 172.21.15.78:0/3811428658 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40b5220 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- 172.21.15.78:0/3811428658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- 172.21.15.78:0/3811428658 >> 172.21.15.78:0/3811428658 conn(0x7faff406ca80 msgr2=0x7faff40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- 172.21.15.78:0/3811428658 shutdown_connections 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- 172.21.15.78:0/3811428658 wait complete. 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 Processor -- start 2024-09-11T12:49:40.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- start start 2024-09-11T12:49:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40aedd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faff4167080 con 0x7faff40707f0 2024-09-11T12:49:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faff4166f00 con 0x7faff40b4db0 2024-09-11T12:49:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faff4167200 con 0x7faff4073ed0 2024-09-11T12:49:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49330/0 (socket says 172.21.15.78:49330) 2024-09-11T12:49:40.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.202+0000 7faff1e00640 1 -- 172.21.15.78:0/1256627069 learned_addr learned my addr 172.21.15.78:0/1256627069 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1400640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 -- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 msgr2=0x7faff415c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff2800640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40aedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 -- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 msgr2=0x7faff40aedd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40aedd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 -- 172.21.15.78:0/1256627069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff40cdb30 con 0x7faff40707f0 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1400640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff2800640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff1e00640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e9f0 tx=0x7fafe400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:40.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe400cd80 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafe4010430 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff40cde10 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe4010860 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff40cf4a0 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff40ce050 con 0x7faff40707f0 2024-09-11T12:49:40.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fafe4010a00 con 0x7faff40707f0 2024-09-11T12:49:40.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 0x7fafd4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.206+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fafe4092630 con 0x7faff40707f0 2024-09-11T12:49:40.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.210+0000 7faff1400640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 0x7fafd4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.210+0000 7faff1400640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 0x7fafd4073b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001230 tx=0x7fafe0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:40.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.210+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fafe405e810 con 0x7faff40707f0 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7faff40cf7c0 con 0x7faff40707f0 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7fafea000640 1 -- 172.21.15.78:0/1256627069 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7fafe405e110 con 0x7faff40707f0 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:49:40.386 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 msgr2=0x7fafd4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 0x7fafd4073b40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fafe0001230 tx=0x7fafe0009040 comp rx=0 tx=0).stop 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 msgr2=0x7faff415d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fafe400e9f0 tx=0x7fafe400eeb0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 shutdown_connections 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fafd4071680 0x7fafd4073b40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff40b4db0 0x7faff40aedd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff4073ed0 0x7faff415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 --2- 172.21.15.78:0/1256627069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40707f0 0x7faff415d1a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 >> 172.21.15.78:0/1256627069 conn(0x7faff406ca80 msgr2=0x7faff40afd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:40.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 shutdown_connections 2024-09-11T12:49:40.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.382+0000 7faff9000640 1 -- 172.21.15.78:0/1256627069 wait complete. 2024-09-11T12:49:40.407 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:49:40.408 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 -- 172.21.15.78:0/1122445974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db40b5c40 msgr2=0x7f3db40cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1122445974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db40b5c40 0x7f3db40cf990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3db406e460 tx=0x7f3dac01cd80 comp rx=0 tx=0).stop 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 -- 172.21.15.78:0/1122445974 shutdown_connections 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1122445974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db40b5c40 0x7f3db40cf990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1122445974 >> v2:172.21.15.157:3300/0 conn(0x7f3db4152990 0x7f3db415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 -- 172.21.15.78:0/1122445974 >> 172.21.15.78:0/1122445974 conn(0x7f3db40b3050 msgr2=0x7f3db40b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 -- 172.21.15.78:0/1122445974 shutdown_connections 2024-09-11T12:49:40.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.546+0000 7f3dbac00640 1 -- 172.21.15.78:0/1122445974 wait complete. 2024-09-11T12:49:40.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 Processor -- start 2024-09-11T12:49:40.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- start start 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3db40b5c40 0x7f3db414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3db40cf520 0x7f3db41f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3db4168900 con 0x7f3db4152990 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3db40cdf80 con 0x7f3db40cf520 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3db4168a80 con 0x7f3db40b5c40 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49350/0 (socket says 172.21.15.78:49350) 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 -- 172.21.15.78:0/1114821780 learned_addr learned my addr 172.21.15.78:0/1114821780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db3400640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3db40cf520 0x7f3db41f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db3e00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3db40b5c40 0x7f3db414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 -- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3db40b5c40 msgr2=0x7f3db414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3db40b5c40 0x7f3db414b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 -- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3db40cf520 msgr2=0x7f3db41f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3db40cf520 0x7f3db41f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 -- 172.21.15.78:0/1114821780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db414ba10 con 0x7f3db4152990 2024-09-11T12:49:40.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3db8c00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dac098770 tx=0x7f3dac005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac0b2040 con 0x7f3db4152990 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dac0bb070 con 0x7f3db4152990 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dac004350 con 0x7f3db4152990 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db4142d10 con 0x7f3db4152990 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db4143d30 con 0x7f3db4152990 2024-09-11T12:49:40.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.550+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3db40cf990 con 0x7f3db4152990 2024-09-11T12:49:40.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.554+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f3dac0044f0 con 0x7f3db4152990 2024-09-11T12:49:40.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.554+0000 7f3dabe00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 0x7f3d94073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:40.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.554+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f3dac12c4f0 con 0x7f3db4152990 2024-09-11T12:49:40.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.554+0000 7f3db3e00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 0x7f3d94073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:40.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.554+0000 7f3db3e00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 0x7f3d94073b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3da40058d0 tx=0x7f3da4005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:40.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.558+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3dac016970 con 0x7f3db4152990 2024-09-11T12:49:40.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:40.722+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f3db415d930 con 0x7f3db4152990 2024-09-11T12:49:41.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:49:41.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:49:41.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.406+0000 7f3dabe00640 1 -- 172.21.15.78:0/1114821780 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f3dac0f86c0 con 0x7f3db4152990 2024-09-11T12:49:41.411 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:49:41.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 msgr2=0x7f3d94073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:41.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 0x7f3d94073b70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f3da40058d0 tx=0x7f3da4005860 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 msgr2=0x7f3db41437f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3dac098770 tx=0x7f3dac005ae0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 shutdown_connections 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d940716b0 0x7f3d94073b70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3db4152990 0x7f3db41437f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3db40cf520 0x7f3db41f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 --2- 172.21.15.78:0/1114821780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3db40b5c40 0x7f3db414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 >> 172.21.15.78:0/1114821780 conn(0x7f3db40b3050 msgr2=0x7f3db4159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 shutdown_connections 2024-09-11T12:49:41.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.410+0000 7f3dbac00640 1 -- 172.21.15.78:0/1114821780 wait complete. 2024-09-11T12:49:41.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- 172.21.15.78:0/272671521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da80cebc0 msgr2=0x7f7da80704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 --2- 172.21.15.78:0/272671521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da80cebc0 0x7f7da80704b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7d98009bc0 tx=0x7f7d9801cba0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- 172.21.15.78:0/272671521 shutdown_connections 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 --2- 172.21.15.78:0/272671521 >> v2:172.21.15.157:3300/0 conn(0x7f7da80cefa0 0x7f7da80737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 --2- 172.21.15.78:0/272671521 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da80cebc0 0x7f7da80704b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- 172.21.15.78:0/272671521 >> 172.21.15.78:0/272671521 conn(0x7f7da80b2ec0 msgr2=0x7f7da80b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- 172.21.15.78:0/272671521 shutdown_connections 2024-09-11T12:49:41.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- 172.21.15.78:0/272671521 wait complete. 2024-09-11T12:49:41.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 Processor -- start 2024-09-11T12:49:41.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.582+0000 7f7dad800640 1 -- start start 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da8073d00 0x7f7da8149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da80cefa0 0x7f7da8154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8168c90 con 0x7f7da80cebc0 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7da8168b10 con 0x7f7da80cefa0 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7da8168e10 con 0x7f7da8073d00 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49352/0 (socket says 172.21.15.78:49352) 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 -- 172.21.15.78:0/3232828256 learned_addr learned my addr 172.21.15.78:0/3232828256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:41.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6a00640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da8073d00 0x7f7da8149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da7400640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da80cefa0 0x7f7da8154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 -- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da8073d00 msgr2=0x7f7da8149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da8073d00 0x7f7da8149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 -- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da80cefa0 msgr2=0x7f7da8154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da80cefa0 0x7f7da8154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:41.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 -- 172.21.15.78:0/3232828256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da8150870 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7da6000640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400e770 tx=0x7f7d9400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400cc60 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d9400ce00 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da8150b50 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d94004280 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da8154770 con 0x7f7da80cebc0 2024-09-11T12:49:41.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da80737c0 con 0x7f7da80cebc0 2024-09-11T12:49:41.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.586+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7d94003600 con 0x7f7da80cebc0 2024-09-11T12:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.590+0000 7f7d9ea00640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 0x7f7d88073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.590+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f7d940923e0 con 0x7f7da80cebc0 2024-09-11T12:49:41.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.590+0000 7f7da6a00640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 0x7f7d88073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:41.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.590+0000 7f7da6a00640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 0x7f7d88073b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d980af000 tx=0x7f7d9801b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:41.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.594+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7d9405e420 con 0x7f7da80cebc0 2024-09-11T12:49:41.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:41.762+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7da8151fa0 con 0x7f7da80cebc0 2024-09-11T12:49:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.410+0000 7f7d9ea00640 1 -- 172.21.15.78:0/3232828256 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f7d9405dd20 con 0x7f7da80cebc0 2024-09-11T12:49:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 msgr2=0x7f7d88073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 0x7f7d88073b40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f7d980af000 tx=0x7f7d9801b3e0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 msgr2=0x7f7da814ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7d9400e770 tx=0x7f7d9400ec30 comp rx=0 tx=0).stop 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 shutdown_connections 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7d88071680 0x7f7d88073b40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7da80cefa0 0x7f7da8154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7da80cebc0 0x7f7da814ff00 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 --2- 172.21.15.78:0/3232828256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7da8073d00 0x7f7da8149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 >> 172.21.15.78:0/3232828256 conn(0x7f7da80b2ec0 msgr2=0x7f7da815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 shutdown_connections 2024-09-11T12:49:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.414+0000 7f7dad800640 1 -- 172.21.15.78:0/3232828256 wait complete. 2024-09-11T12:49:42.460 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- 172.21.15.78:0/3877949755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40cf6e0 msgr2=0x7f93b415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 --2- 172.21.15.78:0/3877949755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40cf6e0 0x7f93b415a700 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009c80 tx=0x7f93a401cd10 comp rx=0 tx=0).stop 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- 172.21.15.78:0/3877949755 shutdown_connections 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 --2- 172.21.15.78:0/3877949755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40cf6e0 0x7f93b415a700 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- 172.21.15.78:0/3877949755 >> 172.21.15.78:0/3877949755 conn(0x7f93b4071dd0 msgr2=0x7f93b40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- 172.21.15.78:0/3877949755 shutdown_connections 2024-09-11T12:49:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- 172.21.15.78:0/3877949755 wait complete. 2024-09-11T12:49:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 Processor -- start 2024-09-11T12:49:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.602+0000 7f93bb400640 1 -- start start 2024-09-11T12:49:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40b1f10 0x7f93b40b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40cf6e0 0x7f93b40cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40af2b0 con 0x7f93b40b1a80 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93b40af130 con 0x7f93b40b1f10 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93b4168ea0 con 0x7f93b40cf6e0 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49368/0 (socket says 172.21.15.78:49368) 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 -- 172.21.15.78:0/580406134 learned_addr learned my addr 172.21.15.78:0/580406134 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b3e00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40b1f10 0x7f93b40b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b9400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40cf6e0 0x7f93b40cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 -- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40cf6e0 msgr2=0x7f93b40cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40cf6e0 0x7f93b40cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 -- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40b1f10 msgr2=0x7f93b40b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40b1f10 0x7f93b40b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 -- 172.21.15.78:0/580406134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b41f55d0 con 0x7f93b40b1a80 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b8a00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93a401c780 tx=0x7f93a40092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a40b2040 con 0x7f93b40b1a80 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93a4004d40 con 0x7f93b40b1a80 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b41f5890 con 0x7f93b40b1a80 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4004280 con 0x7f93b40b1a80 2024-09-11T12:49:42.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b41f5e50 con 0x7f93b40b1a80 2024-09-11T12:49:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93b415a700 con 0x7f93b40b1a80 2024-09-11T12:49:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.606+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f93a40bd060 con 0x7f93b40b1a80 2024-09-11T12:49:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.610+0000 7f93b0c00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 0x7f9394073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.610+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f93a412bd30 con 0x7f93b40b1a80 2024-09-11T12:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.610+0000 7f93b3e00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 0x7f9394073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:42.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.610+0000 7f93b3e00640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 0x7f9394073b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93b4159750 tx=0x7f93a0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:42.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.614+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f93a4016990 con 0x7f93b40b1a80 2024-09-11T12:49:42.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:42.814+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f93b4157170 con 0x7f93b40b1a80 2024-09-11T12:49:43.437 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 2 and data pool 3 2024-09-11T12:49:43.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.430+0000 7f93b0c00640 1 -- 172.21.15.78:0/580406134 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f93a40f7b60 con 0x7f93b40b1a80 2024-09-11T12:49:43.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.438+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 msgr2=0x7f9394073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.438+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 0x7f9394073b40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f93b4159750 tx=0x7f93a0009040 comp rx=0 tx=0).stop 2024-09-11T12:49:43.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.438+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 msgr2=0x7f93b4137630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.438+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f93a401c780 tx=0x7f93a40092c0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 shutdown_connections 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9394071680 0x7f9394073b40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93b40cf6e0 0x7f93b40cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93b40b1f10 0x7f93b40b1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 --2- 172.21.15.78:0/580406134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93b40b1a80 0x7f93b4137630 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 >> 172.21.15.78:0/580406134 conn(0x7f93b4071dd0 msgr2=0x7f93b406e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:43.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 shutdown_connections 2024-09-11T12:49:43.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.442+0000 7f93bb400640 1 -- 172.21.15.78:0/580406134 wait complete. 2024-09-11T12:49:43.482 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:43.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 -- 172.21.15.78:0/3562025562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c004d50 msgr2=0x7f858c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 --2- 172.21.15.78:0/3562025562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f858c004d50 0x7f858c0051d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f85a0009ec0 tx=0x7f85a001d5d0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85a7200640 1 -- 172.21.15.78:0/3562025562 reap_dead start 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 -- 172.21.15.78:0/3562025562 shutdown_connections 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 -- 172.21.15.78:0/3562025562 >> 172.21.15.78:0/3562025562 conn(0x7f85a80b19a0 msgr2=0x7f85a8070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 -- 172.21.15.78:0/3562025562 shutdown_connections 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.618+0000 7f85aca00640 1 -- 172.21.15.78:0/3562025562 wait complete. 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 Processor -- start 2024-09-11T12:49:43.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- start start 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858c004d50 0x7f85a8149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 0x7f85a8157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85a815e0d0 con 0x7f85a8143b40 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85a815d090 con 0x7f858c004d50 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f85a815e250 con 0x7f85a81572a0 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49382/0 (socket says 172.21.15.78:49382) 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 -- 172.21.15.78:0/3283285243 learned_addr learned my addr 172.21.15.78:0/3283285243 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:43.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85a6800640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858c004d50 0x7f85a8149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85a7200640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 0x7f85a8157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 -- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 msgr2=0x7f85a8157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 0x7f85a8157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 -- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858c004d50 msgr2=0x7f85a8149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858c004d50 0x7f85a8149250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 -- 172.21.15.78:0/3283285243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a806d200 con 0x7f85a8143b40 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85a7200640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 0x7f85a8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859fe00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8590011a50 tx=0x7f8590011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859001f070 con 0x7f85a8143b40 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f859000fe80 con 0x7f85a8143b40 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a806d4e0 con 0x7f85a8143b40 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8590012690 con 0x7f85a8143b40 2024-09-11T12:49:43.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a815bad0 con 0x7f85a8143b40 2024-09-11T12:49:43.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.622+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a80b31f0 con 0x7f85a8143b40 2024-09-11T12:49:43.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.626+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f8590022070 con 0x7f85a8143b40 2024-09-11T12:49:43.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.626+0000 7f859ea00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 0x7f8584073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.626+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f8590092920 con 0x7f85a8143b40 2024-09-11T12:49:43.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.626+0000 7f85a6800640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 0x7f8584073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.626+0000 7f85a6800640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 0x7f8584073b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85a80b1640 tx=0x7f85980067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:43.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.630+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f859005e130 con 0x7f85a8143b40 2024-09-11T12:49:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.798+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85a8074e00 con 0x7f85a8143b40 2024-09-11T12:49:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.798+0000 7f859ea00640 1 -- 172.21.15.78:0/3283285243 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16678 (secure 0 0 0) 0x7f8590062680 con 0x7f85a8143b40 2024-09-11T12:49:43.804 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:43.804 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:42.819643+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 msgr2=0x7f8584073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 0x7f8584073b40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f85a80b1640 tx=0x7f85980067b0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 msgr2=0x7f85a815b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8590011a50 tx=0x7f8590011f10 comp rx=0 tx=0).stop 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 shutdown_connections 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8584071680 0x7f8584073b40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f85a81572a0 0x7f85a8157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85a8143b40 0x7f85a815b590 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 --2- 172.21.15.78:0/3283285243 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f858c004d50 0x7f85a8149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 >> 172.21.15.78:0/3283285243 conn(0x7f85a80b19a0 msgr2=0x7f85a815c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 shutdown_connections 2024-09-11T12:49:43.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.802+0000 7f85aca00640 1 -- 172.21.15.78:0/3283285243 wait complete. 2024-09-11T12:49:43.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 -- 172.21.15.78:0/643505236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c0b14b0 msgr2=0x7f049c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 --2- 172.21.15.78:0/643505236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c0b14b0 0x7f049c15c100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f048c009900 tx=0x7f048c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 -- 172.21.15.78:0/643505236 shutdown_connections 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 --2- 172.21.15.78:0/643505236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c06fdf0 0x7f049c0a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 --2- 172.21.15.78:0/643505236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c0b14b0 0x7f049c15c100 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 -- 172.21.15.78:0/643505236 >> 172.21.15.78:0/643505236 conn(0x7f049c074b90 msgr2=0x7f049c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 -- 172.21.15.78:0/643505236 shutdown_connections 2024-09-11T12:49:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.962+0000 7f04a2400640 1 -- 172.21.15.78:0/643505236 wait complete. 2024-09-11T12:49:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 Processor -- start 2024-09-11T12:49:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- start start 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c0b14b0 0x7f049c1f89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049c0d0bf0 0x7f049c148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f049c169330 con 0x7f049c06fdf0 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f049c1691b0 con 0x7f049c0d0bf0 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f049c1694b0 con 0x7f049c0b14b0 2024-09-11T12:49:43.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44354/0 (socket says 172.21.15.78:44354) 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 -- 172.21.15.78:0/1344147973 learned_addr learned my addr 172.21.15.78:0/1344147973 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049aa00640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c0b14b0 0x7f049c1f89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049be00640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049c0d0bf0 0x7f049c148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 -- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c0b14b0 msgr2=0x7f049c1f89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c0b14b0 0x7f049c1f89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 -- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049c0d0bf0 msgr2=0x7f049c148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049c0d0bf0 0x7f049c148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 -- 172.21.15.78:0/1344147973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f049c071710 con 0x7f049c06fdf0 2024-09-11T12:49:43.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f049b400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f048c0b0040 tx=0x7f048c0a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:43.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c0b2040 con 0x7f049c06fdf0 2024-09-11T12:49:43.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f048c004230 con 0x7f049c06fdf0 2024-09-11T12:49:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f048c003600 con 0x7f049c06fdf0 2024-09-11T12:49:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f049c06f110 con 0x7f049c06fdf0 2024-09-11T12:49:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f049c06c6f0 con 0x7f049c06fdf0 2024-09-11T12:49:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.966+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f049c1523c0 con 0x7f049c06fdf0 2024-09-11T12:49:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.970+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f048c0bf020 con 0x7f049c06fdf0 2024-09-11T12:49:43.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.970+0000 7f0493400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 0x7f047c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:43.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.970+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f048c12c170 con 0x7f049c06fdf0 2024-09-11T12:49:43.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.970+0000 7f049aa00640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 0x7f047c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:43.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.970+0000 7f049aa00640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 0x7f047c073b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f049c1562e0 tx=0x7f048800b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:43.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:43.974+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f048c0c6020 con 0x7f049c06fdf0 2024-09-11T12:49:44.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f049c1520e0 con 0x7f049c06fdf0 2024-09-11T12:49:44.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f0493400640 1 -- 172.21.15.78:0/1344147973 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16678 (secure 0 0 0) 0x7f048c0164d0 con 0x7f049c06fdf0 2024-09-11T12:49:44.149 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:44.149 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":22,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:42.819643+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 msgr2=0x7f047c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 0x7f047c073b40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f049c1562e0 tx=0x7f048800b390 comp rx=0 tx=0).stop 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 msgr2=0x7f049c06d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f048c0b0040 tx=0x7f048c0a57c0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 shutdown_connections 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f047c071680 0x7f047c073b40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f049c0d0bf0 0x7f049c148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f049c0b14b0 0x7f049c1f89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 --2- 172.21.15.78:0/1344147973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f049c06fdf0 0x7f049c06d1d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 >> 172.21.15.78:0/1344147973 conn(0x7f049c074b90 msgr2=0x7f049c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:44.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 shutdown_connections 2024-09-11T12:49:44.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.142+0000 7f04a2400640 1 -- 172.21.15.78:0/1344147973 wait complete. 2024-09-11T12:49:44.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 -- 172.21.15.78:0/4061481069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150f60 msgr2=0x7fe4c015aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/4061481069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150f60 0x7fe4c015aa00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009840 tx=0x7fe4ac01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 -- 172.21.15.78:0/4061481069 shutdown_connections 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/4061481069 >> v2:172.21.15.157:3300/0 conn(0x7fe4c015af40 0x7fe4c00b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/4061481069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150f60 0x7fe4c015aa00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 -- 172.21.15.78:0/4061481069 >> 172.21.15.78:0/4061481069 conn(0x7fe4c0071dd0 msgr2=0x7fe4c00721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 -- 172.21.15.78:0/4061481069 shutdown_connections 2024-09-11T12:49:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.306+0000 7fe4c4c00640 1 -- 172.21.15.78:0/4061481069 wait complete. 2024-09-11T12:49:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 Processor -- start 2024-09-11T12:49:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- start start 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150b80 0x7fe4c0143ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c015b330 0x7fe4c0158cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0165000 con 0x7fe4c0150f60 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c0164e80 con 0x7fe4c015b330 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe4c0165180 con 0x7fe4c0150b80 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44364/0 (socket says 172.21.15.78:44364) 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 -- 172.21.15.78:0/3572358629 learned_addr learned my addr 172.21.15.78:0/3572358629 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bde00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150b80 0x7fe4c0143ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4be800640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c015b330 0x7fe4c0158cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 -- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150b80 msgr2=0x7fe4c0143ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150b80 0x7fe4c0143ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 -- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c015b330 msgr2=0x7fe4c0158cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c015b330 0x7fe4c0158cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 -- 172.21.15.78:0/3572358629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c01fb420 con 0x7fe4c0150f60 2024-09-11T12:49:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4bd400640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009790 tx=0x7fe4ac0a57e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0b2040 con 0x7fe4c0150f60 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4ac004430 con 0x7fe4c0150f60 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac005280 con 0x7fe4c0150f60 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c01fb700 con 0x7fe4c0150f60 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c01fbcc0 con 0x7fe4c0150f60 2024-09-11T12:49:44.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.310+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4c0165630 con 0x7fe4c0150f60 2024-09-11T12:49:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.314+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe4ac0be060 con 0x7fe4c0150f60 2024-09-11T12:49:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.314+0000 7fe4b6000640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 0x7fe4a0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:44.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.314+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe4ac12b590 con 0x7fe4c0150f60 2024-09-11T12:49:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.314+0000 7fe4bde00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 0x7fe4a0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:44.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.314+0000 7fe4bde00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 0x7fe4a0073b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0001910 tx=0x7fe4b0001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:44.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.318+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe4ac016410 con 0x7fe4c0150f60 2024-09-11T12:49:44.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:44.522+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe4c0073f70 con 0x7fe4c0150f60 2024-09-11T12:49:45.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4b6000640 1 -- 172.21.15.78:0/3572358629 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7fe4ac0f6da0 con 0x7fe4c0150f60 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 msgr2=0x7fe4a0073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 0x7fe4a0073b40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0001910 tx=0x7fe4b0001420 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 msgr2=0x7fe4c0145080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009790 tx=0x7fe4ac0a57e0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 shutdown_connections 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4a0071680 0x7fe4a0073b40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe4c015b330 0x7fe4c0158cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe4c0150f60 0x7fe4c0145080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 --2- 172.21.15.78:0/3572358629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe4c0150b80 0x7fe4c0143ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 >> 172.21.15.78:0/3572358629 conn(0x7fe4c0071dd0 msgr2=0x7fe4c0070610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 shutdown_connections 2024-09-11T12:49:45.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.442+0000 7fe4c4c00640 1 -- 172.21.15.78:0/3572358629 wait complete. 2024-09-11T12:49:45.471 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:49:45.471 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- 172.21.15.78:0/205746562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c152690 msgr2=0x7fe23c0a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 --2- 172.21.15.78:0/205746562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c152690 0x7fe23c0a9460 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe22c009bc0 tx=0x7fe22c01cba0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- 172.21.15.78:0/205746562 shutdown_connections 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 --2- 172.21.15.78:0/205746562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c152690 0x7fe23c0a9460 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- 172.21.15.78:0/205746562 >> 172.21.15.78:0/205746562 conn(0x7fe23c0742a0 msgr2=0x7fe23c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- 172.21.15.78:0/205746562 shutdown_connections 2024-09-11T12:49:45.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- 172.21.15.78:0/205746562 wait complete. 2024-09-11T12:49:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 Processor -- start 2024-09-11T12:49:45.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- start start 2024-09-11T12:49:45.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c06fcb0 0x7fe23c14ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c152690 0x7fe23c15f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c169ca0 con 0x7fe23c0cebc0 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe23c169b20 con 0x7fe23c152690 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe23c169e20 con 0x7fe23c06fcb0 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44386/0 (socket says 172.21.15.78:44386) 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 -- 172.21.15.78:0/1582477378 learned_addr learned my addr 172.21.15.78:0/1582477378 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239e00640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c06fcb0 0x7fe23c14ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe23a800640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c152690 0x7fe23c15f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 -- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c06fcb0 msgr2=0x7fe23c14ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c06fcb0 0x7fe23c14ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 -- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c152690 msgr2=0x7fe23c15f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:45.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c152690 0x7fe23c15f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:45.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.614+0000 7fe239400640 1 -- 172.21.15.78:0/1582477378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe23c156520 con 0x7fe23c0cebc0 2024-09-11T12:49:45.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe239400640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c480 tx=0x7fe22800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228016020 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe228012920 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe23c156800 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe228012d70 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe23c156dc0 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe23c0a9460 con 0x7fe23c0cebc0 2024-09-11T12:49:45.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe228019020 con 0x7fe23c0cebc0 2024-09-11T12:49:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 0x7fe21c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe228091da0 con 0x7fe23c0cebc0 2024-09-11T12:49:45.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.618+0000 7fe239e00640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 0x7fe21c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:45.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.622+0000 7fe239e00640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 0x7fe21c073b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0af000 tx=0x7fe22c01b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:45.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.622+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe228020020 con 0x7fe23c0cebc0 2024-09-11T12:49:45.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:45.826+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe23c06d7e0 con 0x7fe23c0cebc0 2024-09-11T12:49:46.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.442+0000 7fe232000640 1 -- 172.21.15.78:0/1582477378 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7fe22805dc00 con 0x7fe23c0cebc0 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 msgr2=0x7fe21c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 0x7fe21c073b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe22c0af000 tx=0x7fe22c01b3e0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 msgr2=0x7fe23c151800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe22800c480 tx=0x7fe22800c940 comp rx=0 tx=0).stop 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 shutdown_connections 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe21c071680 0x7fe21c073b40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe23c152690 0x7fe23c15f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe23c0cebc0 0x7fe23c151800 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 --2- 172.21.15.78:0/1582477378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe23c06fcb0 0x7fe23c14ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 >> 172.21.15.78:0/1582477378 conn(0x7fe23c0742a0 msgr2=0x7fe23c0cd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 shutdown_connections 2024-09-11T12:49:46.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.446+0000 7fe23b200640 1 -- 172.21.15.78:0/1582477378 wait complete. 2024-09-11T12:49:46.474 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:49:46.474 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- 172.21.15.78:0/3627961766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f8004da0 msgr2=0x7f77f8005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 --2- 172.21.15.78:0/3627961766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77f8004da0 0x7f77f8005220 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f780c00c630 tx=0x7f780c01d2c0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f7819800640 1 -- 172.21.15.78:0/3627961766 reap_dead start 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- 172.21.15.78:0/3627961766 shutdown_connections 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- 172.21.15.78:0/3627961766 >> 172.21.15.78:0/3627961766 conn(0x7f781406ca80 msgr2=0x7f78140b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- 172.21.15.78:0/3627961766 shutdown_connections 2024-09-11T12:49:46.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- 172.21.15.78:0/3627961766 wait complete. 2024-09-11T12:49:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 Processor -- start 2024-09-11T12:49:46.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.614+0000 7f781b800640 1 -- start start 2024-09-11T12:49:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77f8004da0 0x7f7814158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 0x7f78141f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78140b4990 con 0x7f7814148730 2024-09-11T12:49:46.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f781415d220 con 0x7f7814148bb0 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78140b4b10 con 0x7f77f8004da0 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44392/0 (socket says 172.21.15.78:44392) 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 -- 172.21.15.78:0/3861246908 learned_addr learned my addr 172.21.15.78:0/3861246908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 -- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77f8004da0 msgr2=0x7f7814158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77f8004da0 0x7f7814158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 -- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 msgr2=0x7f78141f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7819800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 0x7f78141f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 0x7f78141f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 -- 172.21.15.78:0/3861246908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78141573b0 con 0x7f7814148730 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7819800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 0x7f78141f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:46.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7813e00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7800003f90 tx=0x7f780000ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800005220 con 0x7f7814148730 2024-09-11T12:49:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78000053c0 con 0x7f7814148730 2024-09-11T12:49:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000047b0 con 0x7f7814148730 2024-09-11T12:49:46.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7814157690 con 0x7f7814148730 2024-09-11T12:49:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7814157c50 con 0x7f7814148730 2024-09-11T12:49:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f781415d3f0 con 0x7f7814148730 2024-09-11T12:49:46.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.618+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7800004970 con 0x7f7814148730 2024-09-11T12:49:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.622+0000 7f7810c00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 0x7f77f4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.622+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7800092970 con 0x7f7814148730 2024-09-11T12:49:46.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.622+0000 7f7818e00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 0x7f77f4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.622+0000 7f7818e00640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 0x7f77f4073b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7804006860 tx=0x7f78040067f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:46.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.626+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f780005e740 con 0x7f7814148730 2024-09-11T12:49:46.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.822+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78140695e0 con 0x7f7814148730 2024-09-11T12:49:46.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.822+0000 7f7810c00640 1 -- 172.21.15.78:0/3861246908 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f780005e040 con 0x7f7814148730 2024-09-11T12:49:46.828 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:46.828 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:49:46.829 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:49:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 msgr2=0x7f77f4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 0x7f77f4073b40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f7804006860 tx=0x7f78040067f0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 msgr2=0x7f78140c5bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7800003f90 tx=0x7f780000ba80 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 shutdown_connections 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f77f4071680 0x7f77f4073b40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7814148bb0 0x7f78141f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7814148730 0x7f78140c5bc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 --2- 172.21.15.78:0/3861246908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77f8004da0 0x7f7814158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 >> 172.21.15.78:0/3861246908 conn(0x7f781406ca80 msgr2=0x7f78140cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 shutdown_connections 2024-09-11T12:49:46.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.826+0000 7f781b800640 1 -- 172.21.15.78:0/3861246908 wait complete. 2024-09-11T12:49:46.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 -- 172.21.15.78:0/20705949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 msgr2=0x7f78b8073600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 --2- 172.21.15.78:0/20705949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b8073600 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f78a4009840 tx=0x7f78a401c820 comp rx=0 tx=0).stop 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 -- 172.21.15.78:0/20705949 shutdown_connections 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 --2- 172.21.15.78:0/20705949 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b8073600 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 -- 172.21.15.78:0/20705949 >> 172.21.15.78:0/20705949 conn(0x7f78b806d870 msgr2=0x7f78b806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 -- 172.21.15.78:0/20705949 shutdown_connections 2024-09-11T12:49:46.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.986+0000 7f78be000640 1 -- 172.21.15.78:0/20705949 wait complete. 2024-09-11T12:49:46.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 Processor -- start 2024-09-11T12:49:46.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- start start 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b814af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b80cf660 0x7f78b81432c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8169e20 con 0x7f78b81526b0 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f78b8169ca0 con 0x7f78b8073190 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f78b8169fa0 con 0x7f78b80cf660 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44398/0 (socket says 172.21.15.78:44398) 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 -- 172.21.15.78:0/1571672410 learned_addr learned my addr 172.21.15.78:0/1571672410 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:46.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bcc00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b814af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78b7e00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b80cf660 0x7f78b81432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 -- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b80cf660 msgr2=0x7f78b81432c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b80cf660 0x7f78b81432c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 -- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 msgr2=0x7f78b814af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b814af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 -- 172.21.15.78:0/1571672410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b8145cd0 con 0x7f78b81526b0 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bcc00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:46.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78bd600640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78b000cbc0 tx=0x7f78b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0016020 con 0x7f78b81526b0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b000ab00 con 0x7f78b81526b0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b0012420 con 0x7f78b81526b0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b814b4b0 con 0x7f78b81526b0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b8146220 con 0x7f78b81526b0 2024-09-11T12:49:46.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.990+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78b8073600 con 0x7f78b81526b0 2024-09-11T12:49:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.994+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f78b0019020 con 0x7f78b81526b0 2024-09-11T12:49:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.994+0000 7f78b4c00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 0x7f7898073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:46.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.994+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f78b0092160 con 0x7f78b81526b0 2024-09-11T12:49:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.994+0000 7f78bcc00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 0x7f7898073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:46.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.994+0000 7f78bcc00640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 0x7f7898073b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c10 tx=0x7f78ac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:47.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:46.998+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f78b0020020 con 0x7f78b81526b0 2024-09-11T12:49:47.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.162+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f78b80cfee0 con 0x7f78b81526b0 2024-09-11T12:49:47.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78b4c00640 1 -- 172.21.15.78:0/1571672410 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f78b0066070 con 0x7f78b81526b0 2024-09-11T12:49:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:47.169 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:47.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 msgr2=0x7f7898073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 0x7f7898073b40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f78ac000c10 tx=0x7f78ac009040 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 msgr2=0x7f78b8144d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78b000cbc0 tx=0x7f78b000ccb0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 shutdown_connections 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7898071680 0x7f7898073b40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f78b81526b0 0x7f78b8144d90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f78b80cf660 0x7f78b81432c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 --2- 172.21.15.78:0/1571672410 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78b8073190 0x7f78b814af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 >> 172.21.15.78:0/1571672410 conn(0x7f78b806d870 msgr2=0x7f78b8070420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 shutdown_connections 2024-09-11T12:49:47.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.166+0000 7f78be000640 1 -- 172.21.15.78:0/1571672410 wait complete. 2024-09-11T12:49:47.191 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:49:47.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 -- 172.21.15.78:0/2149484774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 msgr2=0x7f07780a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 --2- 172.21.15.78:0/2149484774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f07780a9d00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0764009840 tx=0x7f076401c820 comp rx=0 tx=0).stop 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 -- 172.21.15.78:0/2149484774 shutdown_connections 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 --2- 172.21.15.78:0/2149484774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f07780a9d00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 -- 172.21.15.78:0/2149484774 >> 172.21.15.78:0/2149484774 conn(0x7f0778074b90 msgr2=0x7f07780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:47.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 -- 172.21.15.78:0/2149484774 shutdown_connections 2024-09-11T12:49:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.326+0000 7f077e600640 1 -- 172.21.15.78:0/2149484774 wait complete. 2024-09-11T12:49:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 Processor -- start 2024-09-11T12:49:47.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- start start 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f077814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780b14b0 0x7f077815ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f077816a450 con 0x7f07780d0bf0 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f077816a2d0 con 0x7f077806fdf0 2024-09-11T12:49:47.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f077816a5d0 con 0x7f07780b14b0 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44414/0 (socket says 172.21.15.78:44414) 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 -- 172.21.15.78:0/1359601253 learned_addr learned my addr 172.21.15.78:0/1359601253 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777400640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f077814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0776a00640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780b14b0 0x7f077815ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 -- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780b14b0 msgr2=0x7f077815ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780b14b0 0x7f077815ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 -- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 msgr2=0x7f077814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f077814a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 -- 172.21.15.78:0/1359601253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0778151160 con 0x7f07780d0bf0 2024-09-11T12:49:47.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f0777e00640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f077000ca70 tx=0x7f077000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:47.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770016020 con 0x7f07780d0bf0 2024-09-11T12:49:47.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f077000a930 con 0x7f07780d0bf0 2024-09-11T12:49:47.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0778151440 con 0x7f07780d0bf0 2024-09-11T12:49:47.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077000ad80 con 0x7f07780d0bf0 2024-09-11T12:49:47.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.330+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0778151a00 con 0x7f07780d0bf0 2024-09-11T12:49:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07780a9d00 con 0x7f07780d0bf0 2024-09-11T12:49:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f0770019020 con 0x7f07780d0bf0 2024-09-11T12:49:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f076f400640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 0x7f0758073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0770091c60 con 0x7f07780d0bf0 2024-09-11T12:49:47.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f0777400640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 0x7f0758073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.334+0000 7f0777400640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 0x7f0758073b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c10 tx=0x7f0768009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:47.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.338+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0770020030 con 0x7f07780d0bf0 2024-09-11T12:49:47.400 INFO:tasks.ceph.mds.a.smithi078.stderr:2024-09-11T12:49:47.394+0000 7fa867c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:49:47.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.506+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f077814ad80 con 0x7f07580716b0 2024-09-11T12:49:47.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.510+0000 7f076f400640 1 -- 172.21.15.78:0/1359601253 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241621 (secure 0 0 0) 0x7f077814ad80 con 0x7f07580716b0 2024-09-11T12:49:47.518 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:47.520 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 msgr2=0x7f0758073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 0x7f0758073b70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f0768000c10 tx=0x7f0768009040 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 msgr2=0x7f077815b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f077000ca70 tx=0x7f077000cb60 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 shutdown_connections 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07580716b0 0x7f0758073b70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07780d0bf0 0x7f077815b0d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07780b14b0 0x7f077815ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 --2- 172.21.15.78:0/1359601253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f077806fdf0 0x7f077814a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 >> 172.21.15.78:0/1359601253 conn(0x7f0778074b90 msgr2=0x7f0778153010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 shutdown_connections 2024-09-11T12:49:47.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.514+0000 7f077e600640 1 -- 172.21.15.78:0/1359601253 wait complete. 2024-09-11T12:49:47.539 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:49:47.105337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235616,"kb_used_data":2744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739104,"statfs":{"total":773094113280,"available":772852842496,"internally_reserved":0,"allocated":2809856,"data_stored":1693564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691404"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420936+0000","last_change":"2024-09-11T12:49:41.420936+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420936+0000","last_undegraded":"2024-09-11T12:49:41.420936+0000","last_fullsized":"2024-09-11T12:49:41.420936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420195+0000","last_change":"2024-09-11T12:49:41.420195+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420195+0000","last_undegraded":"2024-09-11T12:49:41.420195+0000","last_fullsized":"2024-09-11T12:49:41.420195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422454+0000","last_change":"2024-09-11T12:49:41.422454+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422454+0000","last_undegraded":"2024-09-11T12:49:41.422454+0000","last_fullsized":"2024-09-11T12:49:41.422454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416526+0000","last_change":"2024-09-11T12:49:41.416526+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416526+0000","last_undegraded":"2024-09-11T12:49:41.416526+0000","last_fullsized":"2024-09-11T12:49:41.416526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420940+0000","last_change":"2024-09-11T12:49:41.420940+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420940+0000","last_undegraded":"2024-09-11T12:49:41.420940+0000","last_fullsized":"2024-09-11T12:49:41.420940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416461+0000","last_change":"2024-09-11T12:49:41.416461+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416461+0000","last_undegraded":"2024-09-11T12:49:41.416461+0000","last_fullsized":"2024-09-11T12:49:41.416461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424304+0000","last_change":"2024-09-11T12:49:41.424304+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424304+0000","last_undegraded":"2024-09-11T12:49:41.424304+0000","last_fullsized":"2024-09-11T12:49:41.424304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420150+0000","last_change":"2024-09-11T12:49:41.420150+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420150+0000","last_undegraded":"2024-09-11T12:49:41.420150+0000","last_fullsized":"2024-09-11T12:49:41.420150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.417231+0000","last_change":"2024-09-11T12:49:41.417231+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.417231+0000","last_undegraded":"2024-09-11T12:49:41.417231+0000","last_fullsized":"2024-09-11T12:49:41.417231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421043+0000","last_change":"2024-09-11T12:49:41.421043+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421043+0000","last_undegraded":"2024-09-11T12:49:41.421043+0000","last_fullsized":"2024-09-11T12:49:41.421043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421911+0000","last_change":"2024-09-11T12:49:41.421911+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421911+0000","last_undegraded":"2024-09-11T12:49:41.421911+0000","last_fullsized":"2024-09-11T12:49:41.421911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416699+0000","last_change":"2024-09-11T12:49:41.416699+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416699+0000","last_undegraded":"2024-09-11T12:49:41.416699+0000","last_fullsized":"2024-09-11T12:49:41.416699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422904+0000","last_change":"2024-09-11T12:49:41.422904+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422904+0000","last_undegraded":"2024-09-11T12:49:41.422904+0000","last_fullsized":"2024-09-11T12:49:41.422904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421305+0000","last_change":"2024-09-11T12:49:41.421305+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421305+0000","last_undegraded":"2024-09-11T12:49:41.421305+0000","last_fullsized":"2024-09-11T12:49:41.421305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423234+0000","last_change":"2024-09-11T12:49:41.423234+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423234+0000","last_undegraded":"2024-09-11T12:49:41.423234+0000","last_fullsized":"2024-09-11T12:49:41.423234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T12:49:41.424690+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:41.424690+0000","last_peered":"2024-09-11T12:49:41.424690+0000","last_clean":"2024-09-11T12:49:41.424690+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:41.424690+0000","last_undegraded":"2024-09-11T12:49:41.424690+0000","last_fullsized":"2024-09-11T12:49:41.424690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.418918+0000","last_change":"2024-09-11T12:49:41.418918+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.418918+0000","last_undegraded":"2024-09-11T12:49:41.418918+0000","last_fullsized":"2024-09-11T12:49:41.418918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423038+0000","last_change":"2024-09-11T12:49:41.423038+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423038+0000","last_undegraded":"2024-09-11T12:49:41.423038+0000","last_fullsized":"2024-09-11T12:49:41.423038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422266+0000","last_change":"2024-09-11T12:49:41.422266+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422266+0000","last_undegraded":"2024-09-11T12:49:41.422266+0000","last_fullsized":"2024-09-11T12:49:41.422266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416548+0000","last_change":"2024-09-11T12:49:41.416548+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416548+0000","last_undegraded":"2024-09-11T12:49:41.416548+0000","last_fullsized":"2024-09-11T12:49:41.416548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420959+0000","last_change":"2024-09-11T12:49:41.420959+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420959+0000","last_undegraded":"2024-09-11T12:49:41.420959+0000","last_fullsized":"2024-09-11T12:49:41.420959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423313+0000","last_change":"2024-09-11T12:49:41.423313+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423313+0000","last_undegraded":"2024-09-11T12:49:41.423313+0000","last_fullsized":"2024-09-11T12:49:41.423313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421068+0000","last_change":"2024-09-11T12:49:41.421068+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421068+0000","last_undegraded":"2024-09-11T12:49:41.421068+0000","last_fullsized":"2024-09-11T12:49:41.421068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420228+0000","last_change":"2024-09-11T12:49:41.420228+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420228+0000","last_undegraded":"2024-09-11T12:49:41.420228+0000","last_fullsized":"2024-09-11T12:49:41.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423954+0000","last_change":"2024-09-11T12:49:41.423954+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423954+0000","last_undegraded":"2024-09-11T12:49:41.423954+0000","last_fullsized":"2024-09-11T12:49:41.423954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420014+0000","last_change":"2024-09-11T12:49:41.420014+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420014+0000","last_undegraded":"2024-09-11T12:49:41.420014+0000","last_fullsized":"2024-09-11T12:49:41.420014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.419869+0000","last_change":"2024-09-11T12:49:41.419869+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.419869+0000","last_undegraded":"2024-09-11T12:49:41.419869+0000","last_fullsized":"2024-09-11T12:49:41.419869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423321+0000","last_change":"2024-09-11T12:49:41.423321+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423321+0000","last_undegraded":"2024-09-11T12:49:41.423321+0000","last_fullsized":"2024-09-11T12:49:41.423321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423277+0000","last_change":"2024-09-11T12:49:41.423277+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423277+0000","last_undegraded":"2024-09-11T12:49:41.423277+0000","last_fullsized":"2024-09-11T12:49:41.423277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424152+0000","last_change":"2024-09-11T12:49:41.424152+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424152+0000","last_undegraded":"2024-09-11T12:49:41.424152+0000","last_fullsized":"2024-09-11T12:49:41.424152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421480+0000","last_change":"2024-09-11T12:49:41.421480+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421480+0000","last_undegraded":"2024-09-11T12:49:41.421480+0000","last_fullsized":"2024-09-11T12:49:41.421480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":651973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:47.546 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:47.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- 172.21.15.78:0/3493429121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130070360 msgr2=0x7fc13015ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- 172.21.15.78:0/3493429121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130070360 0x7fc13015ea20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc128009c80 tx=0x7fc12801cd70 comp rx=0 tx=0).stop 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- 172.21.15.78:0/3493429121 shutdown_connections 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- 172.21.15.78:0/3493429121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130070360 0x7fc13015ea20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- 172.21.15.78:0/3493429121 >> v2:172.21.15.157:3300/0 conn(0x7fc130074240 0x7fc13015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- 172.21.15.78:0/3493429121 >> 172.21.15.78:0/3493429121 conn(0x7fc13006ca80 msgr2=0x7fc1300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- 172.21.15.78:0/3493429121 shutdown_connections 2024-09-11T12:49:47.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- 172.21.15.78:0/3493429121 wait complete. 2024-09-11T12:49:47.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 Processor -- start 2024-09-11T12:49:47.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- start start 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130074240 0x7fc13014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 0x7fc130159f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc130167740 con 0x7fc130070360 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc130072910 con 0x7fc13015e5b0 2024-09-11T12:49:47.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc136200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1301678c0 con 0x7fc130074240 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44416/0 (socket says 172.21.15.78:44416) 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 -- 172.21.15.78:0/2206552504 learned_addr learned my addr 172.21.15.78:0/2206552504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12fe00640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 0x7fc130159f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 -- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130074240 msgr2=0x7fc13014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130074240 0x7fc13014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 -- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 msgr2=0x7fc130159f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 0x7fc130159f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 -- 172.21.15.78:0/2206552504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc13015cfa0 con 0x7fc130070360 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12fe00640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 0x7fc130159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.718+0000 7fc12f400640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc12000ea10 tx=0x7fc12000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:47.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc12000cde0 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc120004020 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc120004470 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc13014f8a0 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1301562e0 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1301560f0 con 0x7fc130070360 2024-09-11T12:49:47.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fc120003600 con 0x7fc130070360 2024-09-11T12:49:47.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 0x7fc110073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:47.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc120092780 con 0x7fc130070360 2024-09-11T12:49:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.722+0000 7fc12ea00640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 0x7fc110073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:47.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.726+0000 7fc12ea00640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 0x7fc110073b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc11c000f70 tx=0x7fc11c0079e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:47.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.726+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc12005e680 con 0x7fc130070360 2024-09-11T12:49:47.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.890+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc130165510 con 0x7fc110071680 2024-09-11T12:49:47.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.894+0000 7fc127400640 1 -- 172.21.15.78:0/2206552504 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241621 (secure 0 0 0) 0x7fc130165510 con 0x7fc110071680 2024-09-11T12:49:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:47.904 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 msgr2=0x7fc110073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 0x7fc110073b40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc11c000f70 tx=0x7fc11c0079e0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 msgr2=0x7fc130156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc12000ea10 tx=0x7fc12000eed0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 shutdown_connections 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc110071680 0x7fc110073b40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc13015e5b0 0x7fc130159f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc130074240 0x7fc13014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 --2- 172.21.15.78:0/2206552504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc130070360 0x7fc130156dc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:47.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 >> 172.21.15.78:0/2206552504 conn(0x7fc13006ca80 msgr2=0x7fc130151880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:47.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 shutdown_connections 2024-09-11T12:49:47.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:47.898+0000 7fc136200640 1 -- 172.21.15.78:0/2206552504 wait complete. 2024-09-11T12:49:47.923 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:49:47.105337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235616,"kb_used_data":2744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739104,"statfs":{"total":773094113280,"available":772852842496,"internally_reserved":0,"allocated":2809856,"data_stored":1693564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691404"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420936+0000","last_change":"2024-09-11T12:49:41.420936+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420936+0000","last_undegraded":"2024-09-11T12:49:41.420936+0000","last_fullsized":"2024-09-11T12:49:41.420936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420195+0000","last_change":"2024-09-11T12:49:41.420195+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420195+0000","last_undegraded":"2024-09-11T12:49:41.420195+0000","last_fullsized":"2024-09-11T12:49:41.420195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422454+0000","last_change":"2024-09-11T12:49:41.422454+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422454+0000","last_undegraded":"2024-09-11T12:49:41.422454+0000","last_fullsized":"2024-09-11T12:49:41.422454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416526+0000","last_change":"2024-09-11T12:49:41.416526+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416526+0000","last_undegraded":"2024-09-11T12:49:41.416526+0000","last_fullsized":"2024-09-11T12:49:41.416526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420940+0000","last_change":"2024-09-11T12:49:41.420940+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420940+0000","last_undegraded":"2024-09-11T12:49:41.420940+0000","last_fullsized":"2024-09-11T12:49:41.420940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416461+0000","last_change":"2024-09-11T12:49:41.416461+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416461+0000","last_undegraded":"2024-09-11T12:49:41.416461+0000","last_fullsized":"2024-09-11T12:49:41.416461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424304+0000","last_change":"2024-09-11T12:49:41.424304+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424304+0000","last_undegraded":"2024-09-11T12:49:41.424304+0000","last_fullsized":"2024-09-11T12:49:41.424304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420150+0000","last_change":"2024-09-11T12:49:41.420150+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420150+0000","last_undegraded":"2024-09-11T12:49:41.420150+0000","last_fullsized":"2024-09-11T12:49:41.420150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.417231+0000","last_change":"2024-09-11T12:49:41.417231+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.417231+0000","last_undegraded":"2024-09-11T12:49:41.417231+0000","last_fullsized":"2024-09-11T12:49:41.417231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421043+0000","last_change":"2024-09-11T12:49:41.421043+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421043+0000","last_undegraded":"2024-09-11T12:49:41.421043+0000","last_fullsized":"2024-09-11T12:49:41.421043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421911+0000","last_change":"2024-09-11T12:49:41.421911+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421911+0000","last_undegraded":"2024-09-11T12:49:41.421911+0000","last_fullsized":"2024-09-11T12:49:41.421911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416699+0000","last_change":"2024-09-11T12:49:41.416699+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416699+0000","last_undegraded":"2024-09-11T12:49:41.416699+0000","last_fullsized":"2024-09-11T12:49:41.416699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422904+0000","last_change":"2024-09-11T12:49:41.422904+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422904+0000","last_undegraded":"2024-09-11T12:49:41.422904+0000","last_fullsized":"2024-09-11T12:49:41.422904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421305+0000","last_change":"2024-09-11T12:49:41.421305+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421305+0000","last_undegraded":"2024-09-11T12:49:41.421305+0000","last_fullsized":"2024-09-11T12:49:41.421305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423234+0000","last_change":"2024-09-11T12:49:41.423234+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423234+0000","last_undegraded":"2024-09-11T12:49:41.423234+0000","last_fullsized":"2024-09-11T12:49:41.423234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T12:49:41.424690+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:41.424690+0000","last_peered":"2024-09-11T12:49:41.424690+0000","last_clean":"2024-09-11T12:49:41.424690+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:41.424690+0000","last_undegraded":"2024-09-11T12:49:41.424690+0000","last_fullsized":"2024-09-11T12:49:41.424690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.418918+0000","last_change":"2024-09-11T12:49:41.418918+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.418918+0000","last_undegraded":"2024-09-11T12:49:41.418918+0000","last_fullsized":"2024-09-11T12:49:41.418918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423038+0000","last_change":"2024-09-11T12:49:41.423038+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423038+0000","last_undegraded":"2024-09-11T12:49:41.423038+0000","last_fullsized":"2024-09-11T12:49:41.423038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422266+0000","last_change":"2024-09-11T12:49:41.422266+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422266+0000","last_undegraded":"2024-09-11T12:49:41.422266+0000","last_fullsized":"2024-09-11T12:49:41.422266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416548+0000","last_change":"2024-09-11T12:49:41.416548+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416548+0000","last_undegraded":"2024-09-11T12:49:41.416548+0000","last_fullsized":"2024-09-11T12:49:41.416548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420959+0000","last_change":"2024-09-11T12:49:41.420959+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420959+0000","last_undegraded":"2024-09-11T12:49:41.420959+0000","last_fullsized":"2024-09-11T12:49:41.420959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423313+0000","last_change":"2024-09-11T12:49:41.423313+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423313+0000","last_undegraded":"2024-09-11T12:49:41.423313+0000","last_fullsized":"2024-09-11T12:49:41.423313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421068+0000","last_change":"2024-09-11T12:49:41.421068+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421068+0000","last_undegraded":"2024-09-11T12:49:41.421068+0000","last_fullsized":"2024-09-11T12:49:41.421068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420228+0000","last_change":"2024-09-11T12:49:41.420228+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420228+0000","last_undegraded":"2024-09-11T12:49:41.420228+0000","last_fullsized":"2024-09-11T12:49:41.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423954+0000","last_change":"2024-09-11T12:49:41.423954+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423954+0000","last_undegraded":"2024-09-11T12:49:41.423954+0000","last_fullsized":"2024-09-11T12:49:41.423954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420014+0000","last_change":"2024-09-11T12:49:41.420014+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420014+0000","last_undegraded":"2024-09-11T12:49:41.420014+0000","last_fullsized":"2024-09-11T12:49:41.420014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.419869+0000","last_change":"2024-09-11T12:49:41.419869+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.419869+0000","last_undegraded":"2024-09-11T12:49:41.419869+0000","last_fullsized":"2024-09-11T12:49:41.419869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423321+0000","last_change":"2024-09-11T12:49:41.423321+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423321+0000","last_undegraded":"2024-09-11T12:49:41.423321+0000","last_fullsized":"2024-09-11T12:49:41.423321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423277+0000","last_change":"2024-09-11T12:49:41.423277+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423277+0000","last_undegraded":"2024-09-11T12:49:41.423277+0000","last_fullsized":"2024-09-11T12:49:41.423277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424152+0000","last_change":"2024-09-11T12:49:41.424152+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424152+0000","last_undegraded":"2024-09-11T12:49:41.424152+0000","last_fullsized":"2024-09-11T12:49:41.424152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421480+0000","last_change":"2024-09-11T12:49:41.421480+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421480+0000","last_undegraded":"2024-09-11T12:49:41.421480+0000","last_fullsized":"2024-09-11T12:49:41.421480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":651973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:47.929 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- 172.21.15.78:0/3339043064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 msgr2=0x7f5be815fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- 172.21.15.78:0/3339043064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be815fad0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5be8072e50 tx=0x7f5bd801c890 comp rx=0 tx=0).stop 2024-09-11T12:49:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- 172.21.15.78:0/3339043064 shutdown_connections 2024-09-11T12:49:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- 172.21.15.78:0/3339043064 >> v2:172.21.15.157:3300/0 conn(0x7f5be815f3a0 0x7f5be813ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- 172.21.15.78:0/3339043064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be80d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- 172.21.15.78:0/3339043064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be815fad0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- 172.21.15.78:0/3339043064 >> 172.21.15.78:0/3339043064 conn(0x7f5be80725a0 msgr2=0x7f5be8070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- 172.21.15.78:0/3339043064 shutdown_connections 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- 172.21.15.78:0/3339043064 wait complete. 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 Processor -- start 2024-09-11T12:49:48.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- start start 2024-09-11T12:49:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be815f3a0 0x7f5be81f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be81f8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5be8141560 con 0x7f5be813e620 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5be81413e0 con 0x7f5be815f3a0 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bede00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5be81416e0 con 0x7f5be8160010 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44440/0 (socket says 172.21.15.78:44440) 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 -- 172.21.15.78:0/1424166114 learned_addr learned my addr 172.21.15.78:0/1424166114 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5be7e00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be815f3a0 0x7f5be81f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5bed400640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be81f8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 -- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 msgr2=0x7f5be81f8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be81f8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 -- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be815f3a0 msgr2=0x7f5be81f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.106+0000 7f5beca00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be815f3a0 0x7f5be81f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5beca00640 1 -- 172.21.15.78:0/1424166114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be8144d70 con 0x7f5be813e620 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5bed400640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be81f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5beca00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bd801af40 tx=0x7f5bd8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80b2040 con 0x7f5be813e620 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd8005030 con 0x7f5be813e620 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8005480 con 0x7f5be813e620 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be813bfc0 con 0x7f5be813e620 2024-09-11T12:49:48.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be813c580 con 0x7f5be813e620 2024-09-11T12:49:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be8141b90 con 0x7f5be813e620 2024-09-11T12:49:48.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.110+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f5bd80be060 con 0x7f5be813e620 2024-09-11T12:49:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.114+0000 7f5be4c00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 0x7f5bc8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.114+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd812b570 con 0x7f5be813e620 2024-09-11T12:49:48.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.114+0000 7f5be7e00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 0x7f5bc8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.114+0000 7f5be7e00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 0x7f5bc8073b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005fd0 tx=0x7f5bd4005c90 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.118+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5bd80164b0 con 0x7f5be813e620 2024-09-11T12:49:48.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.278+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5be80d0950 con 0x7f5bc8071680 2024-09-11T12:49:48.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.282+0000 7f5be4c00640 1 -- 172.21.15.78:0/1424166114 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241621 (secure 0 0 0) 0x7f5be80d0950 con 0x7f5bc8071680 2024-09-11T12:49:48.291 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:48.292 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:48.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 msgr2=0x7f5bc8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 0x7f5bc8073b40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4005fd0 tx=0x7f5bd4005c90 comp rx=0 tx=0).stop 2024-09-11T12:49:48.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 msgr2=0x7f5be8154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5bd801af40 tx=0x7f5bd8004040 comp rx=0 tx=0).stop 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 shutdown_connections 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5bc8071680 0x7f5bc8073b40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5be8160010 0x7f5be81f8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5be815f3a0 0x7f5be81f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 --2- 172.21.15.78:0/1424166114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be813e620 0x7f5be8154a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 >> 172.21.15.78:0/1424166114 conn(0x7f5be80725a0 msgr2=0x7f5be806f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 shutdown_connections 2024-09-11T12:49:48.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.286+0000 7f5bede00640 1 -- 172.21.15.78:0/1424166114 wait complete. 2024-09-11T12:49:48.311 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:49:47.105337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235616,"kb_used_data":2744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739104,"statfs":{"total":773094113280,"available":772852842496,"internally_reserved":0,"allocated":2809856,"data_stored":1693564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691404"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420936+0000","last_change":"2024-09-11T12:49:41.420936+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420936+0000","last_undegraded":"2024-09-11T12:49:41.420936+0000","last_fullsized":"2024-09-11T12:49:41.420936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420195+0000","last_change":"2024-09-11T12:49:41.420195+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420195+0000","last_undegraded":"2024-09-11T12:49:41.420195+0000","last_fullsized":"2024-09-11T12:49:41.420195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422454+0000","last_change":"2024-09-11T12:49:41.422454+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422454+0000","last_undegraded":"2024-09-11T12:49:41.422454+0000","last_fullsized":"2024-09-11T12:49:41.422454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416526+0000","last_change":"2024-09-11T12:49:41.416526+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416526+0000","last_undegraded":"2024-09-11T12:49:41.416526+0000","last_fullsized":"2024-09-11T12:49:41.416526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420940+0000","last_change":"2024-09-11T12:49:41.420940+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420940+0000","last_undegraded":"2024-09-11T12:49:41.420940+0000","last_fullsized":"2024-09-11T12:49:41.420940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416461+0000","last_change":"2024-09-11T12:49:41.416461+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416461+0000","last_undegraded":"2024-09-11T12:49:41.416461+0000","last_fullsized":"2024-09-11T12:49:41.416461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424304+0000","last_change":"2024-09-11T12:49:41.424304+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424304+0000","last_undegraded":"2024-09-11T12:49:41.424304+0000","last_fullsized":"2024-09-11T12:49:41.424304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420150+0000","last_change":"2024-09-11T12:49:41.420150+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420150+0000","last_undegraded":"2024-09-11T12:49:41.420150+0000","last_fullsized":"2024-09-11T12:49:41.420150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.417231+0000","last_change":"2024-09-11T12:49:41.417231+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.417231+0000","last_undegraded":"2024-09-11T12:49:41.417231+0000","last_fullsized":"2024-09-11T12:49:41.417231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421043+0000","last_change":"2024-09-11T12:49:41.421043+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421043+0000","last_undegraded":"2024-09-11T12:49:41.421043+0000","last_fullsized":"2024-09-11T12:49:41.421043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421911+0000","last_change":"2024-09-11T12:49:41.421911+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421911+0000","last_undegraded":"2024-09-11T12:49:41.421911+0000","last_fullsized":"2024-09-11T12:49:41.421911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416699+0000","last_change":"2024-09-11T12:49:41.416699+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416699+0000","last_undegraded":"2024-09-11T12:49:41.416699+0000","last_fullsized":"2024-09-11T12:49:41.416699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422904+0000","last_change":"2024-09-11T12:49:41.422904+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422904+0000","last_undegraded":"2024-09-11T12:49:41.422904+0000","last_fullsized":"2024-09-11T12:49:41.422904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421305+0000","last_change":"2024-09-11T12:49:41.421305+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421305+0000","last_undegraded":"2024-09-11T12:49:41.421305+0000","last_fullsized":"2024-09-11T12:49:41.421305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423234+0000","last_change":"2024-09-11T12:49:41.423234+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423234+0000","last_undegraded":"2024-09-11T12:49:41.423234+0000","last_fullsized":"2024-09-11T12:49:41.423234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T12:49:41.424690+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:41.424690+0000","last_peered":"2024-09-11T12:49:41.424690+0000","last_clean":"2024-09-11T12:49:41.424690+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:41.424690+0000","last_undegraded":"2024-09-11T12:49:41.424690+0000","last_fullsized":"2024-09-11T12:49:41.424690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.418918+0000","last_change":"2024-09-11T12:49:41.418918+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.418918+0000","last_undegraded":"2024-09-11T12:49:41.418918+0000","last_fullsized":"2024-09-11T12:49:41.418918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423038+0000","last_change":"2024-09-11T12:49:41.423038+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423038+0000","last_undegraded":"2024-09-11T12:49:41.423038+0000","last_fullsized":"2024-09-11T12:49:41.423038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422266+0000","last_change":"2024-09-11T12:49:41.422266+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422266+0000","last_undegraded":"2024-09-11T12:49:41.422266+0000","last_fullsized":"2024-09-11T12:49:41.422266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416548+0000","last_change":"2024-09-11T12:49:41.416548+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416548+0000","last_undegraded":"2024-09-11T12:49:41.416548+0000","last_fullsized":"2024-09-11T12:49:41.416548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420959+0000","last_change":"2024-09-11T12:49:41.420959+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420959+0000","last_undegraded":"2024-09-11T12:49:41.420959+0000","last_fullsized":"2024-09-11T12:49:41.420959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423313+0000","last_change":"2024-09-11T12:49:41.423313+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423313+0000","last_undegraded":"2024-09-11T12:49:41.423313+0000","last_fullsized":"2024-09-11T12:49:41.423313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421068+0000","last_change":"2024-09-11T12:49:41.421068+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421068+0000","last_undegraded":"2024-09-11T12:49:41.421068+0000","last_fullsized":"2024-09-11T12:49:41.421068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420228+0000","last_change":"2024-09-11T12:49:41.420228+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420228+0000","last_undegraded":"2024-09-11T12:49:41.420228+0000","last_fullsized":"2024-09-11T12:49:41.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423954+0000","last_change":"2024-09-11T12:49:41.423954+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423954+0000","last_undegraded":"2024-09-11T12:49:41.423954+0000","last_fullsized":"2024-09-11T12:49:41.423954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420014+0000","last_change":"2024-09-11T12:49:41.420014+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420014+0000","last_undegraded":"2024-09-11T12:49:41.420014+0000","last_fullsized":"2024-09-11T12:49:41.420014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.419869+0000","last_change":"2024-09-11T12:49:41.419869+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.419869+0000","last_undegraded":"2024-09-11T12:49:41.419869+0000","last_fullsized":"2024-09-11T12:49:41.419869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423321+0000","last_change":"2024-09-11T12:49:41.423321+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423321+0000","last_undegraded":"2024-09-11T12:49:41.423321+0000","last_fullsized":"2024-09-11T12:49:41.423321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423277+0000","last_change":"2024-09-11T12:49:41.423277+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423277+0000","last_undegraded":"2024-09-11T12:49:41.423277+0000","last_fullsized":"2024-09-11T12:49:41.423277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424152+0000","last_change":"2024-09-11T12:49:41.424152+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424152+0000","last_undegraded":"2024-09-11T12:49:41.424152+0000","last_fullsized":"2024-09-11T12:49:41.424152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421480+0000","last_change":"2024-09-11T12:49:41.421480+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421480+0000","last_undegraded":"2024-09-11T12:49:41.421480+0000","last_fullsized":"2024-09-11T12:49:41.421480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":651973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3f is not active+clean 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3e is not active+clean 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3c is not active+clean 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.315 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3b is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3a is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:PG 3.39 is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:PG 3.38 is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:PG 3.37 is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:PG 3.36 is not active+clean 2024-09-11T12:49:48.316 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.35 is not active+clean 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.34 is not active+clean 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.33 is not active+clean 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.31 is not active+clean 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.30 is not active+clean 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.317 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2f is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2e is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2d is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2c is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2a is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:PG 3.29 is not active+clean 2024-09-11T12:49:48.318 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.28 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.27 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.26 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.25 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.24 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:PG 3.22 is not active+clean 2024-09-11T12:49:48.319 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 3.21 is not active+clean 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 3.20 is not active+clean 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1a is not active+clean 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1b is not active+clean 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420936+0000', 'last_change': '2024-09-11T12:49:41.420936+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420936+0000', 'last_undegraded': '2024-09-11T12:49:41.420936+0000', 'last_fullsized': '2024-09-11T12:49:41.420936+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1a is not active+clean 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420195+0000', 'last_change': '2024-09-11T12:49:41.420195+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420195+0000', 'last_undegraded': '2024-09-11T12:49:41.420195+0000', 'last_fullsized': '2024-09-11T12:49:41.420195+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:49:48.320 INFO:tasks.ceph.ceph_manager.ceph:PG 2.19 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.422454+0000', 'last_change': '2024-09-11T12:49:41.422454+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.422454+0000', 'last_undegraded': '2024-09-11T12:49:41.422454+0000', 'last_fullsized': '2024-09-11T12:49:41.422454+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 3.18 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 2.18 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.416526+0000', 'last_change': '2024-09-11T12:49:41.416526+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.416526+0000', 'last_undegraded': '2024-09-11T12:49:41.416526+0000', 'last_fullsized': '2024-09-11T12:49:41.416526+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 3.19 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 2.17 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420940+0000', 'last_change': '2024-09-11T12:49:41.420940+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420940+0000', 'last_undegraded': '2024-09-11T12:49:41.420940+0000', 'last_fullsized': '2024-09-11T12:49:41.420940+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 3.16 is not active+clean 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.321 INFO:tasks.ceph.ceph_manager.ceph:PG 2.16 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.416461+0000', 'last_change': '2024-09-11T12:49:41.416461+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.416461+0000', 'last_undegraded': '2024-09-11T12:49:41.416461+0000', 'last_fullsized': '2024-09-11T12:49:41.416461+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:PG 2.15 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.424304+0000', 'last_change': '2024-09-11T12:49:41.424304+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.424304+0000', 'last_undegraded': '2024-09-11T12:49:41.424304+0000', 'last_fullsized': '2024-09-11T12:49:41.424304+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:PG 3.14 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:PG 2.14 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420150+0000', 'last_change': '2024-09-11T12:49:41.420150+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420150+0000', 'last_undegraded': '2024-09-11T12:49:41.420150+0000', 'last_fullsized': '2024-09-11T12:49:41.420150+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:PG 3.15 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:PG 2.12 is not active+clean 2024-09-11T12:49:48.322 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.417231+0000', 'last_change': '2024-09-11T12:49:41.417231+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.417231+0000', 'last_undegraded': '2024-09-11T12:49:41.417231+0000', 'last_fullsized': '2024-09-11T12:49:41.417231+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 3.13 is not active+clean 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 2.11 is not active+clean 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.421043+0000', 'last_change': '2024-09-11T12:49:41.421043+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.421043+0000', 'last_undegraded': '2024-09-11T12:49:41.421043+0000', 'last_fullsized': '2024-09-11T12:49:41.421043+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 3.10 is not active+clean 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 2.10 is not active+clean 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.421911+0000', 'last_change': '2024-09-11T12:49:41.421911+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.421911+0000', 'last_undegraded': '2024-09-11T12:49:41.421911+0000', 'last_fullsized': '2024-09-11T12:49:41.421911+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 3.11 is not active+clean 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.323 INFO:tasks.ceph.ceph_manager.ceph:PG 2.f is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.416699+0000', 'last_change': '2024-09-11T12:49:41.416699+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.416699+0000', 'last_undegraded': '2024-09-11T12:49:41.416699+0000', 'last_fullsized': '2024-09-11T12:49:41.416699+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:PG 3.e is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:PG 2.e is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.422904+0000', 'last_change': '2024-09-11T12:49:41.422904+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.422904+0000', 'last_undegraded': '2024-09-11T12:49:41.422904+0000', 'last_fullsized': '2024-09-11T12:49:41.422904+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:PG 3.f is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:PG 2.d is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.421305+0000', 'last_change': '2024-09-11T12:49:41.421305+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.421305+0000', 'last_undegraded': '2024-09-11T12:49:41.421305+0000', 'last_fullsized': '2024-09-11T12:49:41.421305+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:PG 3.c is not active+clean 2024-09-11T12:49:48.324 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:PG 2.c is not active+clean 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423234+0000', 'last_change': '2024-09-11T12:49:41.423234+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423234+0000', 'last_undegraded': '2024-09-11T12:49:41.423234+0000', 'last_fullsized': '2024-09-11T12:49:41.423234+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:PG 3.d is not active+clean 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:PG 2.3 is not active+clean 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.418918+0000', 'last_change': '2024-09-11T12:49:41.418918+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.418918+0000', 'last_undegraded': '2024-09-11T12:49:41.418918+0000', 'last_fullsized': '2024-09-11T12:49:41.418918+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:PG 3.2 is not active+clean 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:PG 2.0 is not active+clean 2024-09-11T12:49:48.325 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423038+0000', 'last_change': '2024-09-11T12:49:41.423038+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423038+0000', 'last_undegraded': '2024-09-11T12:49:41.423038+0000', 'last_fullsized': '2024-09-11T12:49:41.423038+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1 is not active+clean 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.422266+0000', 'last_change': '2024-09-11T12:49:41.422266+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.422266+0000', 'last_undegraded': '2024-09-11T12:49:41.422266+0000', 'last_fullsized': '2024-09-11T12:49:41.422266+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:PG 3.0 is not active+clean 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:PG 2.2 is not active+clean 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.416548+0000', 'last_change': '2024-09-11T12:49:41.416548+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.416548+0000', 'last_undegraded': '2024-09-11T12:49:41.416548+0000', 'last_fullsized': '2024-09-11T12:49:41.416548+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:PG 3.3 is not active+clean 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:PG 2.4 is not active+clean 2024-09-11T12:49:48.326 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420959+0000', 'last_change': '2024-09-11T12:49:41.420959+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420959+0000', 'last_undegraded': '2024-09-11T12:49:41.420959+0000', 'last_fullsized': '2024-09-11T12:49:41.420959+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 3.5 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 2.5 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423313+0000', 'last_change': '2024-09-11T12:49:41.423313+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423313+0000', 'last_undegraded': '2024-09-11T12:49:41.423313+0000', 'last_fullsized': '2024-09-11T12:49:41.423313+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 3.4 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 2.6 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.421068+0000', 'last_change': '2024-09-11T12:49:41.421068+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.421068+0000', 'last_undegraded': '2024-09-11T12:49:41.421068+0000', 'last_fullsized': '2024-09-11T12:49:41.421068+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 3.7 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:PG 2.7 is not active+clean 2024-09-11T12:49:48.327 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420228+0000', 'last_change': '2024-09-11T12:49:41.420228+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420228+0000', 'last_undegraded': '2024-09-11T12:49:41.420228+0000', 'last_fullsized': '2024-09-11T12:49:41.420228+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:PG 2.8 is not active+clean 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423954+0000', 'last_change': '2024-09-11T12:49:41.423954+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423954+0000', 'last_undegraded': '2024-09-11T12:49:41.423954+0000', 'last_fullsized': '2024-09-11T12:49:41.423954+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:PG 3.9 is not active+clean 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:PG 2.9 is not active+clean 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.420014+0000', 'last_change': '2024-09-11T12:49:41.420014+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.420014+0000', 'last_undegraded': '2024-09-11T12:49:41.420014+0000', 'last_fullsized': '2024-09-11T12:49:41.420014+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:PG 3.8 is not active+clean 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:PG 2.a is not active+clean 2024-09-11T12:49:48.328 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.419869+0000', 'last_change': '2024-09-11T12:49:41.419869+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.419869+0000', 'last_undegraded': '2024-09-11T12:49:41.419869+0000', 'last_fullsized': '2024-09-11T12:49:41.419869+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 3.b is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 2.b is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423321+0000', 'last_change': '2024-09-11T12:49:41.423321+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423321+0000', 'last_undegraded': '2024-09-11T12:49:41.423321+0000', 'last_fullsized': '2024-09-11T12:49:41.423321+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 3.a is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1d is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1c is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.423277+0000', 'last_change': '2024-09-11T12:49:41.423277+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.423277+0000', 'last_undegraded': '2024-09-11T12:49:41.423277+0000', 'last_fullsized': '2024-09-11T12:49:41.423277+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1c is not active+clean 2024-09-11T12:49:48.329 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1d is not active+clean 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.424152+0000', 'last_change': '2024-09-11T12:49:41.424152+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.424152+0000', 'last_undegraded': '2024-09-11T12:49:41.424152+0000', 'last_fullsized': '2024-09-11T12:49:41.424152+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:PG 2.1e is not active+clean 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '2.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 20, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:49:41.421480+0000', 'last_change': '2024-09-11T12:49:41.421480+0000', 'last_active': '2024-09-11T12:49:41.404553+0000', 'last_peered': '2024-09-11T12:49:41.404553+0000', 'last_clean': '2024-09-11T12:49:41.404553+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:41.421480+0000', 'last_undegraded': '2024-09-11T12:49:41.421480+0000', 'last_fullsized': '2024-09-11T12:49:41.421480+0000', 'mapping_epoch': 20, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 20, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:41.404553+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:PG 3.1e is not active+clean 2024-09-11T12:49:48.330 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '3.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-09-11T12:49:42.411555+0000', 'last_change': '2024-09-11T12:49:42.411555+0000', 'last_active': '2024-09-11T12:49:42.411555+0000', 'last_peered': '2024-09-11T12:49:42.411555+0000', 'last_clean': '2024-09-11T12:49:42.411555+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:49:42.411555+0000', 'last_undegraded': '2024-09-11T12:49:42.411555+0000', 'last_fullsized': '2024-09-11T12:49:42.411555+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'last_clean_scrub_stamp': '2024-09-11T12:49:42.411555+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-09-11T12:49:48.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- 172.21.15.78:0/3825987768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 msgr2=0x7f9de0004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 --2- 172.21.15.78:0/3825987768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9de0004020 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009b60 tx=0x7f9dec01d270 comp rx=0 tx=0).stop 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfbe00640 1 -- 172.21.15.78:0/3825987768 reap_dead start 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- 172.21.15.78:0/3825987768 shutdown_connections 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- 172.21.15.78:0/3825987768 >> 172.21.15.78:0/3825987768 conn(0x7f9dfc06ca80 msgr2=0x7f9dfc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- 172.21.15.78:0/3825987768 shutdown_connections 2024-09-11T12:49:48.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- 172.21.15.78:0/3825987768 wait complete. 2024-09-11T12:49:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 Processor -- start 2024-09-11T12:49:48.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- start start 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dfc164920 0x7f9dfc151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0b0990 0x7f9dfc0b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc1669a0 con 0x7f9de0003c20 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9dfc166820 con 0x7f9dfc0b0990 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9e02200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9dfc166b20 con 0x7f9dfc164920 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44456/0 (socket says 172.21.15.78:44456) 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 -- 172.21.15.78:0/3956174903 learned_addr learned my addr 172.21.15.78:0/3956174903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfbe00640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0b0990 0x7f9dfc0b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfaa00640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dfc164920 0x7f9dfc151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 -- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dfc164920 msgr2=0x7f9dfc151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dfc164920 0x7f9dfc151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 -- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0b0990 msgr2=0x7f9dfc0b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0b0990 0x7f9dfc0b0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.494+0000 7f9dfb400640 1 -- 172.21.15.78:0/3956174903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dfc0ceb70 con 0x7f9de0003c20 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9dfb400640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01d0a0 tx=0x7f9dec005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0b1040 con 0x7f9de0003c20 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9dec0bb070 con 0x7f9de0003c20 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dec004290 con 0x7f9de0003c20 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dfc0cdff0 con 0x7f9de0003c20 2024-09-11T12:49:48.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dfc0ce5b0 con 0x7f9de0003c20 2024-09-11T12:49:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9dfc06ef80 con 0x7f9de0003c20 2024-09-11T12:49:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f9dec0046d0 con 0x7f9de0003c20 2024-09-11T12:49:48.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 0x7f9ddc073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.498+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9dec12cba0 con 0x7f9de0003c20 2024-09-11T12:49:48.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.502+0000 7f9dfaa00640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 0x7f9ddc073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.502+0000 7f9dfaa00640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 0x7f9ddc073b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc1fac90 tx=0x7f9de8001d80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.502+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9dec016f80 con 0x7f9de0003c20 2024-09-11T12:49:48.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9dfc1583b0 con 0x7f9de0003c20 2024-09-11T12:49:48.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9df3400640 1 -- 172.21.15.78:0/3956174903 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1214 (secure 0 0 0) 0x7f9dec0f83b0 con 0x7f9de0003c20 2024-09-11T12:49:48.734 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:48.734 INFO:teuthology.orchestra.run.smithi078.stdout:{"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":30,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":8,"num_up_osds":8,"osd_up_since":1726058972,"num_in_osds":8,"osd_in_since":1726058961,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":54},{"state_name":"creating+peering","count":30},{"state_name":"active+clean","count":13}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":590984,"bytes_used":241270784,"bytes_avail":772852842496,"bytes_total":773094113280,"unknown_pgs_ratio":0.55670100450515747,"inactive_pgs_ratio":0.30927833914756775,"write_bytes_sec":1439,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"a","status":"up:active","gid":4390}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-11T12:49:17.892701+0000","services":{}},"progress_events":{}} 2024-09-11T12:49:48.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 msgr2=0x7f9ddc073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 0x7f9ddc073b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc1fac90 tx=0x7f9de8001d80 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 msgr2=0x7f9dfc0cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9dec01d0a0 tx=0x7f9dec005590 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 shutdown_connections 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9ddc0716b0 0x7f9ddc073b70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9dfc0b0990 0x7f9dfc0b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9dfc164920 0x7f9dfc151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 --2- 172.21.15.78:0/3956174903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9de0003c20 0x7f9dfc0cda10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 >> 172.21.15.78:0/3956174903 conn(0x7f9dfc06ca80 msgr2=0x7f9dfc1538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 shutdown_connections 2024-09-11T12:49:48.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.730+0000 7f9e02200640 1 -- 172.21.15.78:0/3956174903 wait complete. 2024-09-11T12:49:48.755 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-11T12:49:48.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 -- 172.21.15.78:0/3265759919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a015ab00 msgr2=0x7f90a015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 --2- 172.21.15.78:0/3265759919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a015ab00 0x7f90a015af40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9088009840 tx=0x7f908801c820 comp rx=0 tx=0).stop 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 -- 172.21.15.78:0/3265759919 shutdown_connections 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 --2- 172.21.15.78:0/3265759919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a015ab00 0x7f90a015af40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 --2- 172.21.15.78:0/3265759919 >> v2:172.21.15.157:3300/0 conn(0x7f90a0151650 0x7f90a00cfb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 -- 172.21.15.78:0/3265759919 >> 172.21.15.78:0/3265759919 conn(0x7f90a006d870 msgr2=0x7f90a006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:48.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 -- 172.21.15.78:0/3265759919 shutdown_connections 2024-09-11T12:49:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.890+0000 7f909fe00640 1 -- 172.21.15.78:0/3265759919 wait complete. 2024-09-11T12:49:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 Processor -- start 2024-09-11T12:49:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- start start 2024-09-11T12:49:48.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 0x7f90a00ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 0x7f90a01415e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f90a0167960 con 0x7f90a015ab00 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90a01677e0 con 0x7f90a0073260 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f90a0167ae0 con 0x7f90a0151650 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44470/0 (socket says 172.21.15.78:44470) 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 -- 172.21.15.78:0/4174530112 learned_addr learned my addr 172.21.15.78:0/4174530112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909ea00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 0x7f90a00ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f9097e00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 0x7f90a01415e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 -- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 msgr2=0x7f90a01415e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 0x7f90a01415e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 -- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 msgr2=0x7f90a00ce680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 0x7f90a00ce680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 -- 172.21.15.78:0/4174530112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90a0151d80 con 0x7f90a015ab00 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909ea00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 0x7f90a00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f9097e00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 0x7f90a01415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909f400640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f909800cbc0 tx=0x7f909800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098016020 con 0x7f90a015ab00 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f909800ab40 con 0x7f90a015ab00 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90a0152060 con 0x7f90a015ab00 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098012420 con 0x7f90a015ab00 2024-09-11T12:49:48.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.894+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90a0141b20 con 0x7f90a015ab00 2024-09-11T12:49:48.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90a00cfb20 con 0x7f90a015ab00 2024-09-11T12:49:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f9098019020 con 0x7f90a015ab00 2024-09-11T12:49:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f9096a00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 0x7f907c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9098091df0 con 0x7f90a015ab00 2024-09-11T12:49:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f909ea00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 0x7f907c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:48.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.898+0000 7f909ea00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 0x7f907c073b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90900058d0 tx=0x7f9090005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:48.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:48.902+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9098020030 con 0x7f90a015ab00 2024-09-11T12:49:49.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.070+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f90a0152230 con 0x7f907c071680 2024-09-11T12:49:49.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.074+0000 7f9096a00640 1 -- 172.21.15.78:0/4174530112 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+241621 (secure 0 0 0) 0x7f90a0152230 con 0x7f907c071680 2024-09-11T12:49:49.081 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 msgr2=0x7f907c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 0x7f907c073b40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f90900058d0 tx=0x7f9090005860 comp rx=0 tx=0).stop 2024-09-11T12:49:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 msgr2=0x7f90a014e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f909800cbc0 tx=0x7f909800ccb0 comp rx=0 tx=0).stop 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 shutdown_connections 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f907c071680 0x7f907c073b40 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90a015ab00 0x7f90a014e030 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90a0151650 0x7f90a01415e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 --2- 172.21.15.78:0/4174530112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90a0073260 0x7f90a00ce680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 >> 172.21.15.78:0/4174530112 conn(0x7f90a006d870 msgr2=0x7f90a014c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 shutdown_connections 2024-09-11T12:49:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:49.078+0000 7f909fe00640 1 -- 172.21.15.78:0/4174530112 wait complete. 2024-09-11T12:49:49.101 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:49:47.105337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590984,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":108,"num_write_kb":1636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":86,"acting":86,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":37,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235616,"kb_used_data":2744,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739104,"statfs":{"total":773094113280,"available":772852842496,"internally_reserved":0,"allocated":2809856,"data_stored":1693564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.691404"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420936+0000","last_change":"2024-09-11T12:49:41.420936+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420936+0000","last_undegraded":"2024-09-11T12:49:41.420936+0000","last_fullsized":"2024-09-11T12:49:41.420936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420195+0000","last_change":"2024-09-11T12:49:41.420195+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420195+0000","last_undegraded":"2024-09-11T12:49:41.420195+0000","last_fullsized":"2024-09-11T12:49:41.420195+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422454+0000","last_change":"2024-09-11T12:49:41.422454+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422454+0000","last_undegraded":"2024-09-11T12:49:41.422454+0000","last_fullsized":"2024-09-11T12:49:41.422454+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416526+0000","last_change":"2024-09-11T12:49:41.416526+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416526+0000","last_undegraded":"2024-09-11T12:49:41.416526+0000","last_fullsized":"2024-09-11T12:49:41.416526+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420940+0000","last_change":"2024-09-11T12:49:41.420940+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420940+0000","last_undegraded":"2024-09-11T12:49:41.420940+0000","last_fullsized":"2024-09-11T12:49:41.420940+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416461+0000","last_change":"2024-09-11T12:49:41.416461+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416461+0000","last_undegraded":"2024-09-11T12:49:41.416461+0000","last_fullsized":"2024-09-11T12:49:41.416461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424304+0000","last_change":"2024-09-11T12:49:41.424304+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424304+0000","last_undegraded":"2024-09-11T12:49:41.424304+0000","last_fullsized":"2024-09-11T12:49:41.424304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420150+0000","last_change":"2024-09-11T12:49:41.420150+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420150+0000","last_undegraded":"2024-09-11T12:49:41.420150+0000","last_fullsized":"2024-09-11T12:49:41.420150+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.417231+0000","last_change":"2024-09-11T12:49:41.417231+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.417231+0000","last_undegraded":"2024-09-11T12:49:41.417231+0000","last_fullsized":"2024-09-11T12:49:41.417231+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421043+0000","last_change":"2024-09-11T12:49:41.421043+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421043+0000","last_undegraded":"2024-09-11T12:49:41.421043+0000","last_fullsized":"2024-09-11T12:49:41.421043+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421911+0000","last_change":"2024-09-11T12:49:41.421911+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421911+0000","last_undegraded":"2024-09-11T12:49:41.421911+0000","last_fullsized":"2024-09-11T12:49:41.421911+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416699+0000","last_change":"2024-09-11T12:49:41.416699+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416699+0000","last_undegraded":"2024-09-11T12:49:41.416699+0000","last_fullsized":"2024-09-11T12:49:41.416699+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422904+0000","last_change":"2024-09-11T12:49:41.422904+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422904+0000","last_undegraded":"2024-09-11T12:49:41.422904+0000","last_fullsized":"2024-09-11T12:49:41.422904+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421305+0000","last_change":"2024-09-11T12:49:41.421305+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421305+0000","last_undegraded":"2024-09-11T12:49:41.421305+0000","last_fullsized":"2024-09-11T12:49:41.421305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423234+0000","last_change":"2024-09-11T12:49:41.423234+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423234+0000","last_undegraded":"2024-09-11T12:49:41.423234+0000","last_fullsized":"2024-09-11T12:49:41.423234+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T12:49:41.424690+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:41.424690+0000","last_peered":"2024-09-11T12:49:41.424690+0000","last_clean":"2024-09-11T12:49:41.424690+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:41.424690+0000","last_undegraded":"2024-09-11T12:49:41.424690+0000","last_fullsized":"2024-09-11T12:49:41.424690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.418918+0000","last_change":"2024-09-11T12:49:41.418918+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.418918+0000","last_undegraded":"2024-09-11T12:49:41.418918+0000","last_fullsized":"2024-09-11T12:49:41.418918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423038+0000","last_change":"2024-09-11T12:49:41.423038+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423038+0000","last_undegraded":"2024-09-11T12:49:41.423038+0000","last_fullsized":"2024-09-11T12:49:41.423038+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.422266+0000","last_change":"2024-09-11T12:49:41.422266+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.422266+0000","last_undegraded":"2024-09-11T12:49:41.422266+0000","last_fullsized":"2024-09-11T12:49:41.422266+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.416548+0000","last_change":"2024-09-11T12:49:41.416548+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.416548+0000","last_undegraded":"2024-09-11T12:49:41.416548+0000","last_fullsized":"2024-09-11T12:49:41.416548+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420959+0000","last_change":"2024-09-11T12:49:41.420959+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420959+0000","last_undegraded":"2024-09-11T12:49:41.420959+0000","last_fullsized":"2024-09-11T12:49:41.420959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423313+0000","last_change":"2024-09-11T12:49:41.423313+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423313+0000","last_undegraded":"2024-09-11T12:49:41.423313+0000","last_fullsized":"2024-09-11T12:49:41.423313+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421068+0000","last_change":"2024-09-11T12:49:41.421068+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421068+0000","last_undegraded":"2024-09-11T12:49:41.421068+0000","last_fullsized":"2024-09-11T12:49:41.421068+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420228+0000","last_change":"2024-09-11T12:49:41.420228+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420228+0000","last_undegraded":"2024-09-11T12:49:41.420228+0000","last_fullsized":"2024-09-11T12:49:41.420228+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423954+0000","last_change":"2024-09-11T12:49:41.423954+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423954+0000","last_undegraded":"2024-09-11T12:49:41.423954+0000","last_fullsized":"2024-09-11T12:49:41.423954+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.420014+0000","last_change":"2024-09-11T12:49:41.420014+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.420014+0000","last_undegraded":"2024-09-11T12:49:41.420014+0000","last_fullsized":"2024-09-11T12:49:41.420014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.419869+0000","last_change":"2024-09-11T12:49:41.419869+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.419869+0000","last_undegraded":"2024-09-11T12:49:41.419869+0000","last_fullsized":"2024-09-11T12:49:41.419869+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423321+0000","last_change":"2024-09-11T12:49:41.423321+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423321+0000","last_undegraded":"2024-09-11T12:49:41.423321+0000","last_fullsized":"2024-09-11T12:49:41.423321+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.423277+0000","last_change":"2024-09-11T12:49:41.423277+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.423277+0000","last_undegraded":"2024-09-11T12:49:41.423277+0000","last_fullsized":"2024-09-11T12:49:41.423277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.424152+0000","last_change":"2024-09-11T12:49:41.424152+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.424152+0000","last_undegraded":"2024-09-11T12:49:41.424152+0000","last_fullsized":"2024-09-11T12:49:41.424152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-11T12:49:41.421480+0000","last_change":"2024-09-11T12:49:41.421480+0000","last_active":"2024-09-11T12:49:41.404553+0000","last_peered":"2024-09-11T12:49:41.404553+0000","last_clean":"2024-09-11T12:49:41.404553+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:41.421480+0000","last_undegraded":"2024-09-11T12:49:41.421480+0000","last_fullsized":"2024-09-11T12:49:41.421480+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:49:42.411555+0000","last_change":"2024-09-11T12:49:42.411555+0000","last_active":"2024-09-11T12:49:42.411555+0000","last_peered":"2024-09-11T12:49:42.411555+0000","last_clean":"2024-09-11T12:49:42.411555+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:49:42.411555+0000","last_undegraded":"2024-09-11T12:49:42.411555+0000","last_fullsized":"2024-09-11T12:49:42.411555+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":20,"acting":20,"num_store_stats":1},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":616,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":790528,"data_stored":651973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":196608,"data_stored":61605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:52.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:52.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 -- 172.21.15.78:0/532140806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c740cf2f0 msgr2=0x7f9c741413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 --2- 172.21.15.78:0/532140806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c740cf2f0 0x7f9c741413d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9c6c009c80 tx=0x7f9c6c01cd10 comp rx=0 tx=0).stop 2024-09-11T12:49:52.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 -- 172.21.15.78:0/532140806 shutdown_connections 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 --2- 172.21.15.78:0/532140806 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c740cf2f0 0x7f9c741413d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 --2- 172.21.15.78:0/532140806 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c7415f930 0x7f9c7413efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 -- 172.21.15.78:0/532140806 >> 172.21.15.78:0/532140806 conn(0x7f9c740746c0 msgr2=0x7f9c740ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 -- 172.21.15.78:0/532140806 shutdown_connections 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.250+0000 7f9c73e00640 1 -- 172.21.15.78:0/532140806 wait complete. 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 Processor -- start 2024-09-11T12:49:52.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- start start 2024-09-11T12:49:52.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 0x7f9c7415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 0x7f9c74139630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74168bd0 con 0x7f9c7415f930 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9c74168a50 con 0x7f9c7415fd10 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9c74168d50 con 0x7f9c740cf2f0 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44496/0 (socket says 172.21.15.78:44496) 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 -- 172.21.15.78:0/431496044 learned_addr learned my addr 172.21.15.78:0/431496044 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73400640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 0x7f9c74139630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72a00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 0x7f9c7415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 -- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 msgr2=0x7f9c7415b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 0x7f9c7415b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 -- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 msgr2=0x7f9c74139630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 0x7f9c74139630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 -- 172.21.15.78:0/431496044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7415b410 con 0x7f9c7415f930 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73400640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 0x7f9c74139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72a00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 0x7f9c7415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c72000640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c3a0 tx=0x7f9c6000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60017070 con 0x7f9c7415f930 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7415b130 con 0x7f9c7415f930 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c600053e0 con 0x7f9c7415f930 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c60003880 con 0x7f9c7415f930 2024-09-11T12:49:52.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c7413cfa0 con 0x7f9c7415f930 2024-09-11T12:49:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.254+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c741413d0 con 0x7f9c7415f930 2024-09-11T12:49:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.258+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f9c6001a070 con 0x7f9c7415f930 2024-09-11T12:49:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.258+0000 7f9c6aa00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 0x7f9c54073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.258+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9c60091240 con 0x7f9c7415f930 2024-09-11T12:49:52.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.258+0000 7f9c72a00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 0x7f9c54073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.258+0000 7f9c72a00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 0x7f9c54073b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c640058d0 tx=0x7f9c64005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.262+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9c6005d020 con 0x7f9c7415f930 2024-09-11T12:49:52.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.426+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9c74137aa0 con 0x7f9c54071680 2024-09-11T12:49:52.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.430+0000 7f9c6aa00640 1 -- 172.21.15.78:0/431496044 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251937 (secure 0 0 0) 0x7f9c74137aa0 con 0x7f9c54071680 2024-09-11T12:49:52.437 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:52.438 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 msgr2=0x7f9c54073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 0x7f9c54073b40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9c640058d0 tx=0x7f9c64005860 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 msgr2=0x7f9c741390f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000c3a0 tx=0x7f9c6000c860 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 shutdown_connections 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9c54071680 0x7f9c54073b40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9c7415fd10 0x7f9c74139630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9c7415f930 0x7f9c741390f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 --2- 172.21.15.78:0/431496044 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9c740cf2f0 0x7f9c7415b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 >> 172.21.15.78:0/431496044 conn(0x7f9c740746c0 msgr2=0x7f9c7413db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:52.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 shutdown_connections 2024-09-11T12:49:52.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.434+0000 7f9c73e00640 1 -- 172.21.15.78:0/431496044 wait complete. 2024-09-11T12:49:52.458 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-11T12:49:51.106333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001846"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472816+0000","last_change":"2024-09-11T12:49:43.456400+0000","last_active":"2024-09-11T12:49:44.472816+0000","last_peered":"2024-09-11T12:49:44.472816+0000","last_clean":"2024-09-11T12:49:44.472816+0000","last_became_active":"2024-09-11T12:49:43.456074+0000","last_became_peered":"2024-09-11T12:49:43.456074+0000","last_unstale":"2024-09-11T12:49:44.472816+0000","last_undegraded":"2024-09-11T12:49:44.472816+0000","last_fullsized":"2024-09-11T12:49:44.472816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452157+0000","last_change":"2024-09-11T12:49:43.446993+0000","last_active":"2024-09-11T12:49:44.452157+0000","last_peered":"2024-09-11T12:49:44.452157+0000","last_clean":"2024-09-11T12:49:44.452157+0000","last_became_active":"2024-09-11T12:49:43.443935+0000","last_became_peered":"2024-09-11T12:49:43.443935+0000","last_unstale":"2024-09-11T12:49:44.452157+0000","last_undegraded":"2024-09-11T12:49:44.452157+0000","last_fullsized":"2024-09-11T12:49:44.452157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444935+0000","last_change":"2024-09-11T12:49:43.444872+0000","last_active":"2024-09-11T12:49:44.444935+0000","last_peered":"2024-09-11T12:49:44.444935+0000","last_clean":"2024-09-11T12:49:44.444935+0000","last_became_active":"2024-09-11T12:49:43.444523+0000","last_became_peered":"2024-09-11T12:49:43.444523+0000","last_unstale":"2024-09-11T12:49:44.444935+0000","last_undegraded":"2024-09-11T12:49:44.444935+0000","last_fullsized":"2024-09-11T12:49:44.444935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472516+0000","last_change":"2024-09-11T12:49:43.454578+0000","last_active":"2024-09-11T12:49:44.472516+0000","last_peered":"2024-09-11T12:49:44.472516+0000","last_clean":"2024-09-11T12:49:44.472516+0000","last_became_active":"2024-09-11T12:49:43.454355+0000","last_became_peered":"2024-09-11T12:49:43.454355+0000","last_unstale":"2024-09-11T12:49:44.472516+0000","last_undegraded":"2024-09-11T12:49:44.472516+0000","last_fullsized":"2024-09-11T12:49:44.472516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444824+0000","last_change":"2024-09-11T12:49:43.447252+0000","last_active":"2024-09-11T12:49:44.444824+0000","last_peered":"2024-09-11T12:49:44.444824+0000","last_clean":"2024-09-11T12:49:44.444824+0000","last_became_active":"2024-09-11T12:49:43.446860+0000","last_became_peered":"2024-09-11T12:49:43.446860+0000","last_unstale":"2024-09-11T12:49:44.444824+0000","last_undegraded":"2024-09-11T12:49:44.444824+0000","last_fullsized":"2024-09-11T12:49:44.444824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472261+0000","last_change":"2024-09-11T12:49:43.468669+0000","last_active":"2024-09-11T12:49:44.472261+0000","last_peered":"2024-09-11T12:49:44.472261+0000","last_clean":"2024-09-11T12:49:44.472261+0000","last_became_active":"2024-09-11T12:49:43.468316+0000","last_became_peered":"2024-09-11T12:49:43.468316+0000","last_unstale":"2024-09-11T12:49:44.472261+0000","last_undegraded":"2024-09-11T12:49:44.472261+0000","last_fullsized":"2024-09-11T12:49:44.472261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471049+0000","last_change":"2024-09-11T12:49:43.458945+0000","last_active":"2024-09-11T12:49:44.471049+0000","last_peered":"2024-09-11T12:49:44.471049+0000","last_clean":"2024-09-11T12:49:44.471049+0000","last_became_active":"2024-09-11T12:49:43.458677+0000","last_became_peered":"2024-09-11T12:49:43.458677+0000","last_unstale":"2024-09-11T12:49:44.471049+0000","last_undegraded":"2024-09-11T12:49:44.471049+0000","last_fullsized":"2024-09-11T12:49:44.471049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472868+0000","last_change":"2024-09-11T12:49:43.469306+0000","last_active":"2024-09-11T12:49:44.472868+0000","last_peered":"2024-09-11T12:49:44.472868+0000","last_clean":"2024-09-11T12:49:44.472868+0000","last_became_active":"2024-09-11T12:49:43.469031+0000","last_became_peered":"2024-09-11T12:49:43.469031+0000","last_unstale":"2024-09-11T12:49:44.472868+0000","last_undegraded":"2024-09-11T12:49:44.472868+0000","last_fullsized":"2024-09-11T12:49:44.472868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455525+0000","last_change":"2024-09-11T12:49:43.462111+0000","last_active":"2024-09-11T12:49:44.455525+0000","last_peered":"2024-09-11T12:49:44.455525+0000","last_clean":"2024-09-11T12:49:44.455525+0000","last_became_active":"2024-09-11T12:49:43.461818+0000","last_became_peered":"2024-09-11T12:49:43.461818+0000","last_unstale":"2024-09-11T12:49:44.455525+0000","last_undegraded":"2024-09-11T12:49:44.455525+0000","last_fullsized":"2024-09-11T12:49:44.455525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455751+0000","last_change":"2024-09-11T12:49:43.454080+0000","last_active":"2024-09-11T12:49:44.455751+0000","last_peered":"2024-09-11T12:49:44.455751+0000","last_clean":"2024-09-11T12:49:44.455751+0000","last_became_active":"2024-09-11T12:49:43.453788+0000","last_became_peered":"2024-09-11T12:49:43.453788+0000","last_unstale":"2024-09-11T12:49:44.455751+0000","last_undegraded":"2024-09-11T12:49:44.455751+0000","last_fullsized":"2024-09-11T12:49:44.455751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446203+0000","last_change":"2024-09-11T12:49:43.447103+0000","last_active":"2024-09-11T12:49:44.446203+0000","last_peered":"2024-09-11T12:49:44.446203+0000","last_clean":"2024-09-11T12:49:44.446203+0000","last_became_active":"2024-09-11T12:49:43.446839+0000","last_became_peered":"2024-09-11T12:49:43.446839+0000","last_unstale":"2024-09-11T12:49:44.446203+0000","last_undegraded":"2024-09-11T12:49:44.446203+0000","last_fullsized":"2024-09-11T12:49:44.446203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471850+0000","last_change":"2024-09-11T12:49:43.446894+0000","last_active":"2024-09-11T12:49:44.471850+0000","last_peered":"2024-09-11T12:49:44.471850+0000","last_clean":"2024-09-11T12:49:44.471850+0000","last_became_active":"2024-09-11T12:49:43.446045+0000","last_became_peered":"2024-09-11T12:49:43.446045+0000","last_unstale":"2024-09-11T12:49:44.471850+0000","last_undegraded":"2024-09-11T12:49:44.471850+0000","last_fullsized":"2024-09-11T12:49:44.471850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451685+0000","last_change":"2024-09-11T12:49:43.446706+0000","last_active":"2024-09-11T12:49:44.451685+0000","last_peered":"2024-09-11T12:49:44.451685+0000","last_clean":"2024-09-11T12:49:44.451685+0000","last_became_active":"2024-09-11T12:49:43.446340+0000","last_became_peered":"2024-09-11T12:49:43.446340+0000","last_unstale":"2024-09-11T12:49:44.451685+0000","last_undegraded":"2024-09-11T12:49:44.451685+0000","last_fullsized":"2024-09-11T12:49:44.451685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444746+0000","last_change":"2024-09-11T12:49:43.446371+0000","last_active":"2024-09-11T12:49:44.444746+0000","last_peered":"2024-09-11T12:49:44.444746+0000","last_clean":"2024-09-11T12:49:44.444746+0000","last_became_active":"2024-09-11T12:49:43.446163+0000","last_became_peered":"2024-09-11T12:49:43.446163+0000","last_unstale":"2024-09-11T12:49:44.444746+0000","last_undegraded":"2024-09-11T12:49:44.444746+0000","last_fullsized":"2024-09-11T12:49:44.444746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445420+0000","last_change":"2024-09-11T12:49:43.461338+0000","last_active":"2024-09-11T12:49:44.445420+0000","last_peered":"2024-09-11T12:49:44.445420+0000","last_clean":"2024-09-11T12:49:44.445420+0000","last_became_active":"2024-09-11T12:49:43.461065+0000","last_became_peered":"2024-09-11T12:49:43.461065+0000","last_unstale":"2024-09-11T12:49:44.445420+0000","last_undegraded":"2024-09-11T12:49:44.445420+0000","last_fullsized":"2024-09-11T12:49:44.445420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443267+0000","last_change":"2024-09-11T12:49:43.436172+0000","last_active":"2024-09-11T12:49:44.443267+0000","last_peered":"2024-09-11T12:49:44.443267+0000","last_clean":"2024-09-11T12:49:44.443267+0000","last_became_active":"2024-09-11T12:49:43.435740+0000","last_became_peered":"2024-09-11T12:49:43.435740+0000","last_unstale":"2024-09-11T12:49:44.443267+0000","last_undegraded":"2024-09-11T12:49:44.443267+0000","last_fullsized":"2024-09-11T12:49:44.443267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472800+0000","last_change":"2024-09-11T12:49:43.459483+0000","last_active":"2024-09-11T12:49:44.472800+0000","last_peered":"2024-09-11T12:49:44.472800+0000","last_clean":"2024-09-11T12:49:44.472800+0000","last_became_active":"2024-09-11T12:49:43.458688+0000","last_became_peered":"2024-09-11T12:49:43.458688+0000","last_unstale":"2024-09-11T12:49:44.472800+0000","last_undegraded":"2024-09-11T12:49:44.472800+0000","last_fullsized":"2024-09-11T12:49:44.472800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455412+0000","last_change":"2024-09-11T12:49:43.446362+0000","last_active":"2024-09-11T12:49:44.455412+0000","last_peered":"2024-09-11T12:49:44.455412+0000","last_clean":"2024-09-11T12:49:44.455412+0000","last_became_active":"2024-09-11T12:49:43.446092+0000","last_became_peered":"2024-09-11T12:49:43.446092+0000","last_unstale":"2024-09-11T12:49:44.455412+0000","last_undegraded":"2024-09-11T12:49:44.455412+0000","last_fullsized":"2024-09-11T12:49:44.455412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444759+0000","last_change":"2024-09-11T12:49:43.447439+0000","last_active":"2024-09-11T12:49:44.444759+0000","last_peered":"2024-09-11T12:49:44.444759+0000","last_clean":"2024-09-11T12:49:44.444759+0000","last_became_active":"2024-09-11T12:49:43.447044+0000","last_became_peered":"2024-09-11T12:49:43.447044+0000","last_unstale":"2024-09-11T12:49:44.444759+0000","last_undegraded":"2024-09-11T12:49:44.444759+0000","last_fullsized":"2024-09-11T12:49:44.444759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471454+0000","last_change":"2024-09-11T12:49:43.455751+0000","last_active":"2024-09-11T12:49:44.471454+0000","last_peered":"2024-09-11T12:49:44.471454+0000","last_clean":"2024-09-11T12:49:44.471454+0000","last_became_active":"2024-09-11T12:49:43.455514+0000","last_became_peered":"2024-09-11T12:49:43.455514+0000","last_unstale":"2024-09-11T12:49:44.471454+0000","last_undegraded":"2024-09-11T12:49:44.471454+0000","last_fullsized":"2024-09-11T12:49:44.471454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444864+0000","last_change":"2024-09-11T12:49:43.443468+0000","last_active":"2024-09-11T12:49:44.444864+0000","last_peered":"2024-09-11T12:49:44.444864+0000","last_clean":"2024-09-11T12:49:44.444864+0000","last_became_active":"2024-09-11T12:49:43.443158+0000","last_became_peered":"2024-09-11T12:49:43.443158+0000","last_unstale":"2024-09-11T12:49:44.444864+0000","last_undegraded":"2024-09-11T12:49:44.444864+0000","last_fullsized":"2024-09-11T12:49:44.444864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472404+0000","last_change":"2024-09-11T12:49:43.450952+0000","last_active":"2024-09-11T12:49:44.472404+0000","last_peered":"2024-09-11T12:49:44.472404+0000","last_clean":"2024-09-11T12:49:44.472404+0000","last_became_active":"2024-09-11T12:49:43.450620+0000","last_became_peered":"2024-09-11T12:49:43.450620+0000","last_unstale":"2024-09-11T12:49:44.472404+0000","last_undegraded":"2024-09-11T12:49:44.472404+0000","last_fullsized":"2024-09-11T12:49:44.472404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455599+0000","last_change":"2024-09-11T12:49:43.445613+0000","last_active":"2024-09-11T12:49:44.455599+0000","last_peered":"2024-09-11T12:49:44.455599+0000","last_clean":"2024-09-11T12:49:44.455599+0000","last_became_active":"2024-09-11T12:49:43.445164+0000","last_became_peered":"2024-09-11T12:49:43.445164+0000","last_unstale":"2024-09-11T12:49:44.455599+0000","last_undegraded":"2024-09-11T12:49:44.455599+0000","last_fullsized":"2024-09-11T12:49:44.455599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452041+0000","last_change":"2024-09-11T12:49:43.441250+0000","last_active":"2024-09-11T12:49:44.452041+0000","last_peered":"2024-09-11T12:49:44.452041+0000","last_clean":"2024-09-11T12:49:44.452041+0000","last_became_active":"2024-09-11T12:49:43.440991+0000","last_became_peered":"2024-09-11T12:49:43.440991+0000","last_unstale":"2024-09-11T12:49:44.452041+0000","last_undegraded":"2024-09-11T12:49:44.452041+0000","last_fullsized":"2024-09-11T12:49:44.452041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472314+0000","last_change":"2024-09-11T12:49:43.450922+0000","last_active":"2024-09-11T12:49:44.472314+0000","last_peered":"2024-09-11T12:49:44.472314+0000","last_clean":"2024-09-11T12:49:44.472314+0000","last_became_active":"2024-09-11T12:49:43.450635+0000","last_became_peered":"2024-09-11T12:49:43.450635+0000","last_unstale":"2024-09-11T12:49:44.472314+0000","last_undegraded":"2024-09-11T12:49:44.472314+0000","last_fullsized":"2024-09-11T12:49:44.472314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445843+0000","last_change":"2024-09-11T12:49:43.460072+0000","last_active":"2024-09-11T12:49:44.445843+0000","last_peered":"2024-09-11T12:49:44.445843+0000","last_clean":"2024-09-11T12:49:44.445843+0000","last_became_active":"2024-09-11T12:49:43.459042+0000","last_became_peered":"2024-09-11T12:49:43.459042+0000","last_unstale":"2024-09-11T12:49:44.445843+0000","last_undegraded":"2024-09-11T12:49:44.445843+0000","last_fullsized":"2024-09-11T12:49:44.445843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443060+0000","last_change":"2024-09-11T12:49:43.456433+0000","last_active":"2024-09-11T12:49:44.443060+0000","last_peered":"2024-09-11T12:49:44.443060+0000","last_clean":"2024-09-11T12:49:44.443060+0000","last_became_active":"2024-09-11T12:49:43.450398+0000","last_became_peered":"2024-09-11T12:49:43.450398+0000","last_unstale":"2024-09-11T12:49:44.443060+0000","last_undegraded":"2024-09-11T12:49:44.443060+0000","last_fullsized":"2024-09-11T12:49:44.443060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455308+0000","last_change":"2024-09-11T12:49:43.444430+0000","last_active":"2024-09-11T12:49:44.455308+0000","last_peered":"2024-09-11T12:49:44.455308+0000","last_clean":"2024-09-11T12:49:44.455308+0000","last_became_active":"2024-09-11T12:49:43.444155+0000","last_became_peered":"2024-09-11T12:49:43.444155+0000","last_unstale":"2024-09-11T12:49:44.455308+0000","last_undegraded":"2024-09-11T12:49:44.455308+0000","last_fullsized":"2024-09-11T12:49:44.455308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445847+0000","last_change":"2024-09-11T12:49:43.455676+0000","last_active":"2024-09-11T12:49:44.445847+0000","last_peered":"2024-09-11T12:49:44.445847+0000","last_clean":"2024-09-11T12:49:44.445847+0000","last_became_active":"2024-09-11T12:49:43.454645+0000","last_became_peered":"2024-09-11T12:49:43.454645+0000","last_unstale":"2024-09-11T12:49:44.445847+0000","last_undegraded":"2024-09-11T12:49:44.445847+0000","last_fullsized":"2024-09-11T12:49:44.445847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:46.861427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471955+0000","last_change":"2024-09-11T12:49:42.437775+0000","last_active":"2024-09-11T12:49:44.471955+0000","last_peered":"2024-09-11T12:49:44.471955+0000","last_clean":"2024-09-11T12:49:44.471955+0000","last_became_active":"2024-09-11T12:49:42.437412+0000","last_became_peered":"2024-09-11T12:49:42.437412+0000","last_unstale":"2024-09-11T12:49:44.471955+0000","last_undegraded":"2024-09-11T12:49:44.471955+0000","last_fullsized":"2024-09-11T12:49:44.471955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:50.316414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455174+0000","last_change":"2024-09-11T12:49:42.450097+0000","last_active":"2024-09-11T12:49:44.455174+0000","last_peered":"2024-09-11T12:49:44.455174+0000","last_clean":"2024-09-11T12:49:44.455174+0000","last_became_active":"2024-09-11T12:49:42.446539+0000","last_became_peered":"2024-09-11T12:49:42.446539+0000","last_unstale":"2024-09-11T12:49:44.455174+0000","last_undegraded":"2024-09-11T12:49:44.455174+0000","last_fullsized":"2024-09-11T12:49:44.455174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471482+0000","last_change":"2024-09-11T12:49:42.450513+0000","last_active":"2024-09-11T12:49:44.471482+0000","last_peered":"2024-09-11T12:49:44.471482+0000","last_clean":"2024-09-11T12:49:44.471482+0000","last_became_active":"2024-09-11T12:49:42.450293+0000","last_became_peered":"2024-09-11T12:49:42.450293+0000","last_unstale":"2024-09-11T12:49:44.471482+0000","last_undegraded":"2024-09-11T12:49:44.471482+0000","last_fullsized":"2024-09-11T12:49:44.471482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:07:25.000664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471230+0000","last_change":"2024-09-11T12:49:43.445707+0000","last_active":"2024-09-11T12:49:44.471230+0000","last_peered":"2024-09-11T12:49:44.471230+0000","last_clean":"2024-09-11T12:49:44.471230+0000","last_became_active":"2024-09-11T12:49:43.445299+0000","last_became_peered":"2024-09-11T12:49:43.445299+0000","last_unstale":"2024-09-11T12:49:44.471230+0000","last_undegraded":"2024-09-11T12:49:44.471230+0000","last_fullsized":"2024-09-11T12:49:44.471230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444806+0000","last_change":"2024-09-11T12:49:42.438052+0000","last_active":"2024-09-11T12:49:44.444806+0000","last_peered":"2024-09-11T12:49:44.444806+0000","last_clean":"2024-09-11T12:49:44.444806+0000","last_became_active":"2024-09-11T12:49:42.437781+0000","last_became_peered":"2024-09-11T12:49:42.437781+0000","last_unstale":"2024-09-11T12:49:44.444806+0000","last_undegraded":"2024-09-11T12:49:44.444806+0000","last_fullsized":"2024-09-11T12:49:44.444806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451699+0000","last_change":"2024-09-11T12:49:43.440390+0000","last_active":"2024-09-11T12:49:44.451699+0000","last_peered":"2024-09-11T12:49:44.451699+0000","last_clean":"2024-09-11T12:49:44.451699+0000","last_became_active":"2024-09-11T12:49:43.440072+0000","last_became_peered":"2024-09-11T12:49:43.440072+0000","last_unstale":"2024-09-11T12:49:44.451699+0000","last_undegraded":"2024-09-11T12:49:44.451699+0000","last_fullsized":"2024-09-11T12:49:44.451699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455307+0000","last_change":"2024-09-11T12:49:42.444008+0000","last_active":"2024-09-11T12:49:44.455307+0000","last_peered":"2024-09-11T12:49:44.455307+0000","last_clean":"2024-09-11T12:49:44.455307+0000","last_became_active":"2024-09-11T12:49:42.443388+0000","last_became_peered":"2024-09-11T12:49:42.443388+0000","last_unstale":"2024-09-11T12:49:44.455307+0000","last_undegraded":"2024-09-11T12:49:44.455307+0000","last_fullsized":"2024-09-11T12:49:44.455307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445126+0000","last_change":"2024-09-11T12:49:43.461238+0000","last_active":"2024-09-11T12:49:44.445126+0000","last_peered":"2024-09-11T12:49:44.445126+0000","last_clean":"2024-09-11T12:49:44.445126+0000","last_became_active":"2024-09-11T12:49:43.460886+0000","last_became_peered":"2024-09-11T12:49:43.460886+0000","last_unstale":"2024-09-11T12:49:44.445126+0000","last_undegraded":"2024-09-11T12:49:44.445126+0000","last_fullsized":"2024-09-11T12:49:44.445126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445162+0000","last_change":"2024-09-11T12:49:42.446611+0000","last_active":"2024-09-11T12:49:44.445162+0000","last_peered":"2024-09-11T12:49:44.445162+0000","last_clean":"2024-09-11T12:49:44.445162+0000","last_became_active":"2024-09-11T12:49:42.446350+0000","last_became_peered":"2024-09-11T12:49:42.446350+0000","last_unstale":"2024-09-11T12:49:44.445162+0000","last_undegraded":"2024-09-11T12:49:44.445162+0000","last_fullsized":"2024-09-11T12:49:44.445162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451977+0000","last_change":"2024-09-11T12:49:42.440747+0000","last_active":"2024-09-11T12:49:44.451977+0000","last_peered":"2024-09-11T12:49:44.451977+0000","last_clean":"2024-09-11T12:49:44.451977+0000","last_became_active":"2024-09-11T12:49:42.440400+0000","last_became_peered":"2024-09-11T12:49:42.440400+0000","last_unstale":"2024-09-11T12:49:44.451977+0000","last_undegraded":"2024-09-11T12:49:44.451977+0000","last_fullsized":"2024-09-11T12:49:44.451977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446290+0000","last_change":"2024-09-11T12:49:43.455119+0000","last_active":"2024-09-11T12:49:44.446290+0000","last_peered":"2024-09-11T12:49:44.446290+0000","last_clean":"2024-09-11T12:49:44.446290+0000","last_became_active":"2024-09-11T12:49:43.454634+0000","last_became_peered":"2024-09-11T12:49:43.454634+0000","last_unstale":"2024-09-11T12:49:44.446290+0000","last_undegraded":"2024-09-11T12:49:44.446290+0000","last_fullsized":"2024-09-11T12:49:44.446290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455328+0000","last_change":"2024-09-11T12:49:42.445483+0000","last_active":"2024-09-11T12:49:44.455328+0000","last_peered":"2024-09-11T12:49:44.455328+0000","last_clean":"2024-09-11T12:49:44.455328+0000","last_became_active":"2024-09-11T12:49:42.445158+0000","last_became_peered":"2024-09-11T12:49:42.445158+0000","last_unstale":"2024-09-11T12:49:44.455328+0000","last_undegraded":"2024-09-11T12:49:44.455328+0000","last_fullsized":"2024-09-11T12:49:44.455328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472598+0000","last_change":"2024-09-11T12:49:43.470186+0000","last_active":"2024-09-11T12:49:44.472598+0000","last_peered":"2024-09-11T12:49:44.472598+0000","last_clean":"2024-09-11T12:49:44.472598+0000","last_became_active":"2024-09-11T12:49:43.468620+0000","last_became_peered":"2024-09-11T12:49:43.468620+0000","last_unstale":"2024-09-11T12:49:44.472598+0000","last_undegraded":"2024-09-11T12:49:44.472598+0000","last_fullsized":"2024-09-11T12:49:44.472598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444877+0000","last_change":"2024-09-11T12:49:42.445232+0000","last_active":"2024-09-11T12:49:44.444877+0000","last_peered":"2024-09-11T12:49:44.444877+0000","last_clean":"2024-09-11T12:49:44.444877+0000","last_became_active":"2024-09-11T12:49:42.441896+0000","last_became_peered":"2024-09-11T12:49:42.441896+0000","last_unstale":"2024-09-11T12:49:44.444877+0000","last_undegraded":"2024-09-11T12:49:44.444877+0000","last_fullsized":"2024-09-11T12:49:44.444877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472600+0000","last_change":"2024-09-11T12:49:43.463066+0000","last_active":"2024-09-11T12:49:44.472600+0000","last_peered":"2024-09-11T12:49:44.472600+0000","last_clean":"2024-09-11T12:49:44.472600+0000","last_became_active":"2024-09-11T12:49:43.462868+0000","last_became_peered":"2024-09-11T12:49:43.462868+0000","last_unstale":"2024-09-11T12:49:44.472600+0000","last_undegraded":"2024-09-11T12:49:44.472600+0000","last_fullsized":"2024-09-11T12:49:44.472600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455115+0000","last_change":"2024-09-11T12:49:42.447532+0000","last_active":"2024-09-11T12:49:44.455115+0000","last_peered":"2024-09-11T12:49:44.455115+0000","last_clean":"2024-09-11T12:49:44.455115+0000","last_became_active":"2024-09-11T12:49:42.447217+0000","last_became_peered":"2024-09-11T12:49:42.447217+0000","last_unstale":"2024-09-11T12:49:44.455115+0000","last_undegraded":"2024-09-11T12:49:44.455115+0000","last_fullsized":"2024-09-11T12:49:44.455115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455296+0000","last_change":"2024-09-11T12:49:43.444203+0000","last_active":"2024-09-11T12:49:44.455296+0000","last_peered":"2024-09-11T12:49:44.455296+0000","last_clean":"2024-09-11T12:49:44.455296+0000","last_became_active":"2024-09-11T12:49:43.443909+0000","last_became_peered":"2024-09-11T12:49:43.443909+0000","last_unstale":"2024-09-11T12:49:44.455296+0000","last_undegraded":"2024-09-11T12:49:44.455296+0000","last_fullsized":"2024-09-11T12:49:44.455296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.442812+0000","last_change":"2024-09-11T12:49:42.428489+0000","last_active":"2024-09-11T12:49:44.442812+0000","last_peered":"2024-09-11T12:49:44.442812+0000","last_clean":"2024-09-11T12:49:44.442812+0000","last_became_active":"2024-09-11T12:49:42.427771+0000","last_became_peered":"2024-09-11T12:49:42.427771+0000","last_unstale":"2024-09-11T12:49:44.442812+0000","last_undegraded":"2024-09-11T12:49:44.442812+0000","last_fullsized":"2024-09-11T12:49:44.442812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472303+0000","last_change":"2024-09-11T12:49:43.463393+0000","last_active":"2024-09-11T12:49:44.472303+0000","last_peered":"2024-09-11T12:49:44.472303+0000","last_clean":"2024-09-11T12:49:44.472303+0000","last_became_active":"2024-09-11T12:49:43.462936+0000","last_became_peered":"2024-09-11T12:49:43.462936+0000","last_unstale":"2024-09-11T12:49:44.472303+0000","last_undegraded":"2024-09-11T12:49:44.472303+0000","last_fullsized":"2024-09-11T12:49:44.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446267+0000","last_change":"2024-09-11T12:49:42.440111+0000","last_active":"2024-09-11T12:49:44.446267+0000","last_peered":"2024-09-11T12:49:44.446267+0000","last_clean":"2024-09-11T12:49:44.446267+0000","last_became_active":"2024-09-11T12:49:42.438898+0000","last_became_peered":"2024-09-11T12:49:42.438898+0000","last_unstale":"2024-09-11T12:49:44.446267+0000","last_undegraded":"2024-09-11T12:49:44.446267+0000","last_fullsized":"2024-09-11T12:49:44.446267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:05.274573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472602+0000","last_change":"2024-09-11T12:49:43.457973+0000","last_active":"2024-09-11T12:49:44.472602+0000","last_peered":"2024-09-11T12:49:44.472602+0000","last_clean":"2024-09-11T12:49:44.472602+0000","last_became_active":"2024-09-11T12:49:43.457699+0000","last_became_peered":"2024-09-11T12:49:43.457699+0000","last_unstale":"2024-09-11T12:49:44.472602+0000","last_undegraded":"2024-09-11T12:49:44.472602+0000","last_fullsized":"2024-09-11T12:49:44.472602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443194+0000","last_change":"2024-09-11T12:49:42.438768+0000","last_active":"2024-09-11T12:49:44.443194+0000","last_peered":"2024-09-11T12:49:44.443194+0000","last_clean":"2024-09-11T12:49:44.443194+0000","last_became_active":"2024-09-11T12:49:42.438221+0000","last_became_peered":"2024-09-11T12:49:42.438221+0000","last_unstale":"2024-09-11T12:49:44.443194+0000","last_undegraded":"2024-09-11T12:49:44.443194+0000","last_fullsized":"2024-09-11T12:49:44.443194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472978+0000","last_change":"2024-09-11T12:49:43.463216+0000","last_active":"2024-09-11T12:49:44.472978+0000","last_peered":"2024-09-11T12:49:44.472978+0000","last_clean":"2024-09-11T12:49:44.472978+0000","last_became_active":"2024-09-11T12:49:43.462863+0000","last_became_peered":"2024-09-11T12:49:43.462863+0000","last_unstale":"2024-09-11T12:49:44.472978+0000","last_undegraded":"2024-09-11T12:49:44.472978+0000","last_fullsized":"2024-09-11T12:49:44.472978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452079+0000","last_change":"2024-09-11T12:49:42.430270+0000","last_active":"2024-09-11T12:49:44.452079+0000","last_peered":"2024-09-11T12:49:44.452079+0000","last_clean":"2024-09-11T12:49:44.452079+0000","last_became_active":"2024-09-11T12:49:42.429966+0000","last_became_peered":"2024-09-11T12:49:42.429966+0000","last_unstale":"2024-09-11T12:49:44.452079+0000","last_undegraded":"2024-09-11T12:49:44.452079+0000","last_fullsized":"2024-09-11T12:49:44.452079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445265+0000","last_change":"2024-09-11T12:49:43.445039+0000","last_active":"2024-09-11T12:49:44.445265+0000","last_peered":"2024-09-11T12:49:44.445265+0000","last_clean":"2024-09-11T12:49:44.445265+0000","last_became_active":"2024-09-11T12:49:43.443152+0000","last_became_peered":"2024-09-11T12:49:43.443152+0000","last_unstale":"2024-09-11T12:49:44.445265+0000","last_undegraded":"2024-09-11T12:49:44.445265+0000","last_fullsized":"2024-09-11T12:49:44.445265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443025+0000","last_change":"2024-09-11T12:49:42.440447+0000","last_active":"2024-09-11T12:49:44.443025+0000","last_peered":"2024-09-11T12:49:44.443025+0000","last_clean":"2024-09-11T12:49:44.443025+0000","last_became_active":"2024-09-11T12:49:42.439837+0000","last_became_peered":"2024-09-11T12:49:42.439837+0000","last_unstale":"2024-09-11T12:49:44.443025+0000","last_undegraded":"2024-09-11T12:49:44.443025+0000","last_fullsized":"2024-09-11T12:49:44.443025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472502+0000","last_change":"2024-09-11T12:49:43.470639+0000","last_active":"2024-09-11T12:49:44.472502+0000","last_peered":"2024-09-11T12:49:44.472502+0000","last_clean":"2024-09-11T12:49:44.472502+0000","last_became_active":"2024-09-11T12:49:43.468548+0000","last_became_peered":"2024-09-11T12:49:43.468548+0000","last_unstale":"2024-09-11T12:49:44.472502+0000","last_undegraded":"2024-09-11T12:49:44.472502+0000","last_fullsized":"2024-09-11T12:49:44.472502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472244+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:44.472244+0000","last_peered":"2024-09-11T12:49:44.472244+0000","last_clean":"2024-09-11T12:49:44.472244+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:44.472244+0000","last_undegraded":"2024-09-11T12:49:44.472244+0000","last_fullsized":"2024-09-11T12:49:44.472244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444614+0000","last_change":"2024-09-11T12:49:42.436719+0000","last_active":"2024-09-11T12:49:44.444614+0000","last_peered":"2024-09-11T12:49:44.444614+0000","last_clean":"2024-09-11T12:49:44.444614+0000","last_became_active":"2024-09-11T12:49:42.436415+0000","last_became_peered":"2024-09-11T12:49:42.436415+0000","last_unstale":"2024-09-11T12:49:44.444614+0000","last_undegraded":"2024-09-11T12:49:44.444614+0000","last_fullsized":"2024-09-11T12:49:44.444614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471442+0000","last_change":"2024-09-11T12:49:43.462048+0000","last_active":"2024-09-11T12:49:44.471442+0000","last_peered":"2024-09-11T12:49:44.471442+0000","last_clean":"2024-09-11T12:49:44.471442+0000","last_became_active":"2024-09-11T12:49:43.459912+0000","last_became_peered":"2024-09-11T12:49:43.459912+0000","last_unstale":"2024-09-11T12:49:44.471442+0000","last_undegraded":"2024-09-11T12:49:44.471442+0000","last_fullsized":"2024-09-11T12:49:44.471442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472116+0000","last_change":"2024-09-11T12:49:42.448655+0000","last_active":"2024-09-11T12:49:44.472116+0000","last_peered":"2024-09-11T12:49:44.472116+0000","last_clean":"2024-09-11T12:49:44.472116+0000","last_became_active":"2024-09-11T12:49:42.448394+0000","last_became_peered":"2024-09-11T12:49:42.448394+0000","last_unstale":"2024-09-11T12:49:44.472116+0000","last_undegraded":"2024-09-11T12:49:44.472116+0000","last_fullsized":"2024-09-11T12:49:44.472116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443004+0000","last_change":"2024-09-11T12:49:42.439971+0000","last_active":"2024-09-11T12:49:44.443004+0000","last_peered":"2024-09-11T12:49:44.443004+0000","last_clean":"2024-09-11T12:49:44.443004+0000","last_became_active":"2024-09-11T12:49:42.438664+0000","last_became_peered":"2024-09-11T12:49:42.438664+0000","last_unstale":"2024-09-11T12:49:44.443004+0000","last_undegraded":"2024-09-11T12:49:44.443004+0000","last_fullsized":"2024-09-11T12:49:44.443004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451753+0000","last_change":"2024-09-11T12:49:43.437660+0000","last_active":"2024-09-11T12:49:44.451753+0000","last_peered":"2024-09-11T12:49:44.451753+0000","last_clean":"2024-09-11T12:49:44.451753+0000","last_became_active":"2024-09-11T12:49:43.437383+0000","last_became_peered":"2024-09-11T12:49:43.437383+0000","last_unstale":"2024-09-11T12:49:44.451753+0000","last_undegraded":"2024-09-11T12:49:44.451753+0000","last_fullsized":"2024-09-11T12:49:44.451753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444974+0000","last_change":"2024-09-11T12:49:42.442628+0000","last_active":"2024-09-11T12:49:44.444974+0000","last_peered":"2024-09-11T12:49:44.444974+0000","last_clean":"2024-09-11T12:49:44.444974+0000","last_became_active":"2024-09-11T12:49:42.442343+0000","last_became_peered":"2024-09-11T12:49:42.442343+0000","last_unstale":"2024-09-11T12:49:44.444974+0000","last_undegraded":"2024-09-11T12:49:44.444974+0000","last_fullsized":"2024-09-11T12:49:44.444974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445987+0000","last_change":"2024-09-11T12:49:43.467784+0000","last_active":"2024-09-11T12:49:44.445987+0000","last_peered":"2024-09-11T12:49:44.445987+0000","last_clean":"2024-09-11T12:49:44.445987+0000","last_became_active":"2024-09-11T12:49:43.467380+0000","last_became_peered":"2024-09-11T12:49:43.467380+0000","last_unstale":"2024-09-11T12:49:44.445987+0000","last_undegraded":"2024-09-11T12:49:44.445987+0000","last_fullsized":"2024-09-11T12:49:44.445987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:39.392320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451933+0000","last_change":"2024-09-11T12:49:42.436463+0000","last_active":"2024-09-11T12:49:44.451933+0000","last_peered":"2024-09-11T12:49:44.451933+0000","last_clean":"2024-09-11T12:49:44.451933+0000","last_became_active":"2024-09-11T12:49:42.436201+0000","last_became_peered":"2024-09-11T12:49:42.436201+0000","last_unstale":"2024-09-11T12:49:44.451933+0000","last_undegraded":"2024-09-11T12:49:44.451933+0000","last_fullsized":"2024-09-11T12:49:44.451933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445376+0000","last_change":"2024-09-11T12:49:43.449369+0000","last_active":"2024-09-11T12:49:44.445376+0000","last_peered":"2024-09-11T12:49:44.445376+0000","last_clean":"2024-09-11T12:49:44.445376+0000","last_became_active":"2024-09-11T12:49:43.443602+0000","last_became_peered":"2024-09-11T12:49:43.443602+0000","last_unstale":"2024-09-11T12:49:44.445376+0000","last_undegraded":"2024-09-11T12:49:44.445376+0000","last_fullsized":"2024-09-11T12:49:44.445376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472687+0000","last_change":"2024-09-11T12:49:42.449197+0000","last_active":"2024-09-11T12:49:44.472687+0000","last_peered":"2024-09-11T12:49:44.472687+0000","last_clean":"2024-09-11T12:49:44.472687+0000","last_became_active":"2024-09-11T12:49:42.448893+0000","last_became_peered":"2024-09-11T12:49:42.448893+0000","last_unstale":"2024-09-11T12:49:44.472687+0000","last_undegraded":"2024-09-11T12:49:44.472687+0000","last_fullsized":"2024-09-11T12:49:44.472687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:35.796869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451779+0000","last_change":"2024-09-11T12:49:43.439241+0000","last_active":"2024-09-11T12:49:44.451779+0000","last_peered":"2024-09-11T12:49:44.451779+0000","last_clean":"2024-09-11T12:49:44.451779+0000","last_became_active":"2024-09-11T12:49:43.438486+0000","last_became_peered":"2024-09-11T12:49:43.438486+0000","last_unstale":"2024-09-11T12:49:44.451779+0000","last_undegraded":"2024-09-11T12:49:44.451779+0000","last_fullsized":"2024-09-11T12:49:44.451779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452121+0000","last_change":"2024-09-11T12:49:42.445521+0000","last_active":"2024-09-11T12:49:44.452121+0000","last_peered":"2024-09-11T12:49:44.452121+0000","last_clean":"2024-09-11T12:49:44.452121+0000","last_became_active":"2024-09-11T12:49:42.445218+0000","last_became_peered":"2024-09-11T12:49:42.445218+0000","last_unstale":"2024-09-11T12:49:44.452121+0000","last_undegraded":"2024-09-11T12:49:44.452121+0000","last_fullsized":"2024-09-11T12:49:44.452121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472486+0000","last_change":"2024-09-11T12:49:43.454081+0000","last_active":"2024-09-11T12:49:44.472486+0000","last_peered":"2024-09-11T12:49:44.472486+0000","last_clean":"2024-09-11T12:49:44.472486+0000","last_became_active":"2024-09-11T12:49:43.453820+0000","last_became_peered":"2024-09-11T12:49:43.453820+0000","last_unstale":"2024-09-11T12:49:44.472486+0000","last_undegraded":"2024-09-11T12:49:44.472486+0000","last_fullsized":"2024-09-11T12:49:44.472486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455425+0000","last_change":"2024-09-11T12:49:42.446529+0000","last_active":"2024-09-11T12:49:44.455425+0000","last_peered":"2024-09-11T12:49:44.455425+0000","last_clean":"2024-09-11T12:49:44.455425+0000","last_became_active":"2024-09-11T12:49:42.446268+0000","last_became_peered":"2024-09-11T12:49:42.446268+0000","last_unstale":"2024-09-11T12:49:44.455425+0000","last_undegraded":"2024-09-11T12:49:44.455425+0000","last_fullsized":"2024-09-11T12:49:44.455425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472114+0000","last_change":"2024-09-11T12:49:42.445423+0000","last_active":"2024-09-11T12:49:44.472114+0000","last_peered":"2024-09-11T12:49:44.472114+0000","last_clean":"2024-09-11T12:49:44.472114+0000","last_became_active":"2024-09-11T12:49:42.444814+0000","last_became_peered":"2024-09-11T12:49:42.444814+0000","last_unstale":"2024-09-11T12:49:44.472114+0000","last_undegraded":"2024-09-11T12:49:44.472114+0000","last_fullsized":"2024-09-11T12:49:44.472114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445597+0000","last_change":"2024-09-11T12:49:43.459523+0000","last_active":"2024-09-11T12:49:44.445597+0000","last_peered":"2024-09-11T12:49:44.445597+0000","last_clean":"2024-09-11T12:49:44.445597+0000","last_became_active":"2024-09-11T12:49:43.459164+0000","last_became_peered":"2024-09-11T12:49:43.459164+0000","last_unstale":"2024-09-11T12:49:44.445597+0000","last_undegraded":"2024-09-11T12:49:44.445597+0000","last_fullsized":"2024-09-11T12:49:44.445597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:50.017910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451744+0000","last_change":"2024-09-11T12:49:42.431712+0000","last_active":"2024-09-11T12:49:44.451744+0000","last_peered":"2024-09-11T12:49:44.451744+0000","last_clean":"2024-09-11T12:49:44.451744+0000","last_became_active":"2024-09-11T12:49:42.431429+0000","last_became_peered":"2024-09-11T12:49:42.431429+0000","last_unstale":"2024-09-11T12:49:44.451744+0000","last_undegraded":"2024-09-11T12:49:44.451744+0000","last_fullsized":"2024-09-11T12:49:44.451744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471398+0000","last_change":"2024-09-11T12:49:43.444653+0000","last_active":"2024-09-11T12:49:44.471398+0000","last_peered":"2024-09-11T12:49:44.471398+0000","last_clean":"2024-09-11T12:49:44.471398+0000","last_became_active":"2024-09-11T12:49:43.444206+0000","last_became_peered":"2024-09-11T12:49:43.444206+0000","last_unstale":"2024-09-11T12:49:44.471398+0000","last_undegraded":"2024-09-11T12:49:44.471398+0000","last_fullsized":"2024-09-11T12:49:44.471398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451809+0000","last_change":"2024-09-11T12:49:42.439128+0000","last_active":"2024-09-11T12:49:44.451809+0000","last_peered":"2024-09-11T12:49:44.451809+0000","last_clean":"2024-09-11T12:49:44.451809+0000","last_became_active":"2024-09-11T12:49:42.438820+0000","last_became_peered":"2024-09-11T12:49:42.438820+0000","last_unstale":"2024-09-11T12:49:44.451809+0000","last_undegraded":"2024-09-11T12:49:44.451809+0000","last_fullsized":"2024-09-11T12:49:44.451809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471928+0000","last_change":"2024-09-11T12:49:43.447109+0000","last_active":"2024-09-11T12:49:44.471928+0000","last_peered":"2024-09-11T12:49:44.471928+0000","last_clean":"2024-09-11T12:49:44.471928+0000","last_became_active":"2024-09-11T12:49:43.446540+0000","last_became_peered":"2024-09-11T12:49:43.446540+0000","last_unstale":"2024-09-11T12:49:44.471928+0000","last_undegraded":"2024-09-11T12:49:44.471928+0000","last_fullsized":"2024-09-11T12:49:44.471928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472587+0000","last_change":"2024-09-11T12:49:42.447202+0000","last_active":"2024-09-11T12:49:44.472587+0000","last_peered":"2024-09-11T12:49:44.472587+0000","last_clean":"2024-09-11T12:49:44.472587+0000","last_became_active":"2024-09-11T12:49:42.446936+0000","last_became_peered":"2024-09-11T12:49:42.446936+0000","last_unstale":"2024-09-11T12:49:44.472587+0000","last_undegraded":"2024-09-11T12:49:44.472587+0000","last_fullsized":"2024-09-11T12:49:44.472587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455309+0000","last_change":"2024-09-11T12:49:43.454367+0000","last_active":"2024-09-11T12:49:44.455309+0000","last_peered":"2024-09-11T12:49:44.455309+0000","last_clean":"2024-09-11T12:49:44.455309+0000","last_became_active":"2024-09-11T12:49:43.454114+0000","last_became_peered":"2024-09-11T12:49:43.454114+0000","last_unstale":"2024-09-11T12:49:44.455309+0000","last_undegraded":"2024-09-11T12:49:44.455309+0000","last_fullsized":"2024-09-11T12:49:44.455309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445226+0000","last_change":"2024-09-11T12:49:43.449972+0000","last_active":"2024-09-11T12:49:44.445226+0000","last_peered":"2024-09-11T12:49:44.445226+0000","last_clean":"2024-09-11T12:49:44.445226+0000","last_became_active":"2024-09-11T12:49:43.449653+0000","last_became_peered":"2024-09-11T12:49:43.449653+0000","last_unstale":"2024-09-11T12:49:44.445226+0000","last_undegraded":"2024-09-11T12:49:44.445226+0000","last_fullsized":"2024-09-11T12:49:44.445226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472376+0000","last_change":"2024-09-11T12:49:42.443992+0000","last_active":"2024-09-11T12:49:44.472376+0000","last_peered":"2024-09-11T12:49:44.472376+0000","last_clean":"2024-09-11T12:49:44.472376+0000","last_became_active":"2024-09-11T12:49:42.443492+0000","last_became_peered":"2024-09-11T12:49:42.443492+0000","last_unstale":"2024-09-11T12:49:44.472376+0000","last_undegraded":"2024-09-11T12:49:44.472376+0000","last_fullsized":"2024-09-11T12:49:44.472376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:07.781904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445104+0000","last_change":"2024-09-11T12:49:43.449359+0000","last_active":"2024-09-11T12:49:44.445104+0000","last_peered":"2024-09-11T12:49:44.445104+0000","last_clean":"2024-09-11T12:49:44.445104+0000","last_became_active":"2024-09-11T12:49:43.448838+0000","last_became_peered":"2024-09-11T12:49:43.448838+0000","last_unstale":"2024-09-11T12:49:44.445104+0000","last_undegraded":"2024-09-11T12:49:44.445104+0000","last_fullsized":"2024-09-11T12:49:44.445104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472595+0000","last_change":"2024-09-11T12:49:42.450569+0000","last_active":"2024-09-11T12:49:44.472595+0000","last_peered":"2024-09-11T12:49:44.472595+0000","last_clean":"2024-09-11T12:49:44.472595+0000","last_became_active":"2024-09-11T12:49:42.450338+0000","last_became_peered":"2024-09-11T12:49:42.450338+0000","last_unstale":"2024-09-11T12:49:44.472595+0000","last_undegraded":"2024-09-11T12:49:44.472595+0000","last_fullsized":"2024-09-11T12:49:44.472595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472335+0000","last_change":"2024-09-11T12:49:42.446328+0000","last_active":"2024-09-11T12:49:44.472335+0000","last_peered":"2024-09-11T12:49:44.472335+0000","last_clean":"2024-09-11T12:49:44.472335+0000","last_became_active":"2024-09-11T12:49:42.446059+0000","last_became_peered":"2024-09-11T12:49:42.446059+0000","last_unstale":"2024-09-11T12:49:44.472335+0000","last_undegraded":"2024-09-11T12:49:44.472335+0000","last_fullsized":"2024-09-11T12:49:44.472335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472359+0000","last_change":"2024-09-11T12:49:43.443985+0000","last_active":"2024-09-11T12:49:44.472359+0000","last_peered":"2024-09-11T12:49:44.472359+0000","last_clean":"2024-09-11T12:49:44.472359+0000","last_became_active":"2024-09-11T12:49:43.443740+0000","last_became_peered":"2024-09-11T12:49:43.443740+0000","last_unstale":"2024-09-11T12:49:44.472359+0000","last_undegraded":"2024-09-11T12:49:44.472359+0000","last_fullsized":"2024-09-11T12:49:44.472359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509445,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:52.462 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:49:52.463 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-11T12:49:52.463 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- 172.21.15.78:0/2499705643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 msgr2=0x7f1b280b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/2499705643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b280b5640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1b20009900 tx=0x7f1b2001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- 172.21.15.78:0/2499705643 shutdown_connections 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/2499705643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b280b5640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- 172.21.15.78:0/2499705643 >> 172.21.15.78:0/2499705643 conn(0x7f1b280b1b50 msgr2=0x7f1b28070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- 172.21.15.78:0/2499705643 shutdown_connections 2024-09-11T12:49:52.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- 172.21.15.78:0/2499705643 wait complete. 2024-09-11T12:49:52.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 Processor -- start 2024-09-11T12:49:52.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- start start 2024-09-11T12:49:52.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 0x7f1b281323d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b280d1740 0x7f1b28133330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28169ae0 con 0x7f1b280b33e0 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1b28169960 con 0x7f1b280b2f70 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.638+0000 7f1b2cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1b28169c60 con 0x7f1b280d1740 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44508/0 (socket says 172.21.15.78:44508) 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 -- 172.21.15.78:0/3183688436 learned_addr learned my addr 172.21.15.78:0/3183688436 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25e00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 0x7f1b281323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b26800640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b280d1740 0x7f1b28133330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 -- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b280d1740 msgr2=0x7f1b28133330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b280d1740 0x7f1b28133330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 -- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 msgr2=0x7f1b281323d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 0x7f1b281323d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 -- 172.21.15.78:0/3183688436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b281417e0 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25e00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 0x7f1b281323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b25400640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c4b0 tx=0x7f1b1400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b140052d0 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b14005470 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b14012870 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b28141a80 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b28141d60 con 0x7f1b280b33e0 2024-09-11T12:49:52.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2815bfc0 con 0x7f1b280b33e0 2024-09-11T12:49:52.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.642+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f1b14003600 con 0x7f1b280b33e0 2024-09-11T12:49:52.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.646+0000 7f1b1e000640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 0x7f1b08073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.646+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1b14091960 con 0x7f1b280b33e0 2024-09-11T12:49:52.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.646+0000 7f1b25e00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 0x7f1b08073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.646+0000 7f1b25e00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 0x7f1b08073b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b18001db0 tx=0x7f1b18009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.650+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1b1405d730 con 0x7f1b280b33e0 2024-09-11T12:49:52.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.814+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f1b28152c40 con 0x7f1b280b33e0 2024-09-11T12:49:52.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.814+0000 7f1b1e000640 1 -- 172.21.15.78:0/3183688436 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f1b1405d030 con 0x7f1b280b33e0 2024-09-11T12:49:52.820 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 msgr2=0x7f1b08073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 0x7f1b08073b70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1b18001db0 tx=0x7f1b18009040 comp rx=0 tx=0).stop 2024-09-11T12:49:52.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 msgr2=0x7f1b28151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1b1400c4b0 tx=0x7f1b1400c970 comp rx=0 tx=0).stop 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 shutdown_connections 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1b080716b0 0x7f1b08073b70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1b280d1740 0x7f1b28133330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1b280b33e0 0x7f1b28151c90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 --2- 172.21.15.78:0/3183688436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1b280b2f70 0x7f1b281323d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 >> 172.21.15.78:0/3183688436 conn(0x7f1b280b1b50 msgr2=0x7f1b2815b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 shutdown_connections 2024-09-11T12:49:52.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.818+0000 7f1b2cc00640 1 -- 172.21.15.78:0/3183688436 wait complete. 2024-09-11T12:49:52.841 INFO:teuthology.orchestra.run.smithi078.stdout:[{"version":27,"timestamp":"2024-09-11T12:49:31.276767+0000","name":"","changes":[{"name":"osd.1/osd_mclock_max_capacity_iops_ssd","new_value":"4602.351245"}]}] 2024-09-11T12:49:52.842 INFO:tasks.ceph_manager:config epoch is 27 2024-09-11T12:49:52.842 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T12:49:52.842 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T12:49:52.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T12:49:52.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- 172.21.15.78:0/1894684685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073430 msgr2=0x7f64d815fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 --2- 172.21.15.78:0/1894684685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073430 0x7f64d815fd90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f64cc009840 tx=0x7f64cc01c790 comp rx=0 tx=0).stop 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- 172.21.15.78:0/1894684685 shutdown_connections 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 --2- 172.21.15.78:0/1894684685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073430 0x7f64d815fd90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- 172.21.15.78:0/1894684685 >> 172.21.15.78:0/1894684685 conn(0x7f64d806d870 msgr2=0x7f64d806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- 172.21.15.78:0/1894684685 shutdown_connections 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- 172.21.15.78:0/1894684685 wait complete. 2024-09-11T12:49:52.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 Processor -- start 2024-09-11T12:49:52.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.978+0000 7f64dec00640 1 -- start start 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d8073430 0x7f64d80cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d81454d0 0x7f64d81593a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64d81698b0 con 0x7f64d8073810 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64d8169730 con 0x7f64d81454d0 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64d8169a30 con 0x7f64d8073430 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44520/0 (socket says 172.21.15.78:44520) 2024-09-11T12:49:52.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 -- 172.21.15.78:0/1289787425 learned_addr learned my addr 172.21.15.78:0/1289787425 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dd800640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d8073430 0x7f64d80cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64de200640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d81454d0 0x7f64d81593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 -- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d8073430 msgr2=0x7f64d80cefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d8073430 0x7f64d80cefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 -- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d81454d0 msgr2=0x7f64d81593a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d81454d0 0x7f64d81593a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 -- 172.21.15.78:0/1289787425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d81436c0 con 0x7f64d8073810 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dce00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c450 tx=0x7f64c400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4012830 con 0x7f64d8073810 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64c4017070 con 0x7f64d8073810 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64c4012d80 con 0x7f64d8073810 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d813f1f0 con 0x7f64d8073810 2024-09-11T12:49:52.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d813f7b0 con 0x7f64d8073810 2024-09-11T12:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d8069450 con 0x7f64d8073810 2024-09-11T12:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.982+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f64c401c460 con 0x7f64d8073810 2024-09-11T12:49:52.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.986+0000 7f64d5600640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 0x7f64b8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.986+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f64c4091ab0 con 0x7f64d8073810 2024-09-11T12:49:52.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.986+0000 7f64dd800640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 0x7f64b8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.986+0000 7f64dd800640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 0x7f64b8073b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0af000 tx=0x7f64cc01afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:52.990+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f64c405d2c0 con 0x7f64d8073810 2024-09-11T12:49:53.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.198+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f64d815e580 con 0x7f64d8073810 2024-09-11T12:49:53.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.202+0000 7f64d5600640 1 -- 172.21.15.78:0/1289787425 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+182846 (secure 0 0 0) 0x7f64c4061810 con 0x7f64d8073810 2024-09-11T12:49:53.208 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:53.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 msgr2=0x7f64b8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 0x7f64b8073b40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f64cc0af000 tx=0x7f64cc01afd0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 msgr2=0x7f64d8142f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c450 tx=0x7f64c400c910 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 shutdown_connections 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64b8071680 0x7f64b8073b40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64d81454d0 0x7f64d81593a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d8073810 0x7f64d8142f50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 --2- 172.21.15.78:0/1289787425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64d8073430 0x7f64d80cefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 >> 172.21.15.78:0/1289787425 conn(0x7f64d806d870 msgr2=0x7f64d815de70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 shutdown_connections 2024-09-11T12:49:53.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.206+0000 7f64dec00640 1 -- 172.21.15.78:0/1289787425 wait complete. 2024-09-11T12:49:53.229 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":855997071},{"type":"v1","addr":"172.21.15.157:6801","nonce":855997071}]},"active_addr":"172.21.15.157:6801/855997071","active_change":"2024-09-11T12:49:23.091583+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":3934951986}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":2095507505}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":8749197}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.157:0","nonce":4008390568}]}]} 2024-09-11T12:49:53.232 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T12:49:53.232 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T12:49:53.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:53.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- 172.21.15.78:0/1439103617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce40aed20 msgr2=0x7f4ce40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/1439103617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce40aed20 0x7f4ce40cf280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009900 tx=0x7f4cd401c8b0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- 172.21.15.78:0/1439103617 shutdown_connections 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/1439103617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 0x7f4ce415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/1439103617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce40aed20 0x7f4ce40cf280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- 172.21.15.78:0/1439103617 >> 172.21.15.78:0/1439103617 conn(0x7f4ce406ca80 msgr2=0x7f4ce40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- 172.21.15.78:0/1439103617 shutdown_connections 2024-09-11T12:49:53.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- 172.21.15.78:0/1439103617 wait complete. 2024-09-11T12:49:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 Processor -- start 2024-09-11T12:49:53.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- start start 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 0x7f4ce41377a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 0x7f4ce4139800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce40cd0d0 con 0x7f4ce415d010 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce4152da0 con 0x7f4ce40aed20 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.370+0000 7f4ce9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce40cd250 con 0x7f4ce40cf7c0 2024-09-11T12:49:53.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 0x7f4ce4139800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:44988/0 (socket says 172.21.15.78:44988) 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce2a00640 1 -- 172.21.15.78:0/986057820 learned_addr learned my addr 172.21.15.78:0/986057820 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4cda000640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 0x7f4ce41377a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 -- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 msgr2=0x7f4ce41377a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 0x7f4ce41377a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 -- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 msgr2=0x7f4ce41370d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 -- 172.21.15.78:0/986057820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce4068b40 con 0x7f4ce415d010 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce2a00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce3400640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 0x7f4ce4139800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c730 tx=0x7f4cdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc003b80 con 0x7f4ce415d010 2024-09-11T12:49:53.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cdc003d20 con 0x7f4ce415d010 2024-09-11T12:49:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cdc005330 con 0x7f4ce415d010 2024-09-11T12:49:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce40cf590 con 0x7f4ce415d010 2024-09-11T12:49:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce413a210 con 0x7f4ce415d010 2024-09-11T12:49:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce40cf280 con 0x7f4ce415d010 2024-09-11T12:49:53.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.374+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4cdc0054d0 con 0x7f4ce415d010 2024-09-11T12:49:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.378+0000 7f4cdbe00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 0x7f4cc0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.378+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4cdc0919c0 con 0x7f4ce415d010 2024-09-11T12:49:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.378+0000 7f4ce2a00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 0x7f4cc0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.378+0000 7f4ce2a00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 0x7f4cc0073b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cd401af60 tx=0x7f4cd401b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.382+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4cdc05d250 con 0x7f4ce415d010 2024-09-11T12:49:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.550+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ce40b1050 con 0x7f4ce415d010 2024-09-11T12:49:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.550+0000 7f4cdbe00640 1 -- 172.21.15.78:0/986057820 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f4cdc0617a0 con 0x7f4ce415d010 2024-09-11T12:49:53.556 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:53.556 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 msgr2=0x7f4cc0073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 0x7f4cc0073b40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4cd401af60 tx=0x7f4cd401b0f0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 msgr2=0x7f4ce4139800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 0x7f4ce4139800 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4cdc00c730 tx=0x7f4cdc00cbf0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 shutdown_connections 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4cc0071680 0x7f4cc0073b40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce415d010 0x7f4ce4139800 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce40cf7c0 0x7f4ce41377a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 --2- 172.21.15.78:0/986057820 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce40aed20 0x7f4ce41370d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 >> 172.21.15.78:0/986057820 conn(0x7f4ce406ca80 msgr2=0x7f4ce414ec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:53.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 shutdown_connections 2024-09-11T12:49:53.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.554+0000 7f4ce9e00640 1 -- 172.21.15.78:0/986057820 wait complete. 2024-09-11T12:49:53.577 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T12:49:53.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- 172.21.15.78:0/79213629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e580b4780 msgr2=0x7f0e58073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 --2- 172.21.15.78:0/79213629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e580b4780 0x7f0e58073f70 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0e48009bf0 tx=0x7f0e4801cbe0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- 172.21.15.78:0/79213629 shutdown_connections 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 --2- 172.21.15.78:0/79213629 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e580b4780 0x7f0e58073f70 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- 172.21.15.78:0/79213629 >> 172.21.15.78:0/79213629 conn(0x7f0e5806ca80 msgr2=0x7f0e580b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- 172.21.15.78:0/79213629 shutdown_connections 2024-09-11T12:49:53.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- 172.21.15.78:0/79213629 wait complete. 2024-09-11T12:49:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 Processor -- start 2024-09-11T12:49:53.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.714+0000 7f0e5e000640 1 -- start start 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 0x7f0e5813e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815bc60 0x7f0e581541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58169ed0 con 0x7f0e580ccb20 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e58169d50 con 0x7f0e580b4780 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e5816a050 con 0x7f0e5815bc60 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44546/0 (socket says 172.21.15.78:44546) 2024-09-11T12:49:53.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 -- 172.21.15.78:0/3999333807 learned_addr learned my addr 172.21.15.78:0/3999333807 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e57400640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 0x7f0e5813e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e57e00640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815bc60 0x7f0e581541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 -- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815bc60 msgr2=0x7f0e581541f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815bc60 0x7f0e581541f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 -- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 msgr2=0x7f0e5813e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 0x7f0e5813e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 -- 172.21.15.78:0/3999333807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e58157260 con 0x7f0e580ccb20 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e57400640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 0x7f0e5813e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e56a00640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c410 tx=0x7f0e4400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:53.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011790 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e44017070 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e581569c0 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44011d40 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e58156f80 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e58073f70 con 0x7f0e580ccb20 2024-09-11T12:49:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.718+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f0e4401c460 con 0x7f0e580ccb20 2024-09-11T12:49:53.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.722+0000 7f0e4f400640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 0x7f0e38073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:53.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.722+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0e44091d90 con 0x7f0e580ccb20 2024-09-11T12:49:53.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.722+0000 7f0e57400640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 0x7f0e38073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:53.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.722+0000 7f0e57400640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 0x7f0e38073b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e480af000 tx=0x7f0e4801b420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:53.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.726+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0e4405dad0 con 0x7f0e580ccb20 2024-09-11T12:49:53.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.890+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0e58158a10 con 0x7f0e580ccb20 2024-09-11T12:49:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e4f400640 1 -- 172.21.15.78:0/3999333807 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f0e4405d3d0 con 0x7f0e580ccb20 2024-09-11T12:49:53.898 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:53.898 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:53.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 msgr2=0x7f0e38073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 0x7f0e38073b40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e480af000 tx=0x7f0e4801b420 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 msgr2=0x7f0e58153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e4400c410 tx=0x7f0e4400c8d0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 shutdown_connections 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0e38071680 0x7f0e38073b40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e5815bc60 0x7f0e581541f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e580ccb20 0x7f0e58153cb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 --2- 172.21.15.78:0/3999333807 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e580b4780 0x7f0e5813e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 >> 172.21.15.78:0/3999333807 conn(0x7f0e5806ca80 msgr2=0x7f0e58157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 shutdown_connections 2024-09-11T12:49:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:53.894+0000 7f0e5e000640 1 -- 172.21.15.78:0/3999333807 wait complete. 2024-09-11T12:49:53.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T12:49:53.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T12:49:53.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T12:49:53.921 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-11T12:49:53.921 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-11T12:49:53.921 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-11T12:49:53.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-11T12:49:53.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-11T12:49:54.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 -- 172.21.15.78:0/1622389882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc06a7b0 msgr2=0x7f27dc15a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 --2- 172.21.15.78:0/1622389882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc06a7b0 0x7f27dc15a170 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f27dc072d00 tx=0x7f27d001c800 comp rx=0 tx=0).stop 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 -- 172.21.15.78:0/1622389882 shutdown_connections 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 --2- 172.21.15.78:0/1622389882 >> v2:172.21.15.78:3300/0 conn(0x7f27dc070280 0x7f27dc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 --2- 172.21.15.78:0/1622389882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc15a6b0 0x7f27dc06fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 --2- 172.21.15.78:0/1622389882 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc06a7b0 0x7f27dc15a170 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 -- 172.21.15.78:0/1622389882 >> 172.21.15.78:0/1622389882 conn(0x7f27dc075610 msgr2=0x7f27dc075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 -- 172.21.15.78:0/1622389882 shutdown_connections 2024-09-11T12:49:54.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.162+0000 7f27e2800640 1 -- 172.21.15.78:0/1622389882 wait complete. 2024-09-11T12:49:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 Processor -- start 2024-09-11T12:49:54.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 -- start start 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06a7b0 0x7f27dc159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 0x7f27dc13fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15a6b0 0x7f27dc1ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc0b26a0 con 0x7f27dc070280 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc0b2520 con 0x7f27dc15a6b0 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.166+0000 7f27e2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc0b2820 con 0x7f27dc06a7b0 2024-09-11T12:49:54.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 0x7f27dc13fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15a6b0 0x7f27dc1ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 0x7f27dc13fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59878/0 (socket says 172.21.15.78:59878) 2024-09-11T12:49:54.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 -- 172.21.15.78:0/2631386205 learned_addr learned my addr 172.21.15.78:0/2631386205 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e1400640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06a7b0 0x7f27dc159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7ff4b3e00640 1 -- 172.21.15.78:0/535637882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 msgr2=0x7ff4b4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7ff4b3e00640 1 --2- 172.21.15.78:0/535637882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 0x7ff4b4159da0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4ac009c30 tx=0x7ff4ac01cb80 comp rx=0 tx=0).stop 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 -- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06a7b0 msgr2=0x7f27dc159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06a7b0 0x7f27dc159c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 -- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15a6b0 msgr2=0x7f27dc1ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15a6b0 0x7f27dc1ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc0b03c0 con 0x7f27dc070280 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7ff4b3e00640 1 -- 172.21.15.78:0/535637882 shutdown_connections 2024-09-11T12:49:54.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7ff4b3e00640 1 --2- 172.21.15.78:0/535637882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 0x7ff4b4159da0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7ff4b3e00640 1 -- 172.21.15.78:0/535637882 >> 172.21.15.78:0/535637882 conn(0x7ff4b40cfeb0 msgr2=0x7ff4b40d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e0a00640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 0x7f27dc13fb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e7c0 tx=0x7f27cc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc00ccc0 con 0x7f27dc070280 2024-09-11T12:49:54.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27cc00ce80 con 0x7f27dc070280 2024-09-11T12:49:54.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27cc005280 con 0x7f27dc070280 2024-09-11T12:49:54.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc14fec0 con 0x7f27dc070280 2024-09-11T12:49:54.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc14fbe0 con 0x7f27dc070280 2024-09-11T12:49:54.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 -- 172.21.15.78:0/3419368878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 msgr2=0x7fe3980ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 --2- 172.21.15.78:0/3419368878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe3980ad870 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3980d4c70 tx=0x7fe38c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 -- 172.21.15.78:0/3419368878 shutdown_connections 2024-09-11T12:49:54.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f27dc14f6a0 con 0x7f27dc070280 2024-09-11T12:49:54.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 --2- 172.21.15.78:0/3419368878 >> v2:172.21.15.78:3300/0 conn(0x7fe39815e8f0 0x7fe398073410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 --2- 172.21.15.78:0/3419368878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe3980ad870 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.170+0000 7fe39f000640 1 -- 172.21.15.78:0/3419368878 >> 172.21.15.78:0/3419368878 conn(0x7fe39806e800 msgr2=0x7fe398071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7ff4b3e00640 1 -- 172.21.15.78:0/535637882 shutdown_connections 2024-09-11T12:49:54.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7ff4b3e00640 1 -- 172.21.15.78:0/535637882 wait complete. 2024-09-11T12:49:54.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f27cc005420 con 0x7f27dc070280 2024-09-11T12:49:54.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7fe39f000640 1 -- 172.21.15.78:0/3419368878 shutdown_connections 2024-09-11T12:49:54.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7fe39f000640 1 -- 172.21.15.78:0/3419368878 wait complete. 2024-09-11T12:49:54.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 Processor -- start 2024-09-11T12:49:54.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 -- start start 2024-09-11T12:49:54.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.174+0000 7ff4b3e00640 1 Processor -- start 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27d9600640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27c00716d0 0x7f27c0073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f27cc092b90 con 0x7f27dc070280 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27d9600640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7f27c0079950 0x7f27c007bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f27cc022e50 con 0x7f27c0079950 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f27cc092f40 con 0x7f27dc070280 2024-09-11T12:49:54.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27e1e00640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7f27c0079950 0x7f27c007bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe398073950 0x7fe398156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe398158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39815e8f0 0x7fe398135690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe398073950 0x7fe398156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe398073950 0x7fe398156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43546/0 (socket says 172.21.15.78:43546) 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe398158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe39816a910 con 0x7fe39815e8f0 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe39816a790 con 0x7fe3981439c0 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe39816aa90 con 0x7fe398073950 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39dc00640 1 -- 172.21.15.78:0/1546224064 learned_addr learned my addr 172.21.15.78:0/1546224064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39e600640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39815e8f0 0x7fe398135690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 -- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe398073950 msgr2=0x7fe398156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe398073950 0x7fe398156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 -- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39815e8f0 msgr2=0x7fe398135690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe39815e8f0 0x7fe398135690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39813a6a0 con 0x7fe3981439c0 2024-09-11T12:49:54.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7fe39d200640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe398158430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe38400c9a0 tx=0x7fe38400ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27e1400640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27c00716d0 0x7f27c0073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7f27e1e00640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7f27c0079950 0x7f27c007bd70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 -- 172.21.15.78:0/3550682517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 msgr2=0x7fd35000a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 --2- 172.21.15.78:0/3550682517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd35000a860 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd34c0091e0 tx=0x7fd34c01d840 comp rx=0 tx=0).stop 2024-09-11T12:49:54.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe384007bb0 con 0x7fe3981439c0 2024-09-11T12:49:54.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe384007e40 con 0x7fe3981439c0 2024-09-11T12:49:54.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe38400f6b0 con 0x7fe3981439c0 2024-09-11T12:49:54.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.178+0000 7ff4b3e00640 1 -- start start 2024-09-11T12:49:54.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7f27e1400640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27c00716d0 0x7f27c0073b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27d00af000 tx=0x7f27d001b040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== osd.2 v2:172.21.15.78:6816/4205044300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f27cc022e50 con 0x7f27c0079950 2024-09-11T12:49:54.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39813c080 con 0x7fe3981439c0 2024-09-11T12:49:54.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39813c640 con 0x7fe3981439c0 2024-09-11T12:49:54.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 -- 172.21.15.78:0/3550682517 shutdown_connections 2024-09-11T12:49:54.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 --2- 172.21.15.78:0/3550682517 >> v2:172.21.15.157:3300/0 conn(0x7fd350036700 0x7fd3500060c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 --2- 172.21.15.78:0/3550682517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd35000a860 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 -- 172.21.15.78:0/3550682517 >> 172.21.15.78:0/3550682517 conn(0x7fd350054570 msgr2=0x7fd350054980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe39813c840 con 0x7fe3981439c0 2024-09-11T12:49:54.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 -- 172.21.15.78:0/3550682517 shutdown_connections 2024-09-11T12:49:54.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7fd360200640 1 -- 172.21.15.78:0/3550682517 wait complete. 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 0x7ff4b40b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 0x7ff4b414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b40ce3d0 0x7ff4b414a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4168e20 con 0x7ff4b40ce3d0 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b4168ca0 con 0x7ff4b406c8f0 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b4168fa0 con 0x7ff4b40745a0 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 0x7ff4b40b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b2a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 0x7ff4b40b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59098/0 (socket says 172.21.15.78:59098) 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.182+0000 7ff4b2a00640 1 -- 172.21.15.78:0/2832043695 learned_addr learned my addr 172.21.15.78:0/2832043695 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 -- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 msgr2=0x7ff4b414b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4b40745a0 0x7ff4b414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 -- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b40ce3d0 msgr2=0x7ff4b414a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4b40ce3d0 0x7ff4b414a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b4204770 con 0x7ff4b406c8f0 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6277400640 1 -- 172.21.15.78:0/1722037082 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270009ac0 con 0x7f625c003a60 2024-09-11T12:49:54.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b2a00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 0x7ff4b40b13c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c4b0 tx=0x7ff4a400c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 -- 172.21.15.78:0/1722037082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625c003a60 msgr2=0x7f625c003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 --2- 172.21.15.78:0/1722037082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625c003a60 0x7f625c003e40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6270009ef0 tx=0x7f627001d3f0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe38400f850 con 0x7fe3981439c0 2024-09-11T12:49:54.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 -- 172.21.15.78:0/1722037082 shutdown_connections 2024-09-11T12:49:54.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 --2- 172.21.15.78:0/1722037082 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f625c003a60 0x7f625c003e40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 -- 172.21.15.78:0/1722037082 >> 172.21.15.78:0/1722037082 conn(0x7f627806ca80 msgr2=0x7f62780b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 -- 172.21.15.78:0/1722037082 shutdown_connections 2024-09-11T12:49:54.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f627f600640 1 -- 172.21.15.78:0/1722037082 wait complete. 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a40052d0 con 0x7ff4b406c8f0 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4a4005470 con 0x7ff4b406c8f0 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a4012870 con 0x7ff4b406c8f0 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4b414b9b0 con 0x7ff4b406c8f0 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4b4149f00 con 0x7ff4b406c8f0 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 -- 172.21.15.78:0/2295421094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048070190 msgr2=0x7fc048071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 --2- 172.21.15.78:0/2295421094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048070190 0x7fc048071d60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc040009bc0 tx=0x7fc04001cb90 comp rx=0 tx=0).stop 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 -- 172.21.15.78:0/2295421094 shutdown_connections 2024-09-11T12:49:54.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 --2- 172.21.15.78:0/2295421094 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc048070190 0x7fc048071d60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 --2- 172.21.15.78:0/2295421094 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc04815ba80 0x7fc04806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 -- 172.21.15.78:0/2295421094 >> 172.21.15.78:0/2295421094 conn(0x7fc04806ca80 msgr2=0x7fc0480b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fd360200640 1 Processor -- start 2024-09-11T12:49:54.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe378071680 0x7fe378073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe3840923b0 con 0x7fe3981439c0 2024-09-11T12:49:54.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fd360200640 1 -- start start 2024-09-11T12:49:54.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff4b415d800 con 0x7ff4b406c8f0 2024-09-11T12:49:54.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe39dc00640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe378071680 0x7fe378073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fe3780798a0 0x7fe37807bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6e84000640 1 -- 172.21.15.78:0/2786038973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e640073e0 msgr2=0x7f6e6400bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6e84000640 1 --2- 172.21.15.78:0/2786038973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e640073e0 0x7f6e6400bbd0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006fa0 tx=0x7f6e7001ce90 comp rx=0 tx=0).stop 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6e84000640 1 -- 172.21.15.78:0/2786038973 shutdown_connections 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6e84000640 1 --2- 172.21.15.78:0/2786038973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e640073e0 0x7f6e6400bbd0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7f6e84000640 1 -- 172.21.15.78:0/2786038973 >> 172.21.15.78:0/2786038973 conn(0x7f6e7c0ccd10 msgr2=0x7f6e7c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fc04fc00640 1 -- 172.21.15.78:0/2295421094 shutdown_connections 2024-09-11T12:49:54.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fc04fc00640 1 -- 172.21.15.78:0/2295421094 wait complete. 2024-09-11T12:49:54.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe384011e50 con 0x7fe3780798a0 2024-09-11T12:49:54.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.186+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fe384099050 con 0x7fe3981439c0 2024-09-11T12:49:54.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fe39e600640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fe3780798a0 0x7fe37807bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 Processor -- start 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd3500d7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35001c370 0x7fd350183ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd350036700 0x7fd350048010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd350038790 con 0x7fd350036700 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd350038610 con 0x7fd35001c370 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd360200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd350038910 con 0x7fd35000a480 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 -- 172.21.15.78:0/364394771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e6415a1c0 msgr2=0x7f4e640cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 --2- 172.21.15.78:0/364394771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e6415a1c0 0x7f4e640cbed0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4e5400afd0 tx=0x7f4e5401d3f0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd3500d7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 -- start start 2024-09-11T12:49:54.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35ce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35001c370 0x7fd350183ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35e200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd350036700 0x7fd350048010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f6e84000640 1 -- 172.21.15.78:0/2786038973 shutdown_connections 2024-09-11T12:49:54.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fe39e600640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fe3780798a0 0x7fe37807bcc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd3500d7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43588/0 (socket says 172.21.15.78:43588) 2024-09-11T12:49:54.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 -- 172.21.15.78:0/4253777042 learned_addr learned my addr 172.21.15.78:0/4253777042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 -- 172.21.15.78:0/364394771 shutdown_connections 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 --2- 172.21.15.78:0/364394771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e6415a1c0 0x7f4e640cbed0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 --2- 172.21.15.78:0/364394771 >> v2:172.21.15.78:3300/0 conn(0x7f4e640b4780 0x7f4e6415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 -- 172.21.15.78:0/364394771 >> 172.21.15.78:0/364394771 conn(0x7f4e6406ca80 msgr2=0x7f4e640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 -- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35001c370 msgr2=0x7fd350183ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd35001c370 0x7fd350183ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 -- 172.21.15.78:0/4253777042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd350036700 msgr2=0x7fd350048010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd350036700 0x7fd350048010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd350035f30 con 0x7fd35000a480 2024-09-11T12:49:54.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7ff4a4003600 con 0x7ff4b406c8f0 2024-09-11T12:49:54.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f6e84000640 1 -- 172.21.15.78:0/2786038973 wait complete. 2024-09-11T12:49:54.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f625c003a60 0x7f627813a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 -- 172.21.15.78:0/364394771 shutdown_connections 2024-09-11T12:49:54.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fc04fc00640 1 Processor -- start 2024-09-11T12:49:54.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fd35d800640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd3500d7b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a4b0 tx=0x7fd34c0a6610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 0x7f62781f3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62780b0ab0 0x7f62780b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f627815ce60 con 0x7f625c003a60 2024-09-11T12:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f627815cce0 con 0x7f627815e1e0 2024-09-11T12:49:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f627815cfe0 con 0x7f62780b0ab0 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f6277e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 0x7f62781f3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f6277e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 0x7f62781f3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59104/0 (socket says 172.21.15.78:59104) 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f627cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f625c003a60 0x7f627813a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f6277e00640 1 -- 172.21.15.78:0/1905177923 learned_addr learned my addr 172.21.15.78:0/1905177923 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7f4e6c800640 1 -- 172.21.15.78:0/364394771 wait complete. 2024-09-11T12:49:54.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4ab400640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff494071780 0x7ff494073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0bb020 con 0x7fd35000a480 2024-09-11T12:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd34c004490 con 0x7fd35000a480 2024-09-11T12:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd34c0b3040 con 0x7fd35000a480 2024-09-11T12:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 -- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62780b0ab0 msgr2=0x7f62780b0f30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62780b0ab0 0x7f62780b0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 -- 172.21.15.78:0/1905177923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f625c003a60 msgr2=0x7f627813a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f625c003a60 0x7f627813a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 Processor -- start 2024-09-11T12:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6278150f60 con 0x7f627815e1e0 2024-09-11T12:49:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff4a40924e0 con 0x7ff4b406c8f0 2024-09-11T12:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4ab400640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7ff494079a00 0x7ff49407be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff4a401ce50 con 0x7ff494079a00 2024-09-11T12:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff4a40928d0 con 0x7ff4b406c8f0 2024-09-11T12:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== osd.1 v2:172.21.15.78:6824/305322464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe384011e50 con 0x7fe3780798a0 2024-09-11T12:49:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 -- start start 2024-09-11T12:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.190+0000 7fc04fc00640 1 -- start start 2024-09-11T12:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 0x7fc0482040c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6277e00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 0x7f62781f3990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6268007470 tx=0x7f6268005a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fe39dc00640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe378071680 0x7fe378073b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009230 tx=0x7fe38c01b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268018020 con 0x7f627815e1e0 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6268013990 con 0x7f627815e1e0 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6268013dc0 con 0x7f627815e1e0 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc048074240 0x7fc04815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 0x7fc0482040c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 0x7fc0482040c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43604/0 (socket says 172.21.15.78:43604) 2024-09-11T12:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04d200640 1 -- 172.21.15.78:0/1332141731 learned_addr learned my addr 172.21.15.78:0/1332141731 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4b3400640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7ff494079a00 0x7ff49407be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3500d5cd0 con 0x7fd35000a480 2024-09-11T12:49:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3500d40b0 con 0x7fd35000a480 2024-09-11T12:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc04815ba80 0x7fc04815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62781f4100 con 0x7f627815e1e0 2024-09-11T12:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62781f46c0 con 0x7f627815e1e0 2024-09-11T12:49:54.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7ff4b2000640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff494071780 0x7ff494073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc04806ecc0 con 0x7fc048074240 2024-09-11T12:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04dc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc04815ba80 0x7fc04815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e640073e0 0x7f6e7c146510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7c0708c0 0x7f6e7c06f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 0x7f6e7c14bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc04806eb40 con 0x7fc04815ba80 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f4e6c800640 1 Processor -- start 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c0ce290 con 0x7f6e640073e0 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e7c0ce110 con 0x7f6e7c0708c0 2024-09-11T12:49:54.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e84000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e7c0ce410 con 0x7f6e7c144e50 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc04806ee40 con 0x7fc048070190 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7fc047e00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc048074240 0x7fc04815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f4e6c800640 1 -- start start 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e82000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 0x7f6e7c14bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e82000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 0x7f6e7c14bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43610/0 (socket says 172.21.15.78:43610) 2024-09-11T12:49:54.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.194+0000 7f6e82000640 1 -- 172.21.15.78:0/2856956059 learned_addr learned my addr 172.21.15.78:0/2856956059 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7ff4b2000640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff494071780 0x7ff494073c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007fd0 tx=0x7ff4a0007940 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7ff4b3400640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7ff494079a00 0x7ff49407be20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e80c00640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7c0708c0 0x7f6e7c06f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 -- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7c0708c0 msgr2=0x7f6e7c06f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7c0708c0 0x7f6e7c06f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 -- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e640073e0 msgr2=0x7f6e7c146510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e640073e0 0x7f6e7c146510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7c146f40 con 0x7f6e7c144e50 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e80c00640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e7c0708c0 0x7f6e7c06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f62781512e0 con 0x7f627815e1e0 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 -- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc04815ba80 msgr2=0x7fc04815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc04815ba80 0x7fc04815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 -- 172.21.15.78:0/1332141731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc048074240 msgr2=0x7fc04815c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc048074240 0x7fc04815c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc048154be0 con 0x7fc048070190 2024-09-11T12:49:54.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc047e00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc048074240 0x7fc04815c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04dc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc04815ba80 0x7fc04815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04d200640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 0x7fc0482040c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c450 tx=0x7fc03400c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd350038dc0 con 0x7fd35000a480 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 0x7f4e6413c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6413d250 0x7f4e64165220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e6415a1c0 0x7f4e64141ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6413ead0 con 0x7f4e6413d250 2024-09-11T12:49:54.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e6413e950 con 0x7f4e6415a1c0 2024-09-11T12:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6413ec50 con 0x7f4e640b4780 2024-09-11T12:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034016020 con 0x7fc048070190 2024-09-11T12:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e82000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 0x7f6e7c14bd10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e7001c950 tx=0x7f6e70001190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc034012920 con 0x7fc048070190 2024-09-11T12:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== osd.0 v2:172.21.15.78:6800/4284670803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff4a401ce50 con 0x7ff494079a00 2024-09-11T12:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 0x7f4e6413c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc048154ec0 con 0x7fc048070190 2024-09-11T12:49:54.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc048154870 con 0x7fc048070190 2024-09-11T12:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 0x7f4e6413c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43622/0 (socket says 172.21.15.78:43622) 2024-09-11T12:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 -- 172.21.15.78:0/1271385550 learned_addr learned my addr 172.21.15.78:0/1271385550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc034012d70 con 0x7fc048070190 2024-09-11T12:49:54.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f626801b020 con 0x7f627815e1e0 2024-09-11T12:49:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc020001040 con 0x7fc048070190 2024-09-11T12:49:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e6a800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e6415a1c0 0x7f4e64141ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fd34c0be020 con 0x7fd35000a480 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 -- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e6415a1c0 msgr2=0x7f4e64141ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e6415a1c0 0x7f4e64141ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 -- 172.21.15.78:0/1271385550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6413d250 msgr2=0x7f4e64165220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e6413d250 0x7f4e64165220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f4e69e00640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e64165760 con 0x7f4e640b4780 2024-09-11T12:49:54.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70001c40 con 0x7f6e7c144e50 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6274c00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6258071780 0x7f6258073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6268095b20 con 0x7f627815e1e0 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6274c00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f62580799a0 0x7f625807bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f626801de50 con 0x7f62580799a0 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6268095f10 con 0x7f627815e1e0 2024-09-11T12:49:54.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.198+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e700015f0 con 0x7f6e7c144e50 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e70003860 con 0x7f6e7c144e50 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7c06fe80 con 0x7f6e7c144e50 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e7c145a30 con 0x7f6e7c144e50 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6e7c146a50 con 0x7f6e7c144e50 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd355600640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd338071680 0x7fd338073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd34c12b8d0 con 0x7fd35000a480 2024-09-11T12:49:54.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f627d600640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f62580799a0 0x7f625807bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e69e00640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 0x7f4e6413c110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d980 tx=0x7f4e5800de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd35ce00640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd338071680 0x7fd338073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e58017070 con 0x7f4e640b4780 2024-09-11T12:49:54.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fc034019020 con 0x7fc048070190 2024-09-11T12:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd355600640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7fd338079900 0x7fd33807bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e58009d20 con 0x7f4e640b4780 2024-09-11T12:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e580106c0 con 0x7f4e640b4780 2024-09-11T12:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd34c0b0e90 con 0x7fd338079900 2024-09-11T12:49:54.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd34c1302a0 con 0x7fd35000a480 2024-09-11T12:49:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f627cc00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6258071780 0x7f6258073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6413b350 con 0x7f4e640b4780 2024-09-11T12:49:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e6413b910 con 0x7f4e640b4780 2024-09-11T12:49:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd35ce00640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd338071680 0x7fd338073b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd350035cb0 tx=0x7fd344009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7fd35e200640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7fd338079900 0x7fd33807bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f627d600640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f62580799a0 0x7f625807bdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f4e53400640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4e3c001040 con 0x7f4e640b4780 2024-09-11T12:49:54.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.202+0000 7f627cc00640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6258071780 0x7f6258073c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62700b0000 tx=0x7f627001a190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc044c00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc028071680 0x7fc028073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc034091f60 con 0x7fc048070190 2024-09-11T12:49:54.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fd35e200640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7fd338079900 0x7fd33807bd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc044c00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fc028079880 0x7fc02807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc047e00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc028071680 0x7fc028073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc034010e50 con 0x7fc028079880 2024-09-11T12:49:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fc0340923f0 con 0x7fc048070190 2024-09-11T12:49:54.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== osd.6 v2:172.21.15.157:6826/899086460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd34c0b0e90 con 0x7fd338079900 2024-09-11T12:49:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc047e00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc028071680 0x7fc028073b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0480cc5c0 tx=0x7fc038005e30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7fc04dc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fc028079880 0x7fc02807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== osd.4 v2:172.21.15.157:6810/2942152212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f626801de50 con 0x7f62580799a0 2024-09-11T12:49:54.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f6e700b8030 con 0x7f6e7c144e50 2024-09-11T12:49:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4e58010860 con 0x7f4e640b4780 2024-09-11T12:49:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e480716d0 0x7f4e48073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4e5805a130 con 0x7f4e640b4780 2024-09-11T12:49:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e69400640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e480716d0 0x7f4e48073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4e4807a5d0 0x7f4e4807ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4e4807d140 con 0x7f4e4807a5d0 2024-09-11T12:49:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.206+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f4e5805a330 con 0x7f4e640b4780 2024-09-11T12:49:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e79600640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6e5c071750 0x7f6e5c073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f4e69400640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e480716d0 0x7f4e48073b90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e540af000 tx=0x7f4e5401a190 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7fc04dc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fc028079880 0x7fc02807bca0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6e7012c410 con 0x7f6e7c144e50 2024-09-11T12:49:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f4e6a800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4e4807a5d0 0x7f4e4807ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e81600640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6e5c071750 0x7f6e5c073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e79600640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6e5c079970 0x7f6e5c07bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 --> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f27dc14e250 con 0x7f27c0079950 2024-09-11T12:49:54.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6e7000a030 con 0x7f6e5c079970 2024-09-11T12:49:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f6e7012c800 con 0x7f6e7c144e50 2024-09-11T12:49:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e80c00640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6e5c079970 0x7f6e5c07bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e81600640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6e5c071750 0x7f6e5c073c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c10 tx=0x7f6e68006880 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27d9600640 1 -- 172.21.15.78:0/2631386205 <== osd.2 v2:172.21.15.78:6816/4205044300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f27dc14e250 con 0x7f27c0079950 2024-09-11T12:49:54.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== osd.5 v2:172.21.15.157:6802/2089537569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc034010e50 con 0x7fc028079880 2024-09-11T12:49:54.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7f27c0079950 msgr2=0x7f27c007bd70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7f27c0079950 0x7f27c007bd70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f6e80c00640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6e5c079970 0x7f6e5c07bd90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f4e6a800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4e4807a5d0 0x7f4e4807ca70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27c00716d0 msgr2=0x7f27c0073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27c00716d0 0x7f27c0073b90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f27d00af000 tx=0x7f27d001b040 comp rx=0 tx=0).stop 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 msgr2=0x7f27dc13fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e2800640 1 --2- 172.21.15.78:0/2631386205 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc070280 0x7f27dc13fb80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27cc00e7c0 tx=0x7f27cc00ec80 comp rx=0 tx=0).stop 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.210+0000 7f27e1e00640 1 -- 172.21.15.78:0/2631386205 reap_dead start 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.214+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 shutdown_connections 2024-09-11T12:49:54.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.214+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 >> 172.21.15.78:0/2631386205 conn(0x7f27dc075610 msgr2=0x7f27dc0b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.214+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== osd.7 v2:172.21.15.157:6818/3956663176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4e4807d140 con 0x7f4e4807a5d0 2024-09-11T12:49:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.214+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== osd.3 v2:172.21.15.78:6808/2152308697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6e7000a030 con 0x7f6e5c079970 2024-09-11T12:49:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.218+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 shutdown_connections 2024-09-11T12:49:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.218+0000 7f27e2800640 1 -- 172.21.15.78:0/2631386205 wait complete. 2024-09-11T12:49:54.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.218+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 --> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe3980ad870 con 0x7fe3780798a0 2024-09-11T12:49:54.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.218+0000 7fe395600640 1 -- 172.21.15.78:0/1546224064 <== osd.1 v2:172.21.15.78:6824/305322464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe3980ad870 con 0x7fe3780798a0 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fe3780798a0 msgr2=0x7fe37807bcc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fe3780798a0 0x7fe37807bcc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe378071680 msgr2=0x7fe378073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe378071680 0x7fe378073b40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe38c009230 tx=0x7fe38c01b100 comp rx=0 tx=0).stop 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 msgr2=0x7fe398158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 --2- 172.21.15.78:0/1546224064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3981439c0 0x7fe398158430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe38400c9a0 tx=0x7fe38400ce60 comp rx=0 tx=0).stop 2024-09-11T12:49:54.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39e600640 1 -- 172.21.15.78:0/1546224064 reap_dead start 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 shutdown_connections 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 >> 172.21.15.78:0/1546224064 conn(0x7fe39806e800 msgr2=0x7fe398071cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 shutdown_connections 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.222+0000 7fe39f000640 1 -- 172.21.15.78:0/1546224064 wait complete. 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.230+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 --> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6e7c15de40 con 0x7f6e5c079970 2024-09-11T12:49:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.230+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 --> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff4b4156380 con 0x7ff494079a00 2024-09-11T12:49:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.230+0000 7ff4ab400640 1 -- 172.21.15.78:0/2832043695 <== osd.0 v2:172.21.15.78:6800/4284670803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff4b4156380 con 0x7ff494079a00 2024-09-11T12:49:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.230+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7ff494079a00 msgr2=0x7ff49407be20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.230+0000 7ff4b3e00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7ff494079a00 0x7ff49407be20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff494071780 msgr2=0x7ff494073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff494071780 0x7ff494073c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007fd0 tx=0x7ff4a0007940 comp rx=0 tx=0).stop 2024-09-11T12:49:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 msgr2=0x7ff4b40b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 --2- 172.21.15.78:0/2832043695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4b406c8f0 0x7ff4b40b13c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff4a400c4b0 tx=0x7ff4a400c970 comp rx=0 tx=0).stop 2024-09-11T12:49:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3400640 1 -- 172.21.15.78:0/2832043695 reap_dead start 2024-09-11T12:49:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e79600640 1 -- 172.21.15.78:0/2856956059 <== osd.3 v2:172.21.15.78:6808/2152308697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6e7c15de40 con 0x7f6e5c079970 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6e5c079970 msgr2=0x7f6e5c07bd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6e5c079970 0x7f6e5c07bd90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6e5c071750 msgr2=0x7f6e5c073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6e5c071750 0x7f6e5c073c10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6e68000c10 tx=0x7f6e68006880 comp rx=0 tx=0).stop 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 msgr2=0x7f6e7c14bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 --2- 172.21.15.78:0/2856956059 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e7c144e50 0x7f6e7c14bd10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6e7001c950 tx=0x7f6e70001190 comp rx=0 tx=0).stop 2024-09-11T12:49:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 shutdown_connections 2024-09-11T12:49:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 >> 172.21.15.78:0/2832043695 conn(0x7ff4b40cfeb0 msgr2=0x7ff4b4166df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e82000640 1 -- 172.21.15.78:0/2856956059 reap_dead start 2024-09-11T12:49:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 shutdown_connections 2024-09-11T12:49:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 >> 172.21.15.78:0/2856956059 conn(0x7f6e7c0ccd10 msgr2=0x7f6e7c15bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 shutdown_connections 2024-09-11T12:49:54.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7ff4b3e00640 1 -- 172.21.15.78:0/2832043695 wait complete. 2024-09-11T12:49:54.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 shutdown_connections 2024-09-11T12:49:54.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 --> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd35001d990 con 0x7fd338079900 2024-09-11T12:49:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7f6e84000640 1 -- 172.21.15.78:0/2856956059 wait complete. 2024-09-11T12:49:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.234+0000 7fd355600640 1 -- 172.21.15.78:0/4253777042 <== osd.6 v2:172.21.15.157:6826/899086460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd35001d990 con 0x7fd338079900 2024-09-11T12:49:54.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7fd338079900 msgr2=0x7fd33807bd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7fd338079900 0x7fd33807bd20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 --> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f62781f48c0 con 0x7f62580799a0 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd338071680 msgr2=0x7fd338073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd338071680 0x7fd338073b40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd350035cb0 tx=0x7fd344009040 comp rx=0 tx=0).stop 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 msgr2=0x7fd3500d7b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 --2- 172.21.15.78:0/4253777042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd35000a480 0x7fd3500d7b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd34c01a4b0 tx=0x7fd34c0a6610 comp rx=0 tx=0).stop 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd35e200640 1 -- 172.21.15.78:0/4253777042 reap_dead start 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 shutdown_connections 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 >> 172.21.15.78:0/4253777042 conn(0x7fd350054570 msgr2=0x7fd3500e0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.238+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 shutdown_connections 2024-09-11T12:49:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f6274c00640 1 -- 172.21.15.78:0/1905177923 <== osd.4 v2:172.21.15.157:6810/2942152212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f62781f48c0 con 0x7f62580799a0 2024-09-11T12:49:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fd360200640 1 -- 172.21.15.78:0/4253777042 wait complete. 2024-09-11T12:49:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 --> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc020002de0 con 0x7fc028079880 2024-09-11T12:49:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f62580799a0 msgr2=0x7f625807bdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f62580799a0 0x7f625807bdc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6258071780 msgr2=0x7f6258073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6258071780 0x7f6258073c40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f62700b0000 tx=0x7f627001a190 comp rx=0 tx=0).stop 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 msgr2=0x7f62781f3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 --2- 172.21.15.78:0/1905177923 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f627815e1e0 0x7f62781f3990 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6268007470 tx=0x7f6268005a10 comp rx=0 tx=0).stop 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc044c00640 1 -- 172.21.15.78:0/1332141731 <== osd.5 v2:172.21.15.157:6802/2089537569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fc020002de0 con 0x7fc028079880 2024-09-11T12:49:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627d600640 1 -- 172.21.15.78:0/1905177923 reap_dead start 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fc028079880 msgr2=0x7fc02807bca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fc028079880 0x7fc02807bca0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc028071680 msgr2=0x7fc028073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc028071680 0x7fc028073b40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc0480cc5c0 tx=0x7fc038005e30 comp rx=0 tx=0).stop 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 msgr2=0x7fc0482040c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04fc00640 1 --2- 172.21.15.78:0/1332141731 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc048070190 0x7fc0482040c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc03400c450 tx=0x7fc03400c910 comp rx=0 tx=0).stop 2024-09-11T12:49:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7fc04dc00640 1 -- 172.21.15.78:0/1332141731 reap_dead start 2024-09-11T12:49:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.242+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 shutdown_connections 2024-09-11T12:49:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 >> 172.21.15.78:0/1905177923 conn(0x7f627806ca80 msgr2=0x7f62780cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e53400640 1 -- 172.21.15.78:0/1271385550 --> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4e3c002de0 con 0x7f4e4807a5d0 2024-09-11T12:49:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 shutdown_connections 2024-09-11T12:49:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f627f600640 1 -- 172.21.15.78:0/1905177923 wait complete. 2024-09-11T12:49:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e5e000640 1 -- 172.21.15.78:0/1271385550 <== osd.7 v2:172.21.15.157:6818/3956663176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4e3c002de0 con 0x7f4e4807a5d0 2024-09-11T12:49:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 shutdown_connections 2024-09-11T12:49:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4e4807a5d0 msgr2=0x7f4e4807ca70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4e4807a5d0 0x7f4e4807ca70 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e480716d0 msgr2=0x7f4e48073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e480716d0 0x7f4e48073b90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4e540af000 tx=0x7f4e5401a190 comp rx=0 tx=0).stop 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 msgr2=0x7f4e6413c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 --2- 172.21.15.78:0/1271385550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e640b4780 0x7f4e6413c110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4e5800d980 tx=0x7f4e5800de40 comp rx=0 tx=0).stop 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6a800640 1 -- 172.21.15.78:0/1271385550 reap_dead start 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 shutdown_connections 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 >> 172.21.15.78:0/1271385550 conn(0x7f4e6406ca80 msgr2=0x7f4e6413de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 >> 172.21.15.78:0/1332141731 conn(0x7fc04806ca80 msgr2=0x7fc0480b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 shutdown_connections 2024-09-11T12:49:54.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7fc04fc00640 1 -- 172.21.15.78:0/1332141731 wait complete. 2024-09-11T12:49:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 shutdown_connections 2024-09-11T12:49:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.246+0000 7f4e6c800640 1 -- 172.21.15.78:0/1271385550 wait complete. 2024-09-11T12:49:54.264 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.264 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T12:49:54.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T12:49:54.280 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T12:49:54.284 INFO:teuthology.orchestra.run.smithi078.stdout:60129542151 2024-09-11T12:49:54.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T12:49:54.291 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T12:49:54.292 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.292 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T12:49:54.294 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.294 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:54.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T12:49:54.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T12:49:54.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 -- 172.21.15.78:0/1900397558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c0cecd0 msgr2=0x7fe64c0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe64aa00640 1 -- 172.21.15.78:0/1900397558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440b2070 con 0x7fe64c0cecd0 2024-09-11T12:49:54.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 --2- 172.21.15.78:0/1900397558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c0cecd0 0x7fe64c0cf120 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe64c074e80 tx=0x7fe64401cd80 comp rx=0 tx=0).stop 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 -- 172.21.15.78:0/1900397558 shutdown_connections 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 --2- 172.21.15.78:0/1900397558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c0cecd0 0x7fe64c0cf120 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 --2- 172.21.15.78:0/1900397558 >> v2:172.21.15.157:3300/0 conn(0x7fe64c0aec70 0x7fe64c0af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 -- 172.21.15.78:0/1900397558 >> 172.21.15.78:0/1900397558 conn(0x7fe64c0b2c90 msgr2=0x7fe64c0b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 -- 172.21.15.78:0/1900397558 shutdown_connections 2024-09-11T12:49:54.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.482+0000 7fe653000640 1 -- 172.21.15.78:0/1900397558 wait complete. 2024-09-11T12:49:54.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.486+0000 7fe653000640 1 Processor -- start 2024-09-11T12:49:54.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.486+0000 7fe653000640 1 -- start start 2024-09-11T12:49:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.486+0000 7fcde2a00640 1 -- 172.21.15.78:0/4293463688 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40b2070 con 0x7fcde40cd120 2024-09-11T12:49:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.486+0000 7fcdea000640 1 -- 172.21.15.78:0/4293463688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 msgr2=0x7fcde40ceeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.486+0000 7fcdea000640 1 --2- 172.21.15.78:0/4293463688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde40ceeb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcde40cd970 tx=0x7fcdd401c6f0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 -- 172.21.15.78:0/4293463688 shutdown_connections 2024-09-11T12:49:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 --2- 172.21.15.78:0/4293463688 >> v2:172.21.15.157:3300/0 conn(0x7fcde40cd500 0x7fcde40cf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 --2- 172.21.15.78:0/4293463688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde40ceeb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 -- 172.21.15.78:0/4293463688 >> 172.21.15.78:0/4293463688 conn(0x7fcde40b2c90 msgr2=0x7fcde40b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe64c0aec70 0x7fe64c147390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 -- 172.21.15.78:0/4293463688 shutdown_connections 2024-09-11T12:49:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe64c0cecd0 0x7fe64c145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fcdea000640 1 -- 172.21.15.78:0/4293463688 wait complete. 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60004/0 (socket says 172.21.15.78:60004) 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe64be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe64c0aec70 0x7fe64c147390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 -- 172.21.15.78:0/1288562566 learned_addr learned my addr 172.21.15.78:0/1288562566 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c0737b0 con 0x7fe64c15f570 2024-09-11T12:49:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe64c15fb40 con 0x7fe64c0aec70 2024-09-11T12:49:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe64c073930 con 0x7fe64c0cecd0 2024-09-11T12:49:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe64b400640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe64c0cecd0 0x7fe64c145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 -- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe64c0cecd0 msgr2=0x7fe64c145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe64c0cecd0 0x7fe64c145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 -- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe64c0aec70 msgr2=0x7fe64c147390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe64c0aec70 0x7fe64c147390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.490+0000 7fe651000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe64c145fd0 con 0x7fe64c15f570 2024-09-11T12:49:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe651000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe64401d620 tx=0x7fe6440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcdea000640 1 Processor -- start 2024-09-11T12:49:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcdea000640 1 -- start start 2024-09-11T12:49:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440bb070 con 0x7fe64c15f570 2024-09-11T12:49:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe64c1462b0 con 0x7fe64c15f570 2024-09-11T12:49:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6440047d0 con 0x7fe64c15f570 2024-09-11T12:49:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6440bb070 con 0x7fe64c15f570 2024-09-11T12:49:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcdea000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcdea000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 0x7fcde413a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcdea000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcde3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcde3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60014/0 (socket says 172.21.15.78:60014) 2024-09-11T12:49:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 0x7fcde413a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcddbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 0x7fcde413a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59190/0 (socket says 172.21.15.78:59190) 2024-09-11T12:49:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fcde3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43648/0 (socket says 172.21.15.78:43648) 2024-09-11T12:49:54.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 -- 172.21.15.78:0/2672787657 learned_addr learned my addr 172.21.15.78:0/2672787657 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcde415c7a0 con 0x7fcde40cd120 2024-09-11T12:49:54.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcde415c620 con 0x7fcde40cd500 2024-09-11T12:49:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.494+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe64c146870 con 0x7fe64c15f570 2024-09-11T12:49:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64c0b1610 con 0x7fe64c15f570 2024-09-11T12:49:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe6440a5bc0 con 0x7fe64c15f570 2024-09-11T12:49:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcde415c920 con 0x7fcde413fd70 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 -- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 msgr2=0x7fcde4142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 -- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 msgr2=0x7fcde413a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 0x7fcde413a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcde4142360 con 0x7fcde40cd120 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3e00640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:54.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fcde3400640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b0040 tx=0x7fcdd4005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.498+0000 7fe643e00640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 0x7fe62c073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40b2070 con 0x7fcde40cd120 2024-09-11T12:49:54.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fe64be00640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 0x7fe62c073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcde41412c0 con 0x7fcde40cd120 2024-09-11T12:49:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcdd40bb070 con 0x7fcde40cd120 2024-09-11T12:49:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd40b2070 con 0x7fcde40cd120 2024-09-11T12:49:54.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fe64be00640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 0x7fe62c073d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe63c005fd0 tx=0x7fe63c00c460 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcde4141880 con 0x7fcde40cd120 2024-09-11T12:49:54.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe644000fc0 con 0x7fe64c15f570 2024-09-11T12:49:54.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.502+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcde4072570 con 0x7fcde40cd120 2024-09-11T12:49:54.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.510+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe644016970 con 0x7fe64c15f570 2024-09-11T12:49:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.510+0000 7f77a9800640 1 -- 172.21.15.78:0/3717697003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cead0 msgr2=0x7f77a415a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.510+0000 7f77a9800640 1 --2- 172.21.15.78:0/3717697003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cead0 0x7f77a415a630 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f77940097c0 tx=0x7f779401c630 comp rx=0 tx=0).stop 2024-09-11T12:49:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fcdd4005030 con 0x7fcde40cd120 2024-09-11T12:49:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.510+0000 7f77a9800640 1 -- 172.21.15.78:0/3717697003 shutdown_connections 2024-09-11T12:49:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7f77a9800640 1 --2- 172.21.15.78:0/3717697003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a40cead0 0x7f77a415a630 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7f77a9800640 1 -- 172.21.15.78:0/3717697003 >> 172.21.15.78:0/3717697003 conn(0x7f77a40b1930 msgr2=0x7f77a40b1d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7fcddb400640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 0x7fcdc0073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7f77a9800640 1 -- 172.21.15.78:0/3717697003 shutdown_connections 2024-09-11T12:49:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcddc001880 con 0x7fcde40cd120 2024-09-11T12:49:54.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7fcddbe00640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 0x7fcdc0073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.514+0000 7f77a9800640 1 -- 172.21.15.78:0/3717697003 wait complete. 2024-09-11T12:49:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 Processor -- start 2024-09-11T12:49:54.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 -- start start 2024-09-11T12:49:54.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40b2450 0x7f77a41447e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a40cead0 0x7f77a414bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f77a415ef10 con 0x7f77a415a1a0 2024-09-11T12:49:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.518+0000 7f77a9800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77a415ed90 con 0x7f77a40b2450 2024-09-11T12:49:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a9800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77a415f090 con 0x7f77a40cead0 2024-09-11T12:49:54.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7fcddbe00640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 0x7fcdc0073f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcde4159740 tx=0x7fcdcc00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60040/0 (socket says 172.21.15.78:60040) 2024-09-11T12:49:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 -- 172.21.15.78:0/3716181444 learned_addr learned my addr 172.21.15.78:0/3716181444 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a3e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40b2450 0x7f77a41447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcdd4016310 con 0x7fcde40cd120 2024-09-11T12:49:54.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a3400640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a40cead0 0x7f77a414bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 -- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a40cead0 msgr2=0x7f77a414bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a40cead0 0x7f77a414bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 -- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40b2450 msgr2=0x7f77a41447e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40b2450 0x7f77a41447e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 -- 172.21.15.78:0/3716181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a414c3e0 con 0x7f77a415a1a0 2024-09-11T12:49:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a8e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f779401ace0 tx=0x7f77940a8720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f1337e00640 1 -- 172.21.15.78:0/3118658084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f133815df20 msgr2=0x7f1338073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f1337e00640 1 --2- 172.21.15.78:0/3118658084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f133815df20 0x7f1338073190 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f1338072b60 tx=0x7f132801c630 comp rx=0 tx=0).stop 2024-09-11T12:49:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f1337e00640 1 -- 172.21.15.78:0/3118658084 shutdown_connections 2024-09-11T12:49:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77940bf070 con 0x7f77a415a1a0 2024-09-11T12:49:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f1337e00640 1 --2- 172.21.15.78:0/3118658084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f133815df20 0x7f1338073190 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f1337e00640 1 -- 172.21.15.78:0/3118658084 >> 172.21.15.78:0/3118658084 conn(0x7f133806d870 msgr2=0x7f133806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7794005140 con 0x7f77a415a1a0 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005530 con 0x7f77a415a1a0 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.522+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77a40afab0 con 0x7f77a415a1a0 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.526+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77a414c020 con 0x7f77a415a1a0 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.526+0000 7f1337e00640 1 -- 172.21.15.78:0/3118658084 shutdown_connections 2024-09-11T12:49:54.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.526+0000 7f1337e00640 1 -- 172.21.15.78:0/3118658084 wait complete. 2024-09-11T12:49:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.526+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77a415a610 con 0x7f77a415a1a0 2024-09-11T12:49:54.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.530+0000 7f1337e00640 1 Processor -- start 2024-09-11T12:49:54.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.530+0000 7f1337e00640 1 -- start start 2024-09-11T12:49:54.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.530+0000 7f1337e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13380af170 0x7f133813ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.530+0000 7f1337e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13381466a0 0x7f1338158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.530+0000 7f1337e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f133816c300 con 0x7f13380af170 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f133816c180 con 0x7f13380d03c0 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f133816c480 con 0x7f13381466a0 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1336a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13380af170 0x7f133813ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59218/0 (socket says 172.21.15.78:59218) 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 -- 172.21.15.78:0/2812561778 learned_addr learned my addr 172.21.15.78:0/2812561778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1336000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13381466a0 0x7f1338158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 -- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13381466a0 msgr2=0x7f1338158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13381466a0 0x7f1338158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 -- 172.21.15.78:0/2812561778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13380af170 msgr2=0x7f133813ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13380af170 0x7f133813ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.534+0000 7f1337400640 1 -- 172.21.15.78:0/2812561778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13381694d0 con 0x7f13380d03c0 2024-09-11T12:49:54.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f1337400640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f133000c7a0 tx=0x7f133000cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f77940b8070 con 0x7f77a415a1a0 2024-09-11T12:49:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f779be00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 0x7f7784073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330016020 con 0x7f13380d03c0 2024-09-11T12:49:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1330006e50 con 0x7f13380d03c0 2024-09-11T12:49:54.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330003be0 con 0x7f13380d03c0 2024-09-11T12:49:54.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f77940f4a70 con 0x7f77a415a1a0 2024-09-11T12:49:54.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f77a3e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 0x7f7784073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13381faec0 con 0x7f13380d03c0 2024-09-11T12:49:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.538+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381fb480 con 0x7f13380d03c0 2024-09-11T12:49:54.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.542+0000 7f77a3e00640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 0x7f7784073b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f779c000f90 tx=0x7f779c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.542+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f77940b6040 con 0x7f77a415a1a0 2024-09-11T12:49:54.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.546+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f1330019070 con 0x7f13380d03c0 2024-09-11T12:49:54.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.546+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338160e80 con 0x7f13380d03c0 2024-09-11T12:49:54.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.546+0000 7f132ea00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 0x7f1314073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.546+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1330091240 con 0x7f13380d03c0 2024-09-11T12:49:54.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.550+0000 7f1336a00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 0x7f1314073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.550+0000 7f1336a00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 0x7f1314073b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1320000f70 tx=0x7f1320009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.550+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f133005d220 con 0x7f13380d03c0 2024-09-11T12:49:54.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 -- 172.21.15.78:0/2099712669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 msgr2=0x7f4eb80aa230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2099712669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 0x7f4eb80aa230 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea801c840 comp rx=0 tx=0).stop 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 -- 172.21.15.78:0/2099712669 shutdown_connections 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2099712669 >> v2:172.21.15.157:3300/0 conn(0x7f4eb80cf240 0x7f4eb80d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2099712669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 0x7f4eb80aa230 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.586+0000 7f4ebd400640 1 -- 172.21.15.78:0/2099712669 >> 172.21.15.78:0/2099712669 conn(0x7f4eb8075cc0 msgr2=0x7f4eb80742c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.590+0000 7f4ebd400640 1 -- 172.21.15.78:0/2099712669 shutdown_connections 2024-09-11T12:49:54.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.590+0000 7f4ebd400640 1 -- 172.21.15.78:0/2099712669 wait complete. 2024-09-11T12:49:54.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.590+0000 7f4ebd400640 1 Processor -- start 2024-09-11T12:49:54.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.590+0000 7f4ebd400640 1 -- start start 2024-09-11T12:49:54.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 0x7f4eb814a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 0x7f4eb81fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb81686b0 con 0x7f4eb80cf240 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4eb8168530 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4ebd400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4eb8168830 con 0x7f4eb815b7d0 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60072/0 (socket says 172.21.15.78:60072) 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7e00640 1 -- 172.21.15.78:0/2717699643 learned_addr learned my addr 172.21.15.78:0/2717699643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 0x7f4eb814a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 -- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 msgr2=0x7f4eb81fbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 0x7f4eb81fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 -- 172.21.15.78:0/2717699643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 msgr2=0x7f4eb806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 -- 172.21.15.78:0/2717699643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eb81fb510 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7e00640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:54.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.594+0000 7f4eb7400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 0x7f4eb814a910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009fd0 tx=0x7f4ea400b870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4015020 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ea4011940 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4011d90 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 -- 172.21.15.78:0/2666141854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 msgr2=0x7f71980b1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eb80cd0e0 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 --2- 172.21.15.78:0/2666141854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f71980b1d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f719806ab90 tx=0x7f719001cb70 comp rx=0 tx=0).stop 2024-09-11T12:49:54.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eb80cd6a0 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 -- 172.21.15.78:0/2666141854 shutdown_connections 2024-09-11T12:49:54.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 --2- 172.21.15.78:0/2666141854 >> v2:172.21.15.78:3300/0 conn(0x7f71980cf810 0x7f71980725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 --2- 172.21.15.78:0/2666141854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f71980b1d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 --2- 172.21.15.78:0/2666141854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf430 0x7f71980ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 -- 172.21.15.78:0/2666141854 >> 172.21.15.78:0/2666141854 conn(0x7f7198075610 msgr2=0x7f7198075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eb80d09a0 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 -- 172.21.15.78:0/2666141854 shutdown_connections 2024-09-11T12:49:54.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.598+0000 7f7197200640 1 -- 172.21.15.78:0/2666141854 wait complete. 2024-09-11T12:49:54.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 -- 172.21.15.78:0/4168428313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68153f60 msgr2=0x7f9d6815a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/4168428313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68153f60 0x7f9d6815a170 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9d54009840 tx=0x7f9d5401c780 comp rx=0 tx=0).stop 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 -- 172.21.15.78:0/4168428313 shutdown_connections 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/4168428313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d68154340 0x7f9d680b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/4168428313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68153f60 0x7f9d6815a170 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 -- 172.21.15.78:0/4168428313 >> 172.21.15.78:0/4168428313 conn(0x7f9d68075610 msgr2=0x7f9d68075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 -- 172.21.15.78:0/4168428313 shutdown_connections 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f9d6dc00640 1 -- 172.21.15.78:0/4168428313 wait complete. 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4ea4018020 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f4eafe00640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 0x7f4e98073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4ea4091d40 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f4eb7e00640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 0x7f4e98073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f7197200640 1 Processor -- start 2024-09-11T12:49:54.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.602+0000 7f7197200640 1 -- start start 2024-09-11T12:49:54.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f4eb7e00640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 0x7f4e98073c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea801b080 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4ea405db90 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 -- 172.21.15.78:0/167752527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 msgr2=0x7f82580cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 0x7f719814e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f7198157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f719815ebe0 con 0x7f71980cf430 2024-09-11T12:49:54.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f719815ea60 con 0x7f7198161ed0 2024-09-11T12:49:54.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7197200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71980b4520 con 0x7f71980cf810 2024-09-11T12:49:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 --2- 172.21.15.78:0/167752527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f82580cfb20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8248009bc0 tx=0x7f824801d570 comp rx=0 tx=0).stop 2024-09-11T12:49:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f9d6dc00640 1 Processor -- start 2024-09-11T12:49:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f9d6dc00640 1 -- start start 2024-09-11T12:49:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 -- 172.21.15.78:0/167752527 shutdown_connections 2024-09-11T12:49:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 --2- 172.21.15.78:0/167752527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f82580cfb20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 -- 172.21.15.78:0/167752527 >> 172.21.15.78:0/167752527 conn(0x7f825806d870 msgr2=0x7f825806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 -- 172.21.15.78:0/167752527 shutdown_connections 2024-09-11T12:49:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 0x7f719814e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60074/0 (socket says 172.21.15.78:60074) 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7196800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f7198157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195e00640 1 -- 172.21.15.78:0/2261901620 learned_addr learned my addr 172.21.15.78:0/2261901620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 -- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 msgr2=0x7f7198157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f7198157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 -- 172.21.15.78:0/2261901620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 msgr2=0x7f7198156670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 -- 172.21.15.78:0/2261901620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71981555d0 con 0x7f71980cf810 2024-09-11T12:49:54.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195e00640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:54.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f825dc00640 1 -- 172.21.15.78:0/167752527 wait complete. 2024-09-11T12:49:54.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d68153f60 0x7f9d681505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815a6b0 0x7f9d681fb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68166c80 con 0x7f9d68154340 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60076/0 (socket says 172.21.15.78:60076) 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68166b00 con 0x7f9d6815a6b0 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d6dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9d68166e00 con 0x7f9d68153f60 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 -- 172.21.15.78:0/958682310 learned_addr learned my addr 172.21.15.78:0/958682310 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 -- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d68153f60 msgr2=0x7f9d681505f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 Processor -- start 2024-09-11T12:49:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 -- start start 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 0x7f8258159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f824f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f824f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59288/0 (socket says 172.21.15.78:59288) 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f8257e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f8257e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60092/0 (socket says 172.21.15.78:60092) 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f824f400640 1 -- 172.21.15.78:0/837511803 learned_addr learned my addr 172.21.15.78:0/837511803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825d200640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 0x7f8258159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8258072750 con 0x7f8258073970 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82580725d0 con 0x7f8258151650 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f82580728d0 con 0x7f825815ab00 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.606+0000 7f7195400640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 0x7f719814e830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f718400e9f0 tx=0x7f718400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d68153f60 0x7f9d681505f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 -- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815a6b0 msgr2=0x7f9d681fb760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815a6b0 0x7f9d681fb760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 -- 172.21.15.78:0/958682310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d681ff310 con 0x7f9d68154340 2024-09-11T12:49:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f9d67400640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c480 tx=0x7f9d5800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f8257e00640 1 -- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 msgr2=0x7f8258159940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.610+0000 7f8257e00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 0x7f8258159940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f8257e00640 1 -- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 msgr2=0x7f8258159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184004440 con 0x7f71980cf810 2024-09-11T12:49:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f718400c9d0 con 0x7f71980cf810 2024-09-11T12:49:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58016030 con 0x7f9d68154340 2024-09-11T12:49:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d580068a0 con 0x7f9d68154340 2024-09-11T12:49:54.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718400ce20 con 0x7f71980cf810 2024-09-11T12:49:54.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58006d10 con 0x7f9d68154340 2024-09-11T12:49:54.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f825d200640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 0x7f8258159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:54.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f8257e00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f8257e00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82581fafc0 con 0x7f8258073970 2024-09-11T12:49:54.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f824f400640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:54.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71981558b0 con 0x7f71980cf810 2024-09-11T12:49:54.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f8257e00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82480ac000 tx=0x7f82480092f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d681ff5f0 con 0x7f9d68154340 2024-09-11T12:49:54.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d6813be30 con 0x7f9d68154340 2024-09-11T12:49:54.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7198155e70 con 0x7f71980cf810 2024-09-11T12:49:54.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71980725d0 con 0x7f71980cf810 2024-09-11T12:49:54.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.614+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d400052a0 con 0x7f9d68154340 2024-09-11T12:49:54.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480bb070 con 0x7f8258073970 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82480042b0 con 0x7f8258073970 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82480b2040 con 0x7f8258073970 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82581fb2a0 con 0x7f8258073970 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825813f6e0 con 0x7f8258073970 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7184005030 con 0x7f71980cf810 2024-09-11T12:49:54.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f718e000640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 0x7f7178073b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f9d58019070 con 0x7f9d68154340 2024-09-11T12:49:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f9d5fe00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 0x7f9d48073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.618+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9d58091ca0 con 0x7f9d68154340 2024-09-11T12:49:54.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.622+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82300052a0 con 0x7f8258073970 2024-09-11T12:49:54.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.622+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7184092b10 con 0x7f71980cf810 2024-09-11T12:49:54.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.622+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f71840c0e60 con 0x7f71980cf810 2024-09-11T12:49:54.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.622+0000 7f7195e00640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 0x7f7178073b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.622+0000 7f7195e00640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 0x7f7178073b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71981ff3d0 tx=0x7f718800a2d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.626+0000 7f9d67e00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 0x7f9d48073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.626+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f82480be070 con 0x7f8258073970 2024-09-11T12:49:54.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.626+0000 7f9d67e00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 0x7f9d48073c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d540af000 tx=0x7f9d5401afc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.630+0000 7f8254c00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 0x7f8234073c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.630+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f824812bfd0 con 0x7f8258073970 2024-09-11T12:49:54.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.630+0000 7f824f400640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 0x7f8234073c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.630+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f82480c5020 con 0x7f8258073970 2024-09-11T12:49:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.634+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9d58020020 con 0x7f9d68154340 2024-09-11T12:49:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.634+0000 7f824f400640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 0x7f8234073c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8240007fd0 tx=0x7f8240009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.870+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fcde413e610 con 0x7fcde40cd120 2024-09-11T12:49:54.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.874+0000 7fcddb400640 1 -- 172.21.15.78:0/2672787657 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fcdd40f75b0 con 0x7fcde40cd120 2024-09-11T12:49:54.881 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.878+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe64c15ed10 con 0x7fe64c15f570 2024-09-11T12:49:54.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.878+0000 7fe643e00640 1 -- 172.21.15.78:0/1288562566 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6440f7840 con 0x7fe64c15f570 2024-09-11T12:49:54.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.878+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 msgr2=0x7fcdc0073f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.885 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.878+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 0x7fcdc0073f40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcde4159740 tx=0x7fcdcc00b390 comp rx=0 tx=0).stop 2024-09-11T12:49:54.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 msgr2=0x7fcde41409c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdd40b0040 tx=0x7fcdd4005db0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 shutdown_connections 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcdc0071a80 0x7fcdc0073f40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcde413fd70 0x7fcde4142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcde40cd500 0x7fcde413a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 --2- 172.21.15.78:0/2672787657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcde40cd120 0x7fcde41409c0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 >> 172.21.15.78:0/2672787657 conn(0x7fcde40b2c90 msgr2=0x7fcde4074220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 shutdown_connections 2024-09-11T12:49:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fcdea000640 1 -- 172.21.15.78:0/2672787657 wait complete. 2024-09-11T12:49:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 msgr2=0x7fe62c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 0x7fe62c073d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fe63c005fd0 tx=0x7fe63c00c460 comp rx=0 tx=0).stop 2024-09-11T12:49:54.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 msgr2=0x7fe64c154c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe64401d620 tx=0x7fe6440042f0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 shutdown_connections 2024-09-11T12:49:54.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe62c071880 0x7fe62c073d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe64c15f570 0x7fe64c154c50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe64c0cecd0 0x7fe64c145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 --2- 172.21.15.78:0/1288562566 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe64c0aec70 0x7fe64c147390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.882+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 >> 172.21.15.78:0/1288562566 conn(0x7fe64c0b2c90 msgr2=0x7fe64c1621a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.886+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 shutdown_connections 2024-09-11T12:49:54.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.886+0000 7fe653000640 1 -- 172.21.15.78:0/1288562566 wait complete. 2024-09-11T12:49:54.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.902+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f1338073110 con 0x7f13380d03c0 2024-09-11T12:49:54.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f132ea00640 1 -- 172.21.15.78:0/2812561778 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f133005cb20 con 0x7f13380d03c0 2024-09-11T12:49:54.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f77a413cc10 con 0x7f77a415a1a0 2024-09-11T12:49:54.914 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f779be00640 1 -- 172.21.15.78:0/3716181444 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f77940ac030 con 0x7f77a415a1a0 2024-09-11T12:49:54.915 INFO:teuthology.orchestra.run.smithi078.stdout:60129542150 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 msgr2=0x7f1314073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 0x7f1314073b90 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1320000f70 tx=0x7f1320009040 comp rx=0 tx=0).stop 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 msgr2=0x7f13380cd7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.910+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f133000c7a0 tx=0x7f133000cc60 comp rx=0 tx=0).stop 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 shutdown_connections 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13140716d0 0x7f1314073b90 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13380d03c0 0x7f13380cd7a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13381466a0 0x7f1338158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 --2- 172.21.15.78:0/2812561778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13380af170 0x7f133813ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 >> 172.21.15.78:0/2812561778 conn(0x7f133806d870 msgr2=0x7f13380b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 shutdown_connections 2024-09-11T12:49:54.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f1337e00640 1 -- 172.21.15.78:0/2812561778 wait complete. 2024-09-11T12:49:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 msgr2=0x7f7784073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 0x7f7784073b40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f779c000f90 tx=0x7f779c009040 comp rx=0 tx=0).stop 2024-09-11T12:49:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 msgr2=0x7f77a413bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f779401ace0 tx=0x7f77940a8720 comp rx=0 tx=0).stop 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 shutdown_connections 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7784071680 0x7f7784073b40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77a415a1a0 0x7f77a413bb80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77a40cead0 0x7f77a414bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 --2- 172.21.15.78:0/3716181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f77a40b2450 0x7f77a41447e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 >> 172.21.15.78:0/3716181444 conn(0x7f77a40b1930 msgr2=0x7f77a40b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 shutdown_connections 2024-09-11T12:49:54.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.914+0000 7f77a9800640 1 -- 172.21.15.78:0/3716181444 wait complete. 2024-09-11T12:49:54.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.1 2024-09-11T12:49:54.923 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.2 2024-09-11T12:49:54.943 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.0 2024-09-11T12:49:54.953 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.3 2024-09-11T12:49:54.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.950+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4eb80ce0d0 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4eafe00640 1 -- 172.21.15.78:0/2717699643 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4ea405d490 con 0x7f4eb80d0ee0 2024-09-11T12:49:54.957 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f9d40005860 con 0x7f9d68154340 2024-09-11T12:49:54.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d5fe00640 1 -- 172.21.15.78:0/958682310 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d5805db80 con 0x7f9d68154340 2024-09-11T12:49:54.958 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 msgr2=0x7f4e98073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 0x7f4e98073c10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ea8009900 tx=0x7f4ea801b080 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 msgr2=0x7f4eb814a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 0x7f4eb814a910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4009fd0 tx=0x7f4ea400b870 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 shutdown_connections 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e98071750 0x7f4e98073c10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4eb815b7d0 0x7f4eb81fbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4eb80d0ee0 0x7f4eb814a910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 --2- 172.21.15.78:0/2717699643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4eb80cf240 0x7f4eb806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 >> 172.21.15.78:0/2717699643 conn(0x7f4eb8075cc0 msgr2=0x7f4eb815e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 shutdown_connections 2024-09-11T12:49:54.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f4ebd400640 1 -- 172.21.15.78:0/2717699643 wait complete. 2024-09-11T12:49:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 msgr2=0x7f9d48073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 0x7f9d48073c10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9d540af000 tx=0x7f9d5401afc0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 msgr2=0x7f9d680b12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c480 tx=0x7f9d5800c940 comp rx=0 tx=0).stop 2024-09-11T12:49:54.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 shutdown_connections 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9d48071750 0x7f9d48073c10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9d6815a6b0 0x7f9d681fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9d68154340 0x7f9d680b12e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 --2- 172.21.15.78:0/958682310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9d68153f60 0x7f9d681505f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.954+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 >> 172.21.15.78:0/958682310 conn(0x7f9d68075610 msgr2=0x7f9d68069310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.958+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 shutdown_connections 2024-09-11T12:49:54.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.958+0000 7f9d6dc00640 1 -- 172.21.15.78:0/958682310 wait complete. 2024-09-11T12:49:54.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.7 2024-09-11T12:49:54.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-09-11T12:49:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.982+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f71980adba0 con 0x7f71980cf810 2024-09-11T12:49:54.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.982+0000 7f718e000640 1 -- 172.21.15.78:0/2261901620 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f718405e4a0 con 0x7f71980cf810 2024-09-11T12:49:54.988 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 msgr2=0x7f7178073b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 0x7f7178073b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f71981ff3d0 tx=0x7f718800a2d0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 msgr2=0x7f719814e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 0x7f719814e830 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f718400e9f0 tx=0x7f718400eeb0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 shutdown_connections 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71780716d0 0x7f7178073b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7198161ed0 0x7f7198157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71980cf810 0x7f719814e830 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 --2- 172.21.15.78:0/2261901620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71980cf430 0x7f7198156670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 >> 172.21.15.78:0/2261901620 conn(0x7f7198075610 msgr2=0x7f719806ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 shutdown_connections 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f7197200640 1 -- 172.21.15.78:0/2261901620 wait complete. 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f8230005860 con 0x7f8258073970 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.986+0000 7f8254c00640 1 -- 172.21.15.78:0/837511803 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f8248016790 con 0x7f8258073970 2024-09-11T12:49:54.992 INFO:teuthology.orchestra.run.smithi078.stdout:64424509446 2024-09-11T12:49:54.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 msgr2=0x7f8234073c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 0x7f8234073c10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8240007fd0 tx=0x7f8240009040 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 msgr2=0x7f825813f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f82480ac000 tx=0x7f82480092f0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 shutdown_connections 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8234071750 0x7f8234073c10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f825815ab00 0x7f8258159940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8258151650 0x7f8258159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 --2- 172.21.15.78:0/837511803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8258073970 0x7f825813f1a0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 >> 172.21.15.78:0/837511803 conn(0x7f825806d870 msgr2=0x7f825814d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 shutdown_connections 2024-09-11T12:49:54.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:54.990+0000 7f825dc00640 1 -- 172.21.15.78:0/837511803 wait complete. 2024-09-11T12:49:55.016 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.4 2024-09-11T12:49:55.019 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-11T12:49:55.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T12:49:55.923 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T12:49:55.946 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T12:49:55.954 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-11T12:49:55.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-11T12:49:55.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-11T12:49:56.018 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-11T12:49:56.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-11T12:49:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.118+0000 7f45e1800640 1 -- 172.21.15.78:0/2547643857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4006150 msgr2=0x7f45d40e30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.118+0000 7f45e1800640 1 --2- 172.21.15.78:0/2547643857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4006150 0x7f45d40e30e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45c4009840 tx=0x7f45c4020140 comp rx=0 tx=0).stop 2024-09-11T12:49:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 -- 172.21.15.78:0/2547643857 shutdown_connections 2024-09-11T12:49:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 --2- 172.21.15.78:0/2547643857 >> v2:172.21.15.157:3300/0 conn(0x7f45d4007020 0x7f45d40de1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 --2- 172.21.15.78:0/2547643857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4006150 0x7f45d40e30e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 -- 172.21.15.78:0/2547643857 >> 172.21.15.78:0/2547643857 conn(0x7f45d401d7b0 msgr2=0x7f45d40545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 -- 172.21.15.78:0/2547643857 shutdown_connections 2024-09-11T12:49:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 -- 172.21.15.78:0/2547643857 wait complete. 2024-09-11T12:49:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 Processor -- start 2024-09-11T12:49:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.122+0000 7f45e1800640 1 -- start start 2024-09-11T12:49:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 -- 172.21.15.78:0/615720425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c15e160 msgr2=0x7fc01c0735b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 --2- 172.21.15.78:0/615720425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c15e160 0x7fc01c0735b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0d0c70 tx=0x7fc00801c930 comp rx=0 tx=0).stop 2024-09-11T12:49:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 -- 172.21.15.78:0/615720425 shutdown_connections 2024-09-11T12:49:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 --2- 172.21.15.78:0/615720425 >> v2:172.21.15.78:3300/0 conn(0x7fc01c073af0 0x7fc01c15a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 --2- 172.21.15.78:0/615720425 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c15e160 0x7fc01c0735b0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc01c0d0c70 tx=0x7fc00801c930 comp rx=0 tx=0).stop 2024-09-11T12:49:56.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 --2- 172.21.15.78:0/615720425 >> v2:172.21.15.157:3300/0 conn(0x7fc01c13cf10 0x7fc01c15dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 -- 172.21.15.78:0/615720425 >> 172.21.15.78:0/615720425 conn(0x7fc01c0b2ec0 msgr2=0x7fc01c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 -- 172.21.15.78:0/615720425 shutdown_connections 2024-09-11T12:49:56.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 0x7f45d404db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7fc01b200640 1 -- 172.21.15.78:0/615720425 wait complete. 2024-09-11T12:49:56.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 0x7f45d404e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45d40de730 con 0x7f45d4053f20 2024-09-11T12:49:56.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4054ec0 con 0x7f45d4006150 2024-09-11T12:49:56.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45e1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45d40de8b0 con 0x7f45d4007020 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59316/0 (socket says 172.21.15.78:59316) 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 -- 172.21.15.78:0/2447844020 learned_addr learned my addr 172.21.15.78:0/2447844020 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45da000640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 0x7f45d404db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45db400640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 0x7f45d404e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 -- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 msgr2=0x7f45d404db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 0x7f45d404db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 -- 172.21.15.78:0/2447844020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 msgr2=0x7f45d404e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 0x7f45d404e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 -- 172.21.15.78:0/2447844020 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d40d7ac0 con 0x7f45d4006150 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45db400640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 0x7f45d404e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:56.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45da000640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 0x7f45d404db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.126+0000 7f45daa00640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f45dc06bd80 tx=0x7f45dc069a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc0008e0 con 0x7f45d4006150 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45dc068610 con 0x7f45d4006150 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45dc069cf0 con 0x7f45d4006150 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d40d7da0 con 0x7f45d4006150 2024-09-11T12:49:56.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d40d8360 con 0x7f45d4006150 2024-09-11T12:49:56.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7fc01b200640 1 Processor -- start 2024-09-11T12:49:56.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7fc01b200640 1 -- start start 2024-09-11T12:49:56.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.130+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f45dc069e90 con 0x7f45d4006150 2024-09-11T12:49:56.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b40052a0 con 0x7f45d4006150 2024-09-11T12:49:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7f45d2a00640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 0x7f45b8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f45dc10c720 con 0x7f45d4006150 2024-09-11T12:49:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7f45da000640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 0x7f45b8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7f45da000640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 0x7f45b8073b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c40b2000 tx=0x7f45c401e980 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7fc01b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7fc01b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c13cf10 0x7fc01c1381f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.134+0000 7fc01b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c1500d0 0x7fc01c1ec340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43780/0 (socket says 172.21.15.78:43780) 2024-09-11T12:49:56.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 -- 172.21.15.78:0/1674956204 learned_addr learned my addr 172.21.15.78:0/1674956204 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c13e310 con 0x7fc01c1500d0 2024-09-11T12:49:56.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01a800640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c1500d0 0x7fc01c1ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc01c13e190 con 0x7fc01c13cf10 2024-09-11T12:49:56.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc01c13e490 con 0x7fc01c073af0 2024-09-11T12:49:56.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 -- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c13cf10 msgr2=0x7fc01c1381f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c13cf10 0x7fc01c1381f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 -- 172.21.15.78:0/1674956204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c1500d0 msgr2=0x7fc01c1ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c1500d0 0x7fc01c1ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01c138880 con 0x7fc01c073af0 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc019e00640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009830 tx=0x7fc00c009cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c01b070 con 0x7fc01c073af0 2024-09-11T12:49:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc00c00ca60 con 0x7fc01c073af0 2024-09-11T12:49:56.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01c138b60 con 0x7fc01c073af0 2024-09-11T12:49:56.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc00c00ce50 con 0x7fc01c073af0 2024-09-11T12:49:56.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc01c1ebe20 con 0x7fc01c073af0 2024-09-11T12:49:56.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.138+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbff40052a0 con 0x7fc01c073af0 2024-09-11T12:49:56.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.142+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f45dc08f070 con 0x7f45d4006150 2024-09-11T12:49:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.142+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fc00c01e070 con 0x7fc01c073af0 2024-09-11T12:49:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.142+0000 7fc012000640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 0x7fbffc073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.142+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc00c0916c0 con 0x7fc01c073af0 2024-09-11T12:49:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.146+0000 7fc019400640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 0x7fbffc073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.146+0000 7fc019400640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 0x7fbffc073ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0080af000 tx=0x7fc0080196d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.150+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc00c05ced0 con 0x7fc01c073af0 2024-09-11T12:49:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.198+0000 7fe2ad000640 1 -- 172.21.15.78:0/3754280873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 msgr2=0x7fe2a0057f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.198+0000 7fe2ad000640 1 --2- 172.21.15.78:0/3754280873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a0057f70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe2a000fa90 tx=0x7fe29401c8b0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 -- 172.21.15.78:0/3754280873 shutdown_connections 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 --2- 172.21.15.78:0/3754280873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a0057f70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 --2- 172.21.15.78:0/3754280873 >> v2:172.21.15.78:3300/0 conn(0x7fe2a00e6e60 0x7fe2a00530c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 -- 172.21.15.78:0/3754280873 >> 172.21.15.78:0/3754280873 conn(0x7fe2a003a3e0 msgr2=0x7fe2a003a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 -- 172.21.15.78:0/3754280873 shutdown_connections 2024-09-11T12:49:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7fe2ad000640 1 -- 172.21.15.78:0/3754280873 wait complete. 2024-09-11T12:49:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.202+0000 7f4e89200640 1 -- 172.21.15.78:0/879055677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4780 msgr2=0x7f4e8415e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 --2- 172.21.15.78:0/879055677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4780 0x7f4e8415e1a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4e70009880 tx=0x7f4e7001c6c0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 -- 172.21.15.78:0/879055677 shutdown_connections 2024-09-11T12:49:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7fe2ad000640 1 Processor -- start 2024-09-11T12:49:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 --2- 172.21.15.78:0/879055677 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4780 0x7f4e8415e1a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7fe2ad000640 1 -- start start 2024-09-11T12:49:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 -- 172.21.15.78:0/879055677 >> 172.21.15.78:0/879055677 conn(0x7f4e8406c650 msgr2=0x7f4e840b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a004dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 0x7fe2a00527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 -- 172.21.15.78:0/879055677 shutdown_connections 2024-09-11T12:49:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.206+0000 7f4e89200640 1 -- 172.21.15.78:0/879055677 wait complete. 2024-09-11T12:49:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e89200640 1 Processor -- start 2024-09-11T12:49:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 0x7fe2a00e3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e89200640 1 -- start start 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2a5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 0x7fe2a00527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2a5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 0x7fe2a00527e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43790/0 (socket says 172.21.15.78:43790) 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2a5400640 1 -- 172.21.15.78:0/306241855 learned_addr learned my addr 172.21.15.78:0/306241855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00f4180 con 0x7fe2a00502d0 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe2a00f4000 con 0x7fe2a0037e20 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe2a00f4300 con 0x7fe2a00e6e60 2024-09-11T12:49:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7fe2a6800640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 0x7fe2a00e3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e89200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e89200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e840b4780 0x7f4e84155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e89200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4b60 0x7f4e840ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e82800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4b60 0x7f4e840ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e81e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e81e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59362/0 (socket says 172.21.15.78:59362) 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.210+0000 7f4e81e00640 1 -- 172.21.15.78:0/1102359465 learned_addr learned my addr 172.21.15.78:0/1102359465 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e84170520 con 0x7f4e840b4b60 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e841703a0 con 0x7f4e84072f50 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81400640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e840b4780 0x7f4e84155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e841706a0 con 0x7f4e840b4780 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a004dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 -- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 msgr2=0x7fe2a00527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 0x7fe2a00527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 -- 172.21.15.78:0/306241855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 msgr2=0x7fe2a00e3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 0x7fe2a00e3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2a004e440 con 0x7fe2a0037e20 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a6800640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 0x7fe2a00e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:49:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2a5e00640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a004dcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c9d0 tx=0x7fe29800ce90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 -- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e840b4780 msgr2=0x7f4e84155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e840b4780 0x7f4e84155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 -- 172.21.15.78:0/1102359465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4b60 msgr2=0x7f4e840ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4b60 0x7f4e840ce9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe298007bb0 con 0x7fe2a0037e20 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe298007d50 con 0x7fe2a0037e20 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe29800f6b0 con 0x7fe2a0037e20 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2a004e720 con 0x7fe2a0037e20 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2a004ece0 con 0x7fe2a0037e20 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e84155bf0 con 0x7f4e84072f50 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e81e00640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4e7001cba0 tx=0x7f4e70005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e700ac070 con 0x7f4e84072f50 2024-09-11T12:49:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e700043e0 con 0x7f4e84072f50 2024-09-11T12:49:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e70004930 con 0x7f4e84072f50 2024-09-11T12:49:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e84155ed0 con 0x7f4e84072f50 2024-09-11T12:49:56.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.214+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e84155850 con 0x7f4e84072f50 2024-09-11T12:49:56.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.218+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e8415e1a0 con 0x7f4e84072f50 2024-09-11T12:49:56.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.218+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2800052a0 con 0x7fe2a0037e20 2024-09-11T12:49:56.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe29800f850 con 0x7fe2a0037e20 2024-09-11T12:49:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4e700af070 con 0x7f4e84072f50 2024-09-11T12:49:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7fe29e000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 0x7fe288073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe298092160 con 0x7fe2a0037e20 2024-09-11T12:49:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7f4e7a000640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 0x7f4e64073c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7fe2a5400640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 0x7fe288073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4e7012c6a0 con 0x7f4e84072f50 2024-09-11T12:49:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.222+0000 7f4e81400640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 0x7f4e64073c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.226+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4e70016370 con 0x7f4e84072f50 2024-09-11T12:49:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.226+0000 7fe2a5400640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 0x7fe288073b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe294009230 tx=0x7fe29401b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.226+0000 7f4e81400640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 0x7f4e64073c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e840aec40 tx=0x7f4e74007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.230+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe298014030 con 0x7fe2a0037e20 2024-09-11T12:49:56.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.278+0000 7ff413400640 1 -- 172.21.15.78:0/1330182389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 msgr2=0x7ff40c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.278+0000 7ff413400640 1 --2- 172.21.15.78:0/1330182389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c15c100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff40c0d1780 tx=0x7ff3fc01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 -- 172.21.15.78:0/1330182389 shutdown_connections 2024-09-11T12:49:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 --2- 172.21.15.78:0/1330182389 >> v2:172.21.15.78:3300/0 conn(0x7ff40c0d0bf0 0x7ff40c0d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 --2- 172.21.15.78:0/1330182389 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c15c100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 -- 172.21.15.78:0/1330182389 >> 172.21.15.78:0/1330182389 conn(0x7ff40c074b90 msgr2=0x7ff40c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 -- 172.21.15.78:0/1330182389 shutdown_connections 2024-09-11T12:49:56.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 -- 172.21.15.78:0/2336510772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015d2b0 msgr2=0x7f4b100b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 --2- 172.21.15.78:0/2336510772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015d2b0 0x7f4b100b4520 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009840 tx=0x7f4b0401c820 comp rx=0 tx=0).stop 2024-09-11T12:49:56.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 -- 172.21.15.78:0/2336510772 shutdown_connections 2024-09-11T12:49:56.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 --2- 172.21.15.78:0/2336510772 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015d2b0 0x7f4b100b4520 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 --2- 172.21.15.78:0/2336510772 >> v2:172.21.15.78:3300/0 conn(0x7f4b1015ced0 0x7f4b1015e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7f4b18000640 1 -- 172.21.15.78:0/2336510772 >> 172.21.15.78:0/2336510772 conn(0x7f4b1006c650 msgr2=0x7f4b100b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 -- 172.21.15.78:0/1330182389 wait complete. 2024-09-11T12:49:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.278+0000 7fe058800640 1 -- 172.21.15.78:0/337116789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0500b4780 msgr2=0x7fe0500d0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.278+0000 7fe058800640 1 --2- 172.21.15.78:0/337116789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0500b4780 0x7fe0500d0350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe044009900 tx=0x7fe04401c850 comp rx=0 tx=0).stop 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 -- 172.21.15.78:0/337116789 shutdown_connections 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 --2- 172.21.15.78:0/337116789 >> v2:172.21.15.157:3300/0 conn(0x7fe050073c30 0x7fe0500d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 --2- 172.21.15.78:0/337116789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe0500b4780 0x7fe0500d0350 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 -- 172.21.15.78:0/337116789 >> 172.21.15.78:0/337116789 conn(0x7fe05006ea20 msgr2=0x7fe050071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 -- 172.21.15.78:0/337116789 shutdown_connections 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7fe058800640 1 -- 172.21.15.78:0/337116789 wait complete. 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 Processor -- start 2024-09-11T12:49:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.282+0000 7ff413400640 1 -- start start 2024-09-11T12:49:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7f4b18000640 1 -- 172.21.15.78:0/2336510772 shutdown_connections 2024-09-11T12:49:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7f4b18000640 1 -- 172.21.15.78:0/2336510772 wait complete. 2024-09-11T12:49:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7fe058800640 1 Processor -- start 2024-09-11T12:49:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7fe058800640 1 -- start start 2024-09-11T12:49:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c06fdf0 0x7ff40c13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c143b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c168e60 con 0x7ff40c06fdf0 2024-09-11T12:49:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff40c168ce0 con 0x7ff40c0b14b0 2024-09-11T12:49:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff413400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff40c168fe0 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7f4b18000640 1 Processor -- start 2024-09-11T12:49:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7f4b18000640 1 -- start start 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43824/0 (socket says 172.21.15.78:43824) 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 -- 172.21.15.78:0/2233446383 learned_addr learned my addr 172.21.15.78:0/2233446383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff40be00640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 -- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 msgr2=0x7ff40c143b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c143b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 -- 172.21.15.78:0/2233446383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c06fdf0 msgr2=0x7ff40c13d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c06fdf0 0x7ff40c13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7ff411400640 1 -- 172.21.15.78:0/2233446383 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40c142b00 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7fe058800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 0x7fe050157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.286+0000 7fe058800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 0x7fe0501404c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0500b4b60 0x7fe050140a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff411400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff40400ca60 tx=0x7ff40400cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe05008bf90 con 0x7fe0500b4b60 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe0500d32d0 con 0x7fe0500b4780 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 0x7fe050157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 0x7fe0501404c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 0x7fe050157850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:43836/0 (socket says 172.21.15.78:43836) 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055e00640 1 -- 172.21.15.78:0/3132082118 learned_addr learned my addr 172.21.15.78:0/3132082118 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe056800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0500b4b60 0x7fe050140a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe05008c110 con 0x7fe050073c30 2024-09-11T12:49:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 0x7f4b10163450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 0x7f4b1014e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10162e70 con 0x7f4b1015d2b0 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b10162cf0 con 0x7f4b100b2270 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b18000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b10162ff0 con 0x7f4b1015ced0 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404007d90 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40c144050 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff404004c20 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40c142da0 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404007d90 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b15600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b15600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59400/0 (socket says 172.21.15.78:59400) 2024-09-11T12:49:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b15600640 1 -- 172.21.15.78:0/1835115873 learned_addr learned my addr 172.21.15.78:0/1835115873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 -- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 msgr2=0x7fe050157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 0x7fe050157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 -- 172.21.15.78:0/3132082118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0500b4b60 msgr2=0x7fe050140a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0500b4b60 0x7fe050140a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe050139540 con 0x7fe0500b4780 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe055400640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 0x7fe0501404c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b410 tx=0x7fe04000b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040015020 con 0x7fe0500b4780 2024-09-11T12:49:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe040011920 con 0x7fe0500b4780 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe040011d70 con 0x7fe0500b4780 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0501397e0 con 0x7fe0500b4780 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe050139da0 con 0x7fe0500b4780 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b15600640 1 -- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 msgr2=0x7f4b10163450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b14c00640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 0x7f4b10163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b16000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 0x7f4b1014e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe050073520 con 0x7fe0500b4780 2024-09-11T12:49:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7ff40400f450 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7ff408c00640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 0x7ff3ec073d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e40052a0 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe040018020 con 0x7fe0500b4780 2024-09-11T12:49:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff404091e50 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.294+0000 7ff410a00640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 0x7ff3ec073d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.290+0000 7f4b15600640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 0x7f4b10163450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b15600640 1 -- 172.21.15.78:0/1835115873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 msgr2=0x7f4b1014e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b15600640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 0x7f4b1014e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b15600640 1 -- 172.21.15.78:0/1835115873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b101fb500 con 0x7f4b100b2270 2024-09-11T12:49:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7fe04a000640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 0x7fe034073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7ff410a00640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 0x7ff3ec073d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0af000 tx=0x7ff3fc019660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b16000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 0x7f4b1014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b15600640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0099a0 tx=0x7f4afc009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7f4b14c00640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 0x7f4b10163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe040091a90 con 0x7fe0500b4780 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7fe055e00640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 0x7fe034073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.298+0000 7fe055e00640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 0x7fe034073b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe04401c680 tx=0x7fe04401b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff404014030 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc00caa0 con 0x7f4b100b2270 2024-09-11T12:49:56.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4afc00cc40 con 0x7f4b100b2270 2024-09-11T12:49:56.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4afc004020 con 0x7f4b100b2270 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 -- 172.21.15.78:0/2396886660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 msgr2=0x7f2ef415e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 --2- 172.21.15.78:0/2396886660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef415e920 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f2ee400a950 tx=0x7f2ee401d920 comp rx=0 tx=0).stop 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 -- 172.21.15.78:0/2396886660 shutdown_connections 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 --2- 172.21.15.78:0/2396886660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef415e920 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 -- 172.21.15.78:0/2396886660 >> 172.21.15.78:0/2396886660 conn(0x7f2ef40cd010 msgr2=0x7f2ef40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b101fb7e0 con 0x7f4b100b2270 2024-09-11T12:49:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe04001f030 con 0x7fe0500b4780 2024-09-11T12:49:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b101fbda0 con 0x7f4b100b2270 2024-09-11T12:49:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.302+0000 7f2efba00640 1 -- 172.21.15.78:0/2396886660 shutdown_connections 2024-09-11T12:49:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.306+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4afc0041c0 con 0x7f4b100b2270 2024-09-11T12:49:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.306+0000 7f2efba00640 1 -- 172.21.15.78:0/2396886660 wait complete. 2024-09-11T12:49:56.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f4b0d600640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 0x7f4af0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4afc0919f0 con 0x7f4b100b2270 2024-09-11T12:49:56.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f4b14c00640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 0x7f4af0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b1015e210 con 0x7f4b100b2270 2024-09-11T12:49:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f2efba00640 1 Processor -- start 2024-09-11T12:49:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f2efba00640 1 -- start start 2024-09-11T12:49:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.310+0000 7f4b14c00640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 0x7f4af0073b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4b100cc030 tx=0x7f4b08009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 0x7f2ef415cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 0x7f2ef40aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef40af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef41683e0 con 0x7f2ef40aff70 2024-09-11T12:49:56.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef40b2470 con 0x7f2ef40cf680 2024-09-11T12:49:56.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2efba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef4168560 con 0x7f2ef40ca4b0 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 0x7f2ef415cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 0x7f2ef415cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60206/0 (socket says 172.21.15.78:60206) 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef40af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9000640 1 -- 172.21.15.78:0/1562210808 learned_addr learned my addr 172.21.15.78:0/1562210808 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef3e00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 0x7f2ef40aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 -- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 msgr2=0x7f2ef40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 0x7f2ef40aefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 -- 172.21.15.78:0/1562210808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 msgr2=0x7f2ef415cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 0x7f2ef415cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 -- 172.21.15.78:0/1562210808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef4163710 con 0x7f2ef40cf680 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef3e00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 0x7f2ef40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:49:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.314+0000 7f2ef9a00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef40af500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00caa0 tx=0x7f2eec00cb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4afc05d7c0 con 0x7f4b100b2270 2024-09-11T12:49:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec016020 con 0x7f2ef40cf680 2024-09-11T12:49:56.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef41639f0 con 0x7f2ef40cf680 2024-09-11T12:49:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef41644b0 con 0x7f2ef40cf680 2024-09-11T12:49:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2eec00a930 con 0x7f2ef40cf680 2024-09-11T12:49:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec016020 con 0x7f2ef40cf680 2024-09-11T12:49:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.318+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ecc0052a0 con 0x7f2ef40cf680 2024-09-11T12:49:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.322+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f2eec019020 con 0x7f2ef40cf680 2024-09-11T12:49:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.322+0000 7f2ef0c00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 0x7f2ed4073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.322+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2eec0923f0 con 0x7f2ef40cf680 2024-09-11T12:49:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.322+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2eec020020 con 0x7f2ef40cf680 2024-09-11T12:49:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.326+0000 7f2ef9000640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 0x7f2ed4073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.330+0000 7f2ef9000640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 0x7f2ed4073ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ee401b550 tx=0x7f2ee401c160 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.494+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fbff4005590 con 0x7fc01c073af0 2024-09-11T12:49:56.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.498+0000 7fc012000640 1 -- 172.21.15.78:0/1674956204 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc00c061420 con 0x7fc01c073af0 2024-09-11T12:49:56.501 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 msgr2=0x7fbffc073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 0x7fbffc073ce0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc0080af000 tx=0x7fc0080196d0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 msgr2=0x7fc01c150930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009830 tx=0x7fc00c009cf0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 shutdown_connections 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbffc071820 0x7fbffc073ce0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc01c1500d0 0x7fc01c1ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc01c13cf10 0x7fc01c1381f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 --2- 172.21.15.78:0/1674956204 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc01c073af0 0x7fc01c150930 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 >> 172.21.15.78:0/1674956204 conn(0x7fc01c0b2ec0 msgr2=0x7fc01c06feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 shutdown_connections 2024-09-11T12:49:56.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.502+0000 7fc01b200640 1 -- 172.21.15.78:0/1674956204 wait complete. 2024-09-11T12:49:56.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.510+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f45b4005590 con 0x7f45d4006150 2024-09-11T12:49:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.514+0000 7f45d2a00640 1 -- 172.21.15.78:0/2447844020 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f45dc0d8500 con 0x7f45d4006150 2024-09-11T12:49:56.518 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 msgr2=0x7f45b8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 0x7f45b8073b40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c40b2000 tx=0x7f45c401e980 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 msgr2=0x7f45d40d3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f45dc06bd80 tx=0x7f45dc069a40 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 shutdown_connections 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f45b8071680 0x7f45b8073b40 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f45c40b2000 tx=0x7f45c401e980 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45d4053f20 0x7f45d404e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45d4007020 0x7f45d404db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 --2- 172.21.15.78:0/2447844020 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45d4006150 0x7f45d40d3ab0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.518+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 >> 172.21.15.78:0/2447844020 conn(0x7f45d401d7b0 msgr2=0x7f45d404fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.522+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 shutdown_connections 2024-09-11T12:49:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.522+0000 7f45e1800640 1 -- 172.21.15.78:0/2447844020 wait complete. 2024-09-11T12:49:56.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.2 2024-09-11T12:49:56.539 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.557 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2024-09-11T12:49:56.558 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.554+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fe280005860 con 0x7fe2a0037e20 2024-09-11T12:49:56.560 INFO:teuthology.orchestra.run.smithi078.stdout:60129542151 2024-09-11T12:49:56.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.554+0000 7fe29e000640 1 -- 172.21.15.78:0/306241855 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe29805d980 con 0x7fe2a0037e20 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 msgr2=0x7fe288073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 0x7fe288073b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe294009230 tx=0x7fe29401b0f0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 msgr2=0x7fe2a004dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a004dcd0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe29800c9d0 tx=0x7fe29800ce90 comp rx=0 tx=0).stop 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 shutdown_connections 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe288071680 0x7fe288073b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe2a00502d0 0x7fe2a00e3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe2a00e6e60 0x7fe2a00527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 --2- 172.21.15.78:0/306241855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe2a0037e20 0x7fe2a004dcd0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 >> 172.21.15.78:0/306241855 conn(0x7fe2a003a3e0 msgr2=0x7fe2a003c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 shutdown_connections 2024-09-11T12:49:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.558+0000 7fe2ad000640 1 -- 172.21.15.78:0/306241855 wait complete. 2024-09-11T12:49:56.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.0 2024-09-11T12:49:56.583 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.590+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f4e840b1680 con 0x7f4e84072f50 2024-09-11T12:49:56.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e7a000640 1 -- 172.21.15.78:0/1102359465 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e700f7f30 con 0x7f4e84072f50 2024-09-11T12:49:56.598 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 msgr2=0x7f4e64073c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 0x7f4e64073c40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4e840aec40 tx=0x7f4e74007e40 comp rx=0 tx=0).stop 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 msgr2=0x7f4e84151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4e7001cba0 tx=0x7f4e70005760 comp rx=0 tx=0).stop 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 shutdown_connections 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e64071780 0x7f4e64073c40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e840b4b60 0x7f4e840ce9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e840b4780 0x7f4e84155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 --2- 172.21.15.78:0/1102359465 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e84072f50 0x7f4e84151d10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 >> 172.21.15.78:0/1102359465 conn(0x7f4e8406c650 msgr2=0x7f4e840b1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.594+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 shutdown_connections 2024-09-11T12:49:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.598+0000 7f4e89200640 1 -- 172.21.15.78:0/1102359465 wait complete. 2024-09-11T12:49:56.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.614+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f2ecc005860 con 0x7f2ef40cf680 2024-09-11T12:49:56.621 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.3 2024-09-11T12:49:56.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.614+0000 7f2ef0c00640 1 -- 172.21.15.78:0/1562210808 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2eec05e2d0 con 0x7f2ef40cf680 2024-09-11T12:49:56.621 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.621 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 msgr2=0x7f2ed4073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 0x7f2ed4073ce0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2ee401b550 tx=0x7f2ee401c160 comp rx=0 tx=0).stop 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 msgr2=0x7f2ef40af500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef40af500 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f2eec00caa0 tx=0x7f2eec00cb90 comp rx=0 tx=0).stop 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 shutdown_connections 2024-09-11T12:49:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2ed4071820 0x7f2ed4073ce0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ef40cf680 0x7f2ef40af500 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ef40ca4b0 0x7f2ef40aefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 --2- 172.21.15.78:0/1562210808 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2ef40aff70 0x7f2ef415cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 >> 172.21.15.78:0/1562210808 conn(0x7f2ef40cd010 msgr2=0x7f2ef4070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 shutdown_connections 2024-09-11T12:49:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.618+0000 7f2efba00640 1 -- 172.21.15.78:0/1562210808 wait complete. 2024-09-11T12:49:56.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.626+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe0501684e0 con 0x7fe0500b4780 2024-09-11T12:49:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe04a000640 1 -- 172.21.15.78:0/3132082118 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe04005d860 con 0x7fe0500b4780 2024-09-11T12:49:56.633 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 msgr2=0x7fe034073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 0x7fe034073b40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe04401c680 tx=0x7fe04401b090 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 msgr2=0x7fe0501404c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 0x7fe0501404c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe04000b410 tx=0x7fe04000b8d0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 shutdown_connections 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe034071680 0x7fe034073b40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe0500b4b60 0x7fe050140a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe0500b4780 0x7fe0501404c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 --2- 172.21.15.78:0/3132082118 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe050073c30 0x7fe050157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 >> 172.21.15.78:0/3132082118 conn(0x7fe05006ea20 msgr2=0x7fe05006ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 shutdown_connections 2024-09-11T12:49:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.630+0000 7fe058800640 1 -- 172.21.15.78:0/3132082118 wait complete. 2024-09-11T12:49:56.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.6 2024-09-11T12:49:56.643 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.656 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.4 2024-09-11T12:49:56.656 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.658+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ff3e4005590 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff408c00640 1 -- 172.21.15.78:0/2233446383 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff4040640d0 con 0x7ff40c0d0bf0 2024-09-11T12:49:56.666 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 msgr2=0x7ff3ec073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 0x7ff3ec073d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0af000 tx=0x7ff3fc019660 comp rx=0 tx=0).stop 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 msgr2=0x7ff40c148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff40400ca60 tx=0x7ff40400cf20 comp rx=0 tx=0).stop 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 shutdown_connections 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff3ec071880 0x7ff3ec073d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff40c0d0bf0 0x7ff40c148c70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c0b14b0 0x7ff40c143b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 --2- 172.21.15.78:0/2233446383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c06fdf0 0x7ff40c13d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 >> 172.21.15.78:0/2233446383 conn(0x7ff40c074b90 msgr2=0x7ff40c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 shutdown_connections 2024-09-11T12:49:56.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.662+0000 7ff413400640 1 -- 172.21.15.78:0/2233446383 wait complete. 2024-09-11T12:49:56.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f4b100cc870 con 0x7f4b100b2270 2024-09-11T12:49:56.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b0d600640 1 -- 172.21.15.78:0/1835115873 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4afc05d0c0 con 0x7f4b100b2270 2024-09-11T12:49:56.671 INFO:teuthology.orchestra.run.smithi078.stdout:64424509447 2024-09-11T12:49:56.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 msgr2=0x7f4af0073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 0x7f4af0073b40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4b100cc030 tx=0x7f4b08009040 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 msgr2=0x7f4b10146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4afc0099a0 tx=0x7f4afc009e60 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 shutdown_connections 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4af0071680 0x7f4af0073b40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b1015d2b0 0x7f4b1014e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b1015ced0 0x7f4b10163450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 --2- 172.21.15.78:0/1835115873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b100b2270 0x7f4b10146630 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 >> 172.21.15.78:0/1835115873 conn(0x7f4b1006c650 msgr2=0x7f4b1014d460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 shutdown_connections 2024-09-11T12:49:56.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.666+0000 7f4b18000640 1 -- 172.21.15.78:0/1835115873 wait complete. 2024-09-11T12:49:56.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.5 2024-09-11T12:49:56.690 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.695 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.7 2024-09-11T12:49:56.695 DEBUG:teuthology.parallel:result is None 2024-09-11T12:49:56.695 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:49:56.696 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- 172.21.15.78:0/1385401840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8398004d50 msgr2=0x7f83980051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1385401840 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8398004d50 0x7f83980051d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009b60 tx=0x7f83ac01d270 comp rx=0 tx=0).stop 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b3400640 1 -- 172.21.15.78:0/1385401840 reap_dead start 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- 172.21.15.78:0/1385401840 shutdown_connections 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- 172.21.15.78:0/1385401840 >> 172.21.15.78:0/1385401840 conn(0x7f83b406ca80 msgr2=0x7f83b40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:56.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- 172.21.15.78:0/1385401840 shutdown_connections 2024-09-11T12:49:56.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- 172.21.15.78:0/1385401840 wait complete. 2024-09-11T12:49:56.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 Processor -- start 2024-09-11T12:49:56.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- start start 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8398004d50 0x7f83b415b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b41478e0 0x7f83b4154d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83b40b4d00 con 0x7f83b4155260 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83b40b4b80 con 0x7f83b41478e0 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83b4152020 con 0x7f8398004d50 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60234/0 (socket says 172.21.15.78:60234) 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b3400640 1 -- 172.21.15.78:0/1894819257 learned_addr learned my addr 172.21.15.78:0/1894819257 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b2000640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b41478e0 0x7f83b4154d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.878+0000 7f83b2a00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8398004d50 0x7f83b415b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 -- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8398004d50 msgr2=0x7f83b415b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8398004d50 0x7f83b415b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 -- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b41478e0 msgr2=0x7f83b4154d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b41478e0 0x7f83b4154d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 -- 172.21.15.78:0/1894819257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b415a3a0 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b3400640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009b60 tx=0x7f83ac0addc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac0b1040 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83ac0bb070 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ac009730 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b415a680 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b415ac40 con 0x7f83b4155260 2024-09-11T12:49:56.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b415cf60 con 0x7f83b4155260 2024-09-11T12:49:56.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.882+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f83ac004290 con 0x7f83b4155260 2024-09-11T12:49:56.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.886+0000 7f83aaa00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 0x7f8394073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:56.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.886+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f83ac12cb30 con 0x7f83b4155260 2024-09-11T12:49:56.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.886+0000 7f83b2a00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 0x7f8394073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:56.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.886+0000 7f83b2a00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 0x7f8394073b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83a4001340 tx=0x7f83a4006ab0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:56.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:56.890+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f83ac016f80 con 0x7f83b4155260 2024-09-11T12:49:57.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.050+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f83b40af590 con 0x7f8394071680 2024-09-11T12:49:57.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.054+0000 7f83aaa00640 1 -- 172.21.15.78:0/1894819257 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f83b40af590 con 0x7f8394071680 2024-09-11T12:49:57.063 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:57.065 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 msgr2=0x7f8394073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 0x7f8394073b40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f83a4001340 tx=0x7f83a4006ab0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 msgr2=0x7f83b4157ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f83ac009b60 tx=0x7f83ac0addc0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 shutdown_connections 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8394071680 0x7f8394073b40 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83b4155260 0x7f83b4157ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83b41478e0 0x7f83b4154d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 --2- 172.21.15.78:0/1894819257 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8398004d50 0x7f83b415b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 >> 172.21.15.78:0/1894819257 conn(0x7f83b406ca80 msgr2=0x7f83b4070500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 shutdown_connections 2024-09-11T12:49:57.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.058+0000 7f83b9c00640 1 -- 172.21.15.78:0/1894819257 wait complete. 2024-09-11T12:49:57.083 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-11T12:49:55.107308+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.664712"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472816+0000","last_change":"2024-09-11T12:49:43.456400+0000","last_active":"2024-09-11T12:49:44.472816+0000","last_peered":"2024-09-11T12:49:44.472816+0000","last_clean":"2024-09-11T12:49:44.472816+0000","last_became_active":"2024-09-11T12:49:43.456074+0000","last_became_peered":"2024-09-11T12:49:43.456074+0000","last_unstale":"2024-09-11T12:49:44.472816+0000","last_undegraded":"2024-09-11T12:49:44.472816+0000","last_fullsized":"2024-09-11T12:49:44.472816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452157+0000","last_change":"2024-09-11T12:49:43.446993+0000","last_active":"2024-09-11T12:49:44.452157+0000","last_peered":"2024-09-11T12:49:44.452157+0000","last_clean":"2024-09-11T12:49:44.452157+0000","last_became_active":"2024-09-11T12:49:43.443935+0000","last_became_peered":"2024-09-11T12:49:43.443935+0000","last_unstale":"2024-09-11T12:49:44.452157+0000","last_undegraded":"2024-09-11T12:49:44.452157+0000","last_fullsized":"2024-09-11T12:49:44.452157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444935+0000","last_change":"2024-09-11T12:49:43.444872+0000","last_active":"2024-09-11T12:49:44.444935+0000","last_peered":"2024-09-11T12:49:44.444935+0000","last_clean":"2024-09-11T12:49:44.444935+0000","last_became_active":"2024-09-11T12:49:43.444523+0000","last_became_peered":"2024-09-11T12:49:43.444523+0000","last_unstale":"2024-09-11T12:49:44.444935+0000","last_undegraded":"2024-09-11T12:49:44.444935+0000","last_fullsized":"2024-09-11T12:49:44.444935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472516+0000","last_change":"2024-09-11T12:49:43.454578+0000","last_active":"2024-09-11T12:49:44.472516+0000","last_peered":"2024-09-11T12:49:44.472516+0000","last_clean":"2024-09-11T12:49:44.472516+0000","last_became_active":"2024-09-11T12:49:43.454355+0000","last_became_peered":"2024-09-11T12:49:43.454355+0000","last_unstale":"2024-09-11T12:49:44.472516+0000","last_undegraded":"2024-09-11T12:49:44.472516+0000","last_fullsized":"2024-09-11T12:49:44.472516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444824+0000","last_change":"2024-09-11T12:49:43.447252+0000","last_active":"2024-09-11T12:49:44.444824+0000","last_peered":"2024-09-11T12:49:44.444824+0000","last_clean":"2024-09-11T12:49:44.444824+0000","last_became_active":"2024-09-11T12:49:43.446860+0000","last_became_peered":"2024-09-11T12:49:43.446860+0000","last_unstale":"2024-09-11T12:49:44.444824+0000","last_undegraded":"2024-09-11T12:49:44.444824+0000","last_fullsized":"2024-09-11T12:49:44.444824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472261+0000","last_change":"2024-09-11T12:49:43.468669+0000","last_active":"2024-09-11T12:49:44.472261+0000","last_peered":"2024-09-11T12:49:44.472261+0000","last_clean":"2024-09-11T12:49:44.472261+0000","last_became_active":"2024-09-11T12:49:43.468316+0000","last_became_peered":"2024-09-11T12:49:43.468316+0000","last_unstale":"2024-09-11T12:49:44.472261+0000","last_undegraded":"2024-09-11T12:49:44.472261+0000","last_fullsized":"2024-09-11T12:49:44.472261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471049+0000","last_change":"2024-09-11T12:49:43.458945+0000","last_active":"2024-09-11T12:49:44.471049+0000","last_peered":"2024-09-11T12:49:44.471049+0000","last_clean":"2024-09-11T12:49:44.471049+0000","last_became_active":"2024-09-11T12:49:43.458677+0000","last_became_peered":"2024-09-11T12:49:43.458677+0000","last_unstale":"2024-09-11T12:49:44.471049+0000","last_undegraded":"2024-09-11T12:49:44.471049+0000","last_fullsized":"2024-09-11T12:49:44.471049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472868+0000","last_change":"2024-09-11T12:49:43.469306+0000","last_active":"2024-09-11T12:49:44.472868+0000","last_peered":"2024-09-11T12:49:44.472868+0000","last_clean":"2024-09-11T12:49:44.472868+0000","last_became_active":"2024-09-11T12:49:43.469031+0000","last_became_peered":"2024-09-11T12:49:43.469031+0000","last_unstale":"2024-09-11T12:49:44.472868+0000","last_undegraded":"2024-09-11T12:49:44.472868+0000","last_fullsized":"2024-09-11T12:49:44.472868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455525+0000","last_change":"2024-09-11T12:49:43.462111+0000","last_active":"2024-09-11T12:49:44.455525+0000","last_peered":"2024-09-11T12:49:44.455525+0000","last_clean":"2024-09-11T12:49:44.455525+0000","last_became_active":"2024-09-11T12:49:43.461818+0000","last_became_peered":"2024-09-11T12:49:43.461818+0000","last_unstale":"2024-09-11T12:49:44.455525+0000","last_undegraded":"2024-09-11T12:49:44.455525+0000","last_fullsized":"2024-09-11T12:49:44.455525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455751+0000","last_change":"2024-09-11T12:49:43.454080+0000","last_active":"2024-09-11T12:49:44.455751+0000","last_peered":"2024-09-11T12:49:44.455751+0000","last_clean":"2024-09-11T12:49:44.455751+0000","last_became_active":"2024-09-11T12:49:43.453788+0000","last_became_peered":"2024-09-11T12:49:43.453788+0000","last_unstale":"2024-09-11T12:49:44.455751+0000","last_undegraded":"2024-09-11T12:49:44.455751+0000","last_fullsized":"2024-09-11T12:49:44.455751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446203+0000","last_change":"2024-09-11T12:49:43.447103+0000","last_active":"2024-09-11T12:49:44.446203+0000","last_peered":"2024-09-11T12:49:44.446203+0000","last_clean":"2024-09-11T12:49:44.446203+0000","last_became_active":"2024-09-11T12:49:43.446839+0000","last_became_peered":"2024-09-11T12:49:43.446839+0000","last_unstale":"2024-09-11T12:49:44.446203+0000","last_undegraded":"2024-09-11T12:49:44.446203+0000","last_fullsized":"2024-09-11T12:49:44.446203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471850+0000","last_change":"2024-09-11T12:49:43.446894+0000","last_active":"2024-09-11T12:49:44.471850+0000","last_peered":"2024-09-11T12:49:44.471850+0000","last_clean":"2024-09-11T12:49:44.471850+0000","last_became_active":"2024-09-11T12:49:43.446045+0000","last_became_peered":"2024-09-11T12:49:43.446045+0000","last_unstale":"2024-09-11T12:49:44.471850+0000","last_undegraded":"2024-09-11T12:49:44.471850+0000","last_fullsized":"2024-09-11T12:49:44.471850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451685+0000","last_change":"2024-09-11T12:49:43.446706+0000","last_active":"2024-09-11T12:49:44.451685+0000","last_peered":"2024-09-11T12:49:44.451685+0000","last_clean":"2024-09-11T12:49:44.451685+0000","last_became_active":"2024-09-11T12:49:43.446340+0000","last_became_peered":"2024-09-11T12:49:43.446340+0000","last_unstale":"2024-09-11T12:49:44.451685+0000","last_undegraded":"2024-09-11T12:49:44.451685+0000","last_fullsized":"2024-09-11T12:49:44.451685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444746+0000","last_change":"2024-09-11T12:49:43.446371+0000","last_active":"2024-09-11T12:49:44.444746+0000","last_peered":"2024-09-11T12:49:44.444746+0000","last_clean":"2024-09-11T12:49:44.444746+0000","last_became_active":"2024-09-11T12:49:43.446163+0000","last_became_peered":"2024-09-11T12:49:43.446163+0000","last_unstale":"2024-09-11T12:49:44.444746+0000","last_undegraded":"2024-09-11T12:49:44.444746+0000","last_fullsized":"2024-09-11T12:49:44.444746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445420+0000","last_change":"2024-09-11T12:49:43.461338+0000","last_active":"2024-09-11T12:49:44.445420+0000","last_peered":"2024-09-11T12:49:44.445420+0000","last_clean":"2024-09-11T12:49:44.445420+0000","last_became_active":"2024-09-11T12:49:43.461065+0000","last_became_peered":"2024-09-11T12:49:43.461065+0000","last_unstale":"2024-09-11T12:49:44.445420+0000","last_undegraded":"2024-09-11T12:49:44.445420+0000","last_fullsized":"2024-09-11T12:49:44.445420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443267+0000","last_change":"2024-09-11T12:49:43.436172+0000","last_active":"2024-09-11T12:49:44.443267+0000","last_peered":"2024-09-11T12:49:44.443267+0000","last_clean":"2024-09-11T12:49:44.443267+0000","last_became_active":"2024-09-11T12:49:43.435740+0000","last_became_peered":"2024-09-11T12:49:43.435740+0000","last_unstale":"2024-09-11T12:49:44.443267+0000","last_undegraded":"2024-09-11T12:49:44.443267+0000","last_fullsized":"2024-09-11T12:49:44.443267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472800+0000","last_change":"2024-09-11T12:49:43.459483+0000","last_active":"2024-09-11T12:49:44.472800+0000","last_peered":"2024-09-11T12:49:44.472800+0000","last_clean":"2024-09-11T12:49:44.472800+0000","last_became_active":"2024-09-11T12:49:43.458688+0000","last_became_peered":"2024-09-11T12:49:43.458688+0000","last_unstale":"2024-09-11T12:49:44.472800+0000","last_undegraded":"2024-09-11T12:49:44.472800+0000","last_fullsized":"2024-09-11T12:49:44.472800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455412+0000","last_change":"2024-09-11T12:49:43.446362+0000","last_active":"2024-09-11T12:49:44.455412+0000","last_peered":"2024-09-11T12:49:44.455412+0000","last_clean":"2024-09-11T12:49:44.455412+0000","last_became_active":"2024-09-11T12:49:43.446092+0000","last_became_peered":"2024-09-11T12:49:43.446092+0000","last_unstale":"2024-09-11T12:49:44.455412+0000","last_undegraded":"2024-09-11T12:49:44.455412+0000","last_fullsized":"2024-09-11T12:49:44.455412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444759+0000","last_change":"2024-09-11T12:49:43.447439+0000","last_active":"2024-09-11T12:49:44.444759+0000","last_peered":"2024-09-11T12:49:44.444759+0000","last_clean":"2024-09-11T12:49:44.444759+0000","last_became_active":"2024-09-11T12:49:43.447044+0000","last_became_peered":"2024-09-11T12:49:43.447044+0000","last_unstale":"2024-09-11T12:49:44.444759+0000","last_undegraded":"2024-09-11T12:49:44.444759+0000","last_fullsized":"2024-09-11T12:49:44.444759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471454+0000","last_change":"2024-09-11T12:49:43.455751+0000","last_active":"2024-09-11T12:49:44.471454+0000","last_peered":"2024-09-11T12:49:44.471454+0000","last_clean":"2024-09-11T12:49:44.471454+0000","last_became_active":"2024-09-11T12:49:43.455514+0000","last_became_peered":"2024-09-11T12:49:43.455514+0000","last_unstale":"2024-09-11T12:49:44.471454+0000","last_undegraded":"2024-09-11T12:49:44.471454+0000","last_fullsized":"2024-09-11T12:49:44.471454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444864+0000","last_change":"2024-09-11T12:49:43.443468+0000","last_active":"2024-09-11T12:49:44.444864+0000","last_peered":"2024-09-11T12:49:44.444864+0000","last_clean":"2024-09-11T12:49:44.444864+0000","last_became_active":"2024-09-11T12:49:43.443158+0000","last_became_peered":"2024-09-11T12:49:43.443158+0000","last_unstale":"2024-09-11T12:49:44.444864+0000","last_undegraded":"2024-09-11T12:49:44.444864+0000","last_fullsized":"2024-09-11T12:49:44.444864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472404+0000","last_change":"2024-09-11T12:49:43.450952+0000","last_active":"2024-09-11T12:49:44.472404+0000","last_peered":"2024-09-11T12:49:44.472404+0000","last_clean":"2024-09-11T12:49:44.472404+0000","last_became_active":"2024-09-11T12:49:43.450620+0000","last_became_peered":"2024-09-11T12:49:43.450620+0000","last_unstale":"2024-09-11T12:49:44.472404+0000","last_undegraded":"2024-09-11T12:49:44.472404+0000","last_fullsized":"2024-09-11T12:49:44.472404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455599+0000","last_change":"2024-09-11T12:49:43.445613+0000","last_active":"2024-09-11T12:49:44.455599+0000","last_peered":"2024-09-11T12:49:44.455599+0000","last_clean":"2024-09-11T12:49:44.455599+0000","last_became_active":"2024-09-11T12:49:43.445164+0000","last_became_peered":"2024-09-11T12:49:43.445164+0000","last_unstale":"2024-09-11T12:49:44.455599+0000","last_undegraded":"2024-09-11T12:49:44.455599+0000","last_fullsized":"2024-09-11T12:49:44.455599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452041+0000","last_change":"2024-09-11T12:49:43.441250+0000","last_active":"2024-09-11T12:49:44.452041+0000","last_peered":"2024-09-11T12:49:44.452041+0000","last_clean":"2024-09-11T12:49:44.452041+0000","last_became_active":"2024-09-11T12:49:43.440991+0000","last_became_peered":"2024-09-11T12:49:43.440991+0000","last_unstale":"2024-09-11T12:49:44.452041+0000","last_undegraded":"2024-09-11T12:49:44.452041+0000","last_fullsized":"2024-09-11T12:49:44.452041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472314+0000","last_change":"2024-09-11T12:49:43.450922+0000","last_active":"2024-09-11T12:49:44.472314+0000","last_peered":"2024-09-11T12:49:44.472314+0000","last_clean":"2024-09-11T12:49:44.472314+0000","last_became_active":"2024-09-11T12:49:43.450635+0000","last_became_peered":"2024-09-11T12:49:43.450635+0000","last_unstale":"2024-09-11T12:49:44.472314+0000","last_undegraded":"2024-09-11T12:49:44.472314+0000","last_fullsized":"2024-09-11T12:49:44.472314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445843+0000","last_change":"2024-09-11T12:49:43.460072+0000","last_active":"2024-09-11T12:49:44.445843+0000","last_peered":"2024-09-11T12:49:44.445843+0000","last_clean":"2024-09-11T12:49:44.445843+0000","last_became_active":"2024-09-11T12:49:43.459042+0000","last_became_peered":"2024-09-11T12:49:43.459042+0000","last_unstale":"2024-09-11T12:49:44.445843+0000","last_undegraded":"2024-09-11T12:49:44.445843+0000","last_fullsized":"2024-09-11T12:49:44.445843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443060+0000","last_change":"2024-09-11T12:49:43.456433+0000","last_active":"2024-09-11T12:49:44.443060+0000","last_peered":"2024-09-11T12:49:44.443060+0000","last_clean":"2024-09-11T12:49:44.443060+0000","last_became_active":"2024-09-11T12:49:43.450398+0000","last_became_peered":"2024-09-11T12:49:43.450398+0000","last_unstale":"2024-09-11T12:49:44.443060+0000","last_undegraded":"2024-09-11T12:49:44.443060+0000","last_fullsized":"2024-09-11T12:49:44.443060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455308+0000","last_change":"2024-09-11T12:49:43.444430+0000","last_active":"2024-09-11T12:49:44.455308+0000","last_peered":"2024-09-11T12:49:44.455308+0000","last_clean":"2024-09-11T12:49:44.455308+0000","last_became_active":"2024-09-11T12:49:43.444155+0000","last_became_peered":"2024-09-11T12:49:43.444155+0000","last_unstale":"2024-09-11T12:49:44.455308+0000","last_undegraded":"2024-09-11T12:49:44.455308+0000","last_fullsized":"2024-09-11T12:49:44.455308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445847+0000","last_change":"2024-09-11T12:49:43.455676+0000","last_active":"2024-09-11T12:49:44.445847+0000","last_peered":"2024-09-11T12:49:44.445847+0000","last_clean":"2024-09-11T12:49:44.445847+0000","last_became_active":"2024-09-11T12:49:43.454645+0000","last_became_peered":"2024-09-11T12:49:43.454645+0000","last_unstale":"2024-09-11T12:49:44.445847+0000","last_undegraded":"2024-09-11T12:49:44.445847+0000","last_fullsized":"2024-09-11T12:49:44.445847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:46.861427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471955+0000","last_change":"2024-09-11T12:49:42.437775+0000","last_active":"2024-09-11T12:49:44.471955+0000","last_peered":"2024-09-11T12:49:44.471955+0000","last_clean":"2024-09-11T12:49:44.471955+0000","last_became_active":"2024-09-11T12:49:42.437412+0000","last_became_peered":"2024-09-11T12:49:42.437412+0000","last_unstale":"2024-09-11T12:49:44.471955+0000","last_undegraded":"2024-09-11T12:49:44.471955+0000","last_fullsized":"2024-09-11T12:49:44.471955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:50.316414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455174+0000","last_change":"2024-09-11T12:49:42.450097+0000","last_active":"2024-09-11T12:49:44.455174+0000","last_peered":"2024-09-11T12:49:44.455174+0000","last_clean":"2024-09-11T12:49:44.455174+0000","last_became_active":"2024-09-11T12:49:42.446539+0000","last_became_peered":"2024-09-11T12:49:42.446539+0000","last_unstale":"2024-09-11T12:49:44.455174+0000","last_undegraded":"2024-09-11T12:49:44.455174+0000","last_fullsized":"2024-09-11T12:49:44.455174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471482+0000","last_change":"2024-09-11T12:49:42.450513+0000","last_active":"2024-09-11T12:49:44.471482+0000","last_peered":"2024-09-11T12:49:44.471482+0000","last_clean":"2024-09-11T12:49:44.471482+0000","last_became_active":"2024-09-11T12:49:42.450293+0000","last_became_peered":"2024-09-11T12:49:42.450293+0000","last_unstale":"2024-09-11T12:49:44.471482+0000","last_undegraded":"2024-09-11T12:49:44.471482+0000","last_fullsized":"2024-09-11T12:49:44.471482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:07:25.000664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471230+0000","last_change":"2024-09-11T12:49:43.445707+0000","last_active":"2024-09-11T12:49:44.471230+0000","last_peered":"2024-09-11T12:49:44.471230+0000","last_clean":"2024-09-11T12:49:44.471230+0000","last_became_active":"2024-09-11T12:49:43.445299+0000","last_became_peered":"2024-09-11T12:49:43.445299+0000","last_unstale":"2024-09-11T12:49:44.471230+0000","last_undegraded":"2024-09-11T12:49:44.471230+0000","last_fullsized":"2024-09-11T12:49:44.471230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444806+0000","last_change":"2024-09-11T12:49:42.438052+0000","last_active":"2024-09-11T12:49:44.444806+0000","last_peered":"2024-09-11T12:49:44.444806+0000","last_clean":"2024-09-11T12:49:44.444806+0000","last_became_active":"2024-09-11T12:49:42.437781+0000","last_became_peered":"2024-09-11T12:49:42.437781+0000","last_unstale":"2024-09-11T12:49:44.444806+0000","last_undegraded":"2024-09-11T12:49:44.444806+0000","last_fullsized":"2024-09-11T12:49:44.444806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451699+0000","last_change":"2024-09-11T12:49:43.440390+0000","last_active":"2024-09-11T12:49:44.451699+0000","last_peered":"2024-09-11T12:49:44.451699+0000","last_clean":"2024-09-11T12:49:44.451699+0000","last_became_active":"2024-09-11T12:49:43.440072+0000","last_became_peered":"2024-09-11T12:49:43.440072+0000","last_unstale":"2024-09-11T12:49:44.451699+0000","last_undegraded":"2024-09-11T12:49:44.451699+0000","last_fullsized":"2024-09-11T12:49:44.451699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455307+0000","last_change":"2024-09-11T12:49:42.444008+0000","last_active":"2024-09-11T12:49:44.455307+0000","last_peered":"2024-09-11T12:49:44.455307+0000","last_clean":"2024-09-11T12:49:44.455307+0000","last_became_active":"2024-09-11T12:49:42.443388+0000","last_became_peered":"2024-09-11T12:49:42.443388+0000","last_unstale":"2024-09-11T12:49:44.455307+0000","last_undegraded":"2024-09-11T12:49:44.455307+0000","last_fullsized":"2024-09-11T12:49:44.455307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445126+0000","last_change":"2024-09-11T12:49:43.461238+0000","last_active":"2024-09-11T12:49:44.445126+0000","last_peered":"2024-09-11T12:49:44.445126+0000","last_clean":"2024-09-11T12:49:44.445126+0000","last_became_active":"2024-09-11T12:49:43.460886+0000","last_became_peered":"2024-09-11T12:49:43.460886+0000","last_unstale":"2024-09-11T12:49:44.445126+0000","last_undegraded":"2024-09-11T12:49:44.445126+0000","last_fullsized":"2024-09-11T12:49:44.445126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445162+0000","last_change":"2024-09-11T12:49:42.446611+0000","last_active":"2024-09-11T12:49:44.445162+0000","last_peered":"2024-09-11T12:49:44.445162+0000","last_clean":"2024-09-11T12:49:44.445162+0000","last_became_active":"2024-09-11T12:49:42.446350+0000","last_became_peered":"2024-09-11T12:49:42.446350+0000","last_unstale":"2024-09-11T12:49:44.445162+0000","last_undegraded":"2024-09-11T12:49:44.445162+0000","last_fullsized":"2024-09-11T12:49:44.445162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451977+0000","last_change":"2024-09-11T12:49:42.440747+0000","last_active":"2024-09-11T12:49:44.451977+0000","last_peered":"2024-09-11T12:49:44.451977+0000","last_clean":"2024-09-11T12:49:44.451977+0000","last_became_active":"2024-09-11T12:49:42.440400+0000","last_became_peered":"2024-09-11T12:49:42.440400+0000","last_unstale":"2024-09-11T12:49:44.451977+0000","last_undegraded":"2024-09-11T12:49:44.451977+0000","last_fullsized":"2024-09-11T12:49:44.451977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446290+0000","last_change":"2024-09-11T12:49:43.455119+0000","last_active":"2024-09-11T12:49:44.446290+0000","last_peered":"2024-09-11T12:49:44.446290+0000","last_clean":"2024-09-11T12:49:44.446290+0000","last_became_active":"2024-09-11T12:49:43.454634+0000","last_became_peered":"2024-09-11T12:49:43.454634+0000","last_unstale":"2024-09-11T12:49:44.446290+0000","last_undegraded":"2024-09-11T12:49:44.446290+0000","last_fullsized":"2024-09-11T12:49:44.446290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455328+0000","last_change":"2024-09-11T12:49:42.445483+0000","last_active":"2024-09-11T12:49:44.455328+0000","last_peered":"2024-09-11T12:49:44.455328+0000","last_clean":"2024-09-11T12:49:44.455328+0000","last_became_active":"2024-09-11T12:49:42.445158+0000","last_became_peered":"2024-09-11T12:49:42.445158+0000","last_unstale":"2024-09-11T12:49:44.455328+0000","last_undegraded":"2024-09-11T12:49:44.455328+0000","last_fullsized":"2024-09-11T12:49:44.455328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472598+0000","last_change":"2024-09-11T12:49:43.470186+0000","last_active":"2024-09-11T12:49:44.472598+0000","last_peered":"2024-09-11T12:49:44.472598+0000","last_clean":"2024-09-11T12:49:44.472598+0000","last_became_active":"2024-09-11T12:49:43.468620+0000","last_became_peered":"2024-09-11T12:49:43.468620+0000","last_unstale":"2024-09-11T12:49:44.472598+0000","last_undegraded":"2024-09-11T12:49:44.472598+0000","last_fullsized":"2024-09-11T12:49:44.472598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444877+0000","last_change":"2024-09-11T12:49:42.445232+0000","last_active":"2024-09-11T12:49:44.444877+0000","last_peered":"2024-09-11T12:49:44.444877+0000","last_clean":"2024-09-11T12:49:44.444877+0000","last_became_active":"2024-09-11T12:49:42.441896+0000","last_became_peered":"2024-09-11T12:49:42.441896+0000","last_unstale":"2024-09-11T12:49:44.444877+0000","last_undegraded":"2024-09-11T12:49:44.444877+0000","last_fullsized":"2024-09-11T12:49:44.444877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472600+0000","last_change":"2024-09-11T12:49:43.463066+0000","last_active":"2024-09-11T12:49:44.472600+0000","last_peered":"2024-09-11T12:49:44.472600+0000","last_clean":"2024-09-11T12:49:44.472600+0000","last_became_active":"2024-09-11T12:49:43.462868+0000","last_became_peered":"2024-09-11T12:49:43.462868+0000","last_unstale":"2024-09-11T12:49:44.472600+0000","last_undegraded":"2024-09-11T12:49:44.472600+0000","last_fullsized":"2024-09-11T12:49:44.472600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455115+0000","last_change":"2024-09-11T12:49:42.447532+0000","last_active":"2024-09-11T12:49:44.455115+0000","last_peered":"2024-09-11T12:49:44.455115+0000","last_clean":"2024-09-11T12:49:44.455115+0000","last_became_active":"2024-09-11T12:49:42.447217+0000","last_became_peered":"2024-09-11T12:49:42.447217+0000","last_unstale":"2024-09-11T12:49:44.455115+0000","last_undegraded":"2024-09-11T12:49:44.455115+0000","last_fullsized":"2024-09-11T12:49:44.455115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455296+0000","last_change":"2024-09-11T12:49:43.444203+0000","last_active":"2024-09-11T12:49:44.455296+0000","last_peered":"2024-09-11T12:49:44.455296+0000","last_clean":"2024-09-11T12:49:44.455296+0000","last_became_active":"2024-09-11T12:49:43.443909+0000","last_became_peered":"2024-09-11T12:49:43.443909+0000","last_unstale":"2024-09-11T12:49:44.455296+0000","last_undegraded":"2024-09-11T12:49:44.455296+0000","last_fullsized":"2024-09-11T12:49:44.455296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.442812+0000","last_change":"2024-09-11T12:49:42.428489+0000","last_active":"2024-09-11T12:49:44.442812+0000","last_peered":"2024-09-11T12:49:44.442812+0000","last_clean":"2024-09-11T12:49:44.442812+0000","last_became_active":"2024-09-11T12:49:42.427771+0000","last_became_peered":"2024-09-11T12:49:42.427771+0000","last_unstale":"2024-09-11T12:49:44.442812+0000","last_undegraded":"2024-09-11T12:49:44.442812+0000","last_fullsized":"2024-09-11T12:49:44.442812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472303+0000","last_change":"2024-09-11T12:49:43.463393+0000","last_active":"2024-09-11T12:49:44.472303+0000","last_peered":"2024-09-11T12:49:44.472303+0000","last_clean":"2024-09-11T12:49:44.472303+0000","last_became_active":"2024-09-11T12:49:43.462936+0000","last_became_peered":"2024-09-11T12:49:43.462936+0000","last_unstale":"2024-09-11T12:49:44.472303+0000","last_undegraded":"2024-09-11T12:49:44.472303+0000","last_fullsized":"2024-09-11T12:49:44.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446267+0000","last_change":"2024-09-11T12:49:42.440111+0000","last_active":"2024-09-11T12:49:44.446267+0000","last_peered":"2024-09-11T12:49:44.446267+0000","last_clean":"2024-09-11T12:49:44.446267+0000","last_became_active":"2024-09-11T12:49:42.438898+0000","last_became_peered":"2024-09-11T12:49:42.438898+0000","last_unstale":"2024-09-11T12:49:44.446267+0000","last_undegraded":"2024-09-11T12:49:44.446267+0000","last_fullsized":"2024-09-11T12:49:44.446267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:05.274573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472602+0000","last_change":"2024-09-11T12:49:43.457973+0000","last_active":"2024-09-11T12:49:44.472602+0000","last_peered":"2024-09-11T12:49:44.472602+0000","last_clean":"2024-09-11T12:49:44.472602+0000","last_became_active":"2024-09-11T12:49:43.457699+0000","last_became_peered":"2024-09-11T12:49:43.457699+0000","last_unstale":"2024-09-11T12:49:44.472602+0000","last_undegraded":"2024-09-11T12:49:44.472602+0000","last_fullsized":"2024-09-11T12:49:44.472602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443194+0000","last_change":"2024-09-11T12:49:42.438768+0000","last_active":"2024-09-11T12:49:44.443194+0000","last_peered":"2024-09-11T12:49:44.443194+0000","last_clean":"2024-09-11T12:49:44.443194+0000","last_became_active":"2024-09-11T12:49:42.438221+0000","last_became_peered":"2024-09-11T12:49:42.438221+0000","last_unstale":"2024-09-11T12:49:44.443194+0000","last_undegraded":"2024-09-11T12:49:44.443194+0000","last_fullsized":"2024-09-11T12:49:44.443194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472978+0000","last_change":"2024-09-11T12:49:43.463216+0000","last_active":"2024-09-11T12:49:44.472978+0000","last_peered":"2024-09-11T12:49:44.472978+0000","last_clean":"2024-09-11T12:49:44.472978+0000","last_became_active":"2024-09-11T12:49:43.462863+0000","last_became_peered":"2024-09-11T12:49:43.462863+0000","last_unstale":"2024-09-11T12:49:44.472978+0000","last_undegraded":"2024-09-11T12:49:44.472978+0000","last_fullsized":"2024-09-11T12:49:44.472978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452079+0000","last_change":"2024-09-11T12:49:42.430270+0000","last_active":"2024-09-11T12:49:44.452079+0000","last_peered":"2024-09-11T12:49:44.452079+0000","last_clean":"2024-09-11T12:49:44.452079+0000","last_became_active":"2024-09-11T12:49:42.429966+0000","last_became_peered":"2024-09-11T12:49:42.429966+0000","last_unstale":"2024-09-11T12:49:44.452079+0000","last_undegraded":"2024-09-11T12:49:44.452079+0000","last_fullsized":"2024-09-11T12:49:44.452079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445265+0000","last_change":"2024-09-11T12:49:43.445039+0000","last_active":"2024-09-11T12:49:44.445265+0000","last_peered":"2024-09-11T12:49:44.445265+0000","last_clean":"2024-09-11T12:49:44.445265+0000","last_became_active":"2024-09-11T12:49:43.443152+0000","last_became_peered":"2024-09-11T12:49:43.443152+0000","last_unstale":"2024-09-11T12:49:44.445265+0000","last_undegraded":"2024-09-11T12:49:44.445265+0000","last_fullsized":"2024-09-11T12:49:44.445265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443025+0000","last_change":"2024-09-11T12:49:42.440447+0000","last_active":"2024-09-11T12:49:44.443025+0000","last_peered":"2024-09-11T12:49:44.443025+0000","last_clean":"2024-09-11T12:49:44.443025+0000","last_became_active":"2024-09-11T12:49:42.439837+0000","last_became_peered":"2024-09-11T12:49:42.439837+0000","last_unstale":"2024-09-11T12:49:44.443025+0000","last_undegraded":"2024-09-11T12:49:44.443025+0000","last_fullsized":"2024-09-11T12:49:44.443025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472502+0000","last_change":"2024-09-11T12:49:43.470639+0000","last_active":"2024-09-11T12:49:44.472502+0000","last_peered":"2024-09-11T12:49:44.472502+0000","last_clean":"2024-09-11T12:49:44.472502+0000","last_became_active":"2024-09-11T12:49:43.468548+0000","last_became_peered":"2024-09-11T12:49:43.468548+0000","last_unstale":"2024-09-11T12:49:44.472502+0000","last_undegraded":"2024-09-11T12:49:44.472502+0000","last_fullsized":"2024-09-11T12:49:44.472502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472244+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:44.472244+0000","last_peered":"2024-09-11T12:49:44.472244+0000","last_clean":"2024-09-11T12:49:44.472244+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:44.472244+0000","last_undegraded":"2024-09-11T12:49:44.472244+0000","last_fullsized":"2024-09-11T12:49:44.472244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444614+0000","last_change":"2024-09-11T12:49:42.436719+0000","last_active":"2024-09-11T12:49:44.444614+0000","last_peered":"2024-09-11T12:49:44.444614+0000","last_clean":"2024-09-11T12:49:44.444614+0000","last_became_active":"2024-09-11T12:49:42.436415+0000","last_became_peered":"2024-09-11T12:49:42.436415+0000","last_unstale":"2024-09-11T12:49:44.444614+0000","last_undegraded":"2024-09-11T12:49:44.444614+0000","last_fullsized":"2024-09-11T12:49:44.444614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471442+0000","last_change":"2024-09-11T12:49:43.462048+0000","last_active":"2024-09-11T12:49:44.471442+0000","last_peered":"2024-09-11T12:49:44.471442+0000","last_clean":"2024-09-11T12:49:44.471442+0000","last_became_active":"2024-09-11T12:49:43.459912+0000","last_became_peered":"2024-09-11T12:49:43.459912+0000","last_unstale":"2024-09-11T12:49:44.471442+0000","last_undegraded":"2024-09-11T12:49:44.471442+0000","last_fullsized":"2024-09-11T12:49:44.471442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472116+0000","last_change":"2024-09-11T12:49:42.448655+0000","last_active":"2024-09-11T12:49:44.472116+0000","last_peered":"2024-09-11T12:49:44.472116+0000","last_clean":"2024-09-11T12:49:44.472116+0000","last_became_active":"2024-09-11T12:49:42.448394+0000","last_became_peered":"2024-09-11T12:49:42.448394+0000","last_unstale":"2024-09-11T12:49:44.472116+0000","last_undegraded":"2024-09-11T12:49:44.472116+0000","last_fullsized":"2024-09-11T12:49:44.472116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443004+0000","last_change":"2024-09-11T12:49:42.439971+0000","last_active":"2024-09-11T12:49:44.443004+0000","last_peered":"2024-09-11T12:49:44.443004+0000","last_clean":"2024-09-11T12:49:44.443004+0000","last_became_active":"2024-09-11T12:49:42.438664+0000","last_became_peered":"2024-09-11T12:49:42.438664+0000","last_unstale":"2024-09-11T12:49:44.443004+0000","last_undegraded":"2024-09-11T12:49:44.443004+0000","last_fullsized":"2024-09-11T12:49:44.443004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451753+0000","last_change":"2024-09-11T12:49:43.437660+0000","last_active":"2024-09-11T12:49:44.451753+0000","last_peered":"2024-09-11T12:49:44.451753+0000","last_clean":"2024-09-11T12:49:44.451753+0000","last_became_active":"2024-09-11T12:49:43.437383+0000","last_became_peered":"2024-09-11T12:49:43.437383+0000","last_unstale":"2024-09-11T12:49:44.451753+0000","last_undegraded":"2024-09-11T12:49:44.451753+0000","last_fullsized":"2024-09-11T12:49:44.451753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444974+0000","last_change":"2024-09-11T12:49:42.442628+0000","last_active":"2024-09-11T12:49:44.444974+0000","last_peered":"2024-09-11T12:49:44.444974+0000","last_clean":"2024-09-11T12:49:44.444974+0000","last_became_active":"2024-09-11T12:49:42.442343+0000","last_became_peered":"2024-09-11T12:49:42.442343+0000","last_unstale":"2024-09-11T12:49:44.444974+0000","last_undegraded":"2024-09-11T12:49:44.444974+0000","last_fullsized":"2024-09-11T12:49:44.444974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445987+0000","last_change":"2024-09-11T12:49:43.467784+0000","last_active":"2024-09-11T12:49:44.445987+0000","last_peered":"2024-09-11T12:49:44.445987+0000","last_clean":"2024-09-11T12:49:44.445987+0000","last_became_active":"2024-09-11T12:49:43.467380+0000","last_became_peered":"2024-09-11T12:49:43.467380+0000","last_unstale":"2024-09-11T12:49:44.445987+0000","last_undegraded":"2024-09-11T12:49:44.445987+0000","last_fullsized":"2024-09-11T12:49:44.445987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:39.392320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451933+0000","last_change":"2024-09-11T12:49:42.436463+0000","last_active":"2024-09-11T12:49:44.451933+0000","last_peered":"2024-09-11T12:49:44.451933+0000","last_clean":"2024-09-11T12:49:44.451933+0000","last_became_active":"2024-09-11T12:49:42.436201+0000","last_became_peered":"2024-09-11T12:49:42.436201+0000","last_unstale":"2024-09-11T12:49:44.451933+0000","last_undegraded":"2024-09-11T12:49:44.451933+0000","last_fullsized":"2024-09-11T12:49:44.451933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445376+0000","last_change":"2024-09-11T12:49:43.449369+0000","last_active":"2024-09-11T12:49:44.445376+0000","last_peered":"2024-09-11T12:49:44.445376+0000","last_clean":"2024-09-11T12:49:44.445376+0000","last_became_active":"2024-09-11T12:49:43.443602+0000","last_became_peered":"2024-09-11T12:49:43.443602+0000","last_unstale":"2024-09-11T12:49:44.445376+0000","last_undegraded":"2024-09-11T12:49:44.445376+0000","last_fullsized":"2024-09-11T12:49:44.445376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472687+0000","last_change":"2024-09-11T12:49:42.449197+0000","last_active":"2024-09-11T12:49:44.472687+0000","last_peered":"2024-09-11T12:49:44.472687+0000","last_clean":"2024-09-11T12:49:44.472687+0000","last_became_active":"2024-09-11T12:49:42.448893+0000","last_became_peered":"2024-09-11T12:49:42.448893+0000","last_unstale":"2024-09-11T12:49:44.472687+0000","last_undegraded":"2024-09-11T12:49:44.472687+0000","last_fullsized":"2024-09-11T12:49:44.472687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:35.796869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451779+0000","last_change":"2024-09-11T12:49:43.439241+0000","last_active":"2024-09-11T12:49:44.451779+0000","last_peered":"2024-09-11T12:49:44.451779+0000","last_clean":"2024-09-11T12:49:44.451779+0000","last_became_active":"2024-09-11T12:49:43.438486+0000","last_became_peered":"2024-09-11T12:49:43.438486+0000","last_unstale":"2024-09-11T12:49:44.451779+0000","last_undegraded":"2024-09-11T12:49:44.451779+0000","last_fullsized":"2024-09-11T12:49:44.451779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452121+0000","last_change":"2024-09-11T12:49:42.445521+0000","last_active":"2024-09-11T12:49:44.452121+0000","last_peered":"2024-09-11T12:49:44.452121+0000","last_clean":"2024-09-11T12:49:44.452121+0000","last_became_active":"2024-09-11T12:49:42.445218+0000","last_became_peered":"2024-09-11T12:49:42.445218+0000","last_unstale":"2024-09-11T12:49:44.452121+0000","last_undegraded":"2024-09-11T12:49:44.452121+0000","last_fullsized":"2024-09-11T12:49:44.452121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472486+0000","last_change":"2024-09-11T12:49:43.454081+0000","last_active":"2024-09-11T12:49:44.472486+0000","last_peered":"2024-09-11T12:49:44.472486+0000","last_clean":"2024-09-11T12:49:44.472486+0000","last_became_active":"2024-09-11T12:49:43.453820+0000","last_became_peered":"2024-09-11T12:49:43.453820+0000","last_unstale":"2024-09-11T12:49:44.472486+0000","last_undegraded":"2024-09-11T12:49:44.472486+0000","last_fullsized":"2024-09-11T12:49:44.472486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455425+0000","last_change":"2024-09-11T12:49:42.446529+0000","last_active":"2024-09-11T12:49:44.455425+0000","last_peered":"2024-09-11T12:49:44.455425+0000","last_clean":"2024-09-11T12:49:44.455425+0000","last_became_active":"2024-09-11T12:49:42.446268+0000","last_became_peered":"2024-09-11T12:49:42.446268+0000","last_unstale":"2024-09-11T12:49:44.455425+0000","last_undegraded":"2024-09-11T12:49:44.455425+0000","last_fullsized":"2024-09-11T12:49:44.455425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472114+0000","last_change":"2024-09-11T12:49:42.445423+0000","last_active":"2024-09-11T12:49:44.472114+0000","last_peered":"2024-09-11T12:49:44.472114+0000","last_clean":"2024-09-11T12:49:44.472114+0000","last_became_active":"2024-09-11T12:49:42.444814+0000","last_became_peered":"2024-09-11T12:49:42.444814+0000","last_unstale":"2024-09-11T12:49:44.472114+0000","last_undegraded":"2024-09-11T12:49:44.472114+0000","last_fullsized":"2024-09-11T12:49:44.472114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445597+0000","last_change":"2024-09-11T12:49:43.459523+0000","last_active":"2024-09-11T12:49:44.445597+0000","last_peered":"2024-09-11T12:49:44.445597+0000","last_clean":"2024-09-11T12:49:44.445597+0000","last_became_active":"2024-09-11T12:49:43.459164+0000","last_became_peered":"2024-09-11T12:49:43.459164+0000","last_unstale":"2024-09-11T12:49:44.445597+0000","last_undegraded":"2024-09-11T12:49:44.445597+0000","last_fullsized":"2024-09-11T12:49:44.445597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:50.017910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451744+0000","last_change":"2024-09-11T12:49:42.431712+0000","last_active":"2024-09-11T12:49:44.451744+0000","last_peered":"2024-09-11T12:49:44.451744+0000","last_clean":"2024-09-11T12:49:44.451744+0000","last_became_active":"2024-09-11T12:49:42.431429+0000","last_became_peered":"2024-09-11T12:49:42.431429+0000","last_unstale":"2024-09-11T12:49:44.451744+0000","last_undegraded":"2024-09-11T12:49:44.451744+0000","last_fullsized":"2024-09-11T12:49:44.451744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471398+0000","last_change":"2024-09-11T12:49:43.444653+0000","last_active":"2024-09-11T12:49:44.471398+0000","last_peered":"2024-09-11T12:49:44.471398+0000","last_clean":"2024-09-11T12:49:44.471398+0000","last_became_active":"2024-09-11T12:49:43.444206+0000","last_became_peered":"2024-09-11T12:49:43.444206+0000","last_unstale":"2024-09-11T12:49:44.471398+0000","last_undegraded":"2024-09-11T12:49:44.471398+0000","last_fullsized":"2024-09-11T12:49:44.471398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451809+0000","last_change":"2024-09-11T12:49:42.439128+0000","last_active":"2024-09-11T12:49:44.451809+0000","last_peered":"2024-09-11T12:49:44.451809+0000","last_clean":"2024-09-11T12:49:44.451809+0000","last_became_active":"2024-09-11T12:49:42.438820+0000","last_became_peered":"2024-09-11T12:49:42.438820+0000","last_unstale":"2024-09-11T12:49:44.451809+0000","last_undegraded":"2024-09-11T12:49:44.451809+0000","last_fullsized":"2024-09-11T12:49:44.451809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471928+0000","last_change":"2024-09-11T12:49:43.447109+0000","last_active":"2024-09-11T12:49:44.471928+0000","last_peered":"2024-09-11T12:49:44.471928+0000","last_clean":"2024-09-11T12:49:44.471928+0000","last_became_active":"2024-09-11T12:49:43.446540+0000","last_became_peered":"2024-09-11T12:49:43.446540+0000","last_unstale":"2024-09-11T12:49:44.471928+0000","last_undegraded":"2024-09-11T12:49:44.471928+0000","last_fullsized":"2024-09-11T12:49:44.471928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472587+0000","last_change":"2024-09-11T12:49:42.447202+0000","last_active":"2024-09-11T12:49:44.472587+0000","last_peered":"2024-09-11T12:49:44.472587+0000","last_clean":"2024-09-11T12:49:44.472587+0000","last_became_active":"2024-09-11T12:49:42.446936+0000","last_became_peered":"2024-09-11T12:49:42.446936+0000","last_unstale":"2024-09-11T12:49:44.472587+0000","last_undegraded":"2024-09-11T12:49:44.472587+0000","last_fullsized":"2024-09-11T12:49:44.472587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455309+0000","last_change":"2024-09-11T12:49:43.454367+0000","last_active":"2024-09-11T12:49:44.455309+0000","last_peered":"2024-09-11T12:49:44.455309+0000","last_clean":"2024-09-11T12:49:44.455309+0000","last_became_active":"2024-09-11T12:49:43.454114+0000","last_became_peered":"2024-09-11T12:49:43.454114+0000","last_unstale":"2024-09-11T12:49:44.455309+0000","last_undegraded":"2024-09-11T12:49:44.455309+0000","last_fullsized":"2024-09-11T12:49:44.455309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445226+0000","last_change":"2024-09-11T12:49:43.449972+0000","last_active":"2024-09-11T12:49:44.445226+0000","last_peered":"2024-09-11T12:49:44.445226+0000","last_clean":"2024-09-11T12:49:44.445226+0000","last_became_active":"2024-09-11T12:49:43.449653+0000","last_became_peered":"2024-09-11T12:49:43.449653+0000","last_unstale":"2024-09-11T12:49:44.445226+0000","last_undegraded":"2024-09-11T12:49:44.445226+0000","last_fullsized":"2024-09-11T12:49:44.445226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472376+0000","last_change":"2024-09-11T12:49:42.443992+0000","last_active":"2024-09-11T12:49:44.472376+0000","last_peered":"2024-09-11T12:49:44.472376+0000","last_clean":"2024-09-11T12:49:44.472376+0000","last_became_active":"2024-09-11T12:49:42.443492+0000","last_became_peered":"2024-09-11T12:49:42.443492+0000","last_unstale":"2024-09-11T12:49:44.472376+0000","last_undegraded":"2024-09-11T12:49:44.472376+0000","last_fullsized":"2024-09-11T12:49:44.472376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:07.781904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445104+0000","last_change":"2024-09-11T12:49:43.449359+0000","last_active":"2024-09-11T12:49:44.445104+0000","last_peered":"2024-09-11T12:49:44.445104+0000","last_clean":"2024-09-11T12:49:44.445104+0000","last_became_active":"2024-09-11T12:49:43.448838+0000","last_became_peered":"2024-09-11T12:49:43.448838+0000","last_unstale":"2024-09-11T12:49:44.445104+0000","last_undegraded":"2024-09-11T12:49:44.445104+0000","last_fullsized":"2024-09-11T12:49:44.445104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472595+0000","last_change":"2024-09-11T12:49:42.450569+0000","last_active":"2024-09-11T12:49:44.472595+0000","last_peered":"2024-09-11T12:49:44.472595+0000","last_clean":"2024-09-11T12:49:44.472595+0000","last_became_active":"2024-09-11T12:49:42.450338+0000","last_became_peered":"2024-09-11T12:49:42.450338+0000","last_unstale":"2024-09-11T12:49:44.472595+0000","last_undegraded":"2024-09-11T12:49:44.472595+0000","last_fullsized":"2024-09-11T12:49:44.472595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472335+0000","last_change":"2024-09-11T12:49:42.446328+0000","last_active":"2024-09-11T12:49:44.472335+0000","last_peered":"2024-09-11T12:49:44.472335+0000","last_clean":"2024-09-11T12:49:44.472335+0000","last_became_active":"2024-09-11T12:49:42.446059+0000","last_became_peered":"2024-09-11T12:49:42.446059+0000","last_unstale":"2024-09-11T12:49:44.472335+0000","last_undegraded":"2024-09-11T12:49:44.472335+0000","last_fullsized":"2024-09-11T12:49:44.472335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472359+0000","last_change":"2024-09-11T12:49:43.443985+0000","last_active":"2024-09-11T12:49:44.472359+0000","last_peered":"2024-09-11T12:49:44.472359+0000","last_clean":"2024-09-11T12:49:44.472359+0000","last_became_active":"2024-09-11T12:49:43.443740+0000","last_became_peered":"2024-09-11T12:49:43.443740+0000","last_unstale":"2024-09-11T12:49:44.472359+0000","last_undegraded":"2024-09-11T12:49:44.472359+0000","last_fullsized":"2024-09-11T12:49:44.472359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542151,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:57.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- 172.21.15.78:0/2492081002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb28015df00 msgr2=0x7fb28015c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 --2- 172.21.15.78:0/2492081002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb28015df00 0x7fb28015c1d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb2800c6950 tx=0x7fb27401cc40 comp rx=0 tx=0).stop 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- 172.21.15.78:0/2492081002 shutdown_connections 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 --2- 172.21.15.78:0/2492081002 >> v2:172.21.15.157:3300/0 conn(0x7fb2800cfa20 0x7fb2800cfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 --2- 172.21.15.78:0/2492081002 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb28015df00 0x7fb28015c1d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- 172.21.15.78:0/2492081002 >> 172.21.15.78:0/2492081002 conn(0x7fb28006c650 msgr2=0x7fb28006ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:57.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- 172.21.15.78:0/2492081002 shutdown_connections 2024-09-11T12:49:57.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- 172.21.15.78:0/2492081002 wait complete. 2024-09-11T12:49:57.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 Processor -- start 2024-09-11T12:49:57.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.266+0000 7fb288200640 1 -- start start 2024-09-11T12:49:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb28015df00 0x7fb280133280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb28013af40 0x7fb280135150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2801679e0 con 0x7fb2800cfa20 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb280167860 con 0x7fb28013af40 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb280167b60 con 0x7fb28015df00 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60244/0 (socket says 172.21.15.78:60244) 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 -- 172.21.15.78:0/2972437848 learned_addr learned my addr 172.21.15.78:0/2972437848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb286200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb28013af40 0x7fb280135150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb284e00640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb28015df00 0x7fb280133280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 -- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb28015df00 msgr2=0x7fb280133280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb28015df00 0x7fb280133280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 -- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb28013af40 msgr2=0x7fb280135150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb28013af40 0x7fb280135150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 -- 172.21.15.78:0/2972437848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28015a4d0 con 0x7fb2800cfa20 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb285800640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb274000f90 tx=0x7fb274009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740bb070 con 0x7fb2800cfa20 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb274004d40 con 0x7fb2800cfa20 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2740b2040 con 0x7fb2800cfa20 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28013acd0 con 0x7fb2800cfa20 2024-09-11T12:49:57.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2801407a0 con 0x7fb2800cfa20 2024-09-11T12:49:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2800cfeb0 con 0x7fb2800cfa20 2024-09-11T12:49:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.270+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fb2740be070 con 0x7fb2800cfa20 2024-09-11T12:49:57.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.274+0000 7fb27d600640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 0x7fb260073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.274+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb27412bbf0 con 0x7fb2800cfa20 2024-09-11T12:49:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.274+0000 7fb284e00640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 0x7fb260073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.274+0000 7fb284e00640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 0x7fb260073b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb28015b0c0 tx=0x7fb26c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:57.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.278+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb274016950 con 0x7fb2800cfa20 2024-09-11T12:49:57.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.442+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb28015b0f0 con 0x7fb2600716b0 2024-09-11T12:49:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.446+0000 7fb27d600640 1 -- 172.21.15.78:0/2972437848 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7fb28015b0f0 con 0x7fb2600716b0 2024-09-11T12:49:57.452 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:57.454 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 msgr2=0x7fb260073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 0x7fb260073b70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb28015b0c0 tx=0x7fb26c009040 comp rx=0 tx=0).stop 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 msgr2=0x7fb2800cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb274000f90 tx=0x7fb274009360 comp rx=0 tx=0).stop 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 shutdown_connections 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb2600716b0 0x7fb260073b70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb28013af40 0x7fb280135150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb28015df00 0x7fb280133280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 --2- 172.21.15.78:0/2972437848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2800cfa20 0x7fb2800cc370 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 >> 172.21.15.78:0/2972437848 conn(0x7fb28006c650 msgr2=0x7fb28015d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 shutdown_connections 2024-09-11T12:49:57.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.450+0000 7fb288200640 1 -- 172.21.15.78:0/2972437848 wait complete. 2024-09-11T12:49:57.472 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2024-09-11T12:49:57.107978+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235956,"kb_used_data":3048,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738764,"statfs":{"total":773094113280,"available":772852494336,"internally_reserved":0,"allocated":3121152,"data_stored":1833196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003078"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472816+0000","last_change":"2024-09-11T12:49:43.456400+0000","last_active":"2024-09-11T12:49:44.472816+0000","last_peered":"2024-09-11T12:49:44.472816+0000","last_clean":"2024-09-11T12:49:44.472816+0000","last_became_active":"2024-09-11T12:49:43.456074+0000","last_became_peered":"2024-09-11T12:49:43.456074+0000","last_unstale":"2024-09-11T12:49:44.472816+0000","last_undegraded":"2024-09-11T12:49:44.472816+0000","last_fullsized":"2024-09-11T12:49:44.472816+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452157+0000","last_change":"2024-09-11T12:49:43.446993+0000","last_active":"2024-09-11T12:49:44.452157+0000","last_peered":"2024-09-11T12:49:44.452157+0000","last_clean":"2024-09-11T12:49:44.452157+0000","last_became_active":"2024-09-11T12:49:43.443935+0000","last_became_peered":"2024-09-11T12:49:43.443935+0000","last_unstale":"2024-09-11T12:49:44.452157+0000","last_undegraded":"2024-09-11T12:49:44.452157+0000","last_fullsized":"2024-09-11T12:49:44.452157+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472047+0000","last_change":"2024-09-11T12:49:43.440868+0000","last_active":"2024-09-11T12:49:44.472047+0000","last_peered":"2024-09-11T12:49:44.472047+0000","last_clean":"2024-09-11T12:49:44.472047+0000","last_became_active":"2024-09-11T12:49:43.440592+0000","last_became_peered":"2024-09-11T12:49:43.440592+0000","last_unstale":"2024-09-11T12:49:44.472047+0000","last_undegraded":"2024-09-11T12:49:44.472047+0000","last_fullsized":"2024-09-11T12:49:44.472047+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:33.322655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444935+0000","last_change":"2024-09-11T12:49:43.444872+0000","last_active":"2024-09-11T12:49:44.444935+0000","last_peered":"2024-09-11T12:49:44.444935+0000","last_clean":"2024-09-11T12:49:44.444935+0000","last_became_active":"2024-09-11T12:49:43.444523+0000","last_became_peered":"2024-09-11T12:49:43.444523+0000","last_unstale":"2024-09-11T12:49:44.444935+0000","last_undegraded":"2024-09-11T12:49:44.444935+0000","last_fullsized":"2024-09-11T12:49:44.444935+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472516+0000","last_change":"2024-09-11T12:49:43.454578+0000","last_active":"2024-09-11T12:49:44.472516+0000","last_peered":"2024-09-11T12:49:44.472516+0000","last_clean":"2024-09-11T12:49:44.472516+0000","last_became_active":"2024-09-11T12:49:43.454355+0000","last_became_peered":"2024-09-11T12:49:43.454355+0000","last_unstale":"2024-09-11T12:49:44.472516+0000","last_undegraded":"2024-09-11T12:49:44.472516+0000","last_fullsized":"2024-09-11T12:49:44.472516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444824+0000","last_change":"2024-09-11T12:49:43.447252+0000","last_active":"2024-09-11T12:49:44.444824+0000","last_peered":"2024-09-11T12:49:44.444824+0000","last_clean":"2024-09-11T12:49:44.444824+0000","last_became_active":"2024-09-11T12:49:43.446860+0000","last_became_peered":"2024-09-11T12:49:43.446860+0000","last_unstale":"2024-09-11T12:49:44.444824+0000","last_undegraded":"2024-09-11T12:49:44.444824+0000","last_fullsized":"2024-09-11T12:49:44.444824+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472261+0000","last_change":"2024-09-11T12:49:43.468669+0000","last_active":"2024-09-11T12:49:44.472261+0000","last_peered":"2024-09-11T12:49:44.472261+0000","last_clean":"2024-09-11T12:49:44.472261+0000","last_became_active":"2024-09-11T12:49:43.468316+0000","last_became_peered":"2024-09-11T12:49:43.468316+0000","last_unstale":"2024-09-11T12:49:44.472261+0000","last_undegraded":"2024-09-11T12:49:44.472261+0000","last_fullsized":"2024-09-11T12:49:44.472261+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471049+0000","last_change":"2024-09-11T12:49:43.458945+0000","last_active":"2024-09-11T12:49:44.471049+0000","last_peered":"2024-09-11T12:49:44.471049+0000","last_clean":"2024-09-11T12:49:44.471049+0000","last_became_active":"2024-09-11T12:49:43.458677+0000","last_became_peered":"2024-09-11T12:49:43.458677+0000","last_unstale":"2024-09-11T12:49:44.471049+0000","last_undegraded":"2024-09-11T12:49:44.471049+0000","last_fullsized":"2024-09-11T12:49:44.471049+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472868+0000","last_change":"2024-09-11T12:49:43.469306+0000","last_active":"2024-09-11T12:49:44.472868+0000","last_peered":"2024-09-11T12:49:44.472868+0000","last_clean":"2024-09-11T12:49:44.472868+0000","last_became_active":"2024-09-11T12:49:43.469031+0000","last_became_peered":"2024-09-11T12:49:43.469031+0000","last_unstale":"2024-09-11T12:49:44.472868+0000","last_undegraded":"2024-09-11T12:49:44.472868+0000","last_fullsized":"2024-09-11T12:49:44.472868+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455525+0000","last_change":"2024-09-11T12:49:43.462111+0000","last_active":"2024-09-11T12:49:44.455525+0000","last_peered":"2024-09-11T12:49:44.455525+0000","last_clean":"2024-09-11T12:49:44.455525+0000","last_became_active":"2024-09-11T12:49:43.461818+0000","last_became_peered":"2024-09-11T12:49:43.461818+0000","last_unstale":"2024-09-11T12:49:44.455525+0000","last_undegraded":"2024-09-11T12:49:44.455525+0000","last_fullsized":"2024-09-11T12:49:44.455525+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455751+0000","last_change":"2024-09-11T12:49:43.454080+0000","last_active":"2024-09-11T12:49:44.455751+0000","last_peered":"2024-09-11T12:49:44.455751+0000","last_clean":"2024-09-11T12:49:44.455751+0000","last_became_active":"2024-09-11T12:49:43.453788+0000","last_became_peered":"2024-09-11T12:49:43.453788+0000","last_unstale":"2024-09-11T12:49:44.455751+0000","last_undegraded":"2024-09-11T12:49:44.455751+0000","last_fullsized":"2024-09-11T12:49:44.455751+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446203+0000","last_change":"2024-09-11T12:49:43.447103+0000","last_active":"2024-09-11T12:49:44.446203+0000","last_peered":"2024-09-11T12:49:44.446203+0000","last_clean":"2024-09-11T12:49:44.446203+0000","last_became_active":"2024-09-11T12:49:43.446839+0000","last_became_peered":"2024-09-11T12:49:43.446839+0000","last_unstale":"2024-09-11T12:49:44.446203+0000","last_undegraded":"2024-09-11T12:49:44.446203+0000","last_fullsized":"2024-09-11T12:49:44.446203+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471850+0000","last_change":"2024-09-11T12:49:43.446894+0000","last_active":"2024-09-11T12:49:44.471850+0000","last_peered":"2024-09-11T12:49:44.471850+0000","last_clean":"2024-09-11T12:49:44.471850+0000","last_became_active":"2024-09-11T12:49:43.446045+0000","last_became_peered":"2024-09-11T12:49:43.446045+0000","last_unstale":"2024-09-11T12:49:44.471850+0000","last_undegraded":"2024-09-11T12:49:44.471850+0000","last_fullsized":"2024-09-11T12:49:44.471850+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471204+0000","last_change":"2024-09-11T12:49:43.442536+0000","last_active":"2024-09-11T12:49:44.471204+0000","last_peered":"2024-09-11T12:49:44.471204+0000","last_clean":"2024-09-11T12:49:44.471204+0000","last_became_active":"2024-09-11T12:49:43.442257+0000","last_became_peered":"2024-09-11T12:49:43.442257+0000","last_unstale":"2024-09-11T12:49:44.471204+0000","last_undegraded":"2024-09-11T12:49:44.471204+0000","last_fullsized":"2024-09-11T12:49:44.471204+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:35.424245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451685+0000","last_change":"2024-09-11T12:49:43.446706+0000","last_active":"2024-09-11T12:49:44.451685+0000","last_peered":"2024-09-11T12:49:44.451685+0000","last_clean":"2024-09-11T12:49:44.451685+0000","last_became_active":"2024-09-11T12:49:43.446340+0000","last_became_peered":"2024-09-11T12:49:43.446340+0000","last_unstale":"2024-09-11T12:49:44.451685+0000","last_undegraded":"2024-09-11T12:49:44.451685+0000","last_fullsized":"2024-09-11T12:49:44.451685+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444746+0000","last_change":"2024-09-11T12:49:43.446371+0000","last_active":"2024-09-11T12:49:44.444746+0000","last_peered":"2024-09-11T12:49:44.444746+0000","last_clean":"2024-09-11T12:49:44.444746+0000","last_became_active":"2024-09-11T12:49:43.446163+0000","last_became_peered":"2024-09-11T12:49:43.446163+0000","last_unstale":"2024-09-11T12:49:44.444746+0000","last_undegraded":"2024-09-11T12:49:44.444746+0000","last_fullsized":"2024-09-11T12:49:44.444746+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445420+0000","last_change":"2024-09-11T12:49:43.461338+0000","last_active":"2024-09-11T12:49:44.445420+0000","last_peered":"2024-09-11T12:49:44.445420+0000","last_clean":"2024-09-11T12:49:44.445420+0000","last_became_active":"2024-09-11T12:49:43.461065+0000","last_became_peered":"2024-09-11T12:49:43.461065+0000","last_unstale":"2024-09-11T12:49:44.445420+0000","last_undegraded":"2024-09-11T12:49:44.445420+0000","last_fullsized":"2024-09-11T12:49:44.445420+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:02:48.494575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443267+0000","last_change":"2024-09-11T12:49:43.436172+0000","last_active":"2024-09-11T12:49:44.443267+0000","last_peered":"2024-09-11T12:49:44.443267+0000","last_clean":"2024-09-11T12:49:44.443267+0000","last_became_active":"2024-09-11T12:49:43.435740+0000","last_became_peered":"2024-09-11T12:49:43.435740+0000","last_unstale":"2024-09-11T12:49:44.443267+0000","last_undegraded":"2024-09-11T12:49:44.443267+0000","last_fullsized":"2024-09-11T12:49:44.443267+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472800+0000","last_change":"2024-09-11T12:49:43.459483+0000","last_active":"2024-09-11T12:49:44.472800+0000","last_peered":"2024-09-11T12:49:44.472800+0000","last_clean":"2024-09-11T12:49:44.472800+0000","last_became_active":"2024-09-11T12:49:43.458688+0000","last_became_peered":"2024-09-11T12:49:43.458688+0000","last_unstale":"2024-09-11T12:49:44.472800+0000","last_undegraded":"2024-09-11T12:49:44.472800+0000","last_fullsized":"2024-09-11T12:49:44.472800+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455412+0000","last_change":"2024-09-11T12:49:43.446362+0000","last_active":"2024-09-11T12:49:44.455412+0000","last_peered":"2024-09-11T12:49:44.455412+0000","last_clean":"2024-09-11T12:49:44.455412+0000","last_became_active":"2024-09-11T12:49:43.446092+0000","last_became_peered":"2024-09-11T12:49:43.446092+0000","last_unstale":"2024-09-11T12:49:44.455412+0000","last_undegraded":"2024-09-11T12:49:44.455412+0000","last_fullsized":"2024-09-11T12:49:44.455412+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471869+0000","last_change":"2024-09-11T12:49:43.457752+0000","last_active":"2024-09-11T12:49:44.471869+0000","last_peered":"2024-09-11T12:49:44.471869+0000","last_clean":"2024-09-11T12:49:44.471869+0000","last_became_active":"2024-09-11T12:49:43.454852+0000","last_became_peered":"2024-09-11T12:49:43.454852+0000","last_unstale":"2024-09-11T12:49:44.471869+0000","last_undegraded":"2024-09-11T12:49:44.471869+0000","last_fullsized":"2024-09-11T12:49:44.471869+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:45.616829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444759+0000","last_change":"2024-09-11T12:49:43.447439+0000","last_active":"2024-09-11T12:49:44.444759+0000","last_peered":"2024-09-11T12:49:44.444759+0000","last_clean":"2024-09-11T12:49:44.444759+0000","last_became_active":"2024-09-11T12:49:43.447044+0000","last_became_peered":"2024-09-11T12:49:43.447044+0000","last_unstale":"2024-09-11T12:49:44.444759+0000","last_undegraded":"2024-09-11T12:49:44.444759+0000","last_fullsized":"2024-09-11T12:49:44.444759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471454+0000","last_change":"2024-09-11T12:49:43.455751+0000","last_active":"2024-09-11T12:49:44.471454+0000","last_peered":"2024-09-11T12:49:44.471454+0000","last_clean":"2024-09-11T12:49:44.471454+0000","last_became_active":"2024-09-11T12:49:43.455514+0000","last_became_peered":"2024-09-11T12:49:43.455514+0000","last_unstale":"2024-09-11T12:49:44.471454+0000","last_undegraded":"2024-09-11T12:49:44.471454+0000","last_fullsized":"2024-09-11T12:49:44.471454+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444864+0000","last_change":"2024-09-11T12:49:43.443468+0000","last_active":"2024-09-11T12:49:44.444864+0000","last_peered":"2024-09-11T12:49:44.444864+0000","last_clean":"2024-09-11T12:49:44.444864+0000","last_became_active":"2024-09-11T12:49:43.443158+0000","last_became_peered":"2024-09-11T12:49:43.443158+0000","last_unstale":"2024-09-11T12:49:44.444864+0000","last_undegraded":"2024-09-11T12:49:44.444864+0000","last_fullsized":"2024-09-11T12:49:44.444864+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472404+0000","last_change":"2024-09-11T12:49:43.450952+0000","last_active":"2024-09-11T12:49:44.472404+0000","last_peered":"2024-09-11T12:49:44.472404+0000","last_clean":"2024-09-11T12:49:44.472404+0000","last_became_active":"2024-09-11T12:49:43.450620+0000","last_became_peered":"2024-09-11T12:49:43.450620+0000","last_unstale":"2024-09-11T12:49:44.472404+0000","last_undegraded":"2024-09-11T12:49:44.472404+0000","last_fullsized":"2024-09-11T12:49:44.472404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455599+0000","last_change":"2024-09-11T12:49:43.445613+0000","last_active":"2024-09-11T12:49:44.455599+0000","last_peered":"2024-09-11T12:49:44.455599+0000","last_clean":"2024-09-11T12:49:44.455599+0000","last_became_active":"2024-09-11T12:49:43.445164+0000","last_became_peered":"2024-09-11T12:49:43.445164+0000","last_unstale":"2024-09-11T12:49:44.455599+0000","last_undegraded":"2024-09-11T12:49:44.455599+0000","last_fullsized":"2024-09-11T12:49:44.455599+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:13.526065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452041+0000","last_change":"2024-09-11T12:49:43.441250+0000","last_active":"2024-09-11T12:49:44.452041+0000","last_peered":"2024-09-11T12:49:44.452041+0000","last_clean":"2024-09-11T12:49:44.452041+0000","last_became_active":"2024-09-11T12:49:43.440991+0000","last_became_peered":"2024-09-11T12:49:43.440991+0000","last_unstale":"2024-09-11T12:49:44.452041+0000","last_undegraded":"2024-09-11T12:49:44.452041+0000","last_fullsized":"2024-09-11T12:49:44.452041+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:50.908129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472314+0000","last_change":"2024-09-11T12:49:43.450922+0000","last_active":"2024-09-11T12:49:44.472314+0000","last_peered":"2024-09-11T12:49:44.472314+0000","last_clean":"2024-09-11T12:49:44.472314+0000","last_became_active":"2024-09-11T12:49:43.450635+0000","last_became_peered":"2024-09-11T12:49:43.450635+0000","last_unstale":"2024-09-11T12:49:44.472314+0000","last_undegraded":"2024-09-11T12:49:44.472314+0000","last_fullsized":"2024-09-11T12:49:44.472314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:16:43.536586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472021+0000","last_change":"2024-09-11T12:49:43.443298+0000","last_active":"2024-09-11T12:49:44.472021+0000","last_peered":"2024-09-11T12:49:44.472021+0000","last_clean":"2024-09-11T12:49:44.472021+0000","last_became_active":"2024-09-11T12:49:43.442980+0000","last_became_peered":"2024-09-11T12:49:43.442980+0000","last_unstale":"2024-09-11T12:49:44.472021+0000","last_undegraded":"2024-09-11T12:49:44.472021+0000","last_fullsized":"2024-09-11T12:49:44.472021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:13:52.356674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445843+0000","last_change":"2024-09-11T12:49:43.460072+0000","last_active":"2024-09-11T12:49:44.445843+0000","last_peered":"2024-09-11T12:49:44.445843+0000","last_clean":"2024-09-11T12:49:44.445843+0000","last_became_active":"2024-09-11T12:49:43.459042+0000","last_became_peered":"2024-09-11T12:49:43.459042+0000","last_unstale":"2024-09-11T12:49:44.445843+0000","last_undegraded":"2024-09-11T12:49:44.445843+0000","last_fullsized":"2024-09-11T12:49:44.445843+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443060+0000","last_change":"2024-09-11T12:49:43.456433+0000","last_active":"2024-09-11T12:49:44.443060+0000","last_peered":"2024-09-11T12:49:44.443060+0000","last_clean":"2024-09-11T12:49:44.443060+0000","last_became_active":"2024-09-11T12:49:43.450398+0000","last_became_peered":"2024-09-11T12:49:43.450398+0000","last_unstale":"2024-09-11T12:49:44.443060+0000","last_undegraded":"2024-09-11T12:49:44.443060+0000","last_fullsized":"2024-09-11T12:49:44.443060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:07.516897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455308+0000","last_change":"2024-09-11T12:49:43.444430+0000","last_active":"2024-09-11T12:49:44.455308+0000","last_peered":"2024-09-11T12:49:44.455308+0000","last_clean":"2024-09-11T12:49:44.455308+0000","last_became_active":"2024-09-11T12:49:43.444155+0000","last_became_peered":"2024-09-11T12:49:43.444155+0000","last_unstale":"2024-09-11T12:49:44.455308+0000","last_undegraded":"2024-09-11T12:49:44.455308+0000","last_fullsized":"2024-09-11T12:49:44.455308+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445847+0000","last_change":"2024-09-11T12:49:43.455676+0000","last_active":"2024-09-11T12:49:44.445847+0000","last_peered":"2024-09-11T12:49:44.445847+0000","last_clean":"2024-09-11T12:49:44.445847+0000","last_became_active":"2024-09-11T12:49:43.454645+0000","last_became_peered":"2024-09-11T12:49:43.454645+0000","last_unstale":"2024-09-11T12:49:44.445847+0000","last_undegraded":"2024-09-11T12:49:44.445847+0000","last_fullsized":"2024-09-11T12:49:44.445847+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:46.861427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471955+0000","last_change":"2024-09-11T12:49:42.437775+0000","last_active":"2024-09-11T12:49:44.471955+0000","last_peered":"2024-09-11T12:49:44.471955+0000","last_clean":"2024-09-11T12:49:44.471955+0000","last_became_active":"2024-09-11T12:49:42.437412+0000","last_became_peered":"2024-09-11T12:49:42.437412+0000","last_unstale":"2024-09-11T12:49:44.471955+0000","last_undegraded":"2024-09-11T12:49:44.471955+0000","last_fullsized":"2024-09-11T12:49:44.471955+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:32:50.316414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472179+0000","last_change":"2024-09-11T12:49:43.458030+0000","last_active":"2024-09-11T12:49:44.472179+0000","last_peered":"2024-09-11T12:49:44.472179+0000","last_clean":"2024-09-11T12:49:44.472179+0000","last_became_active":"2024-09-11T12:49:43.457513+0000","last_became_peered":"2024-09-11T12:49:43.457513+0000","last_unstale":"2024-09-11T12:49:44.472179+0000","last_undegraded":"2024-09-11T12:49:44.472179+0000","last_fullsized":"2024-09-11T12:49:44.472179+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:48.333848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455174+0000","last_change":"2024-09-11T12:49:42.450097+0000","last_active":"2024-09-11T12:49:44.455174+0000","last_peered":"2024-09-11T12:49:44.455174+0000","last_clean":"2024-09-11T12:49:44.455174+0000","last_became_active":"2024-09-11T12:49:42.446539+0000","last_became_peered":"2024-09-11T12:49:42.446539+0000","last_unstale":"2024-09-11T12:49:44.455174+0000","last_undegraded":"2024-09-11T12:49:44.455174+0000","last_fullsized":"2024-09-11T12:49:44.455174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471482+0000","last_change":"2024-09-11T12:49:42.450513+0000","last_active":"2024-09-11T12:49:44.471482+0000","last_peered":"2024-09-11T12:49:44.471482+0000","last_clean":"2024-09-11T12:49:44.471482+0000","last_became_active":"2024-09-11T12:49:42.450293+0000","last_became_peered":"2024-09-11T12:49:42.450293+0000","last_unstale":"2024-09-11T12:49:44.471482+0000","last_undegraded":"2024-09-11T12:49:44.471482+0000","last_fullsized":"2024-09-11T12:49:44.471482+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:07:25.000664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471230+0000","last_change":"2024-09-11T12:49:43.445707+0000","last_active":"2024-09-11T12:49:44.471230+0000","last_peered":"2024-09-11T12:49:44.471230+0000","last_clean":"2024-09-11T12:49:44.471230+0000","last_became_active":"2024-09-11T12:49:43.445299+0000","last_became_peered":"2024-09-11T12:49:43.445299+0000","last_unstale":"2024-09-11T12:49:44.471230+0000","last_undegraded":"2024-09-11T12:49:44.471230+0000","last_fullsized":"2024-09-11T12:49:44.471230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444806+0000","last_change":"2024-09-11T12:49:42.438052+0000","last_active":"2024-09-11T12:49:44.444806+0000","last_peered":"2024-09-11T12:49:44.444806+0000","last_clean":"2024-09-11T12:49:44.444806+0000","last_became_active":"2024-09-11T12:49:42.437781+0000","last_became_peered":"2024-09-11T12:49:42.437781+0000","last_unstale":"2024-09-11T12:49:44.444806+0000","last_undegraded":"2024-09-11T12:49:44.444806+0000","last_fullsized":"2024-09-11T12:49:44.444806+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451699+0000","last_change":"2024-09-11T12:49:43.440390+0000","last_active":"2024-09-11T12:49:44.451699+0000","last_peered":"2024-09-11T12:49:44.451699+0000","last_clean":"2024-09-11T12:49:44.451699+0000","last_became_active":"2024-09-11T12:49:43.440072+0000","last_became_peered":"2024-09-11T12:49:43.440072+0000","last_unstale":"2024-09-11T12:49:44.451699+0000","last_undegraded":"2024-09-11T12:49:44.451699+0000","last_fullsized":"2024-09-11T12:49:44.451699+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:15.632050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455307+0000","last_change":"2024-09-11T12:49:42.444008+0000","last_active":"2024-09-11T12:49:44.455307+0000","last_peered":"2024-09-11T12:49:44.455307+0000","last_clean":"2024-09-11T12:49:44.455307+0000","last_became_active":"2024-09-11T12:49:42.443388+0000","last_became_peered":"2024-09-11T12:49:42.443388+0000","last_unstale":"2024-09-11T12:49:44.455307+0000","last_undegraded":"2024-09-11T12:49:44.455307+0000","last_fullsized":"2024-09-11T12:49:44.455307+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445126+0000","last_change":"2024-09-11T12:49:43.461238+0000","last_active":"2024-09-11T12:49:44.445126+0000","last_peered":"2024-09-11T12:49:44.445126+0000","last_clean":"2024-09-11T12:49:44.445126+0000","last_became_active":"2024-09-11T12:49:43.460886+0000","last_became_peered":"2024-09-11T12:49:43.460886+0000","last_unstale":"2024-09-11T12:49:44.445126+0000","last_undegraded":"2024-09-11T12:49:44.445126+0000","last_fullsized":"2024-09-11T12:49:44.445126+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445162+0000","last_change":"2024-09-11T12:49:42.446611+0000","last_active":"2024-09-11T12:49:44.445162+0000","last_peered":"2024-09-11T12:49:44.445162+0000","last_clean":"2024-09-11T12:49:44.445162+0000","last_became_active":"2024-09-11T12:49:42.446350+0000","last_became_peered":"2024-09-11T12:49:42.446350+0000","last_unstale":"2024-09-11T12:49:44.445162+0000","last_undegraded":"2024-09-11T12:49:44.445162+0000","last_fullsized":"2024-09-11T12:49:44.445162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472159+0000","last_change":"2024-09-11T12:49:43.451704+0000","last_active":"2024-09-11T12:49:44.472159+0000","last_peered":"2024-09-11T12:49:44.472159+0000","last_clean":"2024-09-11T12:49:44.472159+0000","last_became_active":"2024-09-11T12:49:43.450953+0000","last_became_peered":"2024-09-11T12:49:43.450953+0000","last_unstale":"2024-09-11T12:49:44.472159+0000","last_undegraded":"2024-09-11T12:49:44.472159+0000","last_fullsized":"2024-09-11T12:49:44.472159+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:34.521321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451977+0000","last_change":"2024-09-11T12:49:42.440747+0000","last_active":"2024-09-11T12:49:44.451977+0000","last_peered":"2024-09-11T12:49:44.451977+0000","last_clean":"2024-09-11T12:49:44.451977+0000","last_became_active":"2024-09-11T12:49:42.440400+0000","last_became_peered":"2024-09-11T12:49:42.440400+0000","last_unstale":"2024-09-11T12:49:44.451977+0000","last_undegraded":"2024-09-11T12:49:44.451977+0000","last_fullsized":"2024-09-11T12:49:44.451977+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446290+0000","last_change":"2024-09-11T12:49:43.455119+0000","last_active":"2024-09-11T12:49:44.446290+0000","last_peered":"2024-09-11T12:49:44.446290+0000","last_clean":"2024-09-11T12:49:44.446290+0000","last_became_active":"2024-09-11T12:49:43.454634+0000","last_became_peered":"2024-09-11T12:49:43.454634+0000","last_unstale":"2024-09-11T12:49:44.446290+0000","last_undegraded":"2024-09-11T12:49:44.446290+0000","last_fullsized":"2024-09-11T12:49:44.446290+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:36.803871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455328+0000","last_change":"2024-09-11T12:49:42.445483+0000","last_active":"2024-09-11T12:49:44.455328+0000","last_peered":"2024-09-11T12:49:44.455328+0000","last_clean":"2024-09-11T12:49:44.455328+0000","last_became_active":"2024-09-11T12:49:42.445158+0000","last_became_peered":"2024-09-11T12:49:42.445158+0000","last_unstale":"2024-09-11T12:49:44.455328+0000","last_undegraded":"2024-09-11T12:49:44.455328+0000","last_fullsized":"2024-09-11T12:49:44.455328+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472598+0000","last_change":"2024-09-11T12:49:43.470186+0000","last_active":"2024-09-11T12:49:44.472598+0000","last_peered":"2024-09-11T12:49:44.472598+0000","last_clean":"2024-09-11T12:49:44.472598+0000","last_became_active":"2024-09-11T12:49:43.468620+0000","last_became_peered":"2024-09-11T12:49:43.468620+0000","last_unstale":"2024-09-11T12:49:44.472598+0000","last_undegraded":"2024-09-11T12:49:44.472598+0000","last_fullsized":"2024-09-11T12:49:44.472598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472177+0000","last_change":"2024-09-11T12:49:42.447063+0000","last_active":"2024-09-11T12:49:44.472177+0000","last_peered":"2024-09-11T12:49:44.472177+0000","last_clean":"2024-09-11T12:49:44.472177+0000","last_became_active":"2024-09-11T12:49:42.445739+0000","last_became_peered":"2024-09-11T12:49:42.445739+0000","last_unstale":"2024-09-11T12:49:44.472177+0000","last_undegraded":"2024-09-11T12:49:44.472177+0000","last_fullsized":"2024-09-11T12:49:44.472177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:43.413598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471669+0000","last_change":"2024-09-11T12:49:43.449929+0000","last_active":"2024-09-11T12:49:44.471669+0000","last_peered":"2024-09-11T12:49:44.471669+0000","last_clean":"2024-09-11T12:49:44.471669+0000","last_became_active":"2024-09-11T12:49:43.449581+0000","last_became_peered":"2024-09-11T12:49:43.449581+0000","last_unstale":"2024-09-11T12:49:44.471669+0000","last_undegraded":"2024-09-11T12:49:44.471669+0000","last_fullsized":"2024-09-11T12:49:44.471669+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:34:18.589871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444877+0000","last_change":"2024-09-11T12:49:42.445232+0000","last_active":"2024-09-11T12:49:44.444877+0000","last_peered":"2024-09-11T12:49:44.444877+0000","last_clean":"2024-09-11T12:49:44.444877+0000","last_became_active":"2024-09-11T12:49:42.441896+0000","last_became_peered":"2024-09-11T12:49:42.441896+0000","last_unstale":"2024-09-11T12:49:44.444877+0000","last_undegraded":"2024-09-11T12:49:44.444877+0000","last_fullsized":"2024-09-11T12:49:44.444877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472600+0000","last_change":"2024-09-11T12:49:43.463066+0000","last_active":"2024-09-11T12:49:44.472600+0000","last_peered":"2024-09-11T12:49:44.472600+0000","last_clean":"2024-09-11T12:49:44.472600+0000","last_became_active":"2024-09-11T12:49:43.462868+0000","last_became_peered":"2024-09-11T12:49:43.462868+0000","last_unstale":"2024-09-11T12:49:44.472600+0000","last_undegraded":"2024-09-11T12:49:44.472600+0000","last_fullsized":"2024-09-11T12:49:44.472600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:55.733522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455115+0000","last_change":"2024-09-11T12:49:42.447532+0000","last_active":"2024-09-11T12:49:44.455115+0000","last_peered":"2024-09-11T12:49:44.455115+0000","last_clean":"2024-09-11T12:49:44.455115+0000","last_became_active":"2024-09-11T12:49:42.447217+0000","last_became_peered":"2024-09-11T12:49:42.447217+0000","last_unstale":"2024-09-11T12:49:44.455115+0000","last_undegraded":"2024-09-11T12:49:44.455115+0000","last_fullsized":"2024-09-11T12:49:44.455115+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455296+0000","last_change":"2024-09-11T12:49:43.444203+0000","last_active":"2024-09-11T12:49:44.455296+0000","last_peered":"2024-09-11T12:49:44.455296+0000","last_clean":"2024-09-11T12:49:44.455296+0000","last_became_active":"2024-09-11T12:49:43.443909+0000","last_became_peered":"2024-09-11T12:49:43.443909+0000","last_unstale":"2024-09-11T12:49:44.455296+0000","last_undegraded":"2024-09-11T12:49:44.455296+0000","last_fullsized":"2024-09-11T12:49:44.455296+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:26:19.771132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.442812+0000","last_change":"2024-09-11T12:49:42.428489+0000","last_active":"2024-09-11T12:49:44.442812+0000","last_peered":"2024-09-11T12:49:44.442812+0000","last_clean":"2024-09-11T12:49:44.442812+0000","last_became_active":"2024-09-11T12:49:42.427771+0000","last_became_peered":"2024-09-11T12:49:42.427771+0000","last_unstale":"2024-09-11T12:49:44.442812+0000","last_undegraded":"2024-09-11T12:49:44.442812+0000","last_fullsized":"2024-09-11T12:49:44.442812+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472303+0000","last_change":"2024-09-11T12:49:43.463393+0000","last_active":"2024-09-11T12:49:44.472303+0000","last_peered":"2024-09-11T12:49:44.472303+0000","last_clean":"2024-09-11T12:49:44.472303+0000","last_became_active":"2024-09-11T12:49:43.462936+0000","last_became_peered":"2024-09-11T12:49:43.462936+0000","last_unstale":"2024-09-11T12:49:44.472303+0000","last_undegraded":"2024-09-11T12:49:44.472303+0000","last_fullsized":"2024-09-11T12:49:44.472303+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.446267+0000","last_change":"2024-09-11T12:49:42.440111+0000","last_active":"2024-09-11T12:49:44.446267+0000","last_peered":"2024-09-11T12:49:44.446267+0000","last_clean":"2024-09-11T12:49:44.446267+0000","last_became_active":"2024-09-11T12:49:42.438898+0000","last_became_peered":"2024-09-11T12:49:42.438898+0000","last_unstale":"2024-09-11T12:49:44.446267+0000","last_undegraded":"2024-09-11T12:49:44.446267+0000","last_fullsized":"2024-09-11T12:49:44.446267+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:05.274573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472602+0000","last_change":"2024-09-11T12:49:43.457973+0000","last_active":"2024-09-11T12:49:44.472602+0000","last_peered":"2024-09-11T12:49:44.472602+0000","last_clean":"2024-09-11T12:49:44.472602+0000","last_became_active":"2024-09-11T12:49:43.457699+0000","last_became_peered":"2024-09-11T12:49:43.457699+0000","last_unstale":"2024-09-11T12:49:44.472602+0000","last_undegraded":"2024-09-11T12:49:44.472602+0000","last_fullsized":"2024-09-11T12:49:44.472602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443194+0000","last_change":"2024-09-11T12:49:42.438768+0000","last_active":"2024-09-11T12:49:44.443194+0000","last_peered":"2024-09-11T12:49:44.443194+0000","last_clean":"2024-09-11T12:49:44.443194+0000","last_became_active":"2024-09-11T12:49:42.438221+0000","last_became_peered":"2024-09-11T12:49:42.438221+0000","last_unstale":"2024-09-11T12:49:44.443194+0000","last_undegraded":"2024-09-11T12:49:44.443194+0000","last_fullsized":"2024-09-11T12:49:44.443194+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472978+0000","last_change":"2024-09-11T12:49:43.463216+0000","last_active":"2024-09-11T12:49:44.472978+0000","last_peered":"2024-09-11T12:49:44.472978+0000","last_clean":"2024-09-11T12:49:44.472978+0000","last_became_active":"2024-09-11T12:49:43.462863+0000","last_became_peered":"2024-09-11T12:49:43.462863+0000","last_unstale":"2024-09-11T12:49:44.472978+0000","last_undegraded":"2024-09-11T12:49:44.472978+0000","last_fullsized":"2024-09-11T12:49:44.472978+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:23.404226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452079+0000","last_change":"2024-09-11T12:49:42.430270+0000","last_active":"2024-09-11T12:49:44.452079+0000","last_peered":"2024-09-11T12:49:44.452079+0000","last_clean":"2024-09-11T12:49:44.452079+0000","last_became_active":"2024-09-11T12:49:42.429966+0000","last_became_peered":"2024-09-11T12:49:42.429966+0000","last_unstale":"2024-09-11T12:49:44.452079+0000","last_undegraded":"2024-09-11T12:49:44.452079+0000","last_fullsized":"2024-09-11T12:49:44.452079+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445265+0000","last_change":"2024-09-11T12:49:43.445039+0000","last_active":"2024-09-11T12:49:44.445265+0000","last_peered":"2024-09-11T12:49:44.445265+0000","last_clean":"2024-09-11T12:49:44.445265+0000","last_became_active":"2024-09-11T12:49:43.443152+0000","last_became_peered":"2024-09-11T12:49:43.443152+0000","last_unstale":"2024-09-11T12:49:44.445265+0000","last_undegraded":"2024-09-11T12:49:44.445265+0000","last_fullsized":"2024-09-11T12:49:44.445265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:14.492385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443025+0000","last_change":"2024-09-11T12:49:42.440447+0000","last_active":"2024-09-11T12:49:44.443025+0000","last_peered":"2024-09-11T12:49:44.443025+0000","last_clean":"2024-09-11T12:49:44.443025+0000","last_became_active":"2024-09-11T12:49:42.439837+0000","last_became_peered":"2024-09-11T12:49:42.439837+0000","last_unstale":"2024-09-11T12:49:44.443025+0000","last_undegraded":"2024-09-11T12:49:44.443025+0000","last_fullsized":"2024-09-11T12:49:44.443025+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472502+0000","last_change":"2024-09-11T12:49:43.470639+0000","last_active":"2024-09-11T12:49:44.472502+0000","last_peered":"2024-09-11T12:49:44.472502+0000","last_clean":"2024-09-11T12:49:44.472502+0000","last_became_active":"2024-09-11T12:49:43.468548+0000","last_became_peered":"2024-09-11T12:49:43.468548+0000","last_unstale":"2024-09-11T12:49:44.472502+0000","last_undegraded":"2024-09-11T12:49:44.472502+0000","last_fullsized":"2024-09-11T12:49:44.472502+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:08.118429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472244+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:49:44.472244+0000","last_peered":"2024-09-11T12:49:44.472244+0000","last_clean":"2024-09-11T12:49:44.472244+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:49:44.472244+0000","last_undegraded":"2024-09-11T12:49:44.472244+0000","last_fullsized":"2024-09-11T12:49:44.472244+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444614+0000","last_change":"2024-09-11T12:49:42.436719+0000","last_active":"2024-09-11T12:49:44.444614+0000","last_peered":"2024-09-11T12:49:44.444614+0000","last_clean":"2024-09-11T12:49:44.444614+0000","last_became_active":"2024-09-11T12:49:42.436415+0000","last_became_peered":"2024-09-11T12:49:42.436415+0000","last_unstale":"2024-09-11T12:49:44.444614+0000","last_undegraded":"2024-09-11T12:49:44.444614+0000","last_fullsized":"2024-09-11T12:49:44.444614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471442+0000","last_change":"2024-09-11T12:49:43.462048+0000","last_active":"2024-09-11T12:49:44.471442+0000","last_peered":"2024-09-11T12:49:44.471442+0000","last_clean":"2024-09-11T12:49:44.471442+0000","last_became_active":"2024-09-11T12:49:43.459912+0000","last_became_peered":"2024-09-11T12:49:43.459912+0000","last_unstale":"2024-09-11T12:49:44.471442+0000","last_undegraded":"2024-09-11T12:49:44.471442+0000","last_fullsized":"2024-09-11T12:49:44.471442+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472116+0000","last_change":"2024-09-11T12:49:42.448655+0000","last_active":"2024-09-11T12:49:44.472116+0000","last_peered":"2024-09-11T12:49:44.472116+0000","last_clean":"2024-09-11T12:49:44.472116+0000","last_became_active":"2024-09-11T12:49:42.448394+0000","last_became_peered":"2024-09-11T12:49:42.448394+0000","last_unstale":"2024-09-11T12:49:44.472116+0000","last_undegraded":"2024-09-11T12:49:44.472116+0000","last_fullsized":"2024-09-11T12:49:44.472116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:38.385318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.470982+0000","last_change":"2024-09-11T12:49:43.443422+0000","last_active":"2024-09-11T12:49:44.470982+0000","last_peered":"2024-09-11T12:49:44.470982+0000","last_clean":"2024-09-11T12:49:44.470982+0000","last_became_active":"2024-09-11T12:49:43.443040+0000","last_became_peered":"2024-09-11T12:49:43.443040+0000","last_unstale":"2024-09-11T12:49:44.470982+0000","last_undegraded":"2024-09-11T12:49:44.470982+0000","last_fullsized":"2024-09-11T12:49:44.470982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:29.948503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.443004+0000","last_change":"2024-09-11T12:49:42.439971+0000","last_active":"2024-09-11T12:49:44.443004+0000","last_peered":"2024-09-11T12:49:44.443004+0000","last_clean":"2024-09-11T12:49:44.443004+0000","last_became_active":"2024-09-11T12:49:42.438664+0000","last_became_peered":"2024-09-11T12:49:42.438664+0000","last_unstale":"2024-09-11T12:49:44.443004+0000","last_undegraded":"2024-09-11T12:49:44.443004+0000","last_fullsized":"2024-09-11T12:49:44.443004+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451753+0000","last_change":"2024-09-11T12:49:43.437660+0000","last_active":"2024-09-11T12:49:44.451753+0000","last_peered":"2024-09-11T12:49:44.451753+0000","last_clean":"2024-09-11T12:49:44.451753+0000","last_became_active":"2024-09-11T12:49:43.437383+0000","last_became_peered":"2024-09-11T12:49:43.437383+0000","last_unstale":"2024-09-11T12:49:44.451753+0000","last_undegraded":"2024-09-11T12:49:44.451753+0000","last_fullsized":"2024-09-11T12:49:44.451753+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.444974+0000","last_change":"2024-09-11T12:49:42.442628+0000","last_active":"2024-09-11T12:49:44.444974+0000","last_peered":"2024-09-11T12:49:44.444974+0000","last_clean":"2024-09-11T12:49:44.444974+0000","last_became_active":"2024-09-11T12:49:42.442343+0000","last_became_peered":"2024-09-11T12:49:42.442343+0000","last_unstale":"2024-09-11T12:49:44.444974+0000","last_undegraded":"2024-09-11T12:49:44.444974+0000","last_fullsized":"2024-09-11T12:49:44.444974+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445987+0000","last_change":"2024-09-11T12:49:43.467784+0000","last_active":"2024-09-11T12:49:44.445987+0000","last_peered":"2024-09-11T12:49:44.445987+0000","last_clean":"2024-09-11T12:49:44.445987+0000","last_became_active":"2024-09-11T12:49:43.467380+0000","last_became_peered":"2024-09-11T12:49:43.467380+0000","last_unstale":"2024-09-11T12:49:44.445987+0000","last_undegraded":"2024-09-11T12:49:44.445987+0000","last_fullsized":"2024-09-11T12:49:44.445987+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:39.392320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451933+0000","last_change":"2024-09-11T12:49:42.436463+0000","last_active":"2024-09-11T12:49:44.451933+0000","last_peered":"2024-09-11T12:49:44.451933+0000","last_clean":"2024-09-11T12:49:44.451933+0000","last_became_active":"2024-09-11T12:49:42.436201+0000","last_became_peered":"2024-09-11T12:49:42.436201+0000","last_unstale":"2024-09-11T12:49:44.451933+0000","last_undegraded":"2024-09-11T12:49:44.451933+0000","last_fullsized":"2024-09-11T12:49:44.451933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:22:47.649035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445376+0000","last_change":"2024-09-11T12:49:43.449369+0000","last_active":"2024-09-11T12:49:44.445376+0000","last_peered":"2024-09-11T12:49:44.445376+0000","last_clean":"2024-09-11T12:49:44.445376+0000","last_became_active":"2024-09-11T12:49:43.443602+0000","last_became_peered":"2024-09-11T12:49:43.443602+0000","last_unstale":"2024-09-11T12:49:44.445376+0000","last_undegraded":"2024-09-11T12:49:44.445376+0000","last_fullsized":"2024-09-11T12:49:44.445376+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:19.603542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472687+0000","last_change":"2024-09-11T12:49:42.449197+0000","last_active":"2024-09-11T12:49:44.472687+0000","last_peered":"2024-09-11T12:49:44.472687+0000","last_clean":"2024-09-11T12:49:44.472687+0000","last_became_active":"2024-09-11T12:49:42.448893+0000","last_became_peered":"2024-09-11T12:49:42.448893+0000","last_unstale":"2024-09-11T12:49:44.472687+0000","last_undegraded":"2024-09-11T12:49:44.472687+0000","last_fullsized":"2024-09-11T12:49:44.472687+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:35.796869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451779+0000","last_change":"2024-09-11T12:49:43.439241+0000","last_active":"2024-09-11T12:49:44.451779+0000","last_peered":"2024-09-11T12:49:44.451779+0000","last_clean":"2024-09-11T12:49:44.451779+0000","last_became_active":"2024-09-11T12:49:43.438486+0000","last_became_peered":"2024-09-11T12:49:43.438486+0000","last_unstale":"2024-09-11T12:49:44.451779+0000","last_undegraded":"2024-09-11T12:49:44.451779+0000","last_fullsized":"2024-09-11T12:49:44.451779+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:26.624457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.452121+0000","last_change":"2024-09-11T12:49:42.445521+0000","last_active":"2024-09-11T12:49:44.452121+0000","last_peered":"2024-09-11T12:49:44.452121+0000","last_clean":"2024-09-11T12:49:44.452121+0000","last_became_active":"2024-09-11T12:49:42.445218+0000","last_became_peered":"2024-09-11T12:49:42.445218+0000","last_unstale":"2024-09-11T12:49:44.452121+0000","last_undegraded":"2024-09-11T12:49:44.452121+0000","last_fullsized":"2024-09-11T12:49:44.452121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472486+0000","last_change":"2024-09-11T12:49:43.454081+0000","last_active":"2024-09-11T12:49:44.472486+0000","last_peered":"2024-09-11T12:49:44.472486+0000","last_clean":"2024-09-11T12:49:44.472486+0000","last_became_active":"2024-09-11T12:49:43.453820+0000","last_became_peered":"2024-09-11T12:49:43.453820+0000","last_unstale":"2024-09-11T12:49:44.472486+0000","last_undegraded":"2024-09-11T12:49:44.472486+0000","last_fullsized":"2024-09-11T12:49:44.472486+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:35:37.813446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455425+0000","last_change":"2024-09-11T12:49:42.446529+0000","last_active":"2024-09-11T12:49:44.455425+0000","last_peered":"2024-09-11T12:49:44.455425+0000","last_clean":"2024-09-11T12:49:44.455425+0000","last_became_active":"2024-09-11T12:49:42.446268+0000","last_became_peered":"2024-09-11T12:49:42.446268+0000","last_unstale":"2024-09-11T12:49:44.455425+0000","last_undegraded":"2024-09-11T12:49:44.455425+0000","last_fullsized":"2024-09-11T12:49:44.455425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471706+0000","last_change":"2024-09-11T12:49:43.453755+0000","last_active":"2024-09-11T12:49:44.471706+0000","last_peered":"2024-09-11T12:49:44.471706+0000","last_clean":"2024-09-11T12:49:44.471706+0000","last_became_active":"2024-09-11T12:49:43.453345+0000","last_became_peered":"2024-09-11T12:49:43.453345+0000","last_unstale":"2024-09-11T12:49:44.471706+0000","last_undegraded":"2024-09-11T12:49:44.471706+0000","last_fullsized":"2024-09-11T12:49:44.471706+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:46.327809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472114+0000","last_change":"2024-09-11T12:49:42.445423+0000","last_active":"2024-09-11T12:49:44.472114+0000","last_peered":"2024-09-11T12:49:44.472114+0000","last_clean":"2024-09-11T12:49:44.472114+0000","last_became_active":"2024-09-11T12:49:42.444814+0000","last_became_peered":"2024-09-11T12:49:42.444814+0000","last_unstale":"2024-09-11T12:49:44.472114+0000","last_undegraded":"2024-09-11T12:49:44.472114+0000","last_fullsized":"2024-09-11T12:49:44.472114+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445597+0000","last_change":"2024-09-11T12:49:43.459523+0000","last_active":"2024-09-11T12:49:44.445597+0000","last_peered":"2024-09-11T12:49:44.445597+0000","last_clean":"2024-09-11T12:49:44.445597+0000","last_became_active":"2024-09-11T12:49:43.459164+0000","last_became_peered":"2024-09-11T12:49:43.459164+0000","last_unstale":"2024-09-11T12:49:44.445597+0000","last_undegraded":"2024-09-11T12:49:44.445597+0000","last_fullsized":"2024-09-11T12:49:44.445597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:50.017910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451744+0000","last_change":"2024-09-11T12:49:42.431712+0000","last_active":"2024-09-11T12:49:44.451744+0000","last_peered":"2024-09-11T12:49:44.451744+0000","last_clean":"2024-09-11T12:49:44.451744+0000","last_became_active":"2024-09-11T12:49:42.431429+0000","last_became_peered":"2024-09-11T12:49:42.431429+0000","last_unstale":"2024-09-11T12:49:44.451744+0000","last_undegraded":"2024-09-11T12:49:44.451744+0000","last_fullsized":"2024-09-11T12:49:44.451744+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:20:06.509895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471398+0000","last_change":"2024-09-11T12:49:43.444653+0000","last_active":"2024-09-11T12:49:44.471398+0000","last_peered":"2024-09-11T12:49:44.471398+0000","last_clean":"2024-09-11T12:49:44.471398+0000","last_became_active":"2024-09-11T12:49:43.444206+0000","last_became_peered":"2024-09-11T12:49:43.444206+0000","last_unstale":"2024-09-11T12:49:44.471398+0000","last_undegraded":"2024-09-11T12:49:44.471398+0000","last_fullsized":"2024-09-11T12:49:44.471398+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.451809+0000","last_change":"2024-09-11T12:49:42.439128+0000","last_active":"2024-09-11T12:49:44.451809+0000","last_peered":"2024-09-11T12:49:44.451809+0000","last_clean":"2024-09-11T12:49:44.451809+0000","last_became_active":"2024-09-11T12:49:42.438820+0000","last_became_peered":"2024-09-11T12:49:42.438820+0000","last_unstale":"2024-09-11T12:49:44.451809+0000","last_undegraded":"2024-09-11T12:49:44.451809+0000","last_fullsized":"2024-09-11T12:49:44.451809+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:57.720432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471928+0000","last_change":"2024-09-11T12:49:43.447109+0000","last_active":"2024-09-11T12:49:44.471928+0000","last_peered":"2024-09-11T12:49:44.471928+0000","last_clean":"2024-09-11T12:49:44.471928+0000","last_became_active":"2024-09-11T12:49:43.446540+0000","last_became_peered":"2024-09-11T12:49:43.446540+0000","last_unstale":"2024-09-11T12:49:44.471928+0000","last_undegraded":"2024-09-11T12:49:44.471928+0000","last_fullsized":"2024-09-11T12:49:44.471928+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:43.143441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472587+0000","last_change":"2024-09-11T12:49:42.447202+0000","last_active":"2024-09-11T12:49:44.472587+0000","last_peered":"2024-09-11T12:49:44.472587+0000","last_clean":"2024-09-11T12:49:44.472587+0000","last_became_active":"2024-09-11T12:49:42.446936+0000","last_became_peered":"2024-09-11T12:49:42.446936+0000","last_unstale":"2024-09-11T12:49:44.472587+0000","last_undegraded":"2024-09-11T12:49:44.472587+0000","last_fullsized":"2024-09-11T12:49:44.472587+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:49.010908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.455309+0000","last_change":"2024-09-11T12:49:43.454367+0000","last_active":"2024-09-11T12:49:44.455309+0000","last_peered":"2024-09-11T12:49:44.455309+0000","last_clean":"2024-09-11T12:49:44.455309+0000","last_became_active":"2024-09-11T12:49:43.454114+0000","last_became_peered":"2024-09-11T12:49:43.454114+0000","last_unstale":"2024-09-11T12:49:44.455309+0000","last_undegraded":"2024-09-11T12:49:44.455309+0000","last_fullsized":"2024-09-11T12:49:44.455309+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:15.624283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445226+0000","last_change":"2024-09-11T12:49:43.449972+0000","last_active":"2024-09-11T12:49:44.445226+0000","last_peered":"2024-09-11T12:49:44.445226+0000","last_clean":"2024-09-11T12:49:44.445226+0000","last_became_active":"2024-09-11T12:49:43.449653+0000","last_became_peered":"2024-09-11T12:49:43.449653+0000","last_unstale":"2024-09-11T12:49:44.445226+0000","last_undegraded":"2024-09-11T12:49:44.445226+0000","last_fullsized":"2024-09-11T12:49:44.445226+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:04.898464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472376+0000","last_change":"2024-09-11T12:49:42.443992+0000","last_active":"2024-09-11T12:49:44.472376+0000","last_peered":"2024-09-11T12:49:44.472376+0000","last_clean":"2024-09-11T12:49:44.472376+0000","last_became_active":"2024-09-11T12:49:42.443492+0000","last_became_peered":"2024-09-11T12:49:42.443492+0000","last_unstale":"2024-09-11T12:49:44.472376+0000","last_undegraded":"2024-09-11T12:49:44.472376+0000","last_fullsized":"2024-09-11T12:49:44.472376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:07.781904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.445104+0000","last_change":"2024-09-11T12:49:43.449359+0000","last_active":"2024-09-11T12:49:44.445104+0000","last_peered":"2024-09-11T12:49:44.445104+0000","last_clean":"2024-09-11T12:49:44.445104+0000","last_became_active":"2024-09-11T12:49:43.448838+0000","last_became_peered":"2024-09-11T12:49:43.448838+0000","last_unstale":"2024-09-11T12:49:44.445104+0000","last_undegraded":"2024-09-11T12:49:44.445104+0000","last_fullsized":"2024-09-11T12:49:44.445104+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:00.124999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472595+0000","last_change":"2024-09-11T12:49:42.450569+0000","last_active":"2024-09-11T12:49:44.472595+0000","last_peered":"2024-09-11T12:49:44.472595+0000","last_clean":"2024-09-11T12:49:44.472595+0000","last_became_active":"2024-09-11T12:49:42.450338+0000","last_became_peered":"2024-09-11T12:49:42.450338+0000","last_unstale":"2024-09-11T12:49:44.472595+0000","last_undegraded":"2024-09-11T12:49:44.472595+0000","last_fullsized":"2024-09-11T12:49:44.472595+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:52:45.854425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471712+0000","last_change":"2024-09-11T12:49:43.451951+0000","last_active":"2024-09-11T12:49:44.471712+0000","last_peered":"2024-09-11T12:49:44.471712+0000","last_clean":"2024-09-11T12:49:44.471712+0000","last_became_active":"2024-09-11T12:49:43.451337+0000","last_became_peered":"2024-09-11T12:49:43.451337+0000","last_unstale":"2024-09-11T12:49:44.471712+0000","last_undegraded":"2024-09-11T12:49:44.471712+0000","last_fullsized":"2024-09-11T12:49:44.471712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:26.413762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472335+0000","last_change":"2024-09-11T12:49:42.446328+0000","last_active":"2024-09-11T12:49:44.472335+0000","last_peered":"2024-09-11T12:49:44.472335+0000","last_clean":"2024-09-11T12:49:44.472335+0000","last_became_active":"2024-09-11T12:49:42.446059+0000","last_became_peered":"2024-09-11T12:49:42.446059+0000","last_unstale":"2024-09-11T12:49:44.472335+0000","last_undegraded":"2024-09-11T12:49:44.472335+0000","last_fullsized":"2024-09-11T12:49:44.472335+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:50:50.002894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.472359+0000","last_change":"2024-09-11T12:49:43.443985+0000","last_active":"2024-09-11T12:49:44.472359+0000","last_peered":"2024-09-11T12:49:44.472359+0000","last_clean":"2024-09-11T12:49:44.472359+0000","last_became_active":"2024-09-11T12:49:43.443740+0000","last_became_peered":"2024-09-11T12:49:43.443740+0000","last_unstale":"2024-09-11T12:49:44.472359+0000","last_undegraded":"2024-09-11T12:49:44.472359+0000","last_fullsized":"2024-09-11T12:49:44.472359+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:42.411555+0000","last_clean_scrub_stamp":"2024-09-11T12:49:42.411555+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:07.414149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-11T12:49:44.471815+0000","last_change":"2024-09-11T12:49:42.443079+0000","last_active":"2024-09-11T12:49:44.471815+0000","last_peered":"2024-09-11T12:49:44.471815+0000","last_clean":"2024-09-11T12:49:44.471815+0000","last_became_active":"2024-09-11T12:49:42.442853+0000","last_became_peered":"2024-09-11T12:49:42.442853+0000","last_unstale":"2024-09-11T12:49:44.471815+0000","last_undegraded":"2024-09-11T12:49:44.471815+0000","last_fullsized":"2024-09-11T12:49:44.471815+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:41.404553+0000","last_clean_scrub_stamp":"2024-09-11T12:49:41.404553+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:03.323239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":831488,"data_stored":671905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509447,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31436,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340404,"statfs":{"total":96636764160,"available":96604573696,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31424,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340416,"statfs":{"total":96636764160,"available":96604585984,"internally_reserved":0,"allocated":237568,"data_stored":81045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31432,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340408,"statfs":{"total":96636764160,"available":96604577792,"internally_reserved":0,"allocated":245760,"data_stored":81081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":245760,"data_stored":82567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":233472,"data_stored":80955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542152,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27856,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343984,"statfs":{"total":96636764160,"available":96608239616,"internally_reserved":0,"allocated":839680,"data_stored":671961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:49:57.476 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:49:57.476 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T12:49:57.476 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T12:49:57.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T12:49:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 -- 172.21.15.78:0/633212617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 msgr2=0x7f52a0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 --2- 172.21.15.78:0/633212617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52a0003e80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52ac0016f0 tx=0x7f52ac01c790 comp rx=0 tx=0).stop 2024-09-11T12:49:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 -- 172.21.15.78:0/633212617 shutdown_connections 2024-09-11T12:49:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 --2- 172.21.15.78:0/633212617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52a0003e80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 -- 172.21.15.78:0/633212617 >> 172.21.15.78:0/633212617 conn(0x7f52b8074b90 msgr2=0x7f52b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 -- 172.21.15.78:0/633212617 shutdown_connections 2024-09-11T12:49:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.650+0000 7f52c0800640 1 -- 172.21.15.78:0/633212617 wait complete. 2024-09-11T12:49:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 Processor -- start 2024-09-11T12:49:57.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 -- start start 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b80b0230 0x7f52b80b2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b81fc010 0x7f52b81f17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d1180 con 0x7f52a0003a60 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f52b80d1000 con 0x7f52b81fc010 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f52b80d1300 con 0x7f52b80b0230 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60260/0 (socket says 172.21.15.78:60260) 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 -- 172.21.15.78:0/4241354370 learned_addr learned my addr 172.21.15.78:0/4241354370 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bd400640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b80b0230 0x7f52b80b2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52be800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b81fc010 0x7f52b81f17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 -- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b80b0230 msgr2=0x7f52b80b2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b80b0230 0x7f52b80b2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 -- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b81fc010 msgr2=0x7f52b81f17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b81fc010 0x7f52b81f17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 -- 172.21.15.78:0/4241354370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b81fb570 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52bde00640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52ac01ae90 tx=0x7f52ac003c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0ba070 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52ac0af040 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0ae070 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.654+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b8153600 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b81fb810 con 0x7f52a0003a60 2024-09-11T12:49:57.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52b806e940 con 0x7f52a0003a60 2024-09-11T12:49:57.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f52ac006350 con 0x7f52a0003a60 2024-09-11T12:49:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52b2000640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 0x7f529c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f52ac12b160 con 0x7f52a0003a60 2024-09-11T12:49:57.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.658+0000 7f52bd400640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 0x7f529c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:57.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.662+0000 7f52bd400640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 0x7f529c073b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52b815e2b0 tx=0x7f52a8006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:57.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.662+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f52ac0164f0 con 0x7f52a0003a60 2024-09-11T12:49:57.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.862+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f52b815e680 con 0x7f52a0003a60 2024-09-11T12:49:57.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.862+0000 7f52b2000640 1 -- 172.21.15.78:0/4241354370 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f52ac0ff040 con 0x7f52a0003a60 2024-09-11T12:49:57.869 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:57.869 INFO:teuthology.orchestra.run.smithi078.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 msgr2=0x7f529c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 0x7f529c073b40 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f52b815e2b0 tx=0x7f52a8006040 comp rx=0 tx=0).stop 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 msgr2=0x7f52b81530c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f52ac01ae90 tx=0x7f52ac003c60 comp rx=0 tx=0).stop 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 shutdown_connections 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f529c071680 0x7f529c073b40 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f52b81fc010 0x7f52b81f17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52b80b0230 0x7f52b80b2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 --2- 172.21.15.78:0/4241354370 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f52a0003a60 0x7f52b81530c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:57.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 >> 172.21.15.78:0/4241354370 conn(0x7f52b8074b90 msgr2=0x7f52b80b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:57.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 shutdown_connections 2024-09-11T12:49:57.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:57.866+0000 7f52c0800640 1 -- 172.21.15.78:0/4241354370 wait complete. 2024-09-11T12:49:57.889 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T12:49:57.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 -- 172.21.15.78:0/2879322043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10406fdf0 msgr2=0x7fc1040a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 --2- 172.21.15.78:0/2879322043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10406fdf0 0x7fc1040a9d00 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc1040d1780 tx=0x7fc0f001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 -- 172.21.15.78:0/2879322043 shutdown_connections 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 --2- 172.21.15.78:0/2879322043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc10406fdf0 0x7fc1040a9d00 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 --2- 172.21.15.78:0/2879322043 >> v2:172.21.15.157:3300/0 conn(0x7fc1040b14b0 0x7fc10415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 -- 172.21.15.78:0/2879322043 >> 172.21.15.78:0/2879322043 conn(0x7fc104074b90 msgr2=0x7fc1040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 -- 172.21.15.78:0/2879322043 shutdown_connections 2024-09-11T12:49:58.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.026+0000 7fc10a600640 1 -- 172.21.15.78:0/2879322043 wait complete. 2024-09-11T12:49:58.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 Processor -- start 2024-09-11T12:49:58.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- start start 2024-09-11T12:49:58.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc10406fdf0 0x7fc1040accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1040b14b0 0x7fc10413d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc104169330 con 0x7fc1040d0bf0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1041691b0 con 0x7fc1040b14b0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1041694b0 con 0x7fc10406fdf0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60272/0 (socket says 172.21.15.78:60272) 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 -- 172.21.15.78:0/4017451007 learned_addr learned my addr 172.21.15.78:0/4017451007 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103400640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc10406fdf0 0x7fc1040accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc102a00640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1040b14b0 0x7fc10413d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 -- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc10406fdf0 msgr2=0x7fc1040accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc10406fdf0 0x7fc1040accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 -- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1040b14b0 msgr2=0x7fc10413d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1040b14b0 0x7fc10413d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 -- 172.21.15.78:0/4017451007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10406f000 con 0x7fc1040d0bf0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc103e00640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b990 tx=0x7fc0fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc019070 con 0x7fc1040d0bf0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0fc00cc90 con 0x7fc1040d0bf0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0fc0146a0 con 0x7fc1040d0bf0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1040aec90 con 0x7fc1040d0bf0 2024-09-11T12:49:58.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1040af250 con 0x7fc1040d0bf0 2024-09-11T12:49:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.030+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1040a9d00 con 0x7fc1040d0bf0 2024-09-11T12:49:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.034+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fc0fc01c070 con 0x7fc1040d0bf0 2024-09-11T12:49:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.034+0000 7fc0fb400640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 0x7fc0e4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.034+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc0fc091b50 con 0x7fc1040d0bf0 2024-09-11T12:49:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.034+0000 7fc103400640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 0x7fc0e4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.034+0000 7fc103400640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 0x7fc0e4073b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006fd0 tx=0x7fc0f4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.038+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc0fc05d2d0 con 0x7fc1040d0bf0 2024-09-11T12:49:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc104069070 con 0x7fc1040d0bf0 2024-09-11T12:49:58.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc0fb400640 1 -- 172.21.15.78:0/4017451007 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fc0fc061820 con 0x7fc1040d0bf0 2024-09-11T12:49:58.247 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:58.247 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:49:58.247 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 msgr2=0x7fc0e4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 0x7fc0e4073b40 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4006fd0 tx=0x7fc0f4008040 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 msgr2=0x7fc10413d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc0fc00b990 tx=0x7fc0fc00be50 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 shutdown_connections 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc0e4071680 0x7fc0e4073b40 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1040d0bf0 0x7fc10413d620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1040b14b0 0x7fc10413d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 --2- 172.21.15.78:0/4017451007 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc10406fdf0 0x7fc1040accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 >> 172.21.15.78:0/4017451007 conn(0x7fc104074b90 msgr2=0x7fc10415de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 shutdown_connections 2024-09-11T12:49:58.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.242+0000 7fc10a600640 1 -- 172.21.15.78:0/4017451007 wait complete. 2024-09-11T12:49:58.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 -- 172.21.15.78:0/2790593059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c06fc50 msgr2=0x7faa8c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 --2- 172.21.15.78:0/2790593059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c06fc50 0x7faa8c0700c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7faa8c074d50 tx=0x7faa8401cd50 comp rx=0 tx=0).stop 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 -- 172.21.15.78:0/2790593059 shutdown_connections 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 --2- 172.21.15.78:0/2790593059 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c06fc50 0x7faa8c0700c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 --2- 172.21.15.78:0/2790593059 >> v2:172.21.15.157:3300/0 conn(0x7faa8c0cf760 0x7faa8c074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 -- 172.21.15.78:0/2790593059 >> 172.21.15.78:0/2790593059 conn(0x7faa8c06ca80 msgr2=0x7faa8c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 -- 172.21.15.78:0/2790593059 shutdown_connections 2024-09-11T12:49:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.402+0000 7faa91c00640 1 -- 172.21.15.78:0/2790593059 wait complete. 2024-09-11T12:49:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 Processor -- start 2024-09-11T12:49:58.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- start start 2024-09-11T12:49:58.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8c06fc50 0x7faa8c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa8c15aec0 0x7faa8c149a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c15f320 con 0x7faa8c0cf760 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa8c15f1a0 con 0x7faa8c06fc50 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa8c151010 con 0x7faa8c15aec0 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60276/0 (socket says 172.21.15.78:60276) 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 -- 172.21.15.78:0/2266956526 learned_addr learned my addr 172.21.15.78:0/2266956526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa8aa00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8c06fc50 0x7faa8c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa8b400640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa8c15aec0 0x7faa8c149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 -- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa8c15aec0 msgr2=0x7faa8c149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa8c15aec0 0x7faa8c149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 -- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8c06fc50 msgr2=0x7faa8c159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8c06fc50 0x7faa8c159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 -- 172.21.15.78:0/2266956526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa8c1519b0 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa83e00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b410 tx=0x7faa7400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74015030 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa74004020 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa74004470 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa8c1faa00 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.406+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa8c149fd0 con 0x7faa8c0cf760 2024-09-11T12:49:58.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa8c0700c0 con 0x7faa8c0cf760 2024-09-11T12:49:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7faa74018070 con 0x7faa8c0cf760 2024-09-11T12:49:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa82a00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 0x7faa68073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7faa740918f0 con 0x7faa8c0cf760 2024-09-11T12:49:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa8aa00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 0x7faa68073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.410+0000 7faa8aa00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 0x7faa68073b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa7c004630 tx=0x7faa7c00b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.414+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7faa7405d750 con 0x7faa8c0cf760 2024-09-11T12:49:58.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa8c1fabd0 con 0x7faa8c0cf760 2024-09-11T12:49:58.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa82a00640 1 -- 172.21.15.78:0/2266956526 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7faa7405d050 con 0x7faa8c0cf760 2024-09-11T12:49:58.623 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:58.623 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:49:58.623 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:49:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 msgr2=0x7faa68073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 0x7faa68073b40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7faa7c004630 tx=0x7faa7c00b390 comp rx=0 tx=0).stop 2024-09-11T12:49:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 msgr2=0x7faa8c149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7faa7400b410 tx=0x7faa7400b8d0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 shutdown_connections 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa68071680 0x7faa68073b40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa8c15aec0 0x7faa8c149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa8c0cf760 0x7faa8c149550 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 --2- 172.21.15.78:0/2266956526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa8c06fc50 0x7faa8c159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 >> 172.21.15.78:0/2266956526 conn(0x7faa8c06ca80 msgr2=0x7faa8c06f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 shutdown_connections 2024-09-11T12:49:58.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.618+0000 7faa91c00640 1 -- 172.21.15.78:0/2266956526 wait complete. 2024-09-11T12:49:58.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:49:58.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- 172.21.15.78:0/3833088737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c073c70 msgr2=0x7f5e1c15ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 --2- 172.21.15.78:0/3833088737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c073c70 0x7f5e1c15ae40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e14009900 tx=0x7f5e1401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- 172.21.15.78:0/3833088737 shutdown_connections 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 --2- 172.21.15.78:0/3833088737 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c073c70 0x7f5e1c15ae40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- 172.21.15.78:0/3833088737 >> 172.21.15.78:0/3833088737 conn(0x7f5e1c06f3a0 msgr2=0x7f5e1c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- 172.21.15.78:0/3833088737 shutdown_connections 2024-09-11T12:49:58.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- 172.21.15.78:0/3833088737 wait complete. 2024-09-11T12:49:58.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 Processor -- start 2024-09-11T12:49:58.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- start start 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e1c073c70 0x7f5e1c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e1c0cfeb0 0x7f5e1c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c168960 con 0x7f5e1c1518f0 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5e1c1687e0 con 0x7f5e1c073c70 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e20c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5e1c168ae0 con 0x7f5e1c0cfeb0 2024-09-11T12:49:58.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60302/0 (socket says 172.21.15.78:60302) 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 -- 172.21.15.78:0/2806226965 learned_addr learned my addr 172.21.15.78:0/2806226965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 -- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e1c0cfeb0 msgr2=0x7f5e1c14eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e1c0cfeb0 0x7f5e1c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 -- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e1c073c70 msgr2=0x7f5e1c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e1c073c70 0x7f5e1c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 -- 172.21.15.78:0/2806226965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e1c1f76c0 con 0x7f5e1c1518f0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.782+0000 7f5e1a800640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e140b0040 tx=0x7f5e140041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e140bb070 con 0x7f5e1c1518f0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e14005070 con 0x7f5e1c1518f0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e1c1f79a0 con 0x7f5e1c1518f0 2024-09-11T12:49:58.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e140b2040 con 0x7f5e1c1518f0 2024-09-11T12:49:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e1c1f7f60 con 0x7f5e1c1518f0 2024-09-11T12:49:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e1c15ae40 con 0x7f5e1c1518f0 2024-09-11T12:49:58.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f5e140be070 con 0x7f5e1c1518f0 2024-09-11T12:49:58.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 0x7f5dfc073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:58.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5e1412b7e0 con 0x7f5e1c1518f0 2024-09-11T12:49:58.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.786+0000 7f5e19e00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 0x7f5dfc073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:58.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.790+0000 7f5e19e00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 0x7f5dfc073b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00ab90 tx=0x7f5e0c009230 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:58.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.790+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5e140164d0 con 0x7f5e1c1518f0 2024-09-11T12:49:58.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.958+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e1c155210 con 0x7f5e1c1518f0 2024-09-11T12:49:58.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e12000640 1 -- 172.21.15.78:0/2806226965 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f5e140f75b0 con 0x7f5e1c1518f0 2024-09-11T12:49:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:58.966 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 msgr2=0x7f5dfc073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 0x7f5dfc073b40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5e0c00ab90 tx=0x7f5e0c009230 comp rx=0 tx=0).stop 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 msgr2=0x7f5e1c13fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f5e140b0040 tx=0x7f5e140041a0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 shutdown_connections 2024-09-11T12:49:58.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dfc071680 0x7f5dfc073b40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5e1c1518f0 0x7f5e1c13fc60 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5e1c0cfeb0 0x7f5e1c14eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 --2- 172.21.15.78:0/2806226965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5e1c073c70 0x7f5e1c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 >> 172.21.15.78:0/2806226965 conn(0x7f5e1c06f3a0 msgr2=0x7f5e1c152a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 shutdown_connections 2024-09-11T12:49:58.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:58.962+0000 7f5e20c00640 1 -- 172.21.15.78:0/2806226965 wait complete. 2024-09-11T12:49:58.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 -- 172.21.15.78:0/1847262330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 msgr2=0x7f17b0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 --2- 172.21.15.78:0/1847262330 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17b0003e30 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b90 tx=0x7f17bc01d270 comp rx=0 tx=0).stop 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17ca800640 1 -- 172.21.15.78:0/1847262330 reap_dead start 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 -- 172.21.15.78:0/1847262330 shutdown_connections 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 -- 172.21.15.78:0/1847262330 >> 172.21.15.78:0/1847262330 conn(0x7f17cc074b90 msgr2=0x7f17cc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 -- 172.21.15.78:0/1847262330 shutdown_connections 2024-09-11T12:49:59.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.126+0000 7f17d0e00640 1 -- 172.21.15.78:0/1847262330 wait complete. 2024-09-11T12:49:59.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 Processor -- start 2024-09-11T12:49:59.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- start start 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17cc155610 0x7f17cc151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17cc1fb310 0x7f17cc15abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc15dce0 con 0x7f17b0003a10 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17cc15db60 con 0x7f17cc155610 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17cc166730 con 0x7f17cc1fb310 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60310/0 (socket says 172.21.15.78:60310) 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 -- 172.21.15.78:0/3231460089 learned_addr learned my addr 172.21.15.78:0/3231460089 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9400640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17cc155610 0x7f17cc151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17ca800640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17cc1fb310 0x7f17cc15abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 -- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17cc1fb310 msgr2=0x7f17cc15abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17cc1fb310 0x7f17cc15abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 -- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17cc155610 msgr2=0x7f17cc151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17cc155610 0x7f17cc151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 -- 172.21.15.78:0/3231460089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17cc155a90 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c9e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b90 tx=0x7f17bc005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc0b1040 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17bc0ac400 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17bc004290 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17cc155d10 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.130+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17cc1562d0 con 0x7f17b0003a10 2024-09-11T12:49:59.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17cc0d10a0 con 0x7f17b0003a10 2024-09-11T12:49:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f17bc0be020 con 0x7f17b0003a10 2024-09-11T12:49:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17c2000640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 0x7f17ac073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f17bc12c450 con 0x7f17b0003a10 2024-09-11T12:49:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17c9400640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 0x7f17ac073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.134+0000 7f17c9400640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 0x7f17ac073b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17b8001ec0 tx=0x7f17b8006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.138+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f17bc0c5020 con 0x7f17b0003a10 2024-09-11T12:49:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.342+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17cc1525e0 con 0x7f17b0003a10 2024-09-11T12:49:59.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.342+0000 7f17c2000640 1 -- 172.21.15.78:0/3231460089 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f17cc155a90 con 0x7f17b0003a10 2024-09-11T12:49:59.348 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:49:59.348 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:49:59.348 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:49:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 msgr2=0x7f17ac073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 0x7f17ac073b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f17b8001ec0 tx=0x7f17b8006040 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 msgr2=0x7f17cc06e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f17bc009b90 tx=0x7f17bc005590 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 shutdown_connections 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f17ac071680 0x7f17ac073b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17cc1fb310 0x7f17cc15abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17cc155610 0x7f17cc151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 --2- 172.21.15.78:0/3231460089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17b0003a10 0x7f17cc06e1d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 >> 172.21.15.78:0/3231460089 conn(0x7f17cc074b90 msgr2=0x7f17cc06ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 shutdown_connections 2024-09-11T12:49:59.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.346+0000 7f17d0e00640 1 -- 172.21.15.78:0/3231460089 wait complete. 2024-09-11T12:49:59.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:49:42.819611+0000', 'modified': '2024-09-11T12:49:45.831763+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4390}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4390': {'gid': 4390, 'name': 'a', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6834/1888452692', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 1888452692}, {'type': 'v1', 'addr': '172.21.15.78:6834', 'nonce': 1888452692}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4390, 'qdb_cluster': [4390]} 2024-09-11T12:49:59.771 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:49:59.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- 172.21.15.78:0/2379590453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4780 msgr2=0x7fea4c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 --2- 172.21.15.78:0/2379590453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4780 0x7fea4c073f70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c01c820 comp rx=0 tx=0).stop 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- 172.21.15.78:0/2379590453 shutdown_connections 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 --2- 172.21.15.78:0/2379590453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0ccb20 0x7fea4c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 --2- 172.21.15.78:0/2379590453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4780 0x7fea4c073f70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- 172.21.15.78:0/2379590453 >> 172.21.15.78:0/2379590453 conn(0x7fea4c06ca80 msgr2=0x7fea4c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- 172.21.15.78:0/2379590453 shutdown_connections 2024-09-11T12:49:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- 172.21.15.78:0/2379590453 wait complete. 2024-09-11T12:49:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 Processor -- start 2024-09-11T12:49:59.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.910+0000 7fea50e00640 1 -- start start 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0b4780 0x7fea4c073540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 0x7fea4c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c168fd0 con 0x7fea4c0b4b60 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c168e50 con 0x7fea4c0ccb20 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c169150 con 0x7fea4c0b4780 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60320/0 (socket says 172.21.15.78:60320) 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 -- 172.21.15.78:0/2953298656 learned_addr learned my addr 172.21.15.78:0/2953298656 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea4a800640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 0x7fea4c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0b4780 0x7fea4c073540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 -- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0b4780 msgr2=0x7fea4c073540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0b4780 0x7fea4c073540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 -- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 msgr2=0x7fea4c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 0x7fea4c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 -- 172.21.15.78:0/2953298656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea4c073b90 con 0x7fea4c0b4b60 2024-09-11T12:49:59.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea4a800640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 0x7fea4c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:49:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea49400640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d720 tx=0x7fea3800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea42000640 1 -- 172.21.15.78:0/2953298656 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38017070 con 0x7fea4c0b4b60 2024-09-11T12:49:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea42000640 1 -- 172.21.15.78:0/2953298656 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea38009d20 con 0x7fea4c0b4b60 2024-09-11T12:49:59.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea42000640 1 -- 172.21.15.78:0/2953298656 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea38005280 con 0x7fea4c0b4b60 2024-09-11T12:49:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea4c1f7e30 con 0x7fea4c0b4b60 2024-09-11T12:49:59.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea4c1f83f0 con 0x7fea4c0b4b60 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.914+0000 7fea42000640 1 -- 172.21.15.78:0/2953298656 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fea38005420 con 0x7fea4c0b4b60 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea42000640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 0x7fea2c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea42000640 1 -- 172.21.15.78:0/2953298656 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fea38091e10 con 0x7fea4c0b4b60 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 Processor -- start 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 0x7fea2c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- start start 2024-09-11T12:49:59.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c33c0 0x55eb77875e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77876720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb776d3270 con 0x55eb778c2aa0 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eb776c2770 con 0x55eb77876340 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55eb776c67b0 con 0x55eb778c33c0 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 0x7fea2c073b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0af000 tx=0x7fea3c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c33c0 0x55eb77875e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60330/0 (socket says 172.21.15.78:60330) 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 -- 172.21.15.78:0/308477601 learned_addr learned my addr 172.21.15.78:0/308477601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea4a800640 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77876720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 -- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c33c0 msgr2=0x55eb77875e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c33c0 0x55eb77875e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 -- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 msgr2=0x55eb77876720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77876720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 -- 172.21.15.78:0/308477601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb77661f90 con 0x55eb778c2aa0 2024-09-11T12:49:59.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea49e00640 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009600 tx=0x7fea3c0b7000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb79d6eb6a366640 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea36a00640 1 -- 172.21.15.78:0/308477601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0b2040 con 0x55eb778c2aa0 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea36a00640 1 -- 172.21.15.78:0/308477601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3c003dc0 con 0x55eb778c2aa0 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea36a00640 1 -- 172.21.15.78:0/308477601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0052b0 con 0x55eb778c2aa0 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- 172.21.15.78:0/308477601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 msgr2=0x55eb778c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009600 tx=0x7fea3c0b7000 comp rx=0 tx=0).stop 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- 172.21.15.78:0/308477601 shutdown_connections 2024-09-11T12:49:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77876720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c33c0 0x55eb77875e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 --2- 172.21.15.78:0/308477601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c2aa0 0x55eb778c2e80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.918+0000 7fea54236000 1 -- 172.21.15.78:0/308477601 >> 172.21.15.78:0/308477601 conn(0x55eb778a3b60 msgr2=0x55eb778a5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- 172.21.15.78:0/308477601 shutdown_connections 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- 172.21.15.78:0/308477601 wait complete. 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- start start 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 Processor -- start 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77883d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c2aa0 0x55eb77884290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb77792290 con 0x55eb778c33c0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eb7766eab0 con 0x55eb77876340 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55eb7766c170 con 0x55eb778c2aa0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 10 client.0 _ll_register_callbacks cb 0x55eb77863830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60338/0 (socket says 172.21.15.78:60338) 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 -- 172.21.15.78:0/3253612527 learned_addr learned my addr 172.21.15.78:0/3253612527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea49e00640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77883d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea49400640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c2aa0 0x55eb77884290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 -- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c2aa0 msgr2=0x55eb77884290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c2aa0 0x55eb77884290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 -- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 msgr2=0x55eb77883d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77883d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 -- 172.21.15.78:0/3253612527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb777476f0 con 0x55eb778c33c0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea4a800640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fea440115f0 tx=0x7fea440119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9079673be48253 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44003c50 con 0x55eb778c33c0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fea44003df0 con 0x55eb778c33c0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55eb7765bf10 con 0x55eb778c33c0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fea44022050 con 0x55eb778c33c0 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 10 client.4635 fetch_fsmap learned FSMap version 8 2024-09-11T12:49:59.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55eb7771ec40 con 0x55eb778c33c0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55eb7765bf10 con 0x55eb778c33c0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fea44025020 con 0x55eb778c33c0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 10 client.4635 fetch_fsmap finished waiting for FSMap version 8 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 10 client.4635 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 20 client.4635 populate_metadata read hostname 'smithi078' 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778851e0 0x55eb778d4b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 4 client.4635 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 --> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb776d51c0 con 0x55eb778851e0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.922+0000 7fea49e00640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778851e0 0x55eb778d4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea49e00640 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778851e0 0x55eb778d4b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=516e697c09e64bf8 server_cookie=91f7b3b57292343c in_seq=0 out_seq=0 2024-09-11T12:49:59.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea2b400640 10 client.4635 ms_handle_connect on v2:172.21.15.78:6832/1888452692 2024-09-11T12:49:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea2b400640 1 -- 172.21.15.78:0/3253612527 <== mds.0 v2:172.21.15.78:6832/1888452692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb776d51c0 con 0x55eb778851e0 2024-09-11T12:49:59.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea2b400640 10 client.4635 handle_command_reply: tid=0 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 client.4635 shutdown 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 msgr2=0x55eb778847d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fea440115f0 tx=0x7fea440119f0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 shutdown_connections 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778851e0 0x55eb778d4b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778c33c0 0x55eb778847d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb778c2aa0 0x55eb77884290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 --2- 172.21.15.78:0/3253612527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb77876340 0x55eb77883d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 >> 172.21.15.78:0/3253612527 conn(0x55eb778a3b60 msgr2=0x55eb77875500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea2b400640 0 client.4635 ms_handle_reset on v2:172.21.15.78:6832/1888452692 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 shutdown_connections 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 1 -- 172.21.15.78:0/3253612527 wait complete. 2024-09-11T12:49:59.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.926+0000 7fea54236000 20 client.4635 trim_cache size 0 max 16384 2024-09-11T12:49:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.942+0000 7fea54236000 1 Processor -- start 2024-09-11T12:49:59.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.942+0000 7fea54236000 1 -- start start 2024-09-11T12:49:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb778a3b60 0x55eb779341d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb77934710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c146ea0 con 0x55eb7787e4b0 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea4c146200 con 0x55eb778a3b60 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea4c146850 con 0x55eb77876340 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60340/0 (socket says 172.21.15.78:60340) 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 -- 172.21.15.78:0/3113792670 learned_addr learned my addr 172.21.15.78:0/3113792670 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea4a800640 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb77934710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49400640 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb778a3b60 0x55eb779341d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 -- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 msgr2=0x55eb77934710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb77934710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 -- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb778a3b60 msgr2=0x55eb779341d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb778a3b60 0x55eb779341d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 -- 172.21.15.78:0/3113792670 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb77677e70 con 0x55eb7787e4b0 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea49e00640 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c0195c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e039da9a6e516f server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea36000640 1 -- 172.21.15.78:0/3113792670 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0b2040 con 0x55eb7787e4b0 2024-09-11T12:49:59.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea36000640 1 -- 172.21.15.78:0/3113792670 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3c0d2070 con 0x55eb7787e4b0 2024-09-11T12:49:59.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea36000640 1 -- 172.21.15.78:0/3113792670 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea3c005040 con 0x55eb7787e4b0 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- 172.21.15.78:0/3113792670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 msgr2=0x55eb77933c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009840 tx=0x7fea3c0195c0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- 172.21.15.78:0/3113792670 shutdown_connections 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb77934710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb778a3b60 0x55eb779341d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 --2- 172.21.15.78:0/3113792670 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb7787e4b0 0x55eb77933c90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- 172.21.15.78:0/3113792670 >> 172.21.15.78:0/3113792670 conn(0x55eb778751f0 msgr2=0x55eb778d0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- 172.21.15.78:0/3113792670 shutdown_connections 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- 172.21.15.78:0/3113792670 wait complete. 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 -- start start 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.946+0000 7fea54236000 1 Processor -- start 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb778f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb7787e4b0 0x55eb778f3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55eb776cade0 con 0x55eb778a3b60 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55eb776c5ac0 con 0x55eb7787e4b0 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55eb776d5ea0 con 0x55eb77876340 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 10 client.0 _ll_register_callbacks cb 0x7fea440019e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60344/0 (socket says 172.21.15.78:60344) 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 -- 172.21.15.78:0/279711562 learned_addr learned my addr 172.21.15.78:0/279711562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea49e00640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb778f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea49400640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb7787e4b0 0x55eb778f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 -- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 msgr2=0x55eb778f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb778f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 -- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb7787e4b0 msgr2=0x55eb778f3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb7787e4b0 0x55eb778f3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 -- 172.21.15.78:0/279711562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb7764cbf0 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea4a800640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea44001f50 tx=0x7fea44007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821100e14c5fb4ff server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea44015880 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fea44015a20 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea3c0d2070 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7fea44033050 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 10 client.4641 fetch_fsmap learned FSMap version 8 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55eb77668c60 con 0x55eb778a3b60 2024-09-11T12:49:59.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea3c0d2070 con 0x55eb778a3b60 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7fea44036020 con 0x55eb778a3b60 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 10 client.4641 fetch_fsmap finished waiting for FSMap version 8 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 10 client.4641 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 20 client.4641 populate_metadata read hostname 'smithi078' 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778c2aa0 0x55eb778f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 4 client.4641 mds_command: new command op to 4390 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 --> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] -- command(tid 0: {"prefix": "status"}) -- 0x55eb776d0c60 con 0x55eb778c2aa0 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea49e00640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778c2aa0 0x55eb778f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 10 client.4641 ms_handle_connect on v2:172.21.15.78:6832/1888452692 2024-09-11T12:49:59.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea49e00640 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778c2aa0 0x55eb778f3e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b09f9112b6af5b server_cookie=ef41dfd45e98387b in_seq=0 out_seq=0 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 1 -- 172.21.15.78:0/279711562 <== mds.0 v2:172.21.15.78:6832/1888452692 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55eb776d0c60 con 0x55eb778c2aa0 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 10 client.4641 handle_command_reply: tid=0 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 client.4641 shutdown 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 msgr2=0x55eb778f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fea44001f50 tx=0x7fea44007bd0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 shutdown_connections 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:6832/1888452692,v1:172.21.15.78:6834/1888452692] conn(0x55eb778c2aa0 0x55eb778f3e00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55eb778a3b60 0x55eb778f38c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55eb7787e4b0 0x55eb778f3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea2b400640 0 client.4641 ms_handle_reset on v2:172.21.15.78:6832/1888452692 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 --2- 172.21.15.78:0/279711562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55eb77876340 0x55eb778f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 >> 172.21.15.78:0/279711562 conn(0x55eb778751f0 msgr2=0x55eb778a1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 shutdown_connections 2024-09-11T12:49:59.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 1 -- 172.21.15.78:0/279711562 wait complete. 2024-09-11T12:49:59.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.950+0000 7fea54236000 20 client.4641 trim_cache size 0 max 16384 2024-09-11T12:49:59.963 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:49:59.963 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:49:59.963 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:49:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 4390, 2024-09-11T12:49:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:49:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:49:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:49:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 16.515548883000001, 2024-09-11T12:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 8, 2024-09-11T12:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 23, 2024-09-11T12:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 23, 2024-09-11T12:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 22.569603267000002 2024-09-11T12:49:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:49:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 msgr2=0x7fea2c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 0x7fea2c073b40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fea3c0af000 tx=0x7fea3c01b060 comp rx=0 tx=0).stop 2024-09-11T12:49:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 msgr2=0x7fea4c145550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:49:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fea3800d720 tx=0x7fea3800dbe0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 shutdown_connections 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea2c071680 0x7fea2c073b40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea4c0ccb20 0x7fea4c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea4c0b4b60 0x7fea4c145550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 --2- 172.21.15.78:0/2953298656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea4c0b4780 0x7fea4c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 >> 172.21.15.78:0/2953298656 conn(0x7fea4c06ca80 msgr2=0x7fea4c15cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:49:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 shutdown_connections 2024-09-11T12:49:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:49:59.954+0000 7fea50e00640 1 -- 172.21.15.78:0/2953298656 wait complete. 2024-09-11T12:49:59.977 INFO:teuthology.run_tasks:Running task sequential... 2024-09-11T12:49:59.991 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-11T12:49:59.991 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-11T12:49:59.992 INFO:teuthology.task.print:Enabling mgr modules 2024-09-11T12:49:59.992 INFO:teuthology.run_tasks:Running task kclient... 2024-09-11T12:50:00.024 INFO:tasks.kclient:Mounting kernel clients... 2024-09-11T12:50:00.025 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-11T12:50:00.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.025 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:50:00.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T12:50:00.098 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-09-11T12:50:00.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:50:00.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.101 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:50:00.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.157 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T12:50:00.226 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-09-11T12:50:00.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:50:00.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.229 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:50:00.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.238 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-11T12:50:00.305 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-11T12:50:00.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:50:00.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.307 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:50:00.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.356 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-11T12:50:00.425 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-11T12:50:00.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:50:00.427 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-11T12:50:00.427 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T12:50:00.427 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:00.427 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 -- 172.21.15.78:0/2812436154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680b4a60 msgr2=0x7f2468143350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 --2- 172.21.15.78:0/2812436154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680b4a60 0x7f2468143350 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2468074f20 tx=0x7f246001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 -- 172.21.15.78:0/2812436154 shutdown_connections 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 --2- 172.21.15.78:0/2812436154 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680b4a60 0x7f2468143350 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 --2- 172.21.15.78:0/2812436154 >> v2:172.21.15.157:3300/0 conn(0x7f2468073af0 0x7f24680b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 --2- 172.21.15.78:0/2812436154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24680cf490 0x7f246815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.562+0000 7f246fc00640 1 -- 172.21.15.78:0/2812436154 >> 172.21.15.78:0/2812436154 conn(0x7f246806ca80 msgr2=0x7f24680b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- 172.21.15.78:0/2812436154 shutdown_connections 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- 172.21.15.78:0/2812436154 wait complete. 2024-09-11T12:50:00.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 Processor -- start 2024-09-11T12:50:00.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- start start 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 0x7f24681395d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680b4a60 0x7f2468138a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2468142330 con 0x7f24680cf490 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f24681421b0 con 0x7f24680b4a60 2024-09-11T12:50:00.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f24681424b0 con 0x7f2468073af0 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60352/0 (socket says 172.21.15.78:60352) 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 -- 172.21.15.78:0/3833085329 learned_addr learned my addr 172.21.15.78:0/3833085329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246d200640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 0x7f24681395d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f2467e00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680b4a60 0x7f2468138a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 -- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 msgr2=0x7f24681395d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 0x7f24681395d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 -- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680b4a60 msgr2=0x7f2468138a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680b4a60 0x7f2468138a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 -- 172.21.15.78:0/3833085329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2468138fc0 con 0x7f24680cf490 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246d200640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 0x7f24681395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:00.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246dc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f246001d160 tx=0x7f24600a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24600b2040 con 0x7f24680cf490 2024-09-11T12:50:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2460004270 con 0x7f24680cf490 2024-09-11T12:50:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24681392f0 con 0x7f24680cf490 2024-09-11T12:50:00.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460005070 con 0x7f24680cf490 2024-09-11T12:50:00.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.566+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246813a420 con 0x7f24680cf490 2024-09-11T12:50:00.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.570+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2468143350 con 0x7f24680cf490 2024-09-11T12:50:00.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.570+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f24600bd020 con 0x7f24680cf490 2024-09-11T12:50:00.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.570+0000 7f2464c00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 0x7f2448073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:00.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.570+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f246012b730 con 0x7f24680cf490 2024-09-11T12:50:00.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.570+0000 7f246d200640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 0x7f2448073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:00.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.574+0000 7f246d200640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 0x7f2448073b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2458009f70 tx=0x7f2458009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:00.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.574+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2468138fc0 con 0x7f24680cf490 2024-09-11T12:50:00.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.774+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f24681611d0 con 0x7f24680cf490 2024-09-11T12:50:00.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.774+0000 7f2464c00640 1 -- 172.21.15.78:0/3833085329 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f24600164b0 con 0x7f24680cf490 2024-09-11T12:50:00.780 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 msgr2=0x7f2448073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 0x7f2448073b40 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f2458009f70 tx=0x7f2458009500 comp rx=0 tx=0).stop 2024-09-11T12:50:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 msgr2=0x7f246813bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f246001d160 tx=0x7f24600a56d0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 shutdown_connections 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2448071680 0x7f2448073b40 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24680cf490 0x7f246813bdd0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24680b4a60 0x7f2468138a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 --2- 172.21.15.78:0/3833085329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2468073af0 0x7f24681395d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 >> 172.21.15.78:0/3833085329 conn(0x7f246806ca80 msgr2=0x7f2468070750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 shutdown_connections 2024-09-11T12:50:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00.778+0000 7f246fc00640 1 -- 172.21.15.78:0/3833085329 wait complete. 2024-09-11T12:50:00.801 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T12:50:00.801 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:00.801 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:50:00.801 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:50:00.801 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:00.802 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T12:50:00.802 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T12:50:00.802 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:00.802 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:50:00.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.802 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:50:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42433sec preferred_lft 42433sec 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:00.814 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add name ceph-brx type bridge 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr flush dev ceph-brx 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set ceph-brx up 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T12:50:00.814 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:00.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:00.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:00.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.964 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:00.985 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:50:00.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:00.985 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:50:01.068 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:01.069 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:01.070 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:01.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:01.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:01.209 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:50:01.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:01.219 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-09-11T12:50:01.269 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:01.269 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:01.269 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:50:01.269 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-11T12:50:01.269 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:01.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:01.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:01.395 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-11T12:50:01.395 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:01.395 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:01.395 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-11T12:50:01.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T12:50:01.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-11T12:50:01.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-11T12:50:01.396 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-11T12:50:01.396 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:01.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:01.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:01.600 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:01.600 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:01.600 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:50:01.600 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T12:50:01.600 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:01.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:01.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:01.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:01.685 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:01.741 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:50:01.741 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:01.741 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:01.796 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:50:01.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:01.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:01.884 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:01.885 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:02.535 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:02.536 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:02.536 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:02.536 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:02.536 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:02.539 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:02.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:02.540 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:02.606 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:02.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:02.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:02.721 INFO:teuthology.orchestra.run.smithi078.stdout:client4650 2024-09-11T12:50:02.721 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:02.721 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:02.730 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:02.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:02.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:02.834 INFO:teuthology.orchestra.run.smithi078.stdout:client4650 2024-09-11T12:50:02.834 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:02.897 INFO:teuthology.orchestra.run.smithi078.stdout:client4650 2024-09-11T12:50:02.897 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/163b1675-69a3-4b68-a3e4-8c2fa35bac4f.client4650/status') 2024-09-11T12:50:02.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:02.961 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.4650 (3)192.168.0.1:0/2529206861 2024-09-11T12:50:02.961 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-09-11T12:50:02.961 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-09-11T12:50:02.962 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-09-11T12:50:02.962 INFO:teuthology.orchestra.run.smithi078.stderr:69 bytes copied, 5.019e-05 s, 1.4 MB/s 2024-09-11T12:50:02.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:02.969 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-11T12:50:02.969 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T12:50:02.970 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-11T12:50:02.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:03.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- 172.21.15.78:0/3511812272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 msgr2=0x7f13e0070830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:03.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- 172.21.15.78:0/3511812272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e0070830 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f13e0074d50 tx=0x7f13cc01c890 comp rx=0 tx=0).stop 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- 172.21.15.78:0/3511812272 shutdown_connections 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- 172.21.15.78:0/3511812272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e0070830 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- 172.21.15.78:0/3511812272 >> v2:172.21.15.157:3300/0 conn(0x7f13e00cf760 0x7f13e0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- 172.21.15.78:0/3511812272 >> 172.21.15.78:0/3511812272 conn(0x7f13e006ca80 msgr2=0x7f13e00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- 172.21.15.78:0/3511812272 shutdown_connections 2024-09-11T12:50:03.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- 172.21.15.78:0/3511812272 wait complete. 2024-09-11T12:50:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 Processor -- start 2024-09-11T12:50:03.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- start start 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e006fc50 0x7f13e014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e00cf760 0x7f13e00aab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13e0151820 con 0x7f13e015aec0 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13e00cfd50 con 0x7f13e00cf760 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.154+0000 7f13e4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13e01519a0 con 0x7f13e006fc50 2024-09-11T12:50:03.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60364/0 (socket says 172.21.15.78:60364) 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 -- 172.21.15.78:0/4005683384 learned_addr learned my addr 172.21.15.78:0/4005683384 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13dd400640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e00cf760 0x7f13e00aab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13dde00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e006fc50 0x7f13e014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 -- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e006fc50 msgr2=0x7f13e014dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e006fc50 0x7f13e014dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 -- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e00cf760 msgr2=0x7f13e00aab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e00cf760 0x7f13e00aab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 -- 172.21.15.78:0/4005683384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00a9460 con 0x7f13e015aec0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13de800640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c7b0 tx=0x7f13d800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8004b70 con 0x7f13e015aec0 2024-09-11T12:50:03.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d8004e00 con 0x7f13e015aec0 2024-09-11T12:50:03.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e014e820 con 0x7f13e015aec0 2024-09-11T12:50:03.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d800aab0 con 0x7f13e015aec0 2024-09-11T12:50:03.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e01f7e10 con 0x7f13e015aec0 2024-09-11T12:50:03.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0074240 con 0x7f13e015aec0 2024-09-11T12:50:03.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.158+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f13d800ac50 con 0x7f13e015aec0 2024-09-11T12:50:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.162+0000 7f13d6000640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 0x7f13c0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.162+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f13d8091f30 con 0x7f13e015aec0 2024-09-11T12:50:03.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.162+0000 7f13dde00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 0x7f13c0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:03.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.162+0000 7f13dde00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 0x7f13c0073b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009990 tx=0x7f13d0009110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:03.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.166+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f13d8020070 con 0x7f13e015aec0 2024-09-11T12:50:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13e015e790 con 0x7f13e015aec0 2024-09-11T12:50:03.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13d6000640 1 -- 172.21.15.78:0/4005683384 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f13d8062e20 con 0x7f13e015aec0 2024-09-11T12:50:03.374 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 msgr2=0x7f13c0073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 0x7f13c0073b40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f13d0009990 tx=0x7f13d0009110 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 msgr2=0x7f13e00ab0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f13d800c7b0 tx=0x7f13d800cc70 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 shutdown_connections 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13c0071680 0x7f13c0073b40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13e015aec0 0x7f13e00ab0b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13e00cf760 0x7f13e00aab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 --2- 172.21.15.78:0/4005683384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13e006fc50 0x7f13e014dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 >> 172.21.15.78:0/4005683384 conn(0x7f13e006ca80 msgr2=0x7f13e0159f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:03.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 shutdown_connections 2024-09-11T12:50:03.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03.370+0000 7f13e4e00640 1 -- 172.21.15.78:0/4005683384 wait complete. 2024-09-11T12:50:03.394 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:03.395 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-11T12:50:03.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.395 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:50:03.404 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:03.404 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:03.404 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:03.405 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42431sec preferred_lft 42431sec 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:50:03.406 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.407 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:03.407 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:50:03.407 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:50:03.407 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.408 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:03.466 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:50:03.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.468 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:50:03.520 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:50:03.520 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:03.520 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:03.521 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:03.521 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:03.521 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:03.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:03.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:03.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:03.522 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:03.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:03.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:03.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.654 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:50:03.663 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:50:03.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.664 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list-id 2024-09-11T12:50:03.712 INFO:teuthology.orchestra.run.smithi078.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-11T12:50:03.713 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:03.714 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:03.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:50:03.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-11T12:50:03.714 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:03.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:03.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:03.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.843 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether de:2c:9a:00:63:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-11T12:50:03.863 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-11T12:50:03.864 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.864 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::dc2c:9aff:fe00:6306/64 scope link 2024-09-11T12:50:03.864 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:03.865 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-11T12:50:03.928 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether de:2c:9a:00:63:06 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::dc2c:9aff:fe00:6306/64 scope link 2024-09-11T12:50:03.929 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:03.930 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-11T12:50:03.931 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:04.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:04.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.114 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:04.114 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:04.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.1 up 2024-09-11T12:50:04.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set dev brx.1 master ceph-brx 2024-09-11T12:50:04.114 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:04.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:04.196 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-11T12:50:04.245 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-11T12:50:04.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:04.245 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-11T12:50:04.296 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-11T12:50:04.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:04.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:04.384 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:04.384 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 2@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:04.397 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:04.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:04.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:04.401 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-11T12:50:04.421 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:04.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:04.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.520 INFO:teuthology.orchestra.run.smithi078.stdout:client4659 2024-09-11T12:50:04.520 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:04.521 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:04.529 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:04.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:04.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.629 INFO:teuthology.orchestra.run.smithi078.stdout:client4659 2024-09-11T12:50:04.629 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:04.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:04.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.727 INFO:teuthology.orchestra.run.smithi078.stdout:client4659 2024-09-11T12:50:04.727 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/163b1675-69a3-4b68-a3e4-8c2fa35bac4f.client4659/status') 2024-09-11T12:50:04.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:04.833 INFO:teuthology.orchestra.run.smithi078.stdout:instance: client.4659 (3)192.168.0.2:0/3660421983 2024-09-11T12:50:04.833 INFO:teuthology.orchestra.run.smithi078.stdout:blocklisted: false 2024-09-11T12:50:04.834 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records in 2024-09-11T12:50:04.834 INFO:teuthology.orchestra.run.smithi078.stderr:0+1 records out 2024-09-11T12:50:04.834 INFO:teuthology.orchestra.run.smithi078.stderr:69 bytes copied, 4.9119e-05 s, 1.4 MB/s 2024-09-11T12:50:04.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:04.839 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-11T12:50:04.839 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T12:50:04.839 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:04.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 -- 172.21.15.78:0/973897186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f907c0050f0 msgr2=0x7f907c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:05.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 --2- 172.21.15.78:0/973897186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f907c0050f0 0x7f907c005920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9084009b60 tx=0x7f908401d2b0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909d000640 1 -- 172.21.15.78:0/973897186 reap_dead start 2024-09-11T12:50:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 -- 172.21.15.78:0/973897186 shutdown_connections 2024-09-11T12:50:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 -- 172.21.15.78:0/973897186 >> 172.21.15.78:0/973897186 conn(0x7f909806ca80 msgr2=0x7f90980b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 -- 172.21.15.78:0/973897186 shutdown_connections 2024-09-11T12:50:05.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 -- 172.21.15.78:0/973897186 wait complete. 2024-09-11T12:50:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.022+0000 7f909f000640 1 Processor -- start 2024-09-11T12:50:05.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- start start 2024-09-11T12:50:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907c0050f0 0x7f909815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90981581a0 0x7f9098160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:05.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:05.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f909806f010 con 0x7f9098161310 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f909806ee90 con 0x7f90981581a0 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f909806f190 con 0x7f907c0050f0 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36452/0 (socket says 172.21.15.78:36452) 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 -- 172.21.15.78:0/3223707017 learned_addr learned my addr 172.21.15.78:0/3223707017 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f9097e00640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907c0050f0 0x7f909815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f9097400640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90981581a0 0x7f9098160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:05.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 -- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907c0050f0 msgr2=0x7f909815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907c0050f0 0x7f909815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 -- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90981581a0 msgr2=0x7f9098160dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90981581a0 0x7f9098160dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 -- 172.21.15.78:0/3223707017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9098158680 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909d000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90980b2f70 tx=0x7f9090011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090003980 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9090003b20 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9098158960 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9090004030 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9098158f20 con 0x7f9098161310 2024-09-11T12:50:05.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.026+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90980736d0 con 0x7f9098161310 2024-09-11T12:50:05.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.030+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f90900041d0 con 0x7f9098161310 2024-09-11T12:50:05.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.030+0000 7f908fe00640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 0x7f9078073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:05.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.030+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f90900929e0 con 0x7f9098161310 2024-09-11T12:50:05.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.030+0000 7f9097e00640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 0x7f9078073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:05.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.030+0000 7f9097e00640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 0x7f9078073b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9088001870 tx=0x7f9088001660 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:05.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.034+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f909005e830 con 0x7f9098161310 2024-09-11T12:50:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.238+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f909815b1e0 con 0x7f9098161310 2024-09-11T12:50:05.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.238+0000 7f908fe00640 1 -- 172.21.15.78:0/3223707017 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f909005e130 con 0x7f9098161310 2024-09-11T12:50:05.244 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 msgr2=0x7f9078073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 0x7f9078073b40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9088001870 tx=0x7f9088001660 comp rx=0 tx=0).stop 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 msgr2=0x7f9098145310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90980b2f70 tx=0x7f9090011ef0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 shutdown_connections 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9078071680 0x7f9078073b40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9098161310 0x7f9098145310 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90981581a0 0x7f9098160dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 --2- 172.21.15.78:0/3223707017 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907c0050f0 0x7f909815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 >> 172.21.15.78:0/3223707017 conn(0x7f909806ca80 msgr2=0x7f909815d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 shutdown_connections 2024-09-11T12:50:05.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:05.242+0000 7f909f000640 1 -- 172.21.15.78:0/3223707017 wait complete. 2024-09-11T12:50:05.264 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:05.265 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:50:05.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:05.265 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:50:05.277 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:05.277 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:05.277 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:05.277 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:05.277 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:05.278 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:05.279 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:50:05.279 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:05.279 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:05.279 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:05.279 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42438sec preferred_lft 42438sec 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:05.280 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:05.281 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:05.281 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:05.281 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-11T12:50:05.281 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add name ceph-brx type bridge 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr flush dev ceph-brx 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set ceph-brx up 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-11T12:50:05.282 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:05.372 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:05.434 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:05.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:05.442 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:05.461 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:50:05.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:05.463 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:50:05.530 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:50:05.530 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:05.530 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:05.531 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:05.531 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:05.531 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:05.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:05.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:05.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:05.532 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:05.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:05.658 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:05.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:05.665 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:50:05.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:05.720 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-09-11T12:50:05.772 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:05.772 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:05.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:50:05.772 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-11T12:50:05.772 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:05.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:05.896 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:05.902 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-11T12:50:05.903 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-11T12:50:05.904 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-11T12:50:05.904 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:05.993 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:06.110 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:06.117 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:06.117 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:06.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:50:06.117 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.0 master ceph-brx 2024-09-11T12:50:06.117 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:06.163 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:06.194 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:06.200 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:06.200 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:06.256 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:50:06.256 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:06.256 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:06.311 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:50:06.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:06.311 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:06.400 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:06.400 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:07.114 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:07.114 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:07.115 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:07.115 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:07.115 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:07.119 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:07.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:07.119 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:07.139 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:07.228 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:07.247 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:07.254 INFO:teuthology.orchestra.run.smithi157.stdout:client4747 2024-09-11T12:50:07.254 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:07.254 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:07.263 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:07.349 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:07.353 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:07.359 INFO:teuthology.orchestra.run.smithi157.stdout:client4747 2024-09-11T12:50:07.360 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:07.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:07.409 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:07.416 INFO:teuthology.orchestra.run.smithi157.stdout:client4747 2024-09-11T12:50:07.416 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/163b1675-69a3-4b68-a3e4-8c2fa35bac4f.client4747/status') 2024-09-11T12:50:07.465 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:07.478 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4747 (3)192.168.0.1:0/316732615 2024-09-11T12:50:07.478 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-09-11T12:50:07.478 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-09-11T12:50:07.478 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-09-11T12:50:07.478 INFO:teuthology.orchestra.run.smithi157.stderr:68 bytes copied, 5.0741e-05 s, 1.3 MB/s 2024-09-11T12:50:07.479 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:07.486 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-11T12:50:07.486 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-11T12:50:07.486 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-11T12:50:07.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:07.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 -- 172.21.15.78:0/1189030347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0b4520 msgr2=0x7f0dbc071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:07.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 --2- 172.21.15.78:0/1189030347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0b4520 0x7f0dbc071d90 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0db4009c80 tx=0x7f0db401cda0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 -- 172.21.15.78:0/1189030347 shutdown_connections 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 --2- 172.21.15.78:0/1189030347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc0b4520 0x7f0dbc071d90 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 --2- 172.21.15.78:0/1189030347 >> v2:172.21.15.157:3300/0 conn(0x7f0dbc154e20 0x7f0dbc15ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 -- 172.21.15.78:0/1189030347 >> 172.21.15.78:0/1189030347 conn(0x7f0dbc06c650 msgr2=0x7f0dbc0d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 -- 172.21.15.78:0/1189030347 shutdown_connections 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.630+0000 7f0dc1200640 1 -- 172.21.15.78:0/1189030347 wait complete. 2024-09-11T12:50:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 Processor -- start 2024-09-11T12:50:07.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- start start 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 0x7f0dbc153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 0x7f0dbc0b85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc16b300 con 0x7f0dbc0b4520 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0dbc16b180 con 0x7f0dbc154e20 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0dbc16b480 con 0x7f0dbc06fd00 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36458/0 (socket says 172.21.15.78:36458) 2024-09-11T12:50:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 -- 172.21.15.78:0/2531699383 learned_addr learned my addr 172.21.15.78:0/2531699383 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9e00640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 0x7f0dbc153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dba800640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 0x7f0dbc0b85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 -- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 msgr2=0x7f0dbc153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 0x7f0dbc153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 -- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 msgr2=0x7f0dbc0b85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 0x7f0dbc0b85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 -- 172.21.15.78:0/2531699383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dbc0b8c40 con 0x7f0dbc0b4520 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9e00640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 0x7f0dbc153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dba800640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 0x7f0dbc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db9400640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e9a0 tx=0x7f0da800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da800cd80 con 0x7f0dbc0b4520 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0da8004020 con 0x7f0dbc0b4520 2024-09-11T12:50:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da8004450 con 0x7f0dbc0b4520 2024-09-11T12:50:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dbc0b8f20 con 0x7f0dbc0b4520 2024-09-11T12:50:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.634+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dbc165de0 con 0x7f0dbc0b4520 2024-09-11T12:50:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dbc071d90 con 0x7f0dbc0b4520 2024-09-11T12:50:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f0da8003600 con 0x7f0dbc0b4520 2024-09-11T12:50:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0db2000640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 0x7f0d9c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0da80925d0 con 0x7f0dbc0b4520 2024-09-11T12:50:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0db9e00640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 0x7f0d9c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.638+0000 7f0db9e00640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 0x7f0d9c073b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc166a90 tx=0x7f0dac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:07.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.642+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0da805dcd0 con 0x7f0dbc0b4520 2024-09-11T12:50:07.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.842+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0dbc15c6f0 con 0x7f0dbc0b4520 2024-09-11T12:50:07.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.842+0000 7f0db2000640 1 -- 172.21.15.78:0/2531699383 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f0da8062220 con 0x7f0dbc0b4520 2024-09-11T12:50:07.849 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 msgr2=0x7f0d9c073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 0x7f0d9c073b70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc166a90 tx=0x7f0dac009040 comp rx=0 tx=0).stop 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 msgr2=0x7f0dbc0b8060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0da800e9a0 tx=0x7f0da800ee60 comp rx=0 tx=0).stop 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 shutdown_connections 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0d9c0716b0 0x7f0d9c073b70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0dbc154e20 0x7f0dbc0b85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dbc0b4520 0x7f0dbc0b8060 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 --2- 172.21.15.78:0/2531699383 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0dbc06fd00 0x7f0dbc153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:07.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 >> 172.21.15.78:0/2531699383 conn(0x7f0dbc06c650 msgr2=0x7f0dbc168010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:07.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 shutdown_connections 2024-09-11T12:50:07.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:07.846+0000 7f0dc1200640 1 -- 172.21.15.78:0/2531699383 wait complete. 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-11T12:50:07.870 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-11T12:50:07.871 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:07.871 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-11T12:50:07.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:07.871 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:07.880 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42436sec preferred_lft 42436sec 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.881 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:50:07.882 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:07.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:07.883 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:07.946 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:50:07.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:07.948 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:50:08.007 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:50:08.007 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:08.007 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:08.007 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:08.007 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:08.008 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:08.008 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:08.008 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:08.008 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:08.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:08.009 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:08.009 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:08.096 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:08.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:08.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.145 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:50:08.154 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:50:08.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.155 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list-id 2024-09-11T12:50:08.207 INFO:teuthology.orchestra.run.smithi157.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-11T12:50:08.208 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:08.208 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:08.208 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:50:08.208 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-11T12:50:08.208 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:08.298 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:08.327 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:08.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-11T12:50:08.354 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 8a:2b:14:f9:11:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::882b:14ff:fef9:1168/64 scope link 2024-09-11T12:50:08.355 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.357 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-11T12:50:08.419 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:08.420 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 8a:2b:14:f9:11:68 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::882b:14ff:fef9:1168/64 scope link 2024-09-11T12:50:08.421 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:08.422 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-11T12:50:08.422 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-11T12:50:08.423 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:08.514 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:08.598 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:08.605 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:08.605 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:08.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.1 up 2024-09-11T12:50:08.605 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set dev brx.1 master ceph-brx 2024-09-11T12:50:08.605 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:08.697 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:08.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:08.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:08.734 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-11T12:50:08.742 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-11T12:50:08.743 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:08.743 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-11T12:50:08.795 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-11T12:50:08.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.795 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:08.887 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:08.887 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:08.901 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:08.901 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:08.901 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:08.901 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 3@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:08.901 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:08.905 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:08.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:08.905 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-11T12:50:08.924 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:09.014 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:09.018 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:09.024 INFO:teuthology.orchestra.run.smithi157.stdout:client4759 2024-09-11T12:50:09.024 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:09.024 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:09.033 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:09.121 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:09.125 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:09.131 INFO:teuthology.orchestra.run.smithi157.stdout:client4759 2024-09-11T12:50:09.132 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:09.222 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:09.226 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:09.232 INFO:teuthology.orchestra.run.smithi157.stdout:client4759 2024-09-11T12:50:09.232 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/163b1675-69a3-4b68-a3e4-8c2fa35bac4f.client4759/status') 2024-09-11T12:50:09.279 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:09.290 INFO:teuthology.orchestra.run.smithi157.stdout:instance: client.4759 (3)192.168.0.2:0/3387054660 2024-09-11T12:50:09.291 INFO:teuthology.orchestra.run.smithi157.stdout:blocklisted: false 2024-09-11T12:50:09.291 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records in 2024-09-11T12:50:09.291 INFO:teuthology.orchestra.run.smithi157.stderr:0+1 records out 2024-09-11T12:50:09.291 INFO:teuthology.orchestra.run.smithi157.stderr:69 bytes copied, 4.9624e-05 s, 1.4 MB/s 2024-09-11T12:50:09.292 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:09.299 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-11T12:50:09.392 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-11T12:50:09.392 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:50:09.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 -- 172.21.15.78:0/1648723130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c070a00 msgr2=0x7f6c2c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 --2- 172.21.15.78:0/1648723130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c070a00 0x7f6c2c0736d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c15dad0 tx=0x7f6c1c01caf0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 -- 172.21.15.78:0/1648723130 shutdown_connections 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 --2- 172.21.15.78:0/1648723130 >> v2:172.21.15.157:3300/0 conn(0x7f6c2c06eb40 0x7f6c2c06efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 --2- 172.21.15.78:0/1648723130 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c070a00 0x7f6c2c0736d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 -- 172.21.15.78:0/1648723130 >> 172.21.15.78:0/1648723130 conn(0x7f6c2c06ca80 msgr2=0x7f6c2c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 -- 172.21.15.78:0/1648723130 shutdown_connections 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 -- 172.21.15.78:0/1648723130 wait complete. 2024-09-11T12:50:09.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.538+0000 7f6c32e00640 1 Processor -- start 2024-09-11T12:50:09.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- start start 2024-09-11T12:50:09.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 0x7f6c2c141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2c070a00 0x7f6c2c151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c15f9f0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c2c15f870 con 0x7f6c2c06eb40 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c2c0b4520 con 0x7f6c2c070a00 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c2be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 0x7f6c2c141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36470/0 (socket says 172.21.15.78:36470) 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c2be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 0x7f6c2c141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:48570/0 (socket says 172.21.15.78:48570) 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 -- 172.21.15.78:0/3312268276 learned_addr learned my addr 172.21.15.78:0/3312268276 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c2b400640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2c070a00 0x7f6c2c151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 -- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2c070a00 msgr2=0x7f6c2c151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2c070a00 0x7f6c2c151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 -- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 msgr2=0x7f6c2c141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 0x7f6c2c141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 -- 172.21.15.78:0/3312268276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c2c151e50 con 0x7f6c2c157ae0 2024-09-11T12:50:09.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c30e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d780 tx=0x7f6c2400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24017070 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c2400de90 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c2c1520f0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c24005020 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c2c1526b0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.542+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c2c157f10 con 0x7f6c2c157ae0 2024-09-11T12:50:09.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.546+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f6c240051c0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.546+0000 7f6c23e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 0x7f6c0c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.546+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6c24091da0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.546+0000 7f6c2be00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 0x7f6c0c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.546+0000 7f6c2be00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 0x7f6c0c073b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af000 tx=0x7f6c1c01b330 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:09.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.550+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6c2405d5b0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.714+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c2c153fb0 con 0x7f6c2c157ae0 2024-09-11T12:50:09.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c23e00640 1 -- 172.21.15.78:0/3312268276 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f6c24061b00 con 0x7f6c2c157ae0 2024-09-11T12:50:09.721 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:09.722 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 msgr2=0x7f6c0c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 0x7f6c0c073b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af000 tx=0x7f6c1c01b330 comp rx=0 tx=0).stop 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 msgr2=0x7f6c2c1641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400d780 tx=0x7f6c2400dc40 comp rx=0 tx=0).stop 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 shutdown_connections 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c0c071680 0x7f6c0c073b40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c2c157ae0 0x7f6c2c1641e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c2c070a00 0x7f6c2c151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 --2- 172.21.15.78:0/3312268276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c2c06eb40 0x7f6c2c141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 >> 172.21.15.78:0/3312268276 conn(0x7f6c2c06ca80 msgr2=0x7f6c2c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 shutdown_connections 2024-09-11T12:50:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.718+0000 7f6c32e00640 1 -- 172.21.15.78:0/3312268276 wait complete. 2024-09-11T12:50:09.743 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-11T12:49:40.738186+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-11T12:49:41.775035+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:50:09.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:50:09.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- 172.21.15.78:0/3216590711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba8152690 msgr2=0x7f3ba80a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 --2- 172.21.15.78:0/3216590711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009840 tx=0x7f3b9801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- 172.21.15.78:0/3216590711 shutdown_connections 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 --2- 172.21.15.78:0/3216590711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- 172.21.15.78:0/3216590711 >> 172.21.15.78:0/3216590711 conn(0x7f3ba80742a0 msgr2=0x7f3ba80746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- 172.21.15.78:0/3216590711 shutdown_connections 2024-09-11T12:50:09.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- 172.21.15.78:0/3216590711 wait complete. 2024-09-11T12:50:09.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 Processor -- start 2024-09-11T12:50:09.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- start start 2024-09-11T12:50:09.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba806fcb0 0x7f3ba814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8169cd0 con 0x7f3ba80cebc0 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba8169b50 con 0x7f3ba8152690 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bad800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba8169e50 con 0x7f3ba806fcb0 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36488/0 (socket says 172.21.15.78:36488) 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 -- 172.21.15.78:0/473791119 learned_addr learned my addr 172.21.15.78:0/473791119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7e00640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba806fcb0 0x7f3ba814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3bace00640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 -- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba806fcb0 msgr2=0x7f3ba814c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba806fcb0 0x7f3ba814c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 -- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba8152690 msgr2=0x7f3ba80a9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 -- 172.21.15.78:0/473791119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba814faa0 con 0x7f3ba80cebc0 2024-09-11T12:50:09.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.886+0000 7f3ba7400640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c500 tx=0x7f3b9400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94017070 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b94003600 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94003a90 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba81fab90 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba81fb430 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba81fb630 con 0x7f3ba80cebc0 2024-09-11T12:50:09.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f3b9401a070 con 0x7f3ba80cebc0 2024-09-11T12:50:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 0x7f3b88073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:09.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3b94091260 con 0x7f3ba80cebc0 2024-09-11T12:50:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.890+0000 7f3ba7e00640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 0x7f3b88073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:09.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.894+0000 7f3ba7e00640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 0x7f3b88073b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b980af000 tx=0x7f3b9801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:09.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:09.894+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3ba81fb630 con 0x7f3ba80cebc0 2024-09-11T12:50:10.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.062+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ba80a9460 con 0x7f3ba80cebc0 2024-09-11T12:50:10.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.062+0000 7f3b9fe00640 1 -- 172.21.15.78:0/473791119 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f3b940653c0 con 0x7f3ba80cebc0 2024-09-11T12:50:10.068 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 msgr2=0x7f3b88073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 0x7f3b88073b40 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3b980af000 tx=0x7f3b9801b060 comp rx=0 tx=0).stop 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 msgr2=0x7f3ba81513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c500 tx=0x7f3b9400c9c0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 shutdown_connections 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b88071680 0x7f3b88073b40 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ba8152690 0x7f3ba80a9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ba80cebc0 0x7f3ba81513e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 --2- 172.21.15.78:0/473791119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3ba806fcb0 0x7f3ba814c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 >> 172.21.15.78:0/473791119 conn(0x7f3ba80742a0 msgr2=0x7f3ba80b2570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 shutdown_connections 2024-09-11T12:50:10.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.066+0000 7f3bad800640 1 -- 172.21.15.78:0/473791119 wait complete. 2024-09-11T12:50:10.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-11T12:50:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- 172.21.15.78:0/3054426761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 msgr2=0x7f865015b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- 172.21.15.78:0/3054426761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f865015b650 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8640009bc0 tx=0x7f864001ca50 comp rx=0 tx=0).stop 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- 172.21.15.78:0/3054426761 shutdown_connections 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- 172.21.15.78:0/3054426761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 0x7f865015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- 172.21.15.78:0/3054426761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f865015b650 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- 172.21.15.78:0/3054426761 >> 172.21.15.78:0/3054426761 conn(0x7f8650075610 msgr2=0x7f8650075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- 172.21.15.78:0/3054426761 shutdown_connections 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- 172.21.15.78:0/3054426761 wait complete. 2024-09-11T12:50:10.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 Processor -- start 2024-09-11T12:50:10.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- start start 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 0x7f86501fa790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865015e180 0x7f8650135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86501676a0 con 0x7f86501516c0 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8650167520 con 0x7f865015e180 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8650167820 con 0x7f8650073970 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36520/0 (socket says 172.21.15.78:36520) 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 -- 172.21.15.78:0/2886779589 learned_addr learned my addr 172.21.15.78:0/2886779589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864de00640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 0x7f86501fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864e800640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865015e180 0x7f8650135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 -- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 msgr2=0x7f86501fa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 0x7f86501fa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 -- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865015e180 msgr2=0x7f8650135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865015e180 0x7f8650135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 -- 172.21.15.78:0/2886779589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86501fb6d0 con 0x7f86501516c0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.230+0000 7f864d400640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f863c001ec0 tx=0x7f863c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c017070 con 0x7f86501516c0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f863c003600 con 0x7f86501516c0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863c003a50 con 0x7f86501516c0 2024-09-11T12:50:10.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8650163760 con 0x7f86501516c0 2024-09-11T12:50:10.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86501fb110 con 0x7f86501516c0 2024-09-11T12:50:10.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f865015e5f0 con 0x7f86501516c0 2024-09-11T12:50:10.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f863c01a070 con 0x7f86501516c0 2024-09-11T12:50:10.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 0x7f8630073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f863c091100 con 0x7f86501516c0 2024-09-11T12:50:10.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.234+0000 7f864de00640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 0x7f8630073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.238+0000 7f864de00640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 0x7f8630073b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86400af000 tx=0x7f864001b290 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.238+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f863c05c910 con 0x7f86501516c0 2024-09-11T12:50:10.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.402+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8650163930 con 0x7f86501516c0 2024-09-11T12:50:10.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.402+0000 7f8646000640 1 -- 172.21.15.78:0/2886779589 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7f863c060e60 con 0x7f86501516c0 2024-09-11T12:50:10.409 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 32 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 msgr2=0x7f8630073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 0x7f8630073b40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86400af000 tx=0x7f864001b290 comp rx=0 tx=0).stop 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 msgr2=0x7f8650164c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f863c001ec0 tx=0x7f863c00c890 comp rx=0 tx=0).stop 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 shutdown_connections 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8630071680 0x7f8630073b40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f865015e180 0x7f8650135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86501516c0 0x7f8650164c80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 --2- 172.21.15.78:0/2886779589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8650073970 0x7f86501fa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 >> 172.21.15.78:0/2886779589 conn(0x7f8650075610 msgr2=0x7f8650069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 shutdown_connections 2024-09-11T12:50:10.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.406+0000 7f864f200640 1 -- 172.21.15.78:0/2886779589 wait complete. 2024-09-11T12:50:10.430 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-11T12:50:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- 172.21.15.78:0/2520677902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0b3040 msgr2=0x7f148c0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 --2- 172.21.15.78:0/2520677902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0b3040 0x7f148c0cf6e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1480009900 tx=0x7f148001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- 172.21.15.78:0/2520677902 shutdown_connections 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 --2- 172.21.15.78:0/2520677902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0b3040 0x7f148c0cf6e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- 172.21.15.78:0/2520677902 >> 172.21.15.78:0/2520677902 conn(0x7f148c071dd0 msgr2=0x7f148c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- 172.21.15.78:0/2520677902 shutdown_connections 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- 172.21.15.78:0/2520677902 wait complete. 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 Processor -- start 2024-09-11T12:50:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- start start 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0b3040 0x7f148c1fa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0b3420 0x7f148c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0b1e10 con 0x7f148c0cd770 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f148c0b1c90 con 0x7f148c0b3040 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1494400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f148c0b1f90 con 0x7f148c0b3420 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1492400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1492400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36540/0 (socket says 172.21.15.78:36540) 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.570+0000 7f1492400640 1 -- 172.21.15.78:0/1127583451 learned_addr learned my addr 172.21.15.78:0/1127583451 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1491a00640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0b3040 0x7f148c1fa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1491000640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0b3420 0x7f148c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 -- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0b3420 msgr2=0x7f148c158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0b3420 0x7f148c158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 -- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0b3040 msgr2=0x7f148c1fa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0b3040 0x7f148c1fa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 -- 172.21.15.78:0/1127583451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148c1f74f0 con 0x7f148c0cd770 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1492400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f148400b9e0 tx=0x7f148400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1484003b80 con 0x7f148c0cd770 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1484003e10 con 0x7f148c0cd770 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148400cb00 con 0x7f148c0cd770 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f148c1f77d0 con 0x7f148c0cd770 2024-09-11T12:50:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f148c1f7d90 con 0x7f148c0cd770 2024-09-11T12:50:10.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f148c0cf6e0 con 0x7f148c0cd770 2024-09-11T12:50:10.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f1484006d10 con 0x7f148c0cd770 2024-09-11T12:50:10.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 0x7f146c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.574+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1484091810 con 0x7f148c0cd770 2024-09-11T12:50:10.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.578+0000 7f1491a00640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 0x7f146c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.578+0000 7f1491a00640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 0x7f146c073b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14800af000 tx=0x7f148001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.578+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f148405d020 con 0x7f148c0cd770 2024-09-11T12:50:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.742+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f148c076290 con 0x7f148c0cd770 2024-09-11T12:50:10.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.742+0000 7f1489600640 1 -- 172.21.15.78:0/1127583451 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f1484061570 con 0x7f148c0cd770 2024-09-11T12:50:10.749 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 64 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 msgr2=0x7f146c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 0x7f146c073b40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f14800af000 tx=0x7f148001b120 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 msgr2=0x7f148c159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f148400b9e0 tx=0x7f148400bea0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 shutdown_connections 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f146c071680 0x7f146c073b40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f148c0cd770 0x7f148c159150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f148c0b3420 0x7f148c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 --2- 172.21.15.78:0/1127583451 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f148c0b3040 0x7f148c1fa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 >> 172.21.15.78:0/1127583451 conn(0x7f148c071dd0 msgr2=0x7f148c1414d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 shutdown_connections 2024-09-11T12:50:10.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.746+0000 7f1494400640 1 -- 172.21.15.78:0/1127583451 wait complete. 2024-09-11T12:50:10.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 -- 172.21.15.78:0/3658522706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 msgr2=0x7fcc6c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 --2- 172.21.15.78:0/3658522706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc6c004810 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc88071280 tx=0x7fcc7401f660 comp rx=0 tx=0).stop 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc86800640 1 -- 172.21.15.78:0/3658522706 reap_dead start 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 -- 172.21.15.78:0/3658522706 shutdown_connections 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 -- 172.21.15.78:0/3658522706 >> 172.21.15.78:0/3658522706 conn(0x7fcc88074b90 msgr2=0x7fcc880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 -- 172.21.15.78:0/3658522706 shutdown_connections 2024-09-11T12:50:10.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.906+0000 7fcc8d200640 1 -- 172.21.15.78:0/3658522706 wait complete. 2024-09-11T12:50:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 Processor -- start 2024-09-11T12:50:10.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- start start 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc8814cbc0 0x7fcc88147d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc881503f0 0x7fcc8814bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc880d1220 con 0x7fcc6c0040c0 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc880d10a0 con 0x7fcc8814cbc0 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc880d13a0 con 0x7fcc881503f0 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36552/0 (socket says 172.21.15.78:36552) 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 -- 172.21.15.78:0/1628133751 learned_addr learned my addr 172.21.15.78:0/1628133751 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:10.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85400640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc8814cbc0 0x7fcc88147d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc86800640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc881503f0 0x7fcc8814bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 -- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc881503f0 msgr2=0x7fcc8814bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc881503f0 0x7fcc8814bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 -- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc8814cbc0 msgr2=0x7fcc88147d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc8814cbc0 0x7fcc88147d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 -- 172.21.15.78:0/1628133751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc88148f10 con 0x7fcc6c0040c0 2024-09-11T12:50:10.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc85e00640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800b680 tx=0x7fcc7800bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc78010bb0 con 0x7fcc6c0040c0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc78015070 con 0x7fcc6c0040c0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc881491b0 con 0x7fcc6c0040c0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7801a470 con 0x7fcc6c0040c0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.910+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc88149770 con 0x7fcc6c0040c0 2024-09-11T12:50:10.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc8815c310 con 0x7fcc6c0040c0 2024-09-11T12:50:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fcc7801a610 con 0x7fcc6c0040c0 2024-09-11T12:50:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc7e000640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 0x7fcc68073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:10.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fcc78092a80 con 0x7fcc6c0040c0 2024-09-11T12:50:10.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc85400640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 0x7fcc68073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:10.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.914+0000 7fcc85400640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 0x7fcc68073b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b0000 tx=0x7fcc7401c4a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:10.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:10.918+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcc78094e20 con 0x7fcc6c0040c0 2024-09-11T12:50:11.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.142+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fcc88152970 con 0x7fcc6c0040c0 2024-09-11T12:50:11.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc7e000640 1 -- 172.21.15.78:0/1628133751 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v48) ==== 161+0+0 (secure 0 0 0) 0x7fcc7805e8e0 con 0x7fcc6c0040c0 2024-09-11T12:50:11.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 msgr2=0x7fcc68073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 0x7fcc68073b70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc740b0000 tx=0x7fcc7401c4a0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 msgr2=0x7fcc8814d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc7800b680 tx=0x7fcc7800bb40 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 shutdown_connections 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcc680716b0 0x7fcc68073b70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc881503f0 0x7fcc8814bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc8814cbc0 0x7fcc88147d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 --2- 172.21.15.78:0/1628133751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0040c0 0x7fcc8814d810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 >> 172.21.15.78:0/1628133751 conn(0x7fcc88074b90 msgr2=0x7fcc880b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 shutdown_connections 2024-09-11T12:50:11.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.222+0000 7fcc8d200640 1 -- 172.21.15.78:0/1628133751 wait complete. 2024-09-11T12:50:11.249 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:50:11.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:50:11.354 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- 172.21.15.78:0/4245930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12013fb30 msgr2=0x7fb1200c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 --2- 172.21.15.78:0/4245930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12013fb30 0x7fb1200c5a20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb1200c6520 tx=0x7fb11001c8a0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- 172.21.15.78:0/4245930416 shutdown_connections 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 --2- 172.21.15.78:0/4245930416 >> v2:172.21.15.157:3300/0 conn(0x7fb12015b7c0 0x7fb12013dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 --2- 172.21.15.78:0/4245930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12013fb30 0x7fb1200c5a20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- 172.21.15.78:0/4245930416 >> 172.21.15.78:0/4245930416 conn(0x7fb120073b70 msgr2=0x7fb120073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:11.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- 172.21.15.78:0/4245930416 shutdown_connections 2024-09-11T12:50:11.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- 172.21.15.78:0/4245930416 wait complete. 2024-09-11T12:50:11.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 Processor -- start 2024-09-11T12:50:11.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.386+0000 7fb126200640 1 -- start start 2024-09-11T12:50:11.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1200c5f60 0x7fb1201571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12013fb30 0x7fb120155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb120167a70 con 0x7fb12015b7c0 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb1201678f0 con 0x7fb1200c5f60 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb120167bf0 con 0x7fb12013fb30 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36564/0 (socket says 172.21.15.78:36564) 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 -- 172.21.15.78:0/2097984534 learned_addr learned my addr 172.21.15.78:0/2097984534 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11f400640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1200c5f60 0x7fb1201571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb117e00640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12013fb30 0x7fb120155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 -- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12013fb30 msgr2=0x7fb120155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12013fb30 0x7fb120155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 -- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1200c5f60 msgr2=0x7fb1201571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1200c5f60 0x7fb1201571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 -- 172.21.15.78:0/2097984534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1201fb300 con 0x7fb12015b7c0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb11fe00640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb11800cce0 tx=0x7fb11800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800f450 con 0x7fb12015b7c0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb11800f6e0 con 0x7fb12015b7c0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1201fb5e0 con 0x7fb12015b7c0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb11800fb30 con 0x7fb12015b7c0 2024-09-11T12:50:11.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1201559a0 con 0x7fb12015b7c0 2024-09-11T12:50:11.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb120155ba0 con 0x7fb12015b7c0 2024-09-11T12:50:11.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.390+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fb11800fcd0 con 0x7fb12015b7c0 2024-09-11T12:50:11.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.394+0000 7fb117400640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 0x7fb0fc073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.394+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb118092740 con 0x7fb12015b7c0 2024-09-11T12:50:11.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.394+0000 7fb11f400640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 0x7fb0fc073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.394+0000 7fb11f400640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 0x7fb0fc073b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb11001b0e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:11.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.398+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb120155ba0 con 0x7fb12015b7c0 2024-09-11T12:50:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.566+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fb12013f690 con 0x7fb12015b7c0 2024-09-11T12:50:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.566+0000 7fb117400640 1 -- 172.21.15.78:0/2097984534 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7fb118014030 con 0x7fb12015b7c0 2024-09-11T12:50:11.571 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:50:11.571 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:50:11.572 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:50:11.573 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:50:11.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 msgr2=0x7fb0fc073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 0x7fb0fc073b40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb1100af000 tx=0x7fb11001b0e0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 msgr2=0x7fb1201fa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb11800cce0 tx=0x7fb11800a620 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 shutdown_connections 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb0fc071680 0x7fb0fc073b40 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb12015b7c0 0x7fb1201fa920 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb12013fb30 0x7fb120155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 --2- 172.21.15.78:0/2097984534 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb1200c5f60 0x7fb1201571d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 >> 172.21.15.78:0/2097984534 conn(0x7fb120073b70 msgr2=0x7fb12015c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 shutdown_connections 2024-09-11T12:50:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.570+0000 7fb126200640 1 -- 172.21.15.78:0/2097984534 wait complete. 2024-09-11T12:50:11.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:50:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 -- 172.21.15.78:0/3570057070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 msgr2=0x7f7cf40cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 --2- 172.21.15.78:0/3570057070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf40cdc20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009c80 tx=0x7f7cec01cd70 comp rx=0 tx=0).stop 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 -- 172.21.15.78:0/3570057070 shutdown_connections 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 --2- 172.21.15.78:0/3570057070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf40cdc20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 --2- 172.21.15.78:0/3570057070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 0x7f7cf40cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 -- 172.21.15.78:0/3570057070 >> 172.21.15.78:0/3570057070 conn(0x7f7cf4071dd0 msgr2=0x7f7cf40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 -- 172.21.15.78:0/3570057070 shutdown_connections 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.730+0000 7f7cf9600640 1 -- 172.21.15.78:0/3570057070 wait complete. 2024-09-11T12:50:11.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 Processor -- start 2024-09-11T12:50:11.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- start start 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 0x7f7cf413a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf41f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf40af020 con 0x7f7cf40b3040 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cf40aeea0 con 0x7f7cf40cd770 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7cf40af1a0 con 0x7f7cf40b3420 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36568/0 (socket says 172.21.15.78:36568) 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 -- 172.21.15.78:0/2787275848 learned_addr learned my addr 172.21.15.78:0/2787275848 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf3400640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf41f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2000640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 0x7f7cf413a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 -- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 msgr2=0x7f7cf413a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 0x7f7cf413a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 -- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 msgr2=0x7f7cf41f7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf41f7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 -- 172.21.15.78:0/2787275848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cf41563f0 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf2a00640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c760 tx=0x7f7ce400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce4017070 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ce40051a0 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ce400a640 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cf4156690 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.734+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cf4156c50 con 0x7f7cf40b3040 2024-09-11T12:50:11.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cf40cf6e0 con 0x7f7cf40b3040 2024-09-11T12:50:11.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7ce401a070 con 0x7f7cf40b3040 2024-09-11T12:50:11.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7ceaa00640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 0x7f7cd4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:11.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7ce4091080 con 0x7f7cf40b3040 2024-09-11T12:50:11.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7cf2000640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 0x7f7cd4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:11.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.738+0000 7f7cf2000640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 0x7f7cd4073b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40cfbd0 tx=0x7f7ce0010040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:11.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.742+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7cf40ce160 con 0x7f7cf40b3040 2024-09-11T12:50:11.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.910+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f7cf40722e0 con 0x7f7cf40b3040 2024-09-11T12:50:11.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7ceaa00640 1 -- 172.21.15.78:0/2787275848 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f7ce4065120 con 0x7f7cf40b3040 2024-09-11T12:50:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 msgr2=0x7f7cd4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 0x7f7cd4073b40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f7cf40cfbd0 tx=0x7f7ce0010040 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 msgr2=0x7f7cf4153410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7ce400c760 tx=0x7f7ce400cc20 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 shutdown_connections 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7cd4071680 0x7f7cd4073b40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cf40cd770 0x7f7cf41f7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cf40b3420 0x7f7cf413a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 --2- 172.21.15.78:0/2787275848 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cf40b3040 0x7f7cf4153410 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 >> 172.21.15.78:0/2787275848 conn(0x7f7cf4071dd0 msgr2=0x7f7cf40700d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 shutdown_connections 2024-09-11T12:50:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:11.914+0000 7f7cf9600640 1 -- 172.21.15.78:0/2787275848 wait complete. 2024-09-11T12:50:11.939 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:50:11.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:12.043 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:12.131 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:50:12.131 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:12.131 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:12.140 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:12.140 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:12.188 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:12.188 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:12.245 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:12.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 -- 172.21.15.78:0/971521814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91401518f0 msgr2=0x7f9140073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 --2- 172.21.15.78:0/971521814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91401518f0 0x7f9140073c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f9130009bc0 tx=0x7f913001cba0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 -- 172.21.15.78:0/971521814 shutdown_connections 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 --2- 172.21.15.78:0/971521814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 --2- 172.21.15.78:0/971521814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f91401518f0 0x7f9140073c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 -- 172.21.15.78:0/971521814 >> 172.21.15.78:0/971521814 conn(0x7f914006f3a0 msgr2=0x7f91400cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 -- 172.21.15.78:0/971521814 shutdown_connections 2024-09-11T12:50:12.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.430+0000 7f9146400640 1 -- 172.21.15.78:0/971521814 wait complete. 2024-09-11T12:50:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 Processor -- start 2024-09-11T12:50:12.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- start start 2024-09-11T12:50:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914014a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 0x7f914015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9140165730 con 0x7f91401518f0 2024-09-11T12:50:12.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f91401655b0 con 0x7f914015ae40 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f91401658b0 con 0x7f91400cfeb0 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36578/0 (socket says 172.21.15.78:36578) 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 -- 172.21.15.78:0/1665937294 learned_addr learned my addr 172.21.15.78:0/1665937294 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913fe00640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 0x7f914015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 -- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 msgr2=0x7f914014a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913f400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914014a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914014a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 -- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 msgr2=0x7f914015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 0x7f914015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 -- 172.21.15.78:0/1665937294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f914014dfa0 con 0x7f91401518f0 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913f400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:12.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913fe00640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 0x7f914015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f913ea00640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c430 tx=0x7f912c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c011790 con 0x7f91401518f0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f912c017070 con 0x7f91401518f0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f912c011ce0 con 0x7f91401518f0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91400c3bf0 con 0x7f91401518f0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f914014a5a0 con 0x7f91401518f0 2024-09-11T12:50:12.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.434+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f914015f2a0 con 0x7f91401518f0 2024-09-11T12:50:12.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.438+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f912c01c460 con 0x7f91401518f0 2024-09-11T12:50:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.438+0000 7f9137400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 0x7f9120073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.438+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f912c091c90 con 0x7f91401518f0 2024-09-11T12:50:12.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.438+0000 7f913f400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 0x7f9120073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.438+0000 7f913f400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 0x7f9120073b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91300af000 tx=0x7f913001b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.442+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f912c05db20 con 0x7f91401518f0 2024-09-11T12:50:12.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.618+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f914015c610 con 0x7f91401518f0 2024-09-11T12:50:12.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.618+0000 7f9137400640 1 -- 172.21.15.78:0/1665937294 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f912c05d420 con 0x7f91401518f0 2024-09-11T12:50:12.624 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 msgr2=0x7f9120073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 0x7f9120073b40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f91300af000 tx=0x7f913001b3e0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 msgr2=0x7f9140152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f912c00c430 tx=0x7f912c00c8f0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 shutdown_connections 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9120071680 0x7f9120073b40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f914015ae40 0x7f914015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f91401518f0 0x7f9140152240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 --2- 172.21.15.78:0/1665937294 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f91400cfeb0 0x7f914014a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 >> 172.21.15.78:0/1665937294 conn(0x7f914006f3a0 msgr2=0x7f91401662e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 shutdown_connections 2024-09-11T12:50:12.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:12.622+0000 7f9146400640 1 -- 172.21.15.78:0/1665937294 wait complete. 2024-09-11T12:50:12.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:50:12.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:12.645 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-11T12:50:12.954 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:50:12.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:12.955 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:50:12.970 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:12.970 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:12.979 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:12.979 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:13.028 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:13.028 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:13.080 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 -- 172.21.15.78:0/1740044716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 msgr2=0x7f67c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 --2- 172.21.15.78:0/1740044716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67c8004c90 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67d000c310 tx=0x7f67d001f6c0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 -- 172.21.15.78:0/1740044716 shutdown_connections 2024-09-11T12:50:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 --2- 172.21.15.78:0/1740044716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67c8004c90 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 -- 172.21.15.78:0/1740044716 >> 172.21.15.78:0/1740044716 conn(0x7f67e406ca80 msgr2=0x7f67e40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:13.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 -- 172.21.15.78:0/1740044716 shutdown_connections 2024-09-11T12:50:13.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.218+0000 7f67e8e00640 1 -- 172.21.15.78:0/1740044716 wait complete. 2024-09-11T12:50:13.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 Processor -- start 2024-09-11T12:50:13.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- start start 2024-09-11T12:50:13.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e41f1a40 0x7f67e406eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 0x7f67e41f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40ccca0 con 0x7f67c8004810 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67e40ccb20 con 0x7f67e406f080 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67e4072670 con 0x7f67e41f1a40 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36602/0 (socket says 172.21.15.78:36602) 2024-09-11T12:50:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 -- 172.21.15.78:0/3674112313 learned_addr learned my addr 172.21.15.78:0/3674112313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1400640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e41f1a40 0x7f67e406eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e2800640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 0x7f67e41f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 -- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e41f1a40 msgr2=0x7f67e406eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e41f1a40 0x7f67e406eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 -- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 msgr2=0x7f67e41f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 0x7f67e41f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 -- 172.21.15.78:0/3674112313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e41f3a80 con 0x7f67c8004810 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e2800640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 0x7f67e41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e1e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67d40117b0 tx=0x7f67d4011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d401f070 con 0x7f67c8004810 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d400fe40 con 0x7f67c8004810 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d40126b0 con 0x7f67c8004810 2024-09-11T12:50:13.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e41f3d60 con 0x7f67c8004810 2024-09-11T12:50:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.222+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41f4320 con 0x7f67c8004810 2024-09-11T12:50:13.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e40709f0 con 0x7f67c8004810 2024-09-11T12:50:13.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f67d4022070 con 0x7f67c8004810 2024-09-11T12:50:13.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67da000640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 0x7f67c4073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f67d4092fe0 con 0x7f67c8004810 2024-09-11T12:50:13.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67e1400640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 0x7f67c4073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.226+0000 7f67e1400640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 0x7f67c4073b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67d00b0000 tx=0x7f67d0006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:13.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.230+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f67d405e840 con 0x7f67c8004810 2024-09-11T12:50:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.410+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f67e406f460 con 0x7f67c8004810 2024-09-11T12:50:13.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.410+0000 7f67da000640 1 -- 172.21.15.78:0/3674112313 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f67d4062d90 con 0x7f67c8004810 2024-09-11T12:50:13.416 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 msgr2=0x7f67c4073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 0x7f67c4073b70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f67d00b0000 tx=0x7f67d0006040 comp rx=0 tx=0).stop 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 msgr2=0x7f67e41557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f67d40117b0 tx=0x7f67d4011c70 comp rx=0 tx=0).stop 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 shutdown_connections 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f67c40716b0 0x7f67c4073b70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e406f080 0x7f67e41f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e41f1a40 0x7f67e406eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 --2- 172.21.15.78:0/3674112313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67c8004810 0x7f67e41557c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 >> 172.21.15.78:0/3674112313 conn(0x7f67e406ca80 msgr2=0x7f67e40cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 shutdown_connections 2024-09-11T12:50:13.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.414+0000 7f67e8e00640 1 -- 172.21.15.78:0/3674112313 wait complete. 2024-09-11T12:50:13.437 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:50:13.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:13.438 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-11T12:50:13.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:50:13.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:13.476 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:50:13.533 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:13.533 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:13.542 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:13.542 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:13.593 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:13.593 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:13.645 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:13.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- 172.21.15.78:0/2948511140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc15bc60 msgr2=0x7f69bc06ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 --2- 172.21.15.78:0/2948511140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc15bc60 0x7f69bc06ecf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f69b4009900 tx=0x7f69b401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- 172.21.15.78:0/2948511140 shutdown_connections 2024-09-11T12:50:13.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 --2- 172.21.15.78:0/2948511140 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc15bc60 0x7f69bc06ecf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 --2- 172.21.15.78:0/2948511140 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc0b4780 0x7f69bc073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- 172.21.15.78:0/2948511140 >> 172.21.15.78:0/2948511140 conn(0x7f69bc06ca80 msgr2=0x7f69bc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:13.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- 172.21.15.78:0/2948511140 shutdown_connections 2024-09-11T12:50:13.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- 172.21.15.78:0/2948511140 wait complete. 2024-09-11T12:50:13.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 Processor -- start 2024-09-11T12:50:13.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.826+0000 7f69c1a00640 1 -- start start 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69bc0ccb20 0x7f69bc0c9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc15bc60 0x7f69bc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc168c40 con 0x7f69bc0b4780 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69bc168ac0 con 0x7f69bc0ccb20 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f69bc168dc0 con 0x7f69bc15bc60 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36620/0 (socket says 172.21.15.78:36620) 2024-09-11T12:50:13.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 -- 172.21.15.78:0/1864189453 learned_addr learned my addr 172.21.15.78:0/1864189453 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69bb400640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc15bc60 0x7f69bc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69ba000640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69bc0ccb20 0x7f69bc0c9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 -- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc15bc60 msgr2=0x7f69bc14e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc15bc60 0x7f69bc14e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 -- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69bc0ccb20 msgr2=0x7f69bc0c9f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69bc0ccb20 0x7f69bc0c9f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 -- 172.21.15.78:0/1864189453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69bc0ca4d0 con 0x7f69bc0b4780 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69baa00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00ebd0 tx=0x7f69ac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac004d30 con 0x7f69bc0b4780 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69ac005020 con 0x7f69bc0b4780 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac005470 con 0x7f69bc0b4780 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69bc0ca770 con 0x7f69bc0b4780 2024-09-11T12:50:13.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69bc0cad30 con 0x7f69bc0b4780 2024-09-11T12:50:13.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.830+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69bc14bef0 con 0x7f69bc0b4780 2024-09-11T12:50:13.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.834+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f69ac005610 con 0x7f69bc0b4780 2024-09-11T12:50:13.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.834+0000 7f69b2a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 0x7f699c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:13.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.834+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f69ac092b80 con 0x7f69bc0b4780 2024-09-11T12:50:13.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.834+0000 7f69ba000640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 0x7f699c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:13.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.834+0000 7f69ba000640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 0x7f699c073b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0cbe00 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:13.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:13.838+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f69ac05e950 con 0x7f69bc0b4780 2024-09-11T12:50:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f69bc14b930 con 0x7f69bc0b4780 2024-09-11T12:50:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69b2a00640 1 -- 172.21.15.78:0/1864189453 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f69ac05e250 con 0x7f69bc0b4780 2024-09-11T12:50:14.018 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:14.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 msgr2=0x7f699c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 0x7f699c073b40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69bc0cbe00 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 msgr2=0x7f69bc1f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00ebd0 tx=0x7f69ac00c6a0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 shutdown_connections 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f699c071680 0x7f699c073b40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69bc15bc60 0x7f69bc14e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f69bc0ccb20 0x7f69bc0c9f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 --2- 172.21.15.78:0/1864189453 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f69bc0b4780 0x7f69bc1f8320 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 >> 172.21.15.78:0/1864189453 conn(0x7f69bc06ca80 msgr2=0x7f69bc0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 shutdown_connections 2024-09-11T12:50:14.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.014+0000 7f69c1a00640 1 -- 172.21.15.78:0/1864189453 wait complete. 2024-09-11T12:50:14.040 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-11T12:50:14.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:14.040 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-11T12:50:14.063 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:50:14.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:14.063 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-11T12:50:14.117 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:14.117 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:14.127 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:14.127 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:14.176 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:14.176 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:14.228 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:14.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- 172.21.15.78:0/417249704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 msgr2=0x7fbeec071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 --2- 172.21.15.78:0/417249704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec071d60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009900 tx=0x7fbee401c8b0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- 172.21.15.78:0/417249704 shutdown_connections 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 --2- 172.21.15.78:0/417249704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec071d60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- 172.21.15.78:0/417249704 >> 172.21.15.78:0/417249704 conn(0x7fbeec06ca80 msgr2=0x7fbeec0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- 172.21.15.78:0/417249704 shutdown_connections 2024-09-11T12:50:14.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- 172.21.15.78:0/417249704 wait complete. 2024-09-11T12:50:14.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 Processor -- start 2024-09-11T12:50:14.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- start start 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 0x7fbeec153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 0x7fbeec150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec168ae0 con 0x7fbeec070360 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec168960 con 0x7fbeec15ba80 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.366+0000 7fbef1600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbeec168c60 con 0x7fbeec074240 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54798/0 (socket says 172.21.15.78:54798) 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 -- 172.21.15.78:0/1010711085 learned_addr learned my addr 172.21.15.78:0/1010711085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeb400640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 0x7fbeec150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeea000640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 0x7fbeec153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 -- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 msgr2=0x7fbeec153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 0x7fbeec153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 -- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 msgr2=0x7fbeec150000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 0x7fbeec150000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 -- 172.21.15.78:0/1010711085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1fb110 con 0x7fbeec070360 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeb400640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 0x7fbeec150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:14.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeea000640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 0x7fbeec153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbeeaa00640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c870 tx=0x7fbedc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc003ba0 con 0x7fbeec070360 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbedc003d40 con 0x7fbeec070360 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1fb3f0 con 0x7fbeec070360 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc00aab0 con 0x7fbeec070360 2024-09-11T12:50:14.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec153490 con 0x7fbeec070360 2024-09-11T12:50:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeec071d60 con 0x7fbeec070360 2024-09-11T12:50:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.370+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fbedc00ac50 con 0x7fbeec070360 2024-09-11T12:50:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.374+0000 7fbee2a00640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 0x7fbecc073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.374+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbedc091cf0 con 0x7fbeec070360 2024-09-11T12:50:14.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.374+0000 7fbeea000640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 0x7fbecc073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.374+0000 7fbeea000640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 0x7fbecc073b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbeec15afc0 tx=0x7fbed8008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.378+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbedc05db50 con 0x7fbeec070360 2024-09-11T12:50:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.554+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbeec075b00 con 0x7fbeec070360 2024-09-11T12:50:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.554+0000 7fbee2a00640 1 -- 172.21.15.78:0/1010711085 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fbedc05d450 con 0x7fbeec070360 2024-09-11T12:50:14.560 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 msgr2=0x7fbecc073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 0x7fbecc073b70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fbeec15afc0 tx=0x7fbed8008040 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 msgr2=0x7fbeec1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00c870 tx=0x7fbedc00cd30 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 shutdown_connections 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbecc0716b0 0x7fbecc073b70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeec15ba80 0x7fbeec150000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeec074240 0x7fbeec153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 --2- 172.21.15.78:0/1010711085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeec070360 0x7fbeec1550e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 >> 172.21.15.78:0/1010711085 conn(0x7fbeec06ca80 msgr2=0x7fbeec159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 shutdown_connections 2024-09-11T12:50:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.558+0000 7fbef1600640 1 -- 172.21.15.78:0/1010711085 wait complete. 2024-09-11T12:50:14.582 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-11T12:50:14.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:14.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-11T12:50:14.608 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:50:14.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:14.609 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-11T12:50:14.661 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:50:14.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 -- 172.21.15.78:0/403522227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0c004810 msgr2=0x7fad0c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 --2- 172.21.15.78:0/403522227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0c004810 0x7fad0c004c90 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad140016c0 tx=0x7fad1401c710 comp rx=0 tx=0).stop 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 -- 172.21.15.78:0/403522227 shutdown_connections 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 --2- 172.21.15.78:0/403522227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0c004810 0x7fad0c004c90 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 -- 172.21.15.78:0/403522227 >> 172.21.15.78:0/403522227 conn(0x7fad2806fca0 msgr2=0x7fad280700b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 -- 172.21.15.78:0/403522227 shutdown_connections 2024-09-11T12:50:14.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.798+0000 7fad2e800640 1 -- 172.21.15.78:0/403522227 wait complete. 2024-09-11T12:50:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 Processor -- start 2024-09-11T12:50:14.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- start start 2024-09-11T12:50:14.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0c004810 0x7fad281f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad28159f40 0x7fad281f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad28070b50 con 0x7fad2815a3c0 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad280709d0 con 0x7fad28159f40 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad28070cd0 con 0x7fad0c004810 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0c004810 0x7fad281f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54810/0 (socket says 172.21.15.78:54810) 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 -- 172.21.15.78:0/1169829778 learned_addr learned my addr 172.21.15.78:0/1169829778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad26a00640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad28159f40 0x7fad281f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 -- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0c004810 msgr2=0x7fad281f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0c004810 0x7fad281f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 -- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad28159f40 msgr2=0x7fad281f71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad28159f40 0x7fad281f71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 -- 172.21.15.78:0/1169829778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad281564d0 con 0x7fad2815a3c0 2024-09-11T12:50:14.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad27e00640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad2815e450 tx=0x7fad20004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20004b30 con 0x7fad2815a3c0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad20004cd0 con 0x7fad2815a3c0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20010cb0 con 0x7fad2815a3c0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad28156c90 con 0x7fad2815a3c0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.802+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad28156f30 con 0x7fad2815a3c0 2024-09-11T12:50:14.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad280b58b0 con 0x7fad2815a3c0 2024-09-11T12:50:14.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fad2001c020 con 0x7fad2815a3c0 2024-09-11T12:50:14.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad1f400640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 0x7fad08073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:14.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fad20092210 con 0x7fad2815a3c0 2024-09-11T12:50:14.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad27400640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 0x7fad08073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:14.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.806+0000 7fad27400640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 0x7fad08073b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad18006c70 tx=0x7fad180067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:14.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:14.810+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fad2005da20 con 0x7fad2815a3c0 2024-09-11T12:50:15.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.010+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad280cecd0 con 0x7fad2815a3c0 2024-09-11T12:50:15.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.010+0000 7fad1f400640 1 -- 172.21.15.78:0/1169829778 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7fad200662b0 con 0x7fad2815a3c0 2024-09-11T12:50:15.015 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:15.015 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:50:15.016 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 msgr2=0x7fad08073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 0x7fad08073b70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fad18006c70 tx=0x7fad180067b0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 msgr2=0x7fad281f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fad2815e450 tx=0x7fad20004700 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 shutdown_connections 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fad080716b0 0x7fad08073b70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad2815a3c0 0x7fad281f76e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad28159f40 0x7fad281f71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 --2- 172.21.15.78:0/1169829778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad0c004810 0x7fad281f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 >> 172.21.15.78:0/1169829778 conn(0x7fad2806fca0 msgr2=0x7fad28072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 shutdown_connections 2024-09-11T12:50:15.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.014+0000 7fad2e800640 1 -- 172.21.15.78:0/1169829778 wait complete. 2024-09-11T12:50:15.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 -- 172.21.15.78:0/3058203221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 msgr2=0x7f64d00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 --2- 172.21.15.78:0/3058203221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64d00051d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009ec0 tx=0x7f64e401d5d0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64ea800640 1 -- 172.21.15.78:0/3058203221 reap_dead start 2024-09-11T12:50:15.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 -- 172.21.15.78:0/3058203221 shutdown_connections 2024-09-11T12:50:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 -- 172.21.15.78:0/3058203221 >> 172.21.15.78:0/3058203221 conn(0x7f64ec0b19a0 msgr2=0x7f64ec070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 -- 172.21.15.78:0/3058203221 shutdown_connections 2024-09-11T12:50:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.174+0000 7f64eb200640 1 -- 172.21.15.78:0/3058203221 wait complete. 2024-09-11T12:50:15.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 Processor -- start 2024-09-11T12:50:15.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- start start 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 0x7f64ec145f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 0x7f64ec1e8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec15e0d0 con 0x7f64d0004d50 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec15d090 con 0x7f64ec1e7e50 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec15e250 con 0x7f64ec06d930 2024-09-11T12:50:15.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54832/0 (socket says 172.21.15.78:54832) 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 -- 172.21.15.78:0/1085894555 learned_addr learned my addr 172.21.15.78:0/1085894555 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 -- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 msgr2=0x7f64ec145f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9400640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 0x7f64ec145f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64ea800640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 0x7f64ec1e8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 0x7f64ec145f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 -- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 msgr2=0x7f64ec1e8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 0x7f64ec1e8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 -- 172.21.15.78:0/1085894555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec146440 con 0x7f64d0004d50 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64ea800640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 0x7f64ec1e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9400640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 0x7f64ec145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:15.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e9e00640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e2e0 tx=0x7f64dc00e7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc012850 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64dc012ae0 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc01a400 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec06dea0 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.178+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1f8090 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec0a97f0 con 0x7f64d0004d50 2024-09-11T12:50:15.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f64dc00d030 con 0x7f64d0004d50 2024-09-11T12:50:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64e2000640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 0x7f64cc073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f64dc094f80 con 0x7f64d0004d50 2024-09-11T12:50:15.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64e9400640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 0x7f64cc073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.182+0000 7f64e9400640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 0x7f64cc073b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64ec1f7d10 tx=0x7f64d80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.186+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f64dc016030 con 0x7f64d0004d50 2024-09-11T12:50:15.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.386+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64ec074e00 con 0x7f64d0004d50 2024-09-11T12:50:15.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.386+0000 7f64e2000640 1 -- 172.21.15.78:0/1085894555 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f64dc060de0 con 0x7f64d0004d50 2024-09-11T12:50:15.392 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:15.393 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:50:15.393 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:50:15.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 msgr2=0x7f64cc073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 0x7f64cc073b70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f64ec1f7d10 tx=0x7f64d80091e0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 msgr2=0x7f64ec06d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00e2e0 tx=0x7f64dc00e7a0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 shutdown_connections 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f64cc0716b0 0x7f64cc073b70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64ec1e7e50 0x7f64ec1e8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64ec06d930 0x7f64ec145f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 --2- 172.21.15.78:0/1085894555 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64d0004d50 0x7f64ec06d060 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 >> 172.21.15.78:0/1085894555 conn(0x7f64ec0b19a0 msgr2=0x7f64ec06f040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 shutdown_connections 2024-09-11T12:50:15.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.390+0000 7f64eb200640 1 -- 172.21.15.78:0/1085894555 wait complete. 2024-09-11T12:50:15.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- 172.21.15.78:0/3482239074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 msgr2=0x7fd57406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 --2- 172.21.15.78:0/3482239074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57406fc50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd574152090 tx=0x7fd56001c890 comp rx=0 tx=0).stop 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- 172.21.15.78:0/3482239074 shutdown_connections 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 --2- 172.21.15.78:0/3482239074 >> v2:172.21.15.157:3300/0 conn(0x7fd574070190 0x7fd574071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 --2- 172.21.15.78:0/3482239074 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57406fc50 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- 172.21.15.78:0/3482239074 >> 172.21.15.78:0/3482239074 conn(0x7fd57406ca80 msgr2=0x7fd5740b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- 172.21.15.78:0/3482239074 shutdown_connections 2024-09-11T12:50:15.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- 172.21.15.78:0/3482239074 wait complete. 2024-09-11T12:50:15.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 Processor -- start 2024-09-11T12:50:15.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.554+0000 7fd579200640 1 -- start start 2024-09-11T12:50:15.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd574070190 0x7fd5741484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd574074240 0x7fd5741520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5741678c0 con 0x7fd57415ba80 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd574167740 con 0x7fd574074240 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd574167a40 con 0x7fd574070190 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54862/0 (socket says 172.21.15.78:54862) 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 -- 172.21.15.78:0/2963997742 learned_addr learned my addr 172.21.15.78:0/2963997742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd571400640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd574074240 0x7fd5741520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd571e00640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd574070190 0x7fd5741484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 -- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd574070190 msgr2=0x7fd5741484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd574070190 0x7fd5741484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 -- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd574074240 msgr2=0x7fd5741520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd574074240 0x7fd5741520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 -- 172.21.15.78:0/2963997742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5741fae70 con 0x7fd57415ba80 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd572800640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00cce0 tx=0x7fd56c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00f450 con 0x7fd57415ba80 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd56c00f700 con 0x7fd57415ba80 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd56c00fb50 con 0x7fd57415ba80 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5741fb150 con 0x7fd57415ba80 2024-09-11T12:50:15.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5741fb710 con 0x7fd57415ba80 2024-09-11T12:50:15.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5741fb950 con 0x7fd57415ba80 2024-09-11T12:50:15.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.558+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fd56c00fcf0 con 0x7fd57415ba80 2024-09-11T12:50:15.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.562+0000 7fd56a000640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 0x7fd554073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.562+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd56c092870 con 0x7fd57415ba80 2024-09-11T12:50:15.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.562+0000 7fd571e00640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 0x7fd554073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.562+0000 7fd571e00640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 0x7fd554073b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5640016f0 tx=0x7fd564009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.566+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd5741fb950 con 0x7fd57415ba80 2024-09-11T12:50:15.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.730+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5741489e0 con 0x7fd57415ba80 2024-09-11T12:50:15.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.730+0000 7fd56a000640 1 -- 172.21.15.78:0/2963997742 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7fd56c014030 con 0x7fd57415ba80 2024-09-11T12:50:15.736 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:15.736 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:15.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 msgr2=0x7fd554073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 0x7fd554073b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5640016f0 tx=0x7fd564009040 comp rx=0 tx=0).stop 2024-09-11T12:50:15.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 msgr2=0x7fd57414bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd56c00cce0 tx=0x7fd56c00a620 comp rx=0 tx=0).stop 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 shutdown_connections 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd554071680 0x7fd554073b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd57415ba80 0x7fd57414bde0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd574074240 0x7fd5741520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 --2- 172.21.15.78:0/2963997742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd574070190 0x7fd5741484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 >> 172.21.15.78:0/2963997742 conn(0x7fd57406ca80 msgr2=0x7fd5740b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 shutdown_connections 2024-09-11T12:50:15.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.734+0000 7fd579200640 1 -- 172.21.15.78:0/2963997742 wait complete. 2024-09-11T12:50:15.759 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:50:15.759 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.894+0000 7f0681400640 1 -- 172.21.15.78:0/2119004647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c0741e0 msgr2=0x7f067c070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.894+0000 7f0681400640 1 --2- 172.21.15.78:0/2119004647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c0741e0 0x7f067c070910 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f066c009840 tx=0x7f066c01c820 comp rx=0 tx=0).stop 2024-09-11T12:50:15.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- 172.21.15.78:0/2119004647 shutdown_connections 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 --2- 172.21.15.78:0/2119004647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c0741e0 0x7f067c070910 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- 172.21.15.78:0/2119004647 >> 172.21.15.78:0/2119004647 conn(0x7f067c06ca80 msgr2=0x7f067c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- 172.21.15.78:0/2119004647 shutdown_connections 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- 172.21.15.78:0/2119004647 wait complete. 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 Processor -- start 2024-09-11T12:50:15.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- start start 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f067c070360 0x7f067c150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 0x7f067c0b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f067c169250 con 0x7f067c072230 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f067c1690d0 con 0x7f067c0741e0 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0681400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f067c1693d0 con 0x7f067c070360 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54888/0 (socket says 172.21.15.78:54888) 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 -- 172.21.15.78:0/967942056 learned_addr learned my addr 172.21.15.78:0/967942056 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f067a800640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f067c070360 0x7f067c150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f067b200640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 0x7f067c0b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 -- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f067c070360 msgr2=0x7f067c150ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f067c070360 0x7f067c150ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 -- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 msgr2=0x7f067c0b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 0x7f067c0b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 -- 172.21.15.78:0/967942056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f067c0adef0 con 0x7f067c072230 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f067b200640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 0x7f067c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0679e00640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f066800c490 tx=0x7f066800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668016020 con 0x7f067c072230 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0668012940 con 0x7f067c072230 2024-09-11T12:50:15.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.898+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0668012d90 con 0x7f067c072230 2024-09-11T12:50:15.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f067c0adc10 con 0x7f067c072230 2024-09-11T12:50:15.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f067c1539b0 con 0x7f067c072230 2024-09-11T12:50:15.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f067c070910 con 0x7f067c072230 2024-09-11T12:50:15.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f0668019020 con 0x7f067c072230 2024-09-11T12:50:15.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0672a00640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 0x7f065c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:15.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0668091a70 con 0x7f067c072230 2024-09-11T12:50:15.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.902+0000 7f067a800640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 0x7f065c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:15.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.906+0000 7f067a800640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 0x7f065c073b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:15.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:15.906+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0668020030 con 0x7f067c072230 2024-09-11T12:50:16.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f067c0695e0 con 0x7f067c072230 2024-09-11T12:50:16.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0672a00640 1 -- 172.21.15.78:0/967942056 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f066805d990 con 0x7f067c072230 2024-09-11T12:50:16.110 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:16.110 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:50:16.110 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:50:16.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 msgr2=0x7f065c073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 0x7f065c073b70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f066c0af000 tx=0x7f066c01b060 comp rx=0 tx=0).stop 2024-09-11T12:50:16.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 msgr2=0x7f067c0b1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f066800c490 tx=0x7f066800c950 comp rx=0 tx=0).stop 2024-09-11T12:50:16.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 shutdown_connections 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f065c0716b0 0x7f065c073b70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f067c0741e0 0x7f067c0b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f067c072230 0x7f067c0b1020 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 --2- 172.21.15.78:0/967942056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f067c070360 0x7f067c150ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 >> 172.21.15.78:0/967942056 conn(0x7f067c06ca80 msgr2=0x7f067c15d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 shutdown_connections 2024-09-11T12:50:16.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.106+0000 7f0681400640 1 -- 172.21.15.78:0/967942056 wait complete. 2024-09-11T12:50:16.131 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 -- 172.21.15.78:0/4214947291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98e0004810 msgr2=0x7f98e0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 --2- 172.21.15.78:0/4214947291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98e0004810 0x7f98e0004c90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f98e8001860 tx=0x7f98e801f010 comp rx=0 tx=0).stop 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f98fa800640 1 -- 172.21.15.78:0/4214947291 reap_dead start 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 -- 172.21.15.78:0/4214947291 shutdown_connections 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 -- 172.21.15.78:0/4214947291 >> 172.21.15.78:0/4214947291 conn(0x7f98fc074b90 msgr2=0x7f98fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 -- 172.21.15.78:0/4214947291 shutdown_connections 2024-09-11T12:50:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.266+0000 7f9901000640 1 -- 172.21.15.78:0/4214947291 wait complete. 2024-09-11T12:50:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 Processor -- start 2024-09-11T12:50:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 -- start start 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98e0004810 0x7f98fc1556f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc14ec10 0x7f98fc14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0d1510 con 0x7f98fc1f3840 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0d1390 con 0x7f98fc14ec10 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc15d2c0 con 0x7f98e0004810 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54902/0 (socket says 172.21.15.78:54902) 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 -- 172.21.15.78:0/1206671543 learned_addr learned my addr 172.21.15.78:0/1206671543 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:16.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98f9e00640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98e0004810 0x7f98fc1556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98f9400640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc14ec10 0x7f98fc14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 -- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98e0004810 msgr2=0x7f98fc1556f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98e0004810 0x7f98fc1556f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 -- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc14ec10 msgr2=0x7f98fc14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc14ec10 0x7f98fc14f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 -- 172.21.15.78:0/1206671543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc1f8700 con 0x7f98fc1f3840 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98fa800640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0ce790 tx=0x7f98f4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f400f890 con 0x7f98fc1f3840 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98f400fa30 con 0x7f98fc1f3840 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f400fe40 con 0x7f98fc1f3840 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.270+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc1f4640 con 0x7f98fc1f3840 2024-09-11T12:50:16.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1f8380 con 0x7f98fc1f3840 2024-09-11T12:50:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98fc15d7c0 con 0x7f98fc1f3840 2024-09-11T12:50:16.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f98f4003600 con 0x7f98fc1f3840 2024-09-11T12:50:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f98f2000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 0x7f98dc073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f98f4092890 con 0x7f98fc1f3840 2024-09-11T12:50:16.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.274+0000 7f98f9e00640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 0x7f98dc073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.278+0000 7f98f9e00640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 0x7f98dc073b70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0015d0 tx=0x7f98ec0014e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:16.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.278+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f98f405e6f0 con 0x7f98fc1f3840 2024-09-11T12:50:16.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98fc069070 con 0x7f98fc1f3840 2024-09-11T12:50:16.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f98f2000640 1 -- 172.21.15.78:0/1206671543 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5223 (secure 0 0 0) 0x7f98f405dff0 con 0x7f98fc1f3840 2024-09-11T12:50:16.486 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:16.486 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":8,"btime":"2024-09-11T12:49:46:446493+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4314,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/1767689413","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1767689413},{"type":"v1","addr":"172.21.15.78:6835","nonce":1767689413}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4316,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/4019656545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4019656545},{"type":"v1","addr":"172.21.15.157:6836","nonce":4019656545}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4319,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/3460190724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3460190724},{"type":"v1","addr":"172.21.15.157:6837","nonce":3460190724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:49:42.819611+0000","modified":"2024-09-11T12:49:45.831763+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4390},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4390":{"gid":4390,"name":"a","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/1888452692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1888452692},{"type":"v1","addr":"172.21.15.78:6834","nonce":1888452692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4390,"qdb_cluster":[4390]},"id":1}]} 2024-09-11T12:50:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 8 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 msgr2=0x7f98dc073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 0x7f98dc073b70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f98ec0015d0 tx=0x7f98ec0014e0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 msgr2=0x7f98fc1f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0ce790 tx=0x7f98f4011c40 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 shutdown_connections 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f98dc0716b0 0x7f98dc073b70 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98fc1f3840 0x7f98fc1f3cd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98fc14ec10 0x7f98fc14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 --2- 172.21.15.78:0/1206671543 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f98e0004810 0x7f98fc1556f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 >> 172.21.15.78:0/1206671543 conn(0x7f98fc074b90 msgr2=0x7f98fc0b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 shutdown_connections 2024-09-11T12:50:16.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.482+0000 7f9901000640 1 -- 172.21.15.78:0/1206671543 wait complete. 2024-09-11T12:50:16.507 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:16.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 -- 172.21.15.78:0/1291225148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0a75d0 msgr2=0x7f7cec0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/1291225148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0a75d0 0x7f7cec0b0dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7cd8009900 tx=0x7f7cd801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 -- 172.21.15.78:0/1291225148 shutdown_connections 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/1291225148 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0a75d0 0x7f7cec0b0dc0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/1291225148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 0x7f7cec06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 -- 172.21.15.78:0/1291225148 >> 172.21.15.78:0/1291225148 conn(0x7f7cec0cf830 msgr2=0x7f7cec0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 -- 172.21.15.78:0/1291225148 shutdown_connections 2024-09-11T12:50:16.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.642+0000 7f7cf3a00640 1 -- 172.21.15.78:0/1291225148 wait complete. 2024-09-11T12:50:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 Processor -- start 2024-09-11T12:50:16.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- start start 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cec0a75d0 0x7f7cec149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 0x7f7cec14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec168fb0 con 0x7f7cec0b1300 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7cec168e30 con 0x7f7cec0a75d0 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7cec169130 con 0x7f7cec15f080 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54928/0 (socket says 172.21.15.78:54928) 2024-09-11T12:50:16.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 -- 172.21.15.78:0/2486786657 learned_addr learned my addr 172.21.15.78:0/2486786657 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf1000640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cec0a75d0 0x7f7cec149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf1a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 0x7f7cec14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 -- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 msgr2=0x7f7cec14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 0x7f7cec14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 -- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cec0a75d0 msgr2=0x7f7cec149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cec0a75d0 0x7f7cec149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 -- 172.21.15.78:0/2486786657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cec155190 con 0x7f7cec0b1300 2024-09-11T12:50:16.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cebe00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cd801a450 tx=0x7f7cd80a57d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80b2040 con 0x7f7cec0b1300 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cd8005070 con 0x7f7cec0b1300 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cec155470 con 0x7f7cec0b1300 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cd80044f0 con 0x7f7cec0b1300 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.646+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cec153c90 con 0x7f7cec0b1300 2024-09-11T12:50:16.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cec153ab0 con 0x7f7cec0b1300 2024-09-11T12:50:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7cd80bd060 con 0x7f7cec0b1300 2024-09-11T12:50:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7ce8c00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 0x7f7ccc073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7cd812b660 con 0x7f7cec0b1300 2024-09-11T12:50:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7cf1000640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 0x7f7ccc073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.650+0000 7f7cf1000640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 0x7f7ccc073b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc004630 tx=0x7f7cdc00b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:16.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.654+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7cec153ab0 con 0x7f7cec0b1300 2024-09-11T12:50:16.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.818+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7cec0d15a0 con 0x7f7cec0b1300 2024-09-11T12:50:16.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7ce8c00640 1 -- 172.21.15.78:0/2486786657 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16665 (secure 0 0 0) 0x7f7cd80164d0 con 0x7f7cec0b1300 2024-09-11T12:50:16.825 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:16.825 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":23,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:49:44.434088+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:49:40.738186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-11T12:49:41.775035+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:16.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 msgr2=0x7f7ccc073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 0x7f7ccc073b40 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7cdc004630 tx=0x7f7cdc00b2e0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 msgr2=0x7f7cec0a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f7cd801a450 tx=0x7f7cd80a57d0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 shutdown_connections 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ccc071680 0x7f7ccc073b40 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7cec15f080 0x7f7cec14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7cec0b1300 0x7f7cec0a63d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 --2- 172.21.15.78:0/2486786657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7cec0a75d0 0x7f7cec149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 >> 172.21.15.78:0/2486786657 conn(0x7f7cec0cf830 msgr2=0x7f7cec0b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 shutdown_connections 2024-09-11T12:50:16.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.822+0000 7f7cf3a00640 1 -- 172.21.15.78:0/2486786657 wait complete. 2024-09-11T12:50:16.848 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- 172.21.15.78:0/2016465049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 msgr2=0x7f238c073080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- 172.21.15.78:0/2016465049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c073080 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f237801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- 172.21.15.78:0/2016465049 shutdown_connections 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- 172.21.15.78:0/2016465049 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c073080 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- 172.21.15.78:0/2016465049 >> v2:172.21.15.157:3300/0 conn(0x7f238c15b8e0 0x7f238c160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- 172.21.15.78:0/2016465049 >> 172.21.15.78:0/2016465049 conn(0x7f238c06f3a0 msgr2=0x7f238c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- 172.21.15.78:0/2016465049 shutdown_connections 2024-09-11T12:50:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- 172.21.15.78:0/2016465049 wait complete. 2024-09-11T12:50:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 Processor -- start 2024-09-11T12:50:16.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- start start 2024-09-11T12:50:16.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c0cd430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 0x7f238c164ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f238c168db0 con 0x7f238c15b8e0 2024-09-11T12:50:16.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f238c168c30 con 0x7f238c15c780 2024-09-11T12:50:16.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f2392200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f238c168f30 con 0x7f238c15bcc0 2024-09-11T12:50:16.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f238b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f238b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54944/0 (socket says 172.21.15.78:54944) 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f238b400640 1 -- 172.21.15.78:0/1145591951 learned_addr learned my addr 172.21.15.78:0/1145591951 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.986+0000 7f238aa00640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c0cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238be00640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 0x7f238c164ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 -- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 msgr2=0x7f238c0cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c0cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 -- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 msgr2=0x7f238c164ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 0x7f238c164ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 -- 172.21.15.78:0/1145591951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c0c3030 con 0x7f238c15b8e0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238be00640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 0x7f238c164ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f238b400640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f237c00e9f0 tx=0x7f237c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c00cd80 con 0x7f238c15b8e0 2024-09-11T12:50:16.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f237c010430 con 0x7f238c15b8e0 2024-09-11T12:50:16.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1fb700 con 0x7f238c15b8e0 2024-09-11T12:50:16.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c010860 con 0x7f238c15b8e0 2024-09-11T12:50:16.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1fbfa0 con 0x7f238c15b8e0 2024-09-11T12:50:16.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c1fbaf0 con 0x7f238c15b8e0 2024-09-11T12:50:16.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f237c010a00 con 0x7f238c15b8e0 2024-09-11T12:50:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 0x7f236c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.990+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f237c0928f0 con 0x7f238c15b8e0 2024-09-11T12:50:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.994+0000 7f238aa00640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 0x7f236c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.994+0000 7f238aa00640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 0x7f236c073b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23780af000 tx=0x7f237801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:17.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:16.994+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f237c05e810 con 0x7f238c15b8e0 2024-09-11T12:50:17.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.194+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f238c15fe10 con 0x7f238c15b8e0 2024-09-11T12:50:17.200 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:50:17.194+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:50:17.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.198+0000 7f2383400640 1 -- 172.21.15.78:0/1145591951 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f237c05e110 con 0x7f238c15b8e0 2024-09-11T12:50:17.202 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:50:17.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 msgr2=0x7f236c073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:17.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 0x7f236c073b70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f23780af000 tx=0x7f237801b060 comp rx=0 tx=0).stop 2024-09-11T12:50:17.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 msgr2=0x7f238c0c4a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:17.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f237c00e9f0 tx=0x7f237c00eeb0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 shutdown_connections 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c0716b0 0x7f236c073b70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c15c780 0x7f238c164ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15bcc0 0x7f238c0cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 --2- 172.21.15.78:0/1145591951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15b8e0 0x7f238c0c4a90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 >> 172.21.15.78:0/1145591951 conn(0x7f238c06f3a0 msgr2=0x7f238c156bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 shutdown_connections 2024-09-11T12:50:17.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.202+0000 7f2392200640 1 -- 172.21.15.78:0/1145591951 wait complete. 2024-09-11T12:50:17.218 INFO:tasks.ceph.mds.a.smithi078.stderr: -6970> 2024-09-11T12:49:47.394+0000 7fa867c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:50:17.226 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:50:17.268 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:50:17.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 -- 172.21.15.78:0/4140181838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd640748d0 msgr2=0x7fdd64070690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/4140181838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd640748d0 0x7fdd64070690 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800acd0 tx=0x7fdd5801def0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 -- 172.21.15.78:0/4140181838 shutdown_connections 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/4140181838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd640748d0 0x7fdd64070690 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 -- 172.21.15.78:0/4140181838 >> 172.21.15.78:0/4140181838 conn(0x7fdd640720f0 msgr2=0x7fdd64072500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 -- 172.21.15.78:0/4140181838 shutdown_connections 2024-09-11T12:50:17.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.370+0000 7fdd6aa00640 1 -- 172.21.15.78:0/4140181838 wait complete. 2024-09-11T12:50:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 Processor -- start 2024-09-11T12:50:17.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- start start 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 0x7fdd6414b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 0x7fdd64146b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64167c60 con 0x7fdd6415add0 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64167ae0 con 0x7fdd640748d0 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdd64167de0 con 0x7fdd640cf1f0 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54988/0 (socket says 172.21.15.78:54988) 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 -- 172.21.15.78:0/674170344 learned_addr learned my addr 172.21.15.78:0/674170344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 -- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 msgr2=0x7fdd64146b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:17.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd68c00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 0x7fdd64146b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 0x7fdd64146b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd69600640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 0x7fdd6414b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 -- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 msgr2=0x7fdd6414b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 0x7fdd6414b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 -- 172.21.15.78:0/674170344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd64147ad0 con 0x7fdd6415add0 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd68c00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 0x7fdd64146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd69600640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 0x7fdd6414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:17.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6a000640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00c3c0 tx=0x7fdd5c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011790 con 0x7fdd6415add0 2024-09-11T12:50:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd5c017070 con 0x7fdd6415add0 2024-09-11T12:50:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641f73a0 con 0x7fdd6415add0 2024-09-11T12:50:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c011cc0 con 0x7fdd6415add0 2024-09-11T12:50:17.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6414c360 con 0x7fdd6415add0 2024-09-11T12:50:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.374+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64073230 con 0x7fdd6415add0 2024-09-11T12:50:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.378+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fdd5c01c460 con 0x7fdd6415add0 2024-09-11T12:50:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.378+0000 7fdd61600640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 0x7fdd44073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.378+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7fdd5c091c50 con 0x7fdd6415add0 2024-09-11T12:50:17.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.378+0000 7fdd69600640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 0x7fdd44073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:17.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.378+0000 7fdd69600640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 0x7fdd44073b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd580b0000 tx=0x7fdd5801c730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:17.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.382+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdd64147ad0 con 0x7fdd6415add0 2024-09-11T12:50:17.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:17.582+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdd64070690 con 0x7fdd6415add0 2024-09-11T12:50:18.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.254+0000 7fdd61600640 1 -- 172.21.15.78:0/674170344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7fdd5c05da50 con 0x7fdd6415add0 2024-09-11T12:50:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 msgr2=0x7fdd44073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 0x7fdd44073b70 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdd580b0000 tx=0x7fdd5801c730 comp rx=0 tx=0).stop 2024-09-11T12:50:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 msgr2=0x7fdd64147fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:18.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00c3c0 tx=0x7fdd5c00c880 comp rx=0 tx=0).stop 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 shutdown_connections 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd440716b0 0x7fdd44073b70 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd6415add0 0x7fdd64147fc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd640cf1f0 0x7fdd64146b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 --2- 172.21.15.78:0/674170344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd640748d0 0x7fdd6414b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 >> 172.21.15.78:0/674170344 conn(0x7fdd640720f0 msgr2=0x7fdd64159aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 shutdown_connections 2024-09-11T12:50:18.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.258+0000 7fdd6aa00640 1 -- 172.21.15.78:0/674170344 wait complete. 2024-09-11T12:50:18.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:50:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 -- 172.21.15.78:0/3466713821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee28152990 msgr2=0x7fee2815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 --2- 172.21.15.78:0/3466713821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee28152990 0x7fee2815d060 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009900 tx=0x7fee1c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 -- 172.21.15.78:0/3466713821 shutdown_connections 2024-09-11T12:50:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 --2- 172.21.15.78:0/3466713821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280b5c40 0x7fee280cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 --2- 172.21.15.78:0/3466713821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee28152990 0x7fee2815d060 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 -- 172.21.15.78:0/3466713821 >> 172.21.15.78:0/3466713821 conn(0x7fee280b3050 msgr2=0x7fee280b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 -- 172.21.15.78:0/3466713821 shutdown_connections 2024-09-11T12:50:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.470+0000 7fee30600640 1 -- 172.21.15.78:0/3466713821 wait complete. 2024-09-11T12:50:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 Processor -- start 2024-09-11T12:50:18.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- start start 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 0x7fee28140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 0x7fee28153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee28168a80 con 0x7fee28152990 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee28168900 con 0x7fee280b5c40 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee28168c00 con 0x7fee280cf520 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55004/0 (socket says 172.21.15.78:55004) 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 -- 172.21.15.78:0/3403749968 learned_addr learned my addr 172.21.15.78:0/3403749968 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2dc00640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 0x7fee28140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:18.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2d200640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 0x7fee28153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 -- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 msgr2=0x7fee28153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 0x7fee28153000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 -- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 msgr2=0x7fee28140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 0x7fee28140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 -- 172.21.15.78:0/3403749968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee28140db0 con 0x7fee28152990 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2dc00640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 0x7fee28140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2d200640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 0x7fee28153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee2e600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee2000eb20 tx=0x7fee2000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20004b30 con 0x7fee28152990 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee20004cf0 con 0x7fee28152990 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2814dad0 con 0x7fee28152990 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20005280 con 0x7fee28152990 2024-09-11T12:50:18.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.474+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee28155730 con 0x7fee28152990 2024-09-11T12:50:18.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.478+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee280cf990 con 0x7fee28152990 2024-09-11T12:50:18.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.478+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fee20005420 con 0x7fee28152990 2024-09-11T12:50:18.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.478+0000 7fee25600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 0x7fee08073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:18.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.478+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5795+0+0 (secure 0 0 0) 0x7fee20092a70 con 0x7fee28152990 2024-09-11T12:50:18.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.478+0000 7fee2dc00640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 0x7fee08073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:18.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.482+0000 7fee2dc00640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 0x7fee08073b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0af000 tx=0x7fee1c01b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:18.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.482+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fee2005e870 con 0x7fee28152990 2024-09-11T12:50:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:18.650+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fee280b5220 con 0x7fee28152990 2024-09-11T12:50:19.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.262+0000 7fee25600640 1 -- 172.21.15.78:0/3403749968 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) ==== 181+0+0 (secure 0 0 0) 0x7fee2005e170 con 0x7fee28152990 2024-09-11T12:50:19.266 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:50:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.262+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 msgr2=0x7fee08073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.262+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 0x7fee08073b40 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0af000 tx=0x7fee1c01b120 comp rx=0 tx=0).stop 2024-09-11T12:50:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 msgr2=0x7fee2814f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:19.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fee2000eb20 tx=0x7fee2000c2d0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 shutdown_connections 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee08071680 0x7fee08073b40 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee28152990 0x7fee2814f2a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee280cf520 0x7fee28153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 --2- 172.21.15.78:0/3403749968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee280b5c40 0x7fee28140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 >> 172.21.15.78:0/3403749968 conn(0x7fee280b3050 msgr2=0x7fee280b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 shutdown_connections 2024-09-11T12:50:19.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.266+0000 7fee30600640 1 -- 172.21.15.78:0/3403749968 wait complete. 2024-09-11T12:50:19.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:50:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- 172.21.15.78:0/26707091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015c860 msgr2=0x7f20c013edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- 172.21.15.78:0/26707091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015c860 0x7f20c013edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009900 tx=0x7f20a801c880 comp rx=0 tx=0).stop 2024-09-11T12:50:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- 172.21.15.78:0/26707091 shutdown_connections 2024-09-11T12:50:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- 172.21.15.78:0/26707091 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015f350 0x7f20c015f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- 172.21.15.78:0/26707091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015c860 0x7f20c013edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- 172.21.15.78:0/26707091 >> 172.21.15.78:0/26707091 conn(0x7f20c0074610 msgr2=0x7f20c00d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- 172.21.15.78:0/26707091 shutdown_connections 2024-09-11T12:50:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- 172.21.15.78:0/26707091 wait complete. 2024-09-11T12:50:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 Processor -- start 2024-09-11T12:50:19.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- start start 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015c480 0x7f20c0159100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c015c860 0x7f20c015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20c013e0a0 con 0x7f20c015f350 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20c013df20 con 0x7f20c015c860 2024-09-11T12:50:19.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.438+0000 7f20c4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20c013e220 con 0x7f20c015c480 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55026/0 (socket says 172.21.15.78:55026) 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 -- 172.21.15.78:0/2742503873 learned_addr learned my addr 172.21.15.78:0/2742503873 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c015c860 0x7f20c015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bf400640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015c480 0x7f20c0159100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 -- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015c480 msgr2=0x7f20c0159100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015c480 0x7f20c0159100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 -- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c015c860 msgr2=0x7f20c015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c015c860 0x7f20c015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 -- 172.21.15.78:0/2742503873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c015d930 con 0x7f20c015f350 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20bfe00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e970 tx=0x7f20b800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8004d30 con 0x7f20c015f350 2024-09-11T12:50:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b800cb20 con 0x7f20c015f350 2024-09-11T12:50:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c015dc10 con 0x7f20c015f350 2024-09-11T12:50:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8005020 con 0x7f20c015f350 2024-09-11T12:50:19.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c015e4e0 con 0x7f20c015f350 2024-09-11T12:50:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c013e7c0 con 0x7f20c015f350 2024-09-11T12:50:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f20b80051c0 con 0x7f20c015f350 2024-09-11T12:50:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.442+0000 7f20b7400640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 0x7f209c073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.446+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5357+0+0 (secure 0 0 0) 0x7f20b8092410 con 0x7f20c015f350 2024-09-11T12:50:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.446+0000 7f20bf400640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 0x7f209c073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.446+0000 7f20bf400640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 0x7f209c073b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:19.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.446+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f20b8016030 con 0x7f20c015f350 2024-09-11T12:50:19.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:19.614+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f20c0167190 con 0x7f20c015f350 2024-09-11T12:50:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.262+0000 7f20b7400640 1 -- 172.21.15.78:0/2742503873 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) ==== 169+0+0 (secure 0 0 0) 0x7f20b8090e20 con 0x7f20c015f350 2024-09-11T12:50:20.268 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:50:20.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 msgr2=0x7f209c073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 0x7f209c073b70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f20b0007fd0 tx=0x7f20b0007e40 comp rx=0 tx=0).stop 2024-09-11T12:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 msgr2=0x7f20c015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:20.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f20b800e970 tx=0x7f20b800ee30 comp rx=0 tx=0).stop 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 shutdown_connections 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f209c0716b0 0x7f209c073b70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c015f350 0x7f20c015cf00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c015c860 0x7f20c015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 --2- 172.21.15.78:0/2742503873 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c015c480 0x7f20c0159100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.266+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 >> 172.21.15.78:0/2742503873 conn(0x7f20c0074610 msgr2=0x7f20c006eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.270+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 shutdown_connections 2024-09-11T12:50:20.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.270+0000 7f20c4e00640 1 -- 172.21.15.78:0/2742503873 wait complete. 2024-09-11T12:50:20.298 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:50:20.298 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:50:20.299 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:50:20.299 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:20.299 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:50:20.299 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:50:20.299 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:50:20.299 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:20.300 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:50:20.300 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:50:20.300 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:50:20.300 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:20.300 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:50:20.300 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:50:20.301 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:50:20.301 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:20.312 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:50:20.312 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:50:20.313 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:50:20.313 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:50:20.314 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:50:20.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:50:20.315 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:50:20.315 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:50:20.355 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:50:20.355 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:20.357 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:50:20.357 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:20.358 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:50:20.358 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:20.360 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:50:20.360 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:20.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:50:20.441 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:50:20.442 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:50:20.447 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:50:20.447 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:50:20.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 -- 172.21.15.78:0/3474185038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880aed20 msgr2=0x7f5c880cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/3474185038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880aed20 0x7f5c880cf280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009840 tx=0x7f5c7801c760 comp rx=0 tx=0).stop 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 -- 172.21.15.78:0/3474185038 shutdown_connections 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/3474185038 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880aed20 0x7f5c880cf280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 -- 172.21.15.78:0/3474185038 >> 172.21.15.78:0/3474185038 conn(0x7f5c8806ca80 msgr2=0x7f5c880b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 -- 172.21.15.78:0/3474185038 shutdown_connections 2024-09-11T12:50:20.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.506+0000 7f5c8fa00640 1 -- 172.21.15.78:0/3474185038 wait complete. 2024-09-11T12:50:20.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 Processor -- start 2024-09-11T12:50:20.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- start start 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c880aed20 0x7f5c880738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c8815d1e0 0x7f5c880a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880cd460 con 0x7f5c880cf7c0 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c880cd2e0 con 0x7f5c880aed20 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c880cd5e0 con 0x7f5c8815d1e0 2024-09-11T12:50:20.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55074/0 (socket says 172.21.15.78:55074) 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 -- 172.21.15.78:0/2348300291 learned_addr learned my addr 172.21.15.78:0/2348300291 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8d000640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c880aed20 0x7f5c880738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8da00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c8815d1e0 0x7f5c880a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 -- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c8815d1e0 msgr2=0x7f5c880a99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c8815d1e0 0x7f5c880a99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 -- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c880aed20 msgr2=0x7f5c880738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c880aed20 0x7f5c880738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:20.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 -- 172.21.15.78:0/2348300291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c88073390 con 0x7f5c880cf7c0 2024-09-11T12:50:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c87e00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c460 tx=0x7f5c7400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74016020 con 0x7f5c880cf7c0 2024-09-11T12:50:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c74012940 con 0x7f5c880cf7c0 2024-09-11T12:50:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74012d90 con 0x7f5c880cf7c0 2024-09-11T12:50:20.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c88149d70 con 0x7f5c880cf7c0 2024-09-11T12:50:20.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.510+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c88072af0 con 0x7f5c880cf7c0 2024-09-11T12:50:20.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c880cdb00 con 0x7f5c880cf7c0 2024-09-11T12:50:20.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f5c74019020 con 0x7f5c880cf7c0 2024-09-11T12:50:20.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c84c00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 0x7f5c68073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:20.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 4943+0+0 (secure 0 0 0) 0x7f5c740917b0 con 0x7f5c880cf7c0 2024-09-11T12:50:20.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c8d000640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 0x7f5c68073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:20.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.514+0000 7f5c8d000640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 0x7f5c68073b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c780af000 tx=0x7f5c7801afa0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:20.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.518+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5c74020030 con 0x7f5c880cf7c0 2024-09-11T12:50:20.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:20.698+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5c8815e340 con 0x7f5c880cf7c0 2024-09-11T12:50:21.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.282+0000 7f5c84c00640 1 -- 172.21.15.78:0/2348300291 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) ==== 97+0+0 (secure 0 0 0) 0x7f5c7405d1a0 con 0x7f5c880cf7c0 2024-09-11T12:50:21.288 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:50:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 msgr2=0x7f5c68073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 0x7f5c68073b40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5c780af000 tx=0x7f5c7801afa0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 msgr2=0x7f5c880a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400c460 tx=0x7f5c7400c920 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 shutdown_connections 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5c68071680 0x7f5c68073b40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c8815d1e0 0x7f5c880a99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c880cf7c0 0x7f5c880a9460 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 --2- 172.21.15.78:0/2348300291 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c880aed20 0x7f5c880738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 >> 172.21.15.78:0/2348300291 conn(0x7f5c8806ca80 msgr2=0x7f5c8806feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 shutdown_connections 2024-09-11T12:50:21.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.286+0000 7f5c8fa00640 1 -- 172.21.15.78:0/2348300291 wait complete. 2024-09-11T12:50:21.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:50:21.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 -- 172.21.15.78:0/1016784388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac54072230 msgr2=0x7fac5406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 --2- 172.21.15.78:0/1016784388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac54072230 0x7fac5406fc50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fac5415f450 tx=0x7fac3c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 -- 172.21.15.78:0/1016784388 shutdown_connections 2024-09-11T12:50:21.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 --2- 172.21.15.78:0/1016784388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac54072230 0x7fac5406fc50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 --2- 172.21.15.78:0/1016784388 >> v2:172.21.15.157:3300/0 conn(0x7fac54070190 0x7fac5415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 --2- 172.21.15.78:0/1016784388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac540741e0 0x7fac54070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 -- 172.21.15.78:0/1016784388 >> 172.21.15.78:0/1016784388 conn(0x7fac5406ca80 msgr2=0x7fac540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 -- 172.21.15.78:0/1016784388 shutdown_connections 2024-09-11T12:50:21.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.450+0000 7fac58a00640 1 -- 172.21.15.78:0/1016784388 wait complete. 2024-09-11T12:50:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 Processor -- start 2024-09-11T12:50:21.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- start start 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac54070190 0x7fac54147620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54072230 0x7fac541f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac5415bca0 con 0x7fac540741e0 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac54072880 con 0x7fac54072230 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac5415be20 con 0x7fac54070190 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55092/0 (socket says 172.21.15.78:55092) 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 -- 172.21.15.78:0/638171538 learned_addr learned my addr 172.21.15.78:0/638171538 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac51e00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac54070190 0x7fac54147620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac49400640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54072230 0x7fac541f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 -- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac54070190 msgr2=0x7fac54147620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac54070190 0x7fac54147620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 -- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54072230 msgr2=0x7fac541f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54072230 0x7fac541f4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 -- 172.21.15.78:0/638171538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac541f8590 con 0x7fac540741e0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac52800640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9c0 tx=0x7fac4c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00cd80 con 0x7fac540741e0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac4c004b30 con 0x7fac540741e0 2024-09-11T12:50:21.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c005020 con 0x7fac540741e0 2024-09-11T12:50:21.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac54154510 con 0x7fac540741e0 2024-09-11T12:50:21.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.454+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac54154db0 con 0x7fac540741e0 2024-09-11T12:50:21.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac541f8760 con 0x7fac540741e0 2024-09-11T12:50:21.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fac4c0051c0 con 0x7fac540741e0 2024-09-11T12:50:21.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac4aa00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 0x7fac30073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fac4c0922d0 con 0x7fac540741e0 2024-09-11T12:50:21.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac51e00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 0x7fac30073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.458+0000 7fac51e00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 0x7fac30073b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac54151b40 tx=0x7fac4400a250 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:21.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.462+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fac4c016030 con 0x7fac540741e0 2024-09-11T12:50:21.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.686+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fac540b5040 con 0x7fac540741e0 2024-09-11T12:50:21.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.686+0000 7fac4aa00640 1 -- 172.21.15.78:0/638171538 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) ==== 80+0+6645 (secure 0 0 0) 0x7fac4c0920d0 con 0x7fac540741e0 2024-09-11T12:50:21.691 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.693 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:21.694 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:50:21.696 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.697 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:50:21.698 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.700 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.701 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:50:21.702 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:21.703 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.704 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:21.705 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-mds", 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmveUqNRAAgsNX77fSxUbj846dCTwANA==", 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-mds" 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-mgr", 2024-09-11T12:50:21.706 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmyPAqNRAAXcX6NweuDxGgH1IR/f0X8Q==", 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-osd", 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmbPkqNRAA3VPyjdXg+n6ZF6x6NYrXLw==", 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-osd" 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rbd", 2024-09-11T12:50:21.707 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmxAErNRAAyFO38kgSEkt9AGGiwO7e4g==", 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmaAorNRAABvK4hzSJnPBZpZO0VMGcwg==", 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.708 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.bootstrap-rgw", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDNkeFmRBMrNRAANwjq9KGoXl2p0zMPGlBtGQ==", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:50:21.710 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:50:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 msgr2=0x7fac30073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 0x7fac30073b40 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fac54151b40 tx=0x7fac4400a250 comp rx=0 tx=0).stop 2024-09-11T12:50:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 msgr2=0x7fac540b1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9c0 tx=0x7fac4c00ee80 comp rx=0 tx=0).stop 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 shutdown_connections 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac30071680 0x7fac30073b40 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac540741e0 0x7fac540b1150 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac54072230 0x7fac541f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 --2- 172.21.15.78:0/638171538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac54070190 0x7fac54147620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 >> 172.21.15.78:0/638171538 conn(0x7fac5406ca80 msgr2=0x7fac5415c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 shutdown_connections 2024-09-11T12:50:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.690+0000 7fac58a00640 1 -- 172.21.15.78:0/638171538 wait complete. 2024-09-11T12:50:21.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- 172.21.15.78:0/925925438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 msgr2=0x7f837400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 --2- 172.21.15.78:0/925925438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f837400b9d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f838800c670 tx=0x7f838801d300 comp rx=0 tx=0).stop 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8394c00640 1 -- 172.21.15.78:0/925925438 reap_dead start 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- 172.21.15.78:0/925925438 shutdown_connections 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- 172.21.15.78:0/925925438 >> 172.21.15.78:0/925925438 conn(0x7f839006ca80 msgr2=0x7f83900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- 172.21.15.78:0/925925438 shutdown_connections 2024-09-11T12:50:21.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- 172.21.15.78:0/925925438 wait complete. 2024-09-11T12:50:21.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 Processor -- start 2024-09-11T12:50:21.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- start start 2024-09-11T12:50:21.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83901417d0 0x7f83901f74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83901f7a30 0x7f83900ae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83900cd910 con 0x7f83740051d0 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83900b56d0 con 0x7f83901417d0 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8396c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83900cda90 con 0x7f83901f7a30 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55116/0 (socket says 172.21.15.78:55116) 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 -- 172.21.15.78:0/2331518869 learned_addr learned my addr 172.21.15.78:0/2331518869 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f8394c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83901f7a30 0x7f83900ae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838f400640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83901417d0 0x7f83901f74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 -- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83901f7a30 msgr2=0x7f83900ae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83901f7a30 0x7f83900ae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 -- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83901417d0 msgr2=0x7f83901f74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:21.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83901417d0 0x7f83901f74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:21.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.854+0000 7f838fe00640 1 -- 172.21.15.78:0/2331518869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83901f7f70 con 0x7f83740051d0 2024-09-11T12:50:21.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f838fe00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f838000eb20 tx=0x7f838000d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:21.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380004050 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83800041f0 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83901f8250 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8380003600 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8390157a10 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83900706f0 con 0x7f83740051d0 2024-09-11T12:50:21.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f83800037a0 con 0x7f83740051d0 2024-09-11T12:50:21.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 0x7f8370073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:21.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f8380092870 con 0x7f83740051d0 2024-09-11T12:50:21.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.858+0000 7f838f400640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 0x7f8370073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:21.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.862+0000 7f838f400640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 0x7f8370073b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8390158730 tx=0x7f837c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:21.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:21.862+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f838005e3d0 con 0x7f83740051d0 2024-09-11T12:50:22.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.086+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f83900cf2b0 con 0x7f83740051d0 2024-09-11T12:50:22.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8387e00640 1 -- 172.21.15.78:0/2331518869 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) ==== 90+0+0 (secure 0 0 0) 0x7f8380062920 con 0x7f83740051d0 2024-09-11T12:50:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 msgr2=0x7f8370073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 0x7f8370073b70 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f8390158730 tx=0x7f837c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 msgr2=0x7f8390157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f838000eb20 tx=0x7f838000d3e0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 shutdown_connections 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83700716b0 0x7f8370073b70 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83901f7a30 0x7f83900ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83901417d0 0x7f83901f74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 --2- 172.21.15.78:0/2331518869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83740051d0 0x7f8390157f30 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 >> 172.21.15.78:0/2331518869 conn(0x7f839006ca80 msgr2=0x7f83900cf590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 shutdown_connections 2024-09-11T12:50:22.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.090+0000 7f8396c00640 1 -- 172.21.15.78:0/2331518869 wait complete. 2024-09-11T12:50:22.116 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- 172.21.15.78:0/1679497109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 msgr2=0x7fa15c0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 --2- 172.21.15.78:0/1679497109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c0b5220 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa154009900 tx=0x7fa15401c880 comp rx=0 tx=0).stop 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- 172.21.15.78:0/1679497109 shutdown_connections 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 --2- 172.21.15.78:0/1679497109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c0b5220 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 --2- 172.21.15.78:0/1679497109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c15af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- 172.21.15.78:0/1679497109 >> 172.21.15.78:0/1679497109 conn(0x7fa15c06ca80 msgr2=0x7fa15c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- 172.21.15.78:0/1679497109 shutdown_connections 2024-09-11T12:50:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- 172.21.15.78:0/1679497109 wait complete. 2024-09-11T12:50:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 Processor -- start 2024-09-11T12:50:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.254+0000 7fa161200640 1 -- start start 2024-09-11T12:50:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 0x7fa15c13d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c072650 con 0x7fa15c0b4db0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa15c0724d0 con 0x7fa15c0707f0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa15c0727d0 con 0x7fa15c073ed0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55130/0 (socket says 172.21.15.78:55130) 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 -- 172.21.15.78:0/3945984042 learned_addr learned my addr 172.21.15.78:0/3945984042 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 -- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 msgr2=0x7fa15c1503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa159400640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa159e00640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 0x7fa15c13d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c1503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 -- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 msgr2=0x7fa15c13d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 0x7fa15c13d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 -- 172.21.15.78:0/3945984042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15c14d260 con 0x7fa15c0b4db0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa159e00640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 0x7fa15c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa159400640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa15a800640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa1540b0040 tx=0x7fa1540a6630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1540bc020 con 0x7fa15c0b4db0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1540041f0 con 0x7fa15c0b4db0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa154005030 con 0x7fa15c0b4db0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15c13db50 con 0x7fa15c0b4db0 2024-09-11T12:50:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa15c1fac20 con 0x7fa15c0b4db0 2024-09-11T12:50:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15c15af10 con 0x7fa15c0b4db0 2024-09-11T12:50:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.258+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fa1540bf070 con 0x7fa15c0b4db0 2024-09-11T12:50:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.262+0000 7fa152000640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 0x7fa13c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.262+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fa15412b530 con 0x7fa15c0b4db0 2024-09-11T12:50:22.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.262+0000 7fa159e00640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 0x7fa13c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.262+0000 7fa159e00640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 0x7fa13c073b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009f40 tx=0x7fa14c009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:22.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.266+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa1540164d0 con 0x7fa15c0b4db0 2024-09-11T12:50:22.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.494+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7fa15c0b1e30 con 0x7fa15c0b4db0 2024-09-11T12:50:22.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa152000640 1 -- 172.21.15.78:0/3945984042 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7fa1540f7650 con 0x7fa15c0b4db0 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 msgr2=0x7fa13c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 0x7fa13c073b40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009f40 tx=0x7fa14c009500 comp rx=0 tx=0).stop 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 msgr2=0x7fa15c150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa1540b0040 tx=0x7fa1540a6630 comp rx=0 tx=0).stop 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 shutdown_connections 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa13c071680 0x7fa13c073b40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15c0b4db0 0x7fa15c150920 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa15c073ed0 0x7fa15c1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 --2- 172.21.15.78:0/3945984042 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa15c0707f0 0x7fa15c13d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 >> 172.21.15.78:0/3945984042 conn(0x7fa15c06ca80 msgr2=0x7fa15c0ae550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.498+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 shutdown_connections 2024-09-11T12:50:22.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.502+0000 7fa161200640 1 -- 172.21.15.78:0/3945984042 wait complete. 2024-09-11T12:50:22.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-11T12:50:22.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 -- 172.21.15.78:0/4292448062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d38005eb0 msgr2=0x7f4d38006330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 --2- 172.21.15.78:0/4292448062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d38005eb0 0x7f4d38006330 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c009e00 tx=0x7f4d4c01d130 comp rx=0 tx=0).stop 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d53e00640 1 -- 172.21.15.78:0/4292448062 reap_dead start 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 -- 172.21.15.78:0/4292448062 shutdown_connections 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 -- 172.21.15.78:0/4292448062 >> 172.21.15.78:0/4292448062 conn(0x7f4d5406ca80 msgr2=0x7f4d540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 -- 172.21.15.78:0/4292448062 shutdown_connections 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.662+0000 7f4d5a200640 1 -- 172.21.15.78:0/4292448062 wait complete. 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 Processor -- start 2024-09-11T12:50:22.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- start start 2024-09-11T12:50:22.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 0x7f4d540af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d54152e40 0x7f4d5415d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54073ae0 con 0x7f4d54147b90 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4d54073960 con 0x7f4d54152e40 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4d540b4520 con 0x7f4d38005eb0 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55140/0 (socket says 172.21.15.78:55140) 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 -- 172.21.15.78:0/3395810362 learned_addr learned my addr 172.21.15.78:0/3395810362 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d53400640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 0x7f4d540af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d53e00640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d54152e40 0x7f4d5415d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 -- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 msgr2=0x7f4d540af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 0x7f4d540af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 -- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d54152e40 msgr2=0x7f4d5415d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d54152e40 0x7f4d5415d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 -- 172.21.15.78:0/3395810362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d540ad910 con 0x7f4d54147b90 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d53400640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 0x7f4d540af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d52a00640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d40003f30 tx=0x7f4d40001700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40011640 con 0x7f4d54147b90 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d40016020 con 0x7f4d54147b90 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40011a60 con 0x7f4d54147b90 2024-09-11T12:50:22.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d540cda10 con 0x7f4d54147b90 2024-09-11T12:50:22.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d540ade90 con 0x7f4d54147b90 2024-09-11T12:50:22.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.666+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d54073c60 con 0x7f4d54147b90 2024-09-11T12:50:22.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.670+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4d40011c00 con 0x7f4d54147b90 2024-09-11T12:50:22.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.670+0000 7f4d4b400640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 0x7f4d34073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:22.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.670+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f4d40092730 con 0x7f4d54147b90 2024-09-11T12:50:22.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.670+0000 7f4d53400640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 0x7f4d34073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:22.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.674+0000 7f4d53400640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 0x7f4d34073b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d44003fb0 tx=0x7f4d440067b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:22.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.674+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4d40020020 con 0x7f4d54147b90 2024-09-11T12:50:22.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.902+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f4d540b15b0 con 0x7f4d54147b90 2024-09-11T12:50:22.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d4b400640 1 -- 172.21.15.78:0/3395810362 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) ==== 90+0+0 (secure 0 0 0) 0x7f4d4005e850 con 0x7f4d54147b90 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 msgr2=0x7f4d34073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 0x7f4d34073b40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4d44003fb0 tx=0x7f4d440067b0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 msgr2=0x7f4d54152900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4d40003f30 tx=0x7f4d40001700 comp rx=0 tx=0).stop 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 shutdown_connections 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4d34071680 0x7f4d34073b40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4d54152e40 0x7f4d5415d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4d54147b90 0x7f4d54152900 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 --2- 172.21.15.78:0/3395810362 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4d38005eb0 0x7f4d540af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:22.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 >> 172.21.15.78:0/3395810362 conn(0x7f4d5406ca80 msgr2=0x7f4d54072b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:22.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 shutdown_connections 2024-09-11T12:50:22.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:22.906+0000 7f4d5a200640 1 -- 172.21.15.78:0/3395810362 wait complete. 2024-09-11T12:50:22.933 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-11T12:50:23.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- 172.21.15.78:0/1774323936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 msgr2=0x7f8394074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 --2- 172.21.15.78:0/1774323936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394074260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8384009900 tx=0x7f838401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- 172.21.15.78:0/1774323936 shutdown_connections 2024-09-11T12:50:23.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 --2- 172.21.15.78:0/1774323936 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839406fc50 0x7f83940700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 --2- 172.21.15.78:0/1774323936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394074260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- 172.21.15.78:0/1774323936 >> 172.21.15.78:0/1774323936 conn(0x7f839406ca80 msgr2=0x7f83940b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- 172.21.15.78:0/1774323936 shutdown_connections 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- 172.21.15.78:0/1774323936 wait complete. 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 Processor -- start 2024-09-11T12:50:23.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.070+0000 7f8399e00640 1 -- start start 2024-09-11T12:50:23.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839406fc50 0x7f8394143550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839415aec0 0x7f8394159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8394151010 con 0x7f83940cf760 2024-09-11T12:50:23.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f839415f2d0 con 0x7f839406fc50 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8394151190 con 0x7f839415aec0 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8392a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839406fc50 0x7f8394143550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55156/0 (socket says 172.21.15.78:55156) 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 -- 172.21.15.78:0/1634589402 learned_addr learned my addr 172.21.15.78:0/1634589402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8393400640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839415aec0 0x7f8394159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 -- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839415aec0 msgr2=0x7f8394159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839415aec0 0x7f8394159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 -- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839406fc50 msgr2=0x7f8394143550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839406fc50 0x7f8394143550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 -- 172.21.15.78:0/1634589402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8394146570 con 0x7f83940cf760 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838be00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f837c00b3e0 tx=0x7f837c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c015020 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f837c011920 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8394146850 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f837c011db0 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8394146e10 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.074+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8394147010 con 0x7f83940cf760 2024-09-11T12:50:23.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.078+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f837c018020 con 0x7f83940cf760 2024-09-11T12:50:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.078+0000 7f838b400640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 0x7f8370073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.078+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f837c091630 con 0x7f83940cf760 2024-09-11T12:50:23.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.078+0000 7f8392a00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 0x7f8370073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.078+0000 7f8392a00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 0x7f8370073b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83840af000 tx=0x7f838401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.082+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8394147010 con 0x7f83940cf760 2024-09-11T12:50:23.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.306+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f83940695e0 con 0x7f83940cf760 2024-09-11T12:50:23.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.310+0000 7f838b400640 1 -- 172.21.15.78:0/1634589402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) ==== 90+0+0 (secure 0 0 0) 0x7f837c01f030 con 0x7f83940cf760 2024-09-11T12:50:23.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 msgr2=0x7f8370073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 0x7f8370073b40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83840af000 tx=0x7f838401b120 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 msgr2=0x7f8394159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f837c00b3e0 tx=0x7f837c00b8a0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 shutdown_connections 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8370071680 0x7f8370073b40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f839415aec0 0x7f8394159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83940cf760 0x7f8394159900 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 --2- 172.21.15.78:0/1634589402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f839406fc50 0x7f8394143550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 >> 172.21.15.78:0/1634589402 conn(0x7f839406ca80 msgr2=0x7f839406f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 shutdown_connections 2024-09-11T12:50:23.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.314+0000 7f8399e00640 1 -- 172.21.15.78:0/1634589402 wait complete. 2024-09-11T12:50:23.338 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-11T12:50:23.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- 172.21.15.78:0/64270603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b2f70 msgr2=0x7fd7480b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 --2- 172.21.15.78:0/64270603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b2f70 0x7fd7480b33e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd734009840 tx=0x7fd73401c820 comp rx=0 tx=0).stop 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- 172.21.15.78:0/64270603 shutdown_connections 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 --2- 172.21.15.78:0/64270603 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b2f70 0x7fd7480b33e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- 172.21.15.78:0/64270603 >> 172.21.15.78:0/64270603 conn(0x7fd7480b1b50 msgr2=0x7fd748070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- 172.21.15.78:0/64270603 shutdown_connections 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- 172.21.15.78:0/64270603 wait complete. 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 Processor -- start 2024-09-11T12:50:23.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- start start 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7480b2f70 0x7fd74814f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480d1740 0x7fd74815e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd748169ba0 con 0x7fd7480b4f30 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd748169a20 con 0x7fd7480d1740 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd74de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd748169d20 con 0x7fd7480b2f70 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd746000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd746000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55170/0 (socket says 172.21.15.78:55170) 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd746000640 1 -- 172.21.15.78:0/234096253 learned_addr learned my addr 172.21.15.78:0/234096253 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd746a00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7480b2f70 0x7fd74814f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.478+0000 7fd747400640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480d1740 0x7fd74815e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 -- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7480b2f70 msgr2=0x7fd74814f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7480b2f70 0x7fd74814f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 -- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480d1740 msgr2=0x7fd74815e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480d1740 0x7fd74815e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 -- 172.21.15.78:0/234096253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd748139dd0 con 0x7fd7480b4f30 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd746000640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd73401aed0 tx=0x7fd7340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340bb070 con 0x7fd7480b4f30 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd734005030 con 0x7fd7480b4f30 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd74813a0b0 con 0x7fd7480b4f30 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7340b2040 con 0x7fd7480b4f30 2024-09-11T12:50:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd74814f880 con 0x7fd7480b4f30 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7480b33e0 con 0x7fd7480b4f30 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.482+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fd7340be070 con 0x7fd7480b4f30 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.486+0000 7fd73f400640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 0x7fd728073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.486+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fd73412b4f0 con 0x7fd7480b4f30 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.486+0000 7fd746a00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 0x7fd728073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.486+0000 7fd746a00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 0x7fd728073b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd738009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.486+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd734016410 con 0x7fd7480b4f30 2024-09-11T12:50:23.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.714+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fd748070c70 con 0x7fd7480b4f30 2024-09-11T12:50:23.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd73f400640 1 -- 172.21.15.78:0/234096253 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) ==== 97+0+0 (secure 0 0 0) 0x7fd7340f7050 con 0x7fd7480b4f30 2024-09-11T12:50:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 msgr2=0x7fd728073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 0x7fd728073b40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd738007fd0 tx=0x7fd738009040 comp rx=0 tx=0).stop 2024-09-11T12:50:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 msgr2=0x7fd74815ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd73401aed0 tx=0x7fd7340040c0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 shutdown_connections 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd728071680 0x7fd728073b40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480d1740 0x7fd74815e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7480b4f30 0x7fd74815ddb0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 --2- 172.21.15.78:0/234096253 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7480b2f70 0x7fd74814f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 >> 172.21.15.78:0/234096253 conn(0x7fd7480b1b50 msgr2=0x7fd74815d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 shutdown_connections 2024-09-11T12:50:23.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.718+0000 7fd74de00640 1 -- 172.21.15.78:0/234096253 wait complete. 2024-09-11T12:50:23.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-11T12:50:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- 172.21.15.78:0/395629882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff45415aec0 msgr2=0x7ff454070830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- 172.21.15.78:0/395629882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff45415aec0 0x7ff454070830 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff440009900 tx=0x7ff44001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- 172.21.15.78:0/395629882 shutdown_connections 2024-09-11T12:50:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- 172.21.15.78:0/395629882 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff45415aec0 0x7ff454070830 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- 172.21.15.78:0/395629882 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4540cf760 0x7ff454074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- 172.21.15.78:0/395629882 >> 172.21.15.78:0/395629882 conn(0x7ff45406ca80 msgr2=0x7ff4540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- 172.21.15.78:0/395629882 shutdown_connections 2024-09-11T12:50:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- 172.21.15.78:0/395629882 wait complete. 2024-09-11T12:50:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 Processor -- start 2024-09-11T12:50:23.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- start start 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff45406fc50 0x7ff454149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415aec0 0x7ff454143dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff454151820 con 0x7ff4540cf760 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540cfd50 con 0x7ff45406fc50 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff459e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4541519a0 con 0x7ff45415aec0 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff452000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff452000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60232/0 (socket says 172.21.15.78:60232) 2024-09-11T12:50:23.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.882+0000 7ff452000640 1 -- 172.21.15.78:0/3512690089 learned_addr learned my addr 172.21.15.78:0/3512690089 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 -- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415aec0 msgr2=0x7ff454143dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415aec0 0x7ff454143dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 -- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff45406fc50 msgr2=0x7ff454149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff45406fc50 0x7ff454149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 -- 172.21.15.78:0/3512690089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff454144310 con 0x7ff4540cf760 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff452000640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff440009850 tx=0x7ff440005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400b2040 con 0x7ff4540cf760 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff440005320 con 0x7ff4540cf760 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45414a000 con 0x7ff4540cf760 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4400046d0 con 0x7ff4540cf760 2024-09-11T12:50:23.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45414a5c0 con 0x7ff4540cf760 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff454074260 con 0x7ff4540cf760 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7ff440004870 con 0x7ff4540cf760 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 0x7ff434073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.886+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7ff44012bac0 con 0x7ff4540cf760 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.890+0000 7ff452a00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 0x7ff434073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.890+0000 7ff452a00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 0x7ff434073b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ba20 tx=0x7ff444005f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:23.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:23.890+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff4400bc060 con 0x7ff4540cf760 2024-09-11T12:50:24.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.118+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7ff45414f3a0 con 0x7ff4540cf760 2024-09-11T12:50:24.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff44b400640 1 -- 172.21.15.78:0/3512690089 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) ==== 90+0+0 (secure 0 0 0) 0x7ff4400164d0 con 0x7ff4540cf760 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 msgr2=0x7ff434073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 0x7ff434073b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff44400ba20 tx=0x7ff444005f50 comp rx=0 tx=0).stop 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 msgr2=0x7ff4541f9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff440009850 tx=0x7ff440005760 comp rx=0 tx=0).stop 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 shutdown_connections 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff434071680 0x7ff434073b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff45415aec0 0x7ff454143dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540cf760 0x7ff4541f9540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 --2- 172.21.15.78:0/3512690089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff45406fc50 0x7ff454149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 >> 172.21.15.78:0/3512690089 conn(0x7ff45406ca80 msgr2=0x7ff45414e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:24.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 shutdown_connections 2024-09-11T12:50:24.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.122+0000 7ff459e00640 1 -- 172.21.15.78:0/3512690089 wait complete. 2024-09-11T12:50:24.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:50:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.282+0000 7fcd6d800640 1 -- 172.21.15.78:0/2352398551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd4c005220 msgr2=0x7fcd4c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.282+0000 7fcd6d800640 1 --2- 172.21.15.78:0/2352398551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd4c005220 0x7fcd4c00b9d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcd60009b10 tx=0x7fcd6001d250 comp rx=0 tx=0).stop 2024-09-11T12:50:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- 172.21.15.78:0/2352398551 shutdown_connections 2024-09-11T12:50:24.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 --2- 172.21.15.78:0/2352398551 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd4c005220 0x7fcd4c00b9d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- 172.21.15.78:0/2352398551 >> 172.21.15.78:0/2352398551 conn(0x7fcd6806ca80 msgr2=0x7fcd680b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- 172.21.15.78:0/2352398551 shutdown_connections 2024-09-11T12:50:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- 172.21.15.78:0/2352398551 wait complete. 2024-09-11T12:50:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 Processor -- start 2024-09-11T12:50:24.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- start start 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd4c005220 0x7fcd680acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd6815a180 0x7fcd68071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd68140340 con 0x7fcd68154d90 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd681401c0 con 0x7fcd6815a180 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd68140980 con 0x7fcd4c005220 2024-09-11T12:50:24.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60250/0 (socket says 172.21.15.78:60250) 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 -- 172.21.15.78:0/1557794239 learned_addr learned my addr 172.21.15.78:0/1557794239 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66a00640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd4c005220 0x7fcd680acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd67400640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd6815a180 0x7fcd68071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 -- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd4c005220 msgr2=0x7fcd680acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd4c005220 0x7fcd680acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 -- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd6815a180 msgr2=0x7fcd68071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd6815a180 0x7fcd68071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 -- 172.21.15.78:0/1557794239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd680ad4f0 con 0x7fcd68154d90 2024-09-11T12:50:24.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd66000640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd54001ec0 tx=0x7fcd5400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54015070 con 0x7fcd68154d90 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd54010980 con 0x7fcd68154d90 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54010db0 con 0x7fcd68154d90 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.286+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd680a96b0 con 0x7fcd68154d90 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd680a9c70 con 0x7fcd68154d90 2024-09-11T12:50:24.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd6815a560 con 0x7fcd68154d90 2024-09-11T12:50:24.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fcd54018070 con 0x7fcd68154d90 2024-09-11T12:50:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd5ea00640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 0x7fcd48073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7fcd54091cf0 con 0x7fcd68154d90 2024-09-11T12:50:24.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.290+0000 7fcd66a00640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 0x7fcd48073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.294+0000 7fcd66a00640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 0x7fcd48073b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b2100 tx=0x7fcd58006f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:24.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.294+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcd6815a560 con 0x7fcd68154d90 2024-09-11T12:50:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcd68140e50 con 0x7fcd68154d90 2024-09-11T12:50:24.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd5ea00640 1 -- 172.21.15.78:0/1557794239 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) ==== 84+0+65 (secure 0 0 0) 0x7fcd5405de90 con 0x7fcd68154d90 2024-09-11T12:50:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:24.470 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:50:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:50:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:50:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:24.471 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:50:24.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 msgr2=0x7fcd48073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 0x7fcd48073b70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcd680b2100 tx=0x7fcd58006f00 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 msgr2=0x7fcd68159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcd54001ec0 tx=0x7fcd5400ea20 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 shutdown_connections 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcd480716b0 0x7fcd48073b70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd6815a180 0x7fcd68071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd68154d90 0x7fcd68159c40 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 --2- 172.21.15.78:0/1557794239 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd4c005220 0x7fcd680acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 >> 172.21.15.78:0/1557794239 conn(0x7fcd6806ca80 msgr2=0x7fcd68072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 shutdown_connections 2024-09-11T12:50:24.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.466+0000 7fcd6d800640 1 -- 172.21.15.78:0/1557794239 wait complete. 2024-09-11T12:50:24.491 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:50:24.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- 172.21.15.78:0/934113750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7158003a40 msgr2=0x7f7158003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 --2- 172.21.15.78:0/934113750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7158003a40 0x7f7158003e20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f716400c640 tx=0x7f716401d280 comp rx=0 tx=0).stop 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7176200640 1 -- 172.21.15.78:0/934113750 reap_dead start 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- 172.21.15.78:0/934113750 shutdown_connections 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- 172.21.15.78:0/934113750 >> 172.21.15.78:0/934113750 conn(0x7f71700b3050 msgr2=0x7f71700b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- 172.21.15.78:0/934113750 shutdown_connections 2024-09-11T12:50:24.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- 172.21.15.78:0/934113750 wait complete. 2024-09-11T12:50:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 Processor -- start 2024-09-11T12:50:24.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- start start 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003a40 0x7f7170147080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700b4a30 0x7f7170143600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71701669b0 con 0x7f7170144590 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7170166830 con 0x7f7158003a40 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7178200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7170166b30 con 0x7f71700b4a30 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7176200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7176200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60278/0 (socket says 172.21.15.78:60278) 2024-09-11T12:50:24.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7176200640 1 -- 172.21.15.78:0/3129068812 learned_addr learned my addr 172.21.15.78:0/3129068812 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7175800640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003a40 0x7f7170147080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.634+0000 7f7174e00640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700b4a30 0x7f7170143600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 -- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700b4a30 msgr2=0x7f7170143600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700b4a30 0x7f7170143600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 -- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003a40 msgr2=0x7f7170147080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003a40 0x7f7170147080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 -- 172.21.15.78:0/3129068812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71700746d0 con 0x7f7170144590 2024-09-11T12:50:24.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7176200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71680117b0 tx=0x7f7168011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7168011e60 con 0x7f7170144590 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f716800f8c0 con 0x7f7170144590 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f716800fcd0 con 0x7f7170144590 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f717014ef80 con 0x7f7170144590 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f717014f250 con 0x7f7170144590 2024-09-11T12:50:24.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71700723f0 con 0x7f7170144590 2024-09-11T12:50:24.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.638+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7168003600 con 0x7f7170144590 2024-09-11T12:50:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.642+0000 7f716d600640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 0x7f7150073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.642+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4857+0+0 (secure 0 0 0) 0x7f7168092260 con 0x7f7170144590 2024-09-11T12:50:24.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.642+0000 7f7175800640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 0x7f7150073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:24.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.642+0000 7f7175800640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 0x7f7150073b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71640ac000 tx=0x7f716401a0c0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:24.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.646+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7168021020 con 0x7f7170144590 2024-09-11T12:50:24.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:24.810+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f7170072010 con 0x7f7170144590 2024-09-11T12:50:25.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.302+0000 7f716d600640 1 -- 172.21.15.78:0/3129068812 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) ==== 138+0+0 (secure 0 0 0) 0x7f71700746d0 con 0x7f7170144590 2024-09-11T12:50:25.307 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:50:25.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.302+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 msgr2=0x7f7150073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:25.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 0x7f7150073b70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71640ac000 tx=0x7f716401a0c0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 msgr2=0x7f7170148cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:25.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f71680117b0 tx=0x7f7168011c70 comp rx=0 tx=0).stop 2024-09-11T12:50:25.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 shutdown_connections 2024-09-11T12:50:25.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71500716b0 0x7f7150073b70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7170144590 0x7f7170148cb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f71700b4a30 0x7f7170143600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 --2- 172.21.15.78:0/3129068812 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7158003a40 0x7f7170147080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 >> 172.21.15.78:0/3129068812 conn(0x7f71700b3050 msgr2=0x7f717006eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 shutdown_connections 2024-09-11T12:50:25.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.306+0000 7f7178200640 1 -- 172.21.15.78:0/3129068812 wait complete. 2024-09-11T12:50:25.335 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:50:25.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 -- 172.21.15.78:0/1787316416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 msgr2=0x7fe1cc0cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:25.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/1787316416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc0cd290 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc070ec0 tx=0x7fe1b801dbf0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 -- 172.21.15.78:0/1787316416 shutdown_connections 2024-09-11T12:50:25.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/1787316416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc0cd290 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/1787316416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 0x7fe1cc070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/1787316416 >> v2:172.21.15.157:3300/0 conn(0x7fe1cc140400 0x7fe1cc15e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 -- 172.21.15.78:0/1787316416 >> 172.21.15.78:0/1787316416 conn(0x7fe1cc0b2ec0 msgr2=0x7fe1cc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:25.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 -- 172.21.15.78:0/1787316416 shutdown_connections 2024-09-11T12:50:25.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.478+0000 7fe1d3e00640 1 -- 172.21.15.78:0/1787316416 wait complete. 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 Processor -- start 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- start start 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc140400 0x7fe1cc163600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 0x7fe1cc160690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc073860 con 0x7fe1cc0cf790 2024-09-11T12:50:25.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0736e0 con 0x7fe1cc140400 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0739e0 con 0x7fe1cc1407e0 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60292/0 (socket says 172.21.15.78:60292) 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 -- 172.21.15.78:0/13028866 learned_addr learned my addr 172.21.15.78:0/13028866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d0a00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc140400 0x7fe1cc163600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 0x7fe1cc160690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 -- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 msgr2=0x7fe1cc160690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 0x7fe1cc160690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 -- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc140400 msgr2=0x7fe1cc163600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc140400 0x7fe1cc163600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 -- 172.21.15.78:0/13028866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc12fc00 con 0x7fe1cc0cf790 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d1400640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000ea20 tx=0x7fe1c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000cd80 con 0x7fe1cc0cf790 2024-09-11T12:50:25.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c0010430 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc15d1e0 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0010840 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.482+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc15d7a0 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.486+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc15edd0 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.486+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe1c00109e0 con 0x7fe1cc0cf790 2024-09-11T12:50:25.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.486+0000 7fe1c9600640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 0x7fe1ac073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.486+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5227+0+0 (secure 0 0 0) 0x7fe1c0092e00 con 0x7fe1cc0cf790 2024-09-11T12:50:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.486+0000 7fe1d0a00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 0x7fe1ac073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:25.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.490+0000 7fe1d0a00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 0x7fe1ac073b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80b0000 tx=0x7fe1b801c430 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:25.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.490+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe1cc15edd0 con 0x7fe1cc0cf790 2024-09-11T12:50:25.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:25.654+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe1cc15e6f0 con 0x7fe1cc0cf790 2024-09-11T12:50:26.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.310+0000 7fe1c9600640 1 -- 172.21.15.78:0/13028866 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) ==== 170+0+0 (secure 0 0 0) 0x7fe1c005e7e0 con 0x7fe1cc0cf790 2024-09-11T12:50:26.316 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:50:26.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 msgr2=0x7fe1ac073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 0x7fe1ac073b40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fe1b80b0000 tx=0x7fe1b801c430 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 msgr2=0x7fe1cc12f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe1c000ea20 tx=0x7fe1c000eee0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 shutdown_connections 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ac071680 0x7fe1ac073b40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe1cc1407e0 0x7fe1cc160690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe1cc140400 0x7fe1cc163600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 --2- 172.21.15.78:0/13028866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1cc0cf790 0x7fe1cc12f6c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 >> 172.21.15.78:0/13028866 conn(0x7fe1cc0b2ec0 msgr2=0x7fe1cc159810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 shutdown_connections 2024-09-11T12:50:26.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.314+0000 7fe1d3e00640 1 -- 172.21.15.78:0/13028866 wait complete. 2024-09-11T12:50:26.357 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:50:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- 172.21.15.78:0/3199030964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0b2120 msgr2=0x7fa81c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 --2- 172.21.15.78:0/3199030964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0b2120 0x7fa81c0cdc10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa808009900 tx=0x7fa80801c8a0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- 172.21.15.78:0/3199030964 shutdown_connections 2024-09-11T12:50:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 --2- 172.21.15.78:0/3199030964 >> v2:172.21.15.157:3300/0 conn(0x7fa81c0ce150 0x7fa81c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 --2- 172.21.15.78:0/3199030964 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0b2120 0x7fa81c0cdc10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- 172.21.15.78:0/3199030964 >> 172.21.15.78:0/3199030964 conn(0x7fa81c071dd0 msgr2=0x7fa81c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- 172.21.15.78:0/3199030964 shutdown_connections 2024-09-11T12:50:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- 172.21.15.78:0/3199030964 wait complete. 2024-09-11T12:50:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 Processor -- start 2024-09-11T12:50:26.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.498+0000 7fa824200640 1 -- start start 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c0b2120 0x7fa81c0adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c1518b0 0x7fa81c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c167960 con 0x7fa81c0ce150 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa81c1677e0 con 0x7fa81c1518b0 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa81c167ae0 con 0x7fa81c0b2120 2024-09-11T12:50:26.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60304/0 (socket says 172.21.15.78:60304) 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 -- 172.21.15.78:0/2524200792 learned_addr learned my addr 172.21.15.78:0/2524200792 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa821800640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c0b2120 0x7fa81c0adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa822200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c1518b0 0x7fa81c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 -- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c0b2120 msgr2=0x7fa81c0adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c0b2120 0x7fa81c0adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 -- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c1518b0 msgr2=0x7fa81c158b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c1518b0 0x7fa81c158b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 -- 172.21.15.78:0/2524200792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa81c1510e0 con 0x7fa81c0ce150 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa820e00640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa808009850 tx=0x7fa8080a5790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8080b2040 con 0x7fa81c0ce150 2024-09-11T12:50:26.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa808005070 con 0x7fa81c0ce150 2024-09-11T12:50:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa808005480 con 0x7fa81c0ce150 2024-09-11T12:50:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa81c15a5b0 con 0x7fa81c0ce150 2024-09-11T12:50:26.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa81c141090 con 0x7fa81c0ce150 2024-09-11T12:50:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.502+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa81c15ab80 con 0x7fa81c0ce150 2024-09-11T12:50:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.506+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fa8080bd060 con 0x7fa81c0ce150 2024-09-11T12:50:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.506+0000 7fa819600640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 0x7fa7fc073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.506+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5609+0+0 (secure 0 0 0) 0x7fa80812b1c0 con 0x7fa81c0ce150 2024-09-11T12:50:26.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.506+0000 7fa821800640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 0x7fa7fc073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:26.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.506+0000 7fa821800640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 0x7fa7fc073b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8100098e0 tx=0x7fa810009060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:26.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.510+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa808016490 con 0x7fa81c0ce150 2024-09-11T12:50:26.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:26.710+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa81c068b40 con 0x7fa81c0ce150 2024-09-11T12:50:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 4 and data pool 5 2024-09-11T12:50:27.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.322+0000 7fa819600640 1 -- 172.21.15.78:0/2524200792 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7fa8080f7080 con 0x7fa81c0ce150 2024-09-11T12:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 msgr2=0x7fa7fc073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 0x7fa7fc073b40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa8100098e0 tx=0x7fa810009060 comp rx=0 tx=0).stop 2024-09-11T12:50:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 msgr2=0x7fa81c0ae050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa808009850 tx=0x7fa8080a5790 comp rx=0 tx=0).stop 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 shutdown_connections 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7fc071680 0x7fa7fc073b40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa81c1518b0 0x7fa81c158b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa81c0ce150 0x7fa81c0ae050 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 --2- 172.21.15.78:0/2524200792 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa81c0b2120 0x7fa81c0adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 >> 172.21.15.78:0/2524200792 conn(0x7fa81c071dd0 msgr2=0x7fa81c13f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 shutdown_connections 2024-09-11T12:50:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.330+0000 7fa824200640 1 -- 172.21.15.78:0/2524200792 wait complete. 2024-09-11T12:50:27.374 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- 172.21.15.78:0/1266264661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0aed20 msgr2=0x7faa5c0cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 --2- 172.21.15.78:0/1266264661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0aed20 0x7faa5c0cf280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7faa50009840 tx=0x7faa5001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- 172.21.15.78:0/1266264661 shutdown_connections 2024-09-11T12:50:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 --2- 172.21.15.78:0/1266264661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c0cf7c0 0x7faa5c15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 --2- 172.21.15.78:0/1266264661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0aed20 0x7faa5c0cf280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- 172.21.15.78:0/1266264661 >> 172.21.15.78:0/1266264661 conn(0x7faa5c06ca80 msgr2=0x7faa5c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- 172.21.15.78:0/1266264661 shutdown_connections 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- 172.21.15.78:0/1266264661 wait complete. 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 Processor -- start 2024-09-11T12:50:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.510+0000 7faa63a00640 1 -- start start 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5c0aed20 0x7faa5c0b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 0x7faa5c1f74e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c0cd0d0 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faa5c152da0 con 0x7faa5c0aed20 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faa5c0cd250 con 0x7faa5c15d010 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60332/0 (socket says 172.21.15.78:60332) 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 -- 172.21.15.78:0/3593597136 learned_addr learned my addr 172.21.15.78:0/3593597136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:27.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa61a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 0x7faa5c1f74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa61000640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5c0aed20 0x7faa5c0b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 -- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 msgr2=0x7faa5c1f74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 0x7faa5c1f74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 -- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5c0aed20 msgr2=0x7faa5c0b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5c0aed20 0x7faa5c0b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 -- 172.21.15.78:0/3593597136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5c068b40 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa61a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 0x7faa5c1f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa5be00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faa4800c970 tx=0x7faa4800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48007c70 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa4800f450 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800f8a0 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5c0adf30 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa5c1533f0 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.514+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa5c0cf280 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.518+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7faa4800fa40 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.518+0000 7faa58c00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 0x7faa3c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.518+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7faa480925f0 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.518+0000 7faa61000640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 0x7faa3c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.518+0000 7faa61000640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 0x7faa3c073b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa500af000 tx=0x7faa5001b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:27.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.522+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7faa48014030 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.690+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7faa5c141d70 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.690+0000 7faa58c00640 1 -- 172.21.15.78:0/3593597136 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7faa4805e3c0 con 0x7faa5c0cf7c0 2024-09-11T12:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:27.696 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":33,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:26.715223+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:24.824997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:25.670550+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 msgr2=0x7faa3c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 0x7faa3c073b40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7faa500af000 tx=0x7faa5001b030 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 msgr2=0x7faa5c0b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7faa4800c970 tx=0x7faa4800ce30 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 shutdown_connections 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faa3c071680 0x7faa3c073b40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faa5c15d010 0x7faa5c1f74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faa5c0cf7c0 0x7faa5c0b0ae0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 --2- 172.21.15.78:0/3593597136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faa5c0aed20 0x7faa5c0b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 >> 172.21.15.78:0/3593597136 conn(0x7faa5c06ca80 msgr2=0x7faa5c0af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 shutdown_connections 2024-09-11T12:50:27.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.694+0000 7faa63a00640 1 -- 172.21.15.78:0/3593597136 wait complete. 2024-09-11T12:50:27.717 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:27.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 -- 172.21.15.78:0/1175960457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc40d0bf0 msgr2=0x7f4bc40d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 --2- 172.21.15.78:0/1175960457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc40d1080 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc009bc0 tx=0x7f4bbc01cba0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 -- 172.21.15.78:0/1175960457 shutdown_connections 2024-09-11T12:50:27.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 --2- 172.21.15.78:0/1175960457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc40d1080 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 -- 172.21.15.78:0/1175960457 >> 172.21.15.78:0/1175960457 conn(0x7f4bc4074b90 msgr2=0x7f4bc40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:27.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 -- 172.21.15.78:0/1175960457 shutdown_connections 2024-09-11T12:50:27.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.854+0000 7f4bcb200640 1 -- 172.21.15.78:0/1175960457 wait complete. 2024-09-11T12:50:27.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 Processor -- start 2024-09-11T12:50:27.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- start start 2024-09-11T12:50:27.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc40b14b0 0x7f4bc415e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc416a0a0 con 0x7f4bc406fdf0 2024-09-11T12:50:27.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4bc4169f20 con 0x7f4bc40d0bf0 2024-09-11T12:50:27.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4bc416a220 con 0x7f4bc40b14b0 2024-09-11T12:50:27.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60348/0 (socket says 172.21.15.78:60348) 2024-09-11T12:50:27.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 -- 172.21.15.78:0/2528091472 learned_addr learned my addr 172.21.15.78:0/2528091472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc9200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bbbe00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc40b14b0 0x7f4bc415e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 -- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc40b14b0 msgr2=0x7f4bc415e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc40b14b0 0x7f4bc415e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 -- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40d0bf0 msgr2=0x7f4bc415ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc415ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 -- 172.21.15.78:0/2528091472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4bc4154f70 con 0x7f4bc406fdf0 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc3e00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4b0 tx=0x7f4bb400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb40052d0 con 0x7f4bc406fdf0 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4bb4005470 con 0x7f4bc406fdf0 2024-09-11T12:50:27.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4012870 con 0x7f4bc406fdf0 2024-09-11T12:50:27.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4bc4155250 con 0x7f4bc406fdf0 2024-09-11T12:50:27.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4bc4155810 con 0x7f4bc406fdf0 2024-09-11T12:50:27.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.858+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4bc40d1080 con 0x7f4bc406fdf0 2024-09-11T12:50:27.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.862+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f4bb4003600 con 0x7f4bc406fdf0 2024-09-11T12:50:27.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.862+0000 7f4bc0c00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 0x7f4ba0073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:27.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.862+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f4bb40919b0 con 0x7f4bc406fdf0 2024-09-11T12:50:27.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.862+0000 7f4bbbe00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 0x7f4ba0073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:27.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.862+0000 7f4bbbe00640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 0x7f4ba0073b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bc41facb0 tx=0x7f4bac009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:27.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:27.866+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4bb405d780 con 0x7f4bc406fdf0 2024-09-11T12:50:28.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.034+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4bc414db90 con 0x7f4bc406fdf0 2024-09-11T12:50:28.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bc0c00640 1 -- 172.21.15.78:0/2528091472 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) ==== 74+0+16742 (secure 0 0 0) 0x7f4bb405d080 con 0x7f4bc406fdf0 2024-09-11T12:50:28.041 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:28.041 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":33,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:26.715223+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:24.824997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:25.670550+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:28.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 msgr2=0x7f4ba0073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:28.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 0x7f4ba0073b40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4bc41facb0 tx=0x7f4bac009040 comp rx=0 tx=0).stop 2024-09-11T12:50:28.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 msgr2=0x7f4bc4149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4bb400c4b0 tx=0x7f4bb400c970 comp rx=0 tx=0).stop 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 shutdown_connections 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4ba0071680 0x7f4ba0073b40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4bc40d0bf0 0x7f4bc415ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4bc40b14b0 0x7f4bc415e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 --2- 172.21.15.78:0/2528091472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4bc406fdf0 0x7f4bc4149e00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 >> 172.21.15.78:0/2528091472 conn(0x7f4bc4074b90 msgr2=0x7f4bc414a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 shutdown_connections 2024-09-11T12:50:28.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.038+0000 7f4bcb200640 1 -- 172.21.15.78:0/2528091472 wait complete. 2024-09-11T12:50:28.063 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:50:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- 172.21.15.78:0/3773621726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc1f0 msgr2=0x7f1d940b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3773621726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc1f0 0x7f1d940b2de0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1d940cca10 tx=0x7f1d8401c830 comp rx=0 tx=0).stop 2024-09-11T12:50:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- 172.21.15.78:0/3773621726 shutdown_connections 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3773621726 >> v2:172.21.15.157:3300/0 conn(0x7f1d9413f000 0x7f1d9413f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3773621726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc5d0 0x7f1d9415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3773621726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc1f0 0x7f1d940b2de0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- 172.21.15.78:0/3773621726 >> 172.21.15.78:0/3773621726 conn(0x7f1d94071dd0 msgr2=0x7f1d940721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- 172.21.15.78:0/3773621726 shutdown_connections 2024-09-11T12:50:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- 172.21.15.78:0/3773621726 wait complete. 2024-09-11T12:50:28.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 Processor -- start 2024-09-11T12:50:28.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.202+0000 7f1d98e00640 1 -- start start 2024-09-11T12:50:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc1f0 0x7f1d94155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 0x7f1d9406e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9415fb90 con 0x7f1d940cc5d0 2024-09-11T12:50:28.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d9415fa10 con 0x7f1d9413f000 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d9415fd10 con 0x7f1d940cc1f0 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60374/0 (socket says 172.21.15.78:60374) 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 -- 172.21.15.78:0/3875668652 learned_addr learned my addr 172.21.15.78:0/3875668652 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d92800640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 0x7f1d9406e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:28.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc1f0 0x7f1d94155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 -- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc1f0 msgr2=0x7f1d94155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc1f0 0x7f1d94155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 -- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 msgr2=0x7f1d9406e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 0x7f1d9406e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 -- 172.21.15.78:0/3875668652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d94132b00 con 0x7f1d940cc5d0 2024-09-11T12:50:28.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d92800640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 0x7f1d9406e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d91400640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000b550 tx=0x7f1d8000ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80014070 con 0x7f1d940cc5d0 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d80010980 con 0x7f1d940cc5d0 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d941f8740 con 0x7f1d940cc5d0 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d80010dd0 con 0x7f1d940cc5d0 2024-09-11T12:50:28.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d941f8d00 con 0x7f1d940cc5d0 2024-09-11T12:50:28.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.206+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d94160240 con 0x7f1d940cc5d0 2024-09-11T12:50:28.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.210+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f1d80017070 con 0x7f1d940cc5d0 2024-09-11T12:50:28.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.210+0000 7f1d8a000640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 0x7f1d74073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.210+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5709+0+0 (secure 0 0 0) 0x7f1d80091bd0 con 0x7f1d940cc5d0 2024-09-11T12:50:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.210+0000 7f1d91e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 0x7f1d74073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.210+0000 7f1d91e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 0x7f1d74073b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d8401b070 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.214+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1d8005d350 con 0x7f1d940cc5d0 2024-09-11T12:50:28.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:28.418+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1d9413bf70 con 0x7f1d940cc5d0 2024-09-11T12:50:29.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d8a000640 1 -- 172.21.15.78:0/3875668652 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) ==== 118+0+0 (secure 0 0 0) 0x7f1d800618a0 con 0x7f1d940cc5d0 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 msgr2=0x7f1d74073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 0x7f1d74073b40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f1d840af000 tx=0x7f1d8401b070 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 msgr2=0x7f1d941f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1d8000b550 tx=0x7f1d8000ba10 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 shutdown_connections 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d74071680 0x7f1d74073b40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d9413f000 0x7f1d9406e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d940cc5d0 0x7f1d941f7d60 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 --2- 172.21.15.78:0/3875668652 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d940cc1f0 0x7f1d94155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 >> 172.21.15.78:0/3875668652 conn(0x7f1d94071dd0 msgr2=0x7f1d94131fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 shutdown_connections 2024-09-11T12:50:29.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.338+0000 7f1d98e00640 1 -- 172.21.15.78:0/3875668652 wait complete. 2024-09-11T12:50:29.367 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:50:29.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- 172.21.15.78:0/3665478226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3640cf6e0 msgr2=0x7fe3640734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- 172.21.15.78:0/3665478226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3640cf6e0 0x7fe3640734d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe364069110 tx=0x7fe35c01cb80 comp rx=0 tx=0).stop 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- 172.21.15.78:0/3665478226 shutdown_connections 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- 172.21.15.78:0/3665478226 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe3640cf6e0 0x7fe3640734d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- 172.21.15.78:0/3665478226 >> v2:172.21.15.157:3300/0 conn(0x7fe364144640 0x7fe364072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- 172.21.15.78:0/3665478226 >> 172.21.15.78:0/3665478226 conn(0x7fe3640d0410 msgr2=0x7fe3640d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- 172.21.15.78:0/3665478226 shutdown_connections 2024-09-11T12:50:29.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- 172.21.15.78:0/3665478226 wait complete. 2024-09-11T12:50:29.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 Processor -- start 2024-09-11T12:50:29.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- start start 2024-09-11T12:50:29.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:29.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 0x7fe36413d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:29.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 0x7fe36415e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:29.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe364168e90 con 0x7fe364071d90 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe364168d10 con 0x7fe3640cf6e0 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe36a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe364169010 con 0x7fe364144640 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe363400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe363400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60392/0 (socket says 172.21.15.78:60392) 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe363400640 1 -- 172.21.15.78:0/4151488624 learned_addr learned my addr 172.21.15.78:0/4151488624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe363e00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 0x7fe36415e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.514+0000 7fe362a00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 0x7fe36413d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 -- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 msgr2=0x7fe36415e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 0x7fe36415e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 -- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 msgr2=0x7fe36413d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 0x7fe36413d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 -- 172.21.15.78:0/4151488624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36413dd60 con 0x7fe364071d90 2024-09-11T12:50:29.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363e00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 0x7fe36415e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe362a00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 0x7fe36413d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe363400640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3540058d0 tx=0x7fe35400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe354017070 con 0x7fe364071d90 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe354009d20 con 0x7fe364071d90 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36413b2c0 con 0x7fe364071d90 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3540106c0 con 0x7fe364071d90 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36413bda0 con 0x7fe364071d90 2024-09-11T12:50:29.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36415f590 con 0x7fe364071d90 2024-09-11T12:50:29.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fe35401a070 con 0x7fe364071d90 2024-09-11T12:50:29.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 0x7fe344073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:29.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.518+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fe354091e60 con 0x7fe364071d90 2024-09-11T12:50:29.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.522+0000 7fe362a00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 0x7fe344073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:29.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.522+0000 7fe362a00640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 0x7fe344073b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe364159c70 tx=0x7fe350009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:29.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.522+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe35405dcc0 con 0x7fe364071d90 2024-09-11T12:50:29.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:29.722+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe3640d0920 con 0x7fe364071d90 2024-09-11T12:50:30.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.338+0000 7fe35b400640 1 -- 172.21.15.78:0/4151488624 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) ==== 122+0+0 (secure 0 0 0) 0x7fe35405d5c0 con 0x7fe364071d90 2024-09-11T12:50:30.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 msgr2=0x7fe344073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 0x7fe344073b40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe364159c70 tx=0x7fe350009040 comp rx=0 tx=0).stop 2024-09-11T12:50:30.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 msgr2=0x7fe364158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe3540058d0 tx=0x7fe35400dde0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 shutdown_connections 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe344071680 0x7fe344073b40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe364144640 0x7fe36415e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3640cf6e0 0x7fe36413d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 --2- 172.21.15.78:0/4151488624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe364071d90 0x7fe364158df0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 >> 172.21.15.78:0/4151488624 conn(0x7fe3640d0410 msgr2=0x7fe364165f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 shutdown_connections 2024-09-11T12:50:30.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.342+0000 7fe36a800640 1 -- 172.21.15.78:0/4151488624 wait complete. 2024-09-11T12:50:30.366 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:50:30.367 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:30.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 -- 172.21.15.78:0/2211487964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 msgr2=0x7fcac80736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 --2- 172.21.15.78:0/2211487964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac80736d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcab4009840 tx=0x7fcab401c6b0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 -- 172.21.15.78:0/2211487964 shutdown_connections 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 --2- 172.21.15.78:0/2211487964 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac80736d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 -- 172.21.15.78:0/2211487964 >> 172.21.15.78:0/2211487964 conn(0x7fcac806ca80 msgr2=0x7fcac80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 -- 172.21.15.78:0/2211487964 shutdown_connections 2024-09-11T12:50:30.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.502+0000 7fcacce00640 1 -- 172.21.15.78:0/2211487964 wait complete. 2024-09-11T12:50:30.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 Processor -- start 2024-09-11T12:50:30.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- start start 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac81451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 0x7fcac8155ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8169210 con 0x7fcac8157ae0 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcac8169090 con 0x7fcac8070a00 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcac8169390 con 0x7fcac80b4db0 2024-09-11T12:50:30.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60406/0 (socket says 172.21.15.78:60406) 2024-09-11T12:50:30.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 -- 172.21.15.78:0/648200935 learned_addr learned my addr 172.21.15.78:0/648200935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 -- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 msgr2=0x7fcac8155ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac5e00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac81451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac5400640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 0x7fcac8155ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 0x7fcac8155ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 -- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 msgr2=0x7fcac81451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac81451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 -- 172.21.15.78:0/648200935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcac81f8330 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac5400640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 0x7fcac8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac5e00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcac6800640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcac00015b0 tx=0x7fcac000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0016020 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcac000ab60 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcac81f8610 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac0012420 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.506+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcac8151200 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcac80736d0 con 0x7fcac8157ae0 2024-09-11T12:50:30.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fcac0019020 con 0x7fcac8157ae0 2024-09-11T12:50:30.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcabe000640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 0x7fcaa8073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fcac0091ec0 con 0x7fcac8157ae0 2024-09-11T12:50:30.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcac5e00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 0x7fcaa8073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.510+0000 7fcac5e00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 0x7fcaa8073b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000f70 tx=0x7fcab8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:30.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.514+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcac0020030 con 0x7fcac8157ae0 2024-09-11T12:50:30.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.714+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcac8073940 con 0x7fcac8157ae0 2024-09-11T12:50:30.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.714+0000 7fcabe000640 1 -- 172.21.15.78:0/648200935 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5223 (secure 0 0 0) 0x7fcac005ddc0 con 0x7fcac8157ae0 2024-09-11T12:50:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:30.720 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-09-11T12:50:30:342061+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:30.720 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 msgr2=0x7fcaa8073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 0x7fcaa8073b70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcab8000f70 tx=0x7fcab8009040 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 msgr2=0x7fcac81f7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcac00015b0 tx=0x7fcac000cd20 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 shutdown_connections 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcaa80716b0 0x7fcaa8073b70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcac8157ae0 0x7fcac81f7a50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcac80b4db0 0x7fcac8155ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 --2- 172.21.15.78:0/648200935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcac8070a00 0x7fcac81451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 >> 172.21.15.78:0/648200935 conn(0x7fcac806ca80 msgr2=0x7fcac8072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:30.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 shutdown_connections 2024-09-11T12:50:30.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.718+0000 7fcacce00640 1 -- 172.21.15.78:0/648200935 wait complete. 2024-09-11T12:50:30.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- 172.21.15.78:0/4190345288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45040cf760 msgr2=0x7f4504074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- 172.21.15.78:0/4190345288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45040cf760 0x7f4504074260 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4504072f50 tx=0x7f44f401c7d0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- 172.21.15.78:0/4190345288 shutdown_connections 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- 172.21.15.78:0/4190345288 >> v2:172.21.15.157:3300/0 conn(0x7f450406fc50 0x7f45040700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- 172.21.15.78:0/4190345288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45040cf760 0x7f4504074260 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- 172.21.15.78:0/4190345288 >> 172.21.15.78:0/4190345288 conn(0x7f450406ca80 msgr2=0x7f45040b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- 172.21.15.78:0/4190345288 shutdown_connections 2024-09-11T12:50:30.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- 172.21.15.78:0/4190345288 wait complete. 2024-09-11T12:50:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 Processor -- start 2024-09-11T12:50:30.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- start start 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 0x7f450413b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 0x7f45041521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f450415f290 con 0x7f450406fc50 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f450415f110 con 0x7f45040cf760 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4508a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4504151010 con 0x7f450415aec0 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4501e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4501e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60424/0 (socket says 172.21.15.78:60424) 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4501e00640 1 -- 172.21.15.78:0/3060718288 learned_addr learned my addr 172.21.15.78:0/3060718288 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.878+0000 7f4501400640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 0x7f450413b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4502800640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 0x7f45041521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 -- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 msgr2=0x7f45041521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 0x7f45041521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 -- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 msgr2=0x7f450413b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 0x7f450413b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 -- 172.21.15.78:0/3060718288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040b04c0 con 0x7f450406fc50 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501400640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 0x7f450413b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4502800640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 0x7f45041521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501e00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44f401ccb0 tx=0x7f44f40a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40bb070 con 0x7f450406fc50 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44f40043b0 con 0x7f450406fc50 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450415cf70 con 0x7f450406fc50 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f40b2040 con 0x7f450406fc50 2024-09-11T12:50:30.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450415d530 con 0x7f450406fc50 2024-09-11T12:50:30.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45040700c0 con 0x7f450406fc50 2024-09-11T12:50:30.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f44f40be070 con 0x7f450406fc50 2024-09-11T12:50:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 0x7f44e4073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f44f412b650 con 0x7f450406fc50 2024-09-11T12:50:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.882+0000 7f4501400640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 0x7f44e4073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:30.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.886+0000 7f4501400640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 0x7f44e4073b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001230 tx=0x7f44f0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:30.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:30.886+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f44f40163f0 con 0x7f450406fc50 2024-09-11T12:50:31.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.054+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45040cc390 con 0x7f450406fc50 2024-09-11T12:50:31.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.054+0000 7f44fa000640 1 -- 172.21.15.78:0/3060718288 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7f44f40f7430 con 0x7f450406fc50 2024-09-11T12:50:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:31.061 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:28.327657+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:24.824997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:25.670550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:31.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 msgr2=0x7f44e4073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 0x7f44e4073b70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f44f0001230 tx=0x7f44f0009040 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 msgr2=0x7f4504151570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f44f401ccb0 tx=0x7f44f40a5710 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 shutdown_connections 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f44e40716b0 0x7f44e4073b70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f450415aec0 0x7f45041521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f45040cf760 0x7f450413b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 --2- 172.21.15.78:0/3060718288 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f450406fc50 0x7f4504151570 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 >> 172.21.15.78:0/3060718288 conn(0x7f450406ca80 msgr2=0x7f45040ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 shutdown_connections 2024-09-11T12:50:31.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.058+0000 7f4508a00640 1 -- 172.21.15.78:0/3060718288 wait complete. 2024-09-11T12:50:31.083 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:50:31.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- 172.21.15.78:0/3056883161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9524157ae0 msgr2=0x7f9524070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- 172.21.15.78:0/3056883161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9524157ae0 0x7f9524070a00 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f952415dad0 tx=0x7f951401c770 comp rx=0 tx=0).stop 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- 172.21.15.78:0/3056883161 shutdown_connections 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- 172.21.15.78:0/3056883161 >> v2:172.21.15.157:3300/0 conn(0x7f95240b4db0 0x7f95240b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- 172.21.15.78:0/3056883161 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9524157ae0 0x7f9524070a00 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- 172.21.15.78:0/3056883161 >> 172.21.15.78:0/3056883161 conn(0x7f952406ca80 msgr2=0x7f95240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- 172.21.15.78:0/3056883161 shutdown_connections 2024-09-11T12:50:31.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- 172.21.15.78:0/3056883161 wait complete. 2024-09-11T12:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 Processor -- start 2024-09-11T12:50:31.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- start start 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b4db0 0x7f952414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9524157ae0 0x7f95240ce390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f952406f250 con 0x7f95240736d0 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f952406f0d0 con 0x7f95240b4db0 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9529400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f952406f3d0 con 0x7f9524157ae0 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60452/0 (socket says 172.21.15.78:60452) 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 -- 172.21.15.78:0/432722511 learned_addr learned my addr 172.21.15.78:0/432722511 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:31.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9521e00640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b4db0 0x7f952414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9523200640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9524157ae0 0x7f95240ce390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 -- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9524157ae0 msgr2=0x7f95240ce390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9524157ae0 0x7f95240ce390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 -- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b4db0 msgr2=0x7f952414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.222+0000 7f9522800640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b4db0 0x7f952414c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f9522800640 1 -- 172.21.15.78:0/432722511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f952415a360 con 0x7f95240736d0 2024-09-11T12:50:31.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f9522800640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f951401a3a0 tx=0x7f95140a8610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95140be070 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95140a8b20 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f952415a080 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9514005040 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9524155560 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9524073b00 con 0x7f95240736d0 2024-09-11T12:50:31.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f95140c1070 con 0x7f95240736d0 2024-09-11T12:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 0x7f9504073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.226+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f951412bed0 con 0x7f95240736d0 2024-09-11T12:50:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.230+0000 7f9521e00640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 0x7f9504073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.230+0000 7f9521e00640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 0x7f9504073b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f952415ec00 tx=0x7f9510009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:31.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.230+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f95140ac030 con 0x7f95240736d0 2024-09-11T12:50:31.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.402+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95241669a0 con 0x7f9504071680 2024-09-11T12:50:31.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.406+0000 7f951aa00640 1 -- 172.21.15.78:0/432722511 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251605 (secure 0 0 0) 0x7f95241669a0 con 0x7f9504071680 2024-09-11T12:50:31.415 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:31.417 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 msgr2=0x7f9504073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 0x7f9504073b40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f952415ec00 tx=0x7f9510009040 comp rx=0 tx=0).stop 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 msgr2=0x7f95240cf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f951401a3a0 tx=0x7f95140a8610 comp rx=0 tx=0).stop 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 shutdown_connections 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9504071680 0x7f9504073b40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9524157ae0 0x7f95240ce390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95240b4db0 0x7f952414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 --2- 172.21.15.78:0/432722511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95240736d0 0x7f95240cf260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.410+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 >> 172.21.15.78:0/432722511 conn(0x7f952406ca80 msgr2=0x7f952415af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:31.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.414+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 shutdown_connections 2024-09-11T12:50:31.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.414+0000 7f9529400640 1 -- 172.21.15.78:0/432722511 wait complete. 2024-09-11T12:50:31.436 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-11T12:50:31.116340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236808,"kb_used_data":3912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737912,"statfs":{"total":773094113280,"available":772851621888,"internally_reserved":0,"allocated":4005888,"data_stored":2229312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807871"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.350863+0000","last_change":"2024-09-11T12:50:27.348834+0000","last_active":"2024-09-11T12:50:27.350863+0000","last_peered":"2024-09-11T12:50:27.350863+0000","last_clean":"2024-09-11T12:50:27.350863+0000","last_became_active":"2024-09-11T12:50:27.348574+0000","last_became_peered":"2024-09-11T12:50:27.348574+0000","last_unstale":"2024-09-11T12:50:27.350863+0000","last_undegraded":"2024-09-11T12:50:27.350863+0000","last_fullsized":"2024-09-11T12:50:27.350863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341400+0000","last_change":"2024-09-11T12:50:27.336993+0000","last_active":"2024-09-11T12:50:27.341400+0000","last_peered":"2024-09-11T12:50:27.341400+0000","last_clean":"2024-09-11T12:50:27.341400+0000","last_became_active":"2024-09-11T12:50:27.336588+0000","last_became_peered":"2024-09-11T12:50:27.336588+0000","last_unstale":"2024-09-11T12:50:27.341400+0000","last_undegraded":"2024-09-11T12:50:27.341400+0000","last_fullsized":"2024-09-11T12:50:27.341400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351364+0000","last_change":"2024-09-11T12:50:27.342867+0000","last_active":"2024-09-11T12:50:27.351364+0000","last_peered":"2024-09-11T12:50:27.351364+0000","last_clean":"2024-09-11T12:50:27.351364+0000","last_became_active":"2024-09-11T12:50:27.342576+0000","last_became_peered":"2024-09-11T12:50:27.342576+0000","last_unstale":"2024-09-11T12:50:27.351364+0000","last_undegraded":"2024-09-11T12:50:27.351364+0000","last_fullsized":"2024-09-11T12:50:27.351364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360719+0000","last_change":"2024-09-11T12:50:27.357904+0000","last_active":"2024-09-11T12:50:27.360719+0000","last_peered":"2024-09-11T12:50:27.360719+0000","last_clean":"2024-09-11T12:50:27.360719+0000","last_became_active":"2024-09-11T12:50:27.356836+0000","last_became_peered":"2024-09-11T12:50:27.356836+0000","last_unstale":"2024-09-11T12:50:27.360719+0000","last_undegraded":"2024-09-11T12:50:27.360719+0000","last_fullsized":"2024-09-11T12:50:27.360719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345686+0000","last_change":"2024-09-11T12:50:27.341619+0000","last_active":"2024-09-11T12:50:27.345686+0000","last_peered":"2024-09-11T12:50:27.345686+0000","last_clean":"2024-09-11T12:50:27.345686+0000","last_became_active":"2024-09-11T12:50:27.341345+0000","last_became_peered":"2024-09-11T12:50:27.341345+0000","last_unstale":"2024-09-11T12:50:27.345686+0000","last_undegraded":"2024-09-11T12:50:27.345686+0000","last_fullsized":"2024-09-11T12:50:27.345686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:44.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.339528+0000","last_change":"2024-09-11T12:50:27.335087+0000","last_active":"2024-09-11T12:50:27.339528+0000","last_peered":"2024-09-11T12:50:27.339528+0000","last_clean":"2024-09-11T12:50:27.339528+0000","last_became_active":"2024-09-11T12:50:27.334790+0000","last_became_peered":"2024-09-11T12:50:27.334790+0000","last_unstale":"2024-09-11T12:50:27.339528+0000","last_undegraded":"2024-09-11T12:50:27.339528+0000","last_fullsized":"2024-09-11T12:50:27.339528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364813+0000","last_change":"2024-09-11T12:50:27.355196+0000","last_active":"2024-09-11T12:50:27.364813+0000","last_peered":"2024-09-11T12:50:27.364813+0000","last_clean":"2024-09-11T12:50:27.364813+0000","last_became_active":"2024-09-11T12:50:27.353288+0000","last_became_peered":"2024-09-11T12:50:27.353288+0000","last_unstale":"2024-09-11T12:50:27.364813+0000","last_undegraded":"2024-09-11T12:50:27.364813+0000","last_fullsized":"2024-09-11T12:50:27.364813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364931+0000","last_change":"2024-09-11T12:50:27.358075+0000","last_active":"2024-09-11T12:50:27.364931+0000","last_peered":"2024-09-11T12:50:27.364931+0000","last_clean":"2024-09-11T12:50:27.364931+0000","last_became_active":"2024-09-11T12:50:27.357851+0000","last_became_peered":"2024-09-11T12:50:27.357851+0000","last_unstale":"2024-09-11T12:50:27.364931+0000","last_undegraded":"2024-09-11T12:50:27.364931+0000","last_fullsized":"2024-09-11T12:50:27.364931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352225+0000","last_change":"2024-09-11T12:50:27.343726+0000","last_active":"2024-09-11T12:50:27.352225+0000","last_peered":"2024-09-11T12:50:27.352225+0000","last_clean":"2024-09-11T12:50:27.352225+0000","last_became_active":"2024-09-11T12:50:27.343450+0000","last_became_peered":"2024-09-11T12:50:27.343450+0000","last_unstale":"2024-09-11T12:50:27.352225+0000","last_undegraded":"2024-09-11T12:50:27.352225+0000","last_fullsized":"2024-09-11T12:50:27.352225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342383+0000","last_change":"2024-09-11T12:50:27.339989+0000","last_active":"2024-09-11T12:50:27.342383+0000","last_peered":"2024-09-11T12:50:27.342383+0000","last_clean":"2024-09-11T12:50:27.342383+0000","last_became_active":"2024-09-11T12:50:27.339673+0000","last_became_peered":"2024-09-11T12:50:27.339673+0000","last_unstale":"2024-09-11T12:50:27.342383+0000","last_undegraded":"2024-09-11T12:50:27.342383+0000","last_fullsized":"2024-09-11T12:50:27.342383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365270+0000","last_change":"2024-09-11T12:50:27.363103+0000","last_active":"2024-09-11T12:50:27.365270+0000","last_peered":"2024-09-11T12:50:27.365270+0000","last_clean":"2024-09-11T12:50:27.365270+0000","last_became_active":"2024-09-11T12:50:27.362746+0000","last_became_peered":"2024-09-11T12:50:27.362746+0000","last_unstale":"2024-09-11T12:50:27.365270+0000","last_undegraded":"2024-09-11T12:50:27.365270+0000","last_fullsized":"2024-09-11T12:50:27.365270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347758+0000","last_change":"2024-09-11T12:50:27.329779+0000","last_active":"2024-09-11T12:50:27.347758+0000","last_peered":"2024-09-11T12:50:27.347758+0000","last_clean":"2024-09-11T12:50:27.347758+0000","last_became_active":"2024-09-11T12:50:27.327796+0000","last_became_peered":"2024-09-11T12:50:27.327796+0000","last_unstale":"2024-09-11T12:50:27.347758+0000","last_undegraded":"2024-09-11T12:50:27.347758+0000","last_fullsized":"2024-09-11T12:50:27.347758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352853+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.352853+0000","last_peered":"2024-09-11T12:50:27.352853+0000","last_clean":"2024-09-11T12:50:27.352853+0000","last_became_active":"2024-09-11T12:50:27.344834+0000","last_became_peered":"2024-09-11T12:50:27.344834+0000","last_unstale":"2024-09-11T12:50:27.352853+0000","last_undegraded":"2024-09-11T12:50:27.352853+0000","last_fullsized":"2024-09-11T12:50:27.352853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:48.794525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341535+0000","last_change":"2024-09-11T12:50:27.339363+0000","last_active":"2024-09-11T12:50:27.341535+0000","last_peered":"2024-09-11T12:50:27.341535+0000","last_clean":"2024-09-11T12:50:27.341535+0000","last_became_active":"2024-09-11T12:50:27.339131+0000","last_became_peered":"2024-09-11T12:50:27.339131+0000","last_unstale":"2024-09-11T12:50:27.341535+0000","last_undegraded":"2024-09-11T12:50:27.341535+0000","last_fullsized":"2024-09-11T12:50:27.341535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344599+0000","last_change":"2024-09-11T12:50:27.335582+0000","last_active":"2024-09-11T12:50:27.344599+0000","last_peered":"2024-09-11T12:50:27.344599+0000","last_clean":"2024-09-11T12:50:27.344599+0000","last_became_active":"2024-09-11T12:50:27.335002+0000","last_became_peered":"2024-09-11T12:50:27.335002+0000","last_unstale":"2024-09-11T12:50:27.344599+0000","last_undegraded":"2024-09-11T12:50:27.344599+0000","last_fullsized":"2024-09-11T12:50:27.344599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365195+0000","last_change":"2024-09-11T12:50:27.363062+0000","last_active":"2024-09-11T12:50:27.365195+0000","last_peered":"2024-09-11T12:50:27.365195+0000","last_clean":"2024-09-11T12:50:27.365195+0000","last_became_active":"2024-09-11T12:50:27.362788+0000","last_became_peered":"2024-09-11T12:50:27.362788+0000","last_unstale":"2024-09-11T12:50:27.365195+0000","last_undegraded":"2024-09-11T12:50:27.365195+0000","last_fullsized":"2024-09-11T12:50:27.365195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364886+0000","last_change":"2024-09-11T12:50:27.357790+0000","last_active":"2024-09-11T12:50:27.364886+0000","last_peered":"2024-09-11T12:50:27.364886+0000","last_clean":"2024-09-11T12:50:27.364886+0000","last_became_active":"2024-09-11T12:50:27.353639+0000","last_became_peered":"2024-09-11T12:50:27.353639+0000","last_unstale":"2024-09-11T12:50:27.364886+0000","last_undegraded":"2024-09-11T12:50:27.364886+0000","last_fullsized":"2024-09-11T12:50:27.364886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341377+0000","last_change":"2024-09-11T12:50:27.336825+0000","last_active":"2024-09-11T12:50:27.341377+0000","last_peered":"2024-09-11T12:50:27.341377+0000","last_clean":"2024-09-11T12:50:27.341377+0000","last_became_active":"2024-09-11T12:50:27.336241+0000","last_became_peered":"2024-09-11T12:50:27.336241+0000","last_unstale":"2024-09-11T12:50:27.341377+0000","last_undegraded":"2024-09-11T12:50:27.341377+0000","last_fullsized":"2024-09-11T12:50:27.341377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346581+0000","last_change":"2024-09-11T12:50:27.344954+0000","last_active":"2024-09-11T12:50:27.346581+0000","last_peered":"2024-09-11T12:50:27.346581+0000","last_clean":"2024-09-11T12:50:27.346581+0000","last_became_active":"2024-09-11T12:50:27.344681+0000","last_became_peered":"2024-09-11T12:50:27.344681+0000","last_unstale":"2024-09-11T12:50:27.346581+0000","last_undegraded":"2024-09-11T12:50:27.346581+0000","last_fullsized":"2024-09-11T12:50:27.346581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357472+0000","last_change":"2024-09-11T12:50:27.352798+0000","last_active":"2024-09-11T12:50:27.357472+0000","last_peered":"2024-09-11T12:50:27.357472+0000","last_clean":"2024-09-11T12:50:27.357472+0000","last_became_active":"2024-09-11T12:50:27.349194+0000","last_became_peered":"2024-09-11T12:50:27.349194+0000","last_unstale":"2024-09-11T12:50:27.357472+0000","last_undegraded":"2024-09-11T12:50:27.357472+0000","last_fullsized":"2024-09-11T12:50:27.357472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:32.390636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351801+0000","last_change":"2024-09-11T12:50:27.343151+0000","last_active":"2024-09-11T12:50:27.351801+0000","last_peered":"2024-09-11T12:50:27.351801+0000","last_clean":"2024-09-11T12:50:27.351801+0000","last_became_active":"2024-09-11T12:50:27.339404+0000","last_became_peered":"2024-09-11T12:50:27.339404+0000","last_unstale":"2024-09-11T12:50:27.351801+0000","last_undegraded":"2024-09-11T12:50:27.351801+0000","last_fullsized":"2024-09-11T12:50:27.351801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.336093+0000","last_change":"2024-09-11T12:50:27.330086+0000","last_active":"2024-09-11T12:50:27.336093+0000","last_peered":"2024-09-11T12:50:27.336093+0000","last_clean":"2024-09-11T12:50:27.336093+0000","last_became_active":"2024-09-11T12:50:27.329846+0000","last_became_peered":"2024-09-11T12:50:27.329846+0000","last_unstale":"2024-09-11T12:50:27.336093+0000","last_undegraded":"2024-09-11T12:50:27.336093+0000","last_fullsized":"2024-09-11T12:50:27.336093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:12.211705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324523+0000","last_change":"2024-09-11T12:50:26.324523+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324523+0000","last_undegraded":"2024-09-11T12:50:26.324523+0000","last_fullsized":"2024-09-11T12:50:26.324523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343833+0000","last_change":"2024-09-11T12:50:27.337469+0000","last_active":"2024-09-11T12:50:27.343833+0000","last_peered":"2024-09-11T12:50:27.343833+0000","last_clean":"2024-09-11T12:50:27.343833+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.343833+0000","last_undegraded":"2024-09-11T12:50:27.343833+0000","last_fullsized":"2024-09-11T12:50:27.343833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359468+0000","last_change":"2024-09-11T12:50:27.355625+0000","last_active":"2024-09-11T12:50:27.359468+0000","last_peered":"2024-09-11T12:50:27.359468+0000","last_clean":"2024-09-11T12:50:27.359468+0000","last_became_active":"2024-09-11T12:50:27.355274+0000","last_became_peered":"2024-09-11T12:50:27.355274+0000","last_unstale":"2024-09-11T12:50:27.359468+0000","last_undegraded":"2024-09-11T12:50:27.359468+0000","last_fullsized":"2024-09-11T12:50:27.359468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359556+0000","last_change":"2024-09-11T12:50:27.356361+0000","last_active":"2024-09-11T12:50:27.359556+0000","last_peered":"2024-09-11T12:50:27.359556+0000","last_clean":"2024-09-11T12:50:27.359556+0000","last_became_active":"2024-09-11T12:50:27.356074+0000","last_became_peered":"2024-09-11T12:50:27.356074+0000","last_unstale":"2024-09-11T12:50:27.359556+0000","last_undegraded":"2024-09-11T12:50:27.359556+0000","last_fullsized":"2024-09-11T12:50:27.359556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359308+0000","last_change":"2024-09-11T12:50:27.357026+0000","last_active":"2024-09-11T12:50:27.359308+0000","last_peered":"2024-09-11T12:50:27.359308+0000","last_clean":"2024-09-11T12:50:27.359308+0000","last_became_active":"2024-09-11T12:50:27.356785+0000","last_became_peered":"2024-09-11T12:50:27.356785+0000","last_unstale":"2024-09-11T12:50:27.359308+0000","last_undegraded":"2024-09-11T12:50:27.359308+0000","last_fullsized":"2024-09-11T12:50:27.359308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347829+0000","last_change":"2024-09-11T12:50:27.336803+0000","last_active":"2024-09-11T12:50:27.347829+0000","last_peered":"2024-09-11T12:50:27.347829+0000","last_clean":"2024-09-11T12:50:27.347829+0000","last_became_active":"2024-09-11T12:50:27.334739+0000","last_became_peered":"2024-09-11T12:50:27.334739+0000","last_unstale":"2024-09-11T12:50:27.347829+0000","last_undegraded":"2024-09-11T12:50:27.347829+0000","last_fullsized":"2024-09-11T12:50:27.347829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335097+0000","last_change":"2024-09-11T12:50:27.329208+0000","last_active":"2024-09-11T12:50:27.335097+0000","last_peered":"2024-09-11T12:50:27.335097+0000","last_clean":"2024-09-11T12:50:27.335097+0000","last_became_active":"2024-09-11T12:50:27.328939+0000","last_became_peered":"2024-09-11T12:50:27.328939+0000","last_unstale":"2024-09-11T12:50:27.335097+0000","last_undegraded":"2024-09-11T12:50:27.335097+0000","last_fullsized":"2024-09-11T12:50:27.335097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:25.059566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353911+0000","last_change":"2024-09-11T12:50:27.348968+0000","last_active":"2024-09-11T12:50:27.353911+0000","last_peered":"2024-09-11T12:50:27.353911+0000","last_clean":"2024-09-11T12:50:27.353911+0000","last_became_active":"2024-09-11T12:50:27.348693+0000","last_became_peered":"2024-09-11T12:50:27.348693+0000","last_unstale":"2024-09-11T12:50:27.353911+0000","last_undegraded":"2024-09-11T12:50:27.353911+0000","last_fullsized":"2024-09-11T12:50:27.353911+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345469+0000","last_change":"2024-09-11T12:50:27.341331+0000","last_active":"2024-09-11T12:50:27.345469+0000","last_peered":"2024-09-11T12:50:27.345469+0000","last_clean":"2024-09-11T12:50:27.345469+0000","last_became_active":"2024-09-11T12:50:27.341038+0000","last_became_peered":"2024-09-11T12:50:27.341038+0000","last_unstale":"2024-09-11T12:50:27.345469+0000","last_undegraded":"2024-09-11T12:50:27.345469+0000","last_fullsized":"2024-09-11T12:50:27.345469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364720+0000","last_change":"2024-09-11T12:50:27.356104+0000","last_active":"2024-09-11T12:50:27.364720+0000","last_peered":"2024-09-11T12:50:27.364720+0000","last_clean":"2024-09-11T12:50:27.364720+0000","last_became_active":"2024-09-11T12:50:27.355113+0000","last_became_peered":"2024-09-11T12:50:27.355113+0000","last_unstale":"2024-09-11T12:50:27.364720+0000","last_undegraded":"2024-09-11T12:50:27.364720+0000","last_fullsized":"2024-09-11T12:50:27.364720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324419+0000","last_change":"2024-09-11T12:50:26.324419+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324419+0000","last_undegraded":"2024-09-11T12:50:26.324419+0000","last_fullsized":"2024-09-11T12:50:26.324419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365295+0000","last_change":"2024-09-11T12:50:26.343869+0000","last_active":"2024-09-11T12:50:27.365295+0000","last_peered":"2024-09-11T12:50:27.365295+0000","last_clean":"2024-09-11T12:50:27.365295+0000","last_became_active":"2024-09-11T12:50:26.343588+0000","last_became_peered":"2024-09-11T12:50:26.343588+0000","last_unstale":"2024-09-11T12:50:27.365295+0000","last_undegraded":"2024-09-11T12:50:27.365295+0000","last_fullsized":"2024-09-11T12:50:27.365295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344316+0000","last_change":"2024-09-11T12:50:27.340558+0000","last_active":"2024-09-11T12:50:27.344316+0000","last_peered":"2024-09-11T12:50:27.344316+0000","last_clean":"2024-09-11T12:50:27.344316+0000","last_became_active":"2024-09-11T12:50:27.337140+0000","last_became_peered":"2024-09-11T12:50:27.337140+0000","last_unstale":"2024-09-11T12:50:27.344316+0000","last_undegraded":"2024-09-11T12:50:27.344316+0000","last_fullsized":"2024-09-11T12:50:27.344316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.341744+0000","last_change":"2024-09-11T12:50:26.335186+0000","last_active":"2024-09-11T12:50:26.341744+0000","last_peered":"2024-09-11T12:50:26.341744+0000","last_clean":"2024-09-11T12:50:26.341744+0000","last_became_active":"2024-09-11T12:50:26.334825+0000","last_became_peered":"2024-09-11T12:50:26.334825+0000","last_unstale":"2024-09-11T12:50:26.341744+0000","last_undegraded":"2024-09-11T12:50:26.341744+0000","last_fullsized":"2024-09-11T12:50:26.341744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:54.260958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335753+0000","last_change":"2024-09-11T12:50:27.328627+0000","last_active":"2024-09-11T12:50:27.335753+0000","last_peered":"2024-09-11T12:50:27.335753+0000","last_clean":"2024-09-11T12:50:27.335753+0000","last_became_active":"2024-09-11T12:50:27.328292+0000","last_became_peered":"2024-09-11T12:50:27.328292+0000","last_unstale":"2024-09-11T12:50:27.335753+0000","last_undegraded":"2024-09-11T12:50:27.335753+0000","last_fullsized":"2024-09-11T12:50:27.335753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:27.421789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360603+0000","last_change":"2024-09-11T12:50:26.342258+0000","last_active":"2024-09-11T12:50:27.360603+0000","last_peered":"2024-09-11T12:50:27.360603+0000","last_clean":"2024-09-11T12:50:27.360603+0000","last_became_active":"2024-09-11T12:50:26.341986+0000","last_became_peered":"2024-09-11T12:50:26.341986+0000","last_unstale":"2024-09-11T12:50:27.360603+0000","last_undegraded":"2024-09-11T12:50:27.360603+0000","last_fullsized":"2024-09-11T12:50:27.360603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:46.357747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359368+0000","last_change":"2024-09-11T12:50:27.357376+0000","last_active":"2024-09-11T12:50:27.359368+0000","last_peered":"2024-09-11T12:50:27.359368+0000","last_clean":"2024-09-11T12:50:27.359368+0000","last_became_active":"2024-09-11T12:50:27.357016+0000","last_became_peered":"2024-09-11T12:50:27.357016+0000","last_unstale":"2024-09-11T12:50:27.359368+0000","last_undegraded":"2024-09-11T12:50:27.359368+0000","last_fullsized":"2024-09-11T12:50:27.359368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360981+0000","last_change":"2024-09-11T12:50:26.343854+0000","last_active":"2024-09-11T12:50:27.360981+0000","last_peered":"2024-09-11T12:50:27.360981+0000","last_clean":"2024-09-11T12:50:27.360981+0000","last_became_active":"2024-09-11T12:50:26.341771+0000","last_became_peered":"2024-09-11T12:50:26.341771+0000","last_unstale":"2024-09-11T12:50:27.360981+0000","last_undegraded":"2024-09-11T12:50:27.360981+0000","last_fullsized":"2024-09-11T12:50:27.360981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:51.008327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364638+0000","last_change":"2024-09-11T12:50:27.352862+0000","last_active":"2024-09-11T12:50:27.364638+0000","last_peered":"2024-09-11T12:50:27.364638+0000","last_clean":"2024-09-11T12:50:27.364638+0000","last_became_active":"2024-09-11T12:50:27.352467+0000","last_became_peered":"2024-09-11T12:50:27.352467+0000","last_unstale":"2024-09-11T12:50:27.364638+0000","last_undegraded":"2024-09-11T12:50:27.364638+0000","last_fullsized":"2024-09-11T12:50:27.364638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363377+0000","last_change":"2024-09-11T12:50:26.342659+0000","last_active":"2024-09-11T12:50:27.363377+0000","last_peered":"2024-09-11T12:50:27.363377+0000","last_clean":"2024-09-11T12:50:27.363377+0000","last_became_active":"2024-09-11T12:50:26.342331+0000","last_became_peered":"2024-09-11T12:50:26.342331+0000","last_unstale":"2024-09-11T12:50:27.363377+0000","last_undegraded":"2024-09-11T12:50:27.363377+0000","last_fullsized":"2024-09-11T12:50:27.363377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:18.698135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352409+0000","last_change":"2024-09-11T12:50:27.347574+0000","last_active":"2024-09-11T12:50:27.352409+0000","last_peered":"2024-09-11T12:50:27.352409+0000","last_clean":"2024-09-11T12:50:27.352409+0000","last_became_active":"2024-09-11T12:50:27.347302+0000","last_became_peered":"2024-09-11T12:50:27.347302+0000","last_unstale":"2024-09-11T12:50:27.352409+0000","last_undegraded":"2024-09-11T12:50:27.352409+0000","last_fullsized":"2024-09-11T12:50:27.352409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.366007+0000","last_change":"2024-09-11T12:50:26.341601+0000","last_active":"2024-09-11T12:50:27.366007+0000","last_peered":"2024-09-11T12:50:27.366007+0000","last_clean":"2024-09-11T12:50:27.366007+0000","last_became_active":"2024-09-11T12:50:26.341238+0000","last_became_peered":"2024-09-11T12:50:26.341238+0000","last_unstale":"2024-09-11T12:50:27.366007+0000","last_undegraded":"2024-09-11T12:50:27.366007+0000","last_fullsized":"2024-09-11T12:50:27.366007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:46.542517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335996+0000","last_change":"2024-09-11T12:50:27.333846+0000","last_active":"2024-09-11T12:50:27.335996+0000","last_peered":"2024-09-11T12:50:27.335996+0000","last_clean":"2024-09-11T12:50:27.335996+0000","last_became_active":"2024-09-11T12:50:27.333500+0000","last_became_peered":"2024-09-11T12:50:27.333500+0000","last_unstale":"2024-09-11T12:50:27.335996+0000","last_undegraded":"2024-09-11T12:50:27.335996+0000","last_fullsized":"2024-09-11T12:50:27.335996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323107+0000","last_change":"2024-09-11T12:50:26.339433+0000","last_active":"2024-09-11T12:50:27.323107+0000","last_peered":"2024-09-11T12:50:27.323107+0000","last_clean":"2024-09-11T12:50:27.323107+0000","last_became_active":"2024-09-11T12:50:26.338991+0000","last_became_peered":"2024-09-11T12:50:26.338991+0000","last_unstale":"2024-09-11T12:50:27.323107+0000","last_undegraded":"2024-09-11T12:50:27.323107+0000","last_fullsized":"2024-09-11T12:50:27.323107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354941+0000","last_change":"2024-09-11T12:50:27.351561+0000","last_active":"2024-09-11T12:50:27.354941+0000","last_peered":"2024-09-11T12:50:27.354941+0000","last_clean":"2024-09-11T12:50:27.354941+0000","last_became_active":"2024-09-11T12:50:27.351277+0000","last_became_peered":"2024-09-11T12:50:27.351277+0000","last_unstale":"2024-09-11T12:50:27.354941+0000","last_undegraded":"2024-09-11T12:50:27.354941+0000","last_fullsized":"2024-09-11T12:50:27.354941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323830+0000","last_change":"2024-09-11T12:50:26.337847+0000","last_active":"2024-09-11T12:50:27.323830+0000","last_peered":"2024-09-11T12:50:27.323830+0000","last_clean":"2024-09-11T12:50:27.323830+0000","last_became_active":"2024-09-11T12:50:26.337533+0000","last_became_peered":"2024-09-11T12:50:26.337533+0000","last_unstale":"2024-09-11T12:50:27.323830+0000","last_undegraded":"2024-09-11T12:50:27.323830+0000","last_fullsized":"2024-09-11T12:50:27.323830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:06.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346859+0000","last_change":"2024-09-11T12:50:27.330135+0000","last_active":"2024-09-11T12:50:27.346859+0000","last_peered":"2024-09-11T12:50:27.346859+0000","last_clean":"2024-09-11T12:50:27.346859+0000","last_became_active":"2024-09-11T12:50:27.329324+0000","last_became_peered":"2024-09-11T12:50:27.329324+0000","last_unstale":"2024-09-11T12:50:27.346859+0000","last_undegraded":"2024-09-11T12:50:27.346859+0000","last_fullsized":"2024-09-11T12:50:27.346859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324111+0000","last_change":"2024-09-11T12:50:26.343684+0000","last_active":"2024-09-11T12:50:27.324111+0000","last_peered":"2024-09-11T12:50:27.324111+0000","last_clean":"2024-09-11T12:50:27.324111+0000","last_became_active":"2024-09-11T12:50:26.343386+0000","last_became_peered":"2024-09-11T12:50:26.343386+0000","last_unstale":"2024-09-11T12:50:27.324111+0000","last_undegraded":"2024-09-11T12:50:27.324111+0000","last_fullsized":"2024-09-11T12:50:27.324111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:06.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.348808+0000","last_change":"2024-09-11T12:50:27.346100+0000","last_active":"2024-09-11T12:50:27.348808+0000","last_peered":"2024-09-11T12:50:27.348808+0000","last_clean":"2024-09-11T12:50:27.348808+0000","last_became_active":"2024-09-11T12:50:27.345380+0000","last_became_peered":"2024-09-11T12:50:27.345380+0000","last_unstale":"2024-09-11T12:50:27.348808+0000","last_undegraded":"2024-09-11T12:50:27.348808+0000","last_fullsized":"2024-09-11T12:50:27.348808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345851+0000","last_change":"2024-09-11T12:50:26.340838+0000","last_active":"2024-09-11T12:50:26.345851+0000","last_peered":"2024-09-11T12:50:26.345851+0000","last_clean":"2024-09-11T12:50:26.345851+0000","last_became_active":"2024-09-11T12:50:26.340531+0000","last_became_peered":"2024-09-11T12:50:26.340531+0000","last_unstale":"2024-09-11T12:50:26.345851+0000","last_undegraded":"2024-09-11T12:50:26.345851+0000","last_fullsized":"2024-09-11T12:50:26.345851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:03.491437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359623+0000","last_change":"2024-09-11T12:50:27.356884+0000","last_active":"2024-09-11T12:50:27.359623+0000","last_peered":"2024-09-11T12:50:27.359623+0000","last_clean":"2024-09-11T12:50:27.359623+0000","last_became_active":"2024-09-11T12:50:27.355050+0000","last_became_peered":"2024-09-11T12:50:27.355050+0000","last_unstale":"2024-09-11T12:50:27.359623+0000","last_undegraded":"2024-09-11T12:50:27.359623+0000","last_fullsized":"2024-09-11T12:50:27.359623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365551+0000","last_change":"2024-09-11T12:50:26.334350+0000","last_active":"2024-09-11T12:50:27.365551+0000","last_peered":"2024-09-11T12:50:27.365551+0000","last_clean":"2024-09-11T12:50:27.365551+0000","last_became_active":"2024-09-11T12:50:26.334014+0000","last_became_peered":"2024-09-11T12:50:26.334014+0000","last_unstale":"2024-09-11T12:50:27.365551+0000","last_undegraded":"2024-09-11T12:50:27.365551+0000","last_fullsized":"2024-09-11T12:50:27.365551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351493+0000","last_change":"2024-09-11T12:50:27.350052+0000","last_active":"2024-09-11T12:50:27.351493+0000","last_peered":"2024-09-11T12:50:27.351493+0000","last_clean":"2024-09-11T12:50:27.351493+0000","last_became_active":"2024-09-11T12:50:27.349787+0000","last_became_peered":"2024-09-11T12:50:27.349787+0000","last_unstale":"2024-09-11T12:50:27.351493+0000","last_undegraded":"2024-09-11T12:50:27.351493+0000","last_fullsized":"2024-09-11T12:50:27.351493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.371174+0000","last_change":"2024-09-11T12:50:26.342451+0000","last_active":"2024-09-11T12:50:27.371174+0000","last_peered":"2024-09-11T12:50:27.371174+0000","last_clean":"2024-09-11T12:50:27.371174+0000","last_became_active":"2024-09-11T12:50:26.341973+0000","last_became_peered":"2024-09-11T12:50:26.341973+0000","last_unstale":"2024-09-11T12:50:27.371174+0000","last_undegraded":"2024-09-11T12:50:27.371174+0000","last_fullsized":"2024-09-11T12:50:27.371174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:54.047639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354826+0000","last_change":"2024-09-11T12:50:27.349605+0000","last_active":"2024-09-11T12:50:27.354826+0000","last_peered":"2024-09-11T12:50:27.354826+0000","last_clean":"2024-09-11T12:50:27.354826+0000","last_became_active":"2024-09-11T12:50:27.349304+0000","last_became_peered":"2024-09-11T12:50:27.349304+0000","last_unstale":"2024-09-11T12:50:27.354826+0000","last_undegraded":"2024-09-11T12:50:27.354826+0000","last_fullsized":"2024-09-11T12:50:27.354826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362391+0000","last_change":"2024-09-11T12:50:26.337760+0000","last_active":"2024-09-11T12:50:27.362391+0000","last_peered":"2024-09-11T12:50:27.362391+0000","last_clean":"2024-09-11T12:50:27.362391+0000","last_became_active":"2024-09-11T12:50:26.337496+0000","last_became_peered":"2024-09-11T12:50:26.337496+0000","last_unstale":"2024-09-11T12:50:27.362391+0000","last_undegraded":"2024-09-11T12:50:27.362391+0000","last_fullsized":"2024-09-11T12:50:27.362391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:09.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346498+0000","last_change":"2024-09-11T12:50:27.342946+0000","last_active":"2024-09-11T12:50:27.346498+0000","last_peered":"2024-09-11T12:50:27.346498+0000","last_clean":"2024-09-11T12:50:27.346498+0000","last_became_active":"2024-09-11T12:50:27.342692+0000","last_became_peered":"2024-09-11T12:50:27.342692+0000","last_unstale":"2024-09-11T12:50:27.346498+0000","last_undegraded":"2024-09-11T12:50:27.346498+0000","last_fullsized":"2024-09-11T12:50:27.346498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337180+0000","last_change":"2024-09-11T12:50:26.337698+0000","last_active":"2024-09-11T12:50:27.337180+0000","last_peered":"2024-09-11T12:50:27.337180+0000","last_clean":"2024-09-11T12:50:27.337180+0000","last_became_active":"2024-09-11T12:50:26.337145+0000","last_became_peered":"2024-09-11T12:50:26.337145+0000","last_unstale":"2024-09-11T12:50:27.337180+0000","last_undegraded":"2024-09-11T12:50:27.337180+0000","last_fullsized":"2024-09-11T12:50:27.337180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364743+0000","last_change":"2024-09-11T12:50:27.356536+0000","last_active":"2024-09-11T12:50:27.364743+0000","last_peered":"2024-09-11T12:50:27.364743+0000","last_clean":"2024-09-11T12:50:27.364743+0000","last_became_active":"2024-09-11T12:50:27.356235+0000","last_became_peered":"2024-09-11T12:50:27.356235+0000","last_unstale":"2024-09-11T12:50:27.364743+0000","last_undegraded":"2024-09-11T12:50:27.364743+0000","last_fullsized":"2024-09-11T12:50:27.364743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323110+0000","last_change":"2024-09-11T12:50:26.339598+0000","last_active":"2024-09-11T12:50:27.323110+0000","last_peered":"2024-09-11T12:50:27.323110+0000","last_clean":"2024-09-11T12:50:27.323110+0000","last_became_active":"2024-09-11T12:50:26.339322+0000","last_became_peered":"2024-09-11T12:50:26.339322+0000","last_unstale":"2024-09-11T12:50:27.323110+0000","last_undegraded":"2024-09-11T12:50:27.323110+0000","last_fullsized":"2024-09-11T12:50:27.323110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:45.625861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344441+0000","last_change":"2024-09-11T12:50:27.341021+0000","last_active":"2024-09-11T12:50:27.344441+0000","last_peered":"2024-09-11T12:50:27.344441+0000","last_clean":"2024-09-11T12:50:27.344441+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.344441+0000","last_undegraded":"2024-09-11T12:50:27.344441+0000","last_fullsized":"2024-09-11T12:50:27.344441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365719+0000","last_change":"2024-09-11T12:50:26.341481+0000","last_active":"2024-09-11T12:50:27.365719+0000","last_peered":"2024-09-11T12:50:27.365719+0000","last_clean":"2024-09-11T12:50:27.365719+0000","last_became_active":"2024-09-11T12:50:26.341148+0000","last_became_peered":"2024-09-11T12:50:26.341148+0000","last_unstale":"2024-09-11T12:50:27.365719+0000","last_undegraded":"2024-09-11T12:50:27.365719+0000","last_fullsized":"2024-09-11T12:50:27.365719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:41.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352075+0000","last_change":"2024-09-11T12:50:27.343792+0000","last_active":"2024-09-11T12:50:27.352075+0000","last_peered":"2024-09-11T12:50:27.352075+0000","last_clean":"2024-09-11T12:50:27.352075+0000","last_became_active":"2024-09-11T12:50:27.343522+0000","last_became_peered":"2024-09-11T12:50:27.343522+0000","last_unstale":"2024-09-11T12:50:27.352075+0000","last_undegraded":"2024-09-11T12:50:27.352075+0000","last_fullsized":"2024-09-11T12:50:27.352075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365752+0000","last_change":"2024-09-11T12:50:26.342891+0000","last_active":"2024-09-11T12:50:27.365752+0000","last_peered":"2024-09-11T12:50:27.365752+0000","last_clean":"2024-09-11T12:50:27.365752+0000","last_became_active":"2024-09-11T12:50:26.342639+0000","last_became_peered":"2024-09-11T12:50:26.342639+0000","last_unstale":"2024-09-11T12:50:27.365752+0000","last_undegraded":"2024-09-11T12:50:27.365752+0000","last_fullsized":"2024-09-11T12:50:27.365752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344203+0000","last_change":"2024-09-11T12:50:27.335239+0000","last_active":"2024-09-11T12:50:27.344203+0000","last_peered":"2024-09-11T12:50:27.344203+0000","last_clean":"2024-09-11T12:50:27.344203+0000","last_became_active":"2024-09-11T12:50:27.333463+0000","last_became_peered":"2024-09-11T12:50:27.333463+0000","last_unstale":"2024-09-11T12:50:27.344203+0000","last_undegraded":"2024-09-11T12:50:27.344203+0000","last_fullsized":"2024-09-11T12:50:27.344203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.361835+0000","last_change":"2024-09-11T12:50:26.337827+0000","last_active":"2024-09-11T12:50:27.361835+0000","last_peered":"2024-09-11T12:50:27.361835+0000","last_clean":"2024-09-11T12:50:27.361835+0000","last_became_active":"2024-09-11T12:50:26.337466+0000","last_became_peered":"2024-09-11T12:50:26.337466+0000","last_unstale":"2024-09-11T12:50:27.361835+0000","last_undegraded":"2024-09-11T12:50:27.361835+0000","last_fullsized":"2024-09-11T12:50:27.361835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:12.538604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342628+0000","last_change":"2024-09-11T12:50:27.338996+0000","last_active":"2024-09-11T12:50:27.342628+0000","last_peered":"2024-09-11T12:50:27.342628+0000","last_clean":"2024-09-11T12:50:27.342628+0000","last_became_active":"2024-09-11T12:50:27.338705+0000","last_became_peered":"2024-09-11T12:50:27.338705+0000","last_unstale":"2024-09-11T12:50:27.342628+0000","last_undegraded":"2024-09-11T12:50:27.342628+0000","last_fullsized":"2024-09-11T12:50:27.342628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363280+0000","last_change":"2024-09-11T12:50:26.339176+0000","last_active":"2024-09-11T12:50:27.363280+0000","last_peered":"2024-09-11T12:50:27.363280+0000","last_clean":"2024-09-11T12:50:27.363280+0000","last_became_active":"2024-09-11T12:50:26.338882+0000","last_became_peered":"2024-09-11T12:50:26.338882+0000","last_unstale":"2024-09-11T12:50:27.363280+0000","last_undegraded":"2024-09-11T12:50:27.363280+0000","last_fullsized":"2024-09-11T12:50:27.363280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:11.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.358426+0000","last_change":"2024-09-11T12:50:27.353963+0000","last_active":"2024-09-11T12:50:27.358426+0000","last_peered":"2024-09-11T12:50:27.358426+0000","last_clean":"2024-09-11T12:50:27.358426+0000","last_became_active":"2024-09-11T12:50:27.353684+0000","last_became_peered":"2024-09-11T12:50:27.353684+0000","last_unstale":"2024-09-11T12:50:27.358426+0000","last_undegraded":"2024-09-11T12:50:27.358426+0000","last_fullsized":"2024-09-11T12:50:27.358426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:07.300287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324319+0000","last_change":"2024-09-11T12:50:26.344464+0000","last_active":"2024-09-11T12:50:27.324319+0000","last_peered":"2024-09-11T12:50:27.324319+0000","last_clean":"2024-09-11T12:50:27.324319+0000","last_became_active":"2024-09-11T12:50:26.344093+0000","last_became_peered":"2024-09-11T12:50:26.344093+0000","last_unstale":"2024-09-11T12:50:27.324319+0000","last_undegraded":"2024-09-11T12:50:27.324319+0000","last_fullsized":"2024-09-11T12:50:27.324319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:47.700330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343140+0000","last_change":"2024-09-11T12:50:27.338133+0000","last_active":"2024-09-11T12:50:27.343140+0000","last_peered":"2024-09-11T12:50:27.343140+0000","last_clean":"2024-09-11T12:50:27.343140+0000","last_became_active":"2024-09-11T12:50:27.337816+0000","last_became_peered":"2024-09-11T12:50:27.337816+0000","last_unstale":"2024-09-11T12:50:27.343140+0000","last_undegraded":"2024-09-11T12:50:27.343140+0000","last_fullsized":"2024-09-11T12:50:27.343140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:36:21.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.344835+0000","last_change":"2024-09-11T12:50:26.339488+0000","last_active":"2024-09-11T12:50:26.344835+0000","last_peered":"2024-09-11T12:50:26.344835+0000","last_clean":"2024-09-11T12:50:26.344835+0000","last_became_active":"2024-09-11T12:50:26.339267+0000","last_became_peered":"2024-09-11T12:50:26.339267+0000","last_unstale":"2024-09-11T12:50:26.344835+0000","last_undegraded":"2024-09-11T12:50:26.344835+0000","last_fullsized":"2024-09-11T12:50:26.344835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:54:09.177646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353434+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.353434+0000","last_peered":"2024-09-11T12:50:27.353434+0000","last_clean":"2024-09-11T12:50:27.353434+0000","last_became_active":"2024-09-11T12:50:27.345261+0000","last_became_peered":"2024-09-11T12:50:27.345261+0000","last_unstale":"2024-09-11T12:50:27.353434+0000","last_undegraded":"2024-09-11T12:50:27.353434+0000","last_fullsized":"2024-09-11T12:50:27.353434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:58.388446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.330245+0000","last_change":"2024-09-11T12:50:26.346500+0000","last_active":"2024-09-11T12:50:27.330245+0000","last_peered":"2024-09-11T12:50:27.330245+0000","last_clean":"2024-09-11T12:50:27.330245+0000","last_became_active":"2024-09-11T12:50:26.346237+0000","last_became_peered":"2024-09-11T12:50:26.346237+0000","last_unstale":"2024-09-11T12:50:27.330245+0000","last_undegraded":"2024-09-11T12:50:27.330245+0000","last_fullsized":"2024-09-11T12:50:27.330245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346072+0000","last_change":"2024-09-11T12:50:27.342983+0000","last_active":"2024-09-11T12:50:27.346072+0000","last_peered":"2024-09-11T12:50:27.346072+0000","last_clean":"2024-09-11T12:50:27.346072+0000","last_became_active":"2024-09-11T12:50:27.342726+0000","last_became_peered":"2024-09-11T12:50:27.342726+0000","last_unstale":"2024-09-11T12:50:27.346072+0000","last_undegraded":"2024-09-11T12:50:27.346072+0000","last_fullsized":"2024-09-11T12:50:27.346072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:39.629583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325749+0000","last_change":"2024-09-11T12:50:26.339279+0000","last_active":"2024-09-11T12:50:27.325749+0000","last_peered":"2024-09-11T12:50:27.325749+0000","last_clean":"2024-09-11T12:50:27.325749+0000","last_became_active":"2024-09-11T12:50:26.338240+0000","last_became_peered":"2024-09-11T12:50:26.338240+0000","last_unstale":"2024-09-11T12:50:27.325749+0000","last_undegraded":"2024-09-11T12:50:27.325749+0000","last_fullsized":"2024-09-11T12:50:27.325749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:52.150201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357527+0000","last_change":"2024-09-11T12:50:27.349271+0000","last_active":"2024-09-11T12:50:27.357527+0000","last_peered":"2024-09-11T12:50:27.357527+0000","last_clean":"2024-09-11T12:50:27.357527+0000","last_became_active":"2024-09-11T12:50:27.348581+0000","last_became_peered":"2024-09-11T12:50:27.348581+0000","last_unstale":"2024-09-11T12:50:27.357527+0000","last_undegraded":"2024-09-11T12:50:27.357527+0000","last_fullsized":"2024-09-11T12:50:27.357527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324027+0000","last_change":"2024-09-11T12:50:26.336396+0000","last_active":"2024-09-11T12:50:27.324027+0000","last_peered":"2024-09-11T12:50:27.324027+0000","last_clean":"2024-09-11T12:50:27.324027+0000","last_became_active":"2024-09-11T12:50:26.335998+0000","last_became_peered":"2024-09-11T12:50:26.335998+0000","last_unstale":"2024-09-11T12:50:27.324027+0000","last_undegraded":"2024-09-11T12:50:27.324027+0000","last_fullsized":"2024-09-11T12:50:27.324027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357673+0000","last_change":"2024-09-11T12:50:27.349951+0000","last_active":"2024-09-11T12:50:27.357673+0000","last_peered":"2024-09-11T12:50:27.357673+0000","last_clean":"2024-09-11T12:50:27.357673+0000","last_became_active":"2024-09-11T12:50:27.349174+0000","last_became_peered":"2024-09-11T12:50:27.349174+0000","last_unstale":"2024-09-11T12:50:27.357673+0000","last_undegraded":"2024-09-11T12:50:27.357673+0000","last_fullsized":"2024-09-11T12:50:27.357673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:03.499603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364811+0000","last_change":"2024-09-11T12:50:26.337423+0000","last_active":"2024-09-11T12:50:27.364811+0000","last_peered":"2024-09-11T12:50:27.364811+0000","last_clean":"2024-09-11T12:50:27.364811+0000","last_became_active":"2024-09-11T12:50:26.337076+0000","last_became_peered":"2024-09-11T12:50:26.337076+0000","last_unstale":"2024-09-11T12:50:27.364811+0000","last_undegraded":"2024-09-11T12:50:27.364811+0000","last_fullsized":"2024-09-11T12:50:27.364811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351691+0000","last_change":"2024-09-11T12:50:27.349463+0000","last_active":"2024-09-11T12:50:27.351691+0000","last_peered":"2024-09-11T12:50:27.351691+0000","last_clean":"2024-09-11T12:50:27.351691+0000","last_became_active":"2024-09-11T12:50:27.349183+0000","last_became_peered":"2024-09-11T12:50:27.349183+0000","last_unstale":"2024-09-11T12:50:27.351691+0000","last_undegraded":"2024-09-11T12:50:27.351691+0000","last_fullsized":"2024-09-11T12:50:27.351691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345829+0000","last_change":"2024-09-11T12:50:26.340196+0000","last_active":"2024-09-11T12:50:26.345829+0000","last_peered":"2024-09-11T12:50:26.345829+0000","last_clean":"2024-09-11T12:50:26.345829+0000","last_became_active":"2024-09-11T12:50:26.339927+0000","last_became_peered":"2024-09-11T12:50:26.339927+0000","last_unstale":"2024-09-11T12:50:26.345829+0000","last_undegraded":"2024-09-11T12:50:26.345829+0000","last_fullsized":"2024-09-11T12:50:26.345829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:57.749644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.325394+0000","last_change":"2024-09-11T12:50:26.325394+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.325394+0000","last_undegraded":"2024-09-11T12:50:26.325394+0000","last_fullsized":"2024-09-11T12:50:26.325394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337747+0000","last_change":"2024-09-11T12:50:26.338822+0000","last_active":"2024-09-11T12:50:27.337747+0000","last_peered":"2024-09-11T12:50:27.337747+0000","last_clean":"2024-09-11T12:50:27.337747+0000","last_became_active":"2024-09-11T12:50:26.338459+0000","last_became_peered":"2024-09-11T12:50:26.338459+0000","last_unstale":"2024-09-11T12:50:27.337747+0000","last_undegraded":"2024-09-11T12:50:27.337747+0000","last_fullsized":"2024-09-11T12:50:27.337747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:01.412980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347818+0000","last_change":"2024-09-11T12:50:27.344966+0000","last_active":"2024-09-11T12:50:27.347818+0000","last_peered":"2024-09-11T12:50:27.347818+0000","last_clean":"2024-09-11T12:50:27.347818+0000","last_became_active":"2024-09-11T12:50:27.344634+0000","last_became_peered":"2024-09-11T12:50:27.344634+0000","last_unstale":"2024-09-11T12:50:27.347818+0000","last_undegraded":"2024-09-11T12:50:27.347818+0000","last_fullsized":"2024-09-11T12:50:27.347818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.337771+0000","last_change":"2024-09-11T12:50:26.330959+0000","last_active":"2024-09-11T12:50:26.337771+0000","last_peered":"2024-09-11T12:50:26.337771+0000","last_clean":"2024-09-11T12:50:26.337771+0000","last_became_active":"2024-09-11T12:50:26.330722+0000","last_became_peered":"2024-09-11T12:50:26.330722+0000","last_unstale":"2024-09-11T12:50:26.337771+0000","last_undegraded":"2024-09-11T12:50:26.337771+0000","last_fullsized":"2024-09-11T12:50:26.337771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:51.012754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324459+0000","last_change":"2024-09-11T12:50:26.324459+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324459+0000","last_undegraded":"2024-09-11T12:50:26.324459+0000","last_fullsized":"2024-09-11T12:50:26.324459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337362+0000","last_change":"2024-09-11T12:50:26.337961+0000","last_active":"2024-09-11T12:50:27.337362+0000","last_peered":"2024-09-11T12:50:27.337362+0000","last_clean":"2024-09-11T12:50:27.337362+0000","last_became_active":"2024-09-11T12:50:26.337461+0000","last_became_peered":"2024-09-11T12:50:26.337461+0000","last_unstale":"2024-09-11T12:50:27.337362+0000","last_undegraded":"2024-09-11T12:50:27.337362+0000","last_fullsized":"2024-09-11T12:50:27.337362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345998+0000","last_change":"2024-09-11T12:50:27.342392+0000","last_active":"2024-09-11T12:50:27.345998+0000","last_peered":"2024-09-11T12:50:27.345998+0000","last_clean":"2024-09-11T12:50:27.345998+0000","last_became_active":"2024-09-11T12:50:27.342116+0000","last_became_peered":"2024-09-11T12:50:27.342116+0000","last_unstale":"2024-09-11T12:50:27.345998+0000","last_undegraded":"2024-09-11T12:50:27.345998+0000","last_fullsized":"2024-09-11T12:50:27.345998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325924+0000","last_change":"2024-09-11T12:50:26.330978+0000","last_active":"2024-09-11T12:50:27.325924+0000","last_peered":"2024-09-11T12:50:27.325924+0000","last_clean":"2024-09-11T12:50:27.325924+0000","last_became_active":"2024-09-11T12:50:26.330681+0000","last_became_peered":"2024-09-11T12:50:26.330681+0000","last_unstale":"2024-09-11T12:50:27.325924+0000","last_undegraded":"2024-09-11T12:50:27.325924+0000","last_fullsized":"2024-09-11T12:50:27.325924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:43.803456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.334967+0000","last_change":"2024-09-11T12:50:27.328606+0000","last_active":"2024-09-11T12:50:27.334967+0000","last_peered":"2024-09-11T12:50:27.334967+0000","last_clean":"2024-09-11T12:50:27.334967+0000","last_became_active":"2024-09-11T12:50:27.328276+0000","last_became_peered":"2024-09-11T12:50:27.328276+0000","last_unstale":"2024-09-11T12:50:27.334967+0000","last_undegraded":"2024-09-11T12:50:27.334967+0000","last_fullsized":"2024-09-11T12:50:27.334967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:36.902883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323120+0000","last_change":"2024-09-11T12:50:26.337651+0000","last_active":"2024-09-11T12:50:27.323120+0000","last_peered":"2024-09-11T12:50:27.323120+0000","last_clean":"2024-09-11T12:50:27.323120+0000","last_became_active":"2024-09-11T12:50:26.337387+0000","last_became_peered":"2024-09-11T12:50:26.337387+0000","last_unstale":"2024-09-11T12:50:27.323120+0000","last_undegraded":"2024-09-11T12:50:27.323120+0000","last_fullsized":"2024-09-11T12:50:27.323120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:52.922609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362603+0000","last_change":"2024-09-11T12:50:27.351937+0000","last_active":"2024-09-11T12:50:27.362603+0000","last_peered":"2024-09-11T12:50:27.362603+0000","last_clean":"2024-09-11T12:50:27.362603+0000","last_became_active":"2024-09-11T12:50:27.350570+0000","last_became_peered":"2024-09-11T12:50:27.350570+0000","last_unstale":"2024-09-11T12:50:27.362603+0000","last_undegraded":"2024-09-11T12:50:27.362603+0000","last_fullsized":"2024-09-11T12:50:27.362603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.328966+0000","last_change":"2024-09-11T12:50:26.344815+0000","last_active":"2024-09-11T12:50:27.328966+0000","last_peered":"2024-09-11T12:50:27.328966+0000","last_clean":"2024-09-11T12:50:27.328966+0000","last_became_active":"2024-09-11T12:50:26.344518+0000","last_became_peered":"2024-09-11T12:50:26.344518+0000","last_unstale":"2024-09-11T12:50:27.328966+0000","last_undegraded":"2024-09-11T12:50:27.328966+0000","last_fullsized":"2024-09-11T12:50:27.328966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:24.053403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347970+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:50:27.347970+0000","last_peered":"2024-09-11T12:50:27.347970+0000","last_clean":"2024-09-11T12:50:27.347970+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:50:27.347970+0000","last_undegraded":"2024-09-11T12:50:27.347970+0000","last_fullsized":"2024-09-11T12:50:27.347970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:50:31.440 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:50:31.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- 172.21.15.78:0/455519695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 msgr2=0x7f415015ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- 172.21.15.78:0/455519695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f415015ea20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41500b5790 tx=0x7f413c01c890 comp rx=0 tx=0).stop 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- 172.21.15.78:0/455519695 shutdown_connections 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- 172.21.15.78:0/455519695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f415015ea20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- 172.21.15.78:0/455519695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 0x7f41500700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- 172.21.15.78:0/455519695 >> v2:172.21.15.157:3300/0 conn(0x7f4150074240 0x7f415015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- 172.21.15.78:0/455519695 >> 172.21.15.78:0/455519695 conn(0x7f415006ca80 msgr2=0x7f41500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- 172.21.15.78:0/455519695 shutdown_connections 2024-09-11T12:50:31.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- 172.21.15.78:0/455519695 wait complete. 2024-09-11T12:50:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 Processor -- start 2024-09-11T12:50:31.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- start start 2024-09-11T12:50:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 0x7f415014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150074240 0x7f4150154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4150072900 con 0x7f415015e5b0 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4150072780 con 0x7f4150074240 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f4155200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41500b4520 con 0x7f415006fc50 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60464/0 (socket says 172.21.15.78:60464) 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 -- 172.21.15.78:0/849217310 learned_addr learned my addr 172.21.15.78:0/849217310 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414de00640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 0x7f415014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414d400640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150074240 0x7f4150154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 -- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 msgr2=0x7f415014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 0x7f415014de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 -- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150074240 msgr2=0x7f4150154c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150074240 0x7f4150154c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.614+0000 7f414e800640 1 -- 172.21.15.78:0/849217310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4150070930 con 0x7f415015e5b0 2024-09-11T12:50:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f414e800640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f414800e990 tx=0x7f414800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:31.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148004d30 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f414800cb00 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148005020 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41501fa460 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41501faa20 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415015ba80 con 0x7f415015e5b0 2024-09-11T12:50:31.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f41480051c0 con 0x7f415015e5b0 2024-09-11T12:50:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 0x7f4130073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f4148092520 con 0x7f415015e5b0 2024-09-11T12:50:31.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.618+0000 7f414de00640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 0x7f4130073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:31.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.622+0000 7f414de00640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 0x7f4130073b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f414000bc00 tx=0x7f414000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:31.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.622+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4148016030 con 0x7f415015e5b0 2024-09-11T12:50:31.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.786+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41501fb0e0 con 0x7f4130071680 2024-09-11T12:50:31.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4146000640 1 -- 172.21.15.78:0/849217310 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251605 (secure 0 0 0) 0x7f41501fb0e0 con 0x7f4130071680 2024-09-11T12:50:31.798 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:31.800 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 msgr2=0x7f4130073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 0x7f4130073b40 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f414000bc00 tx=0x7f414000b390 comp rx=0 tx=0).stop 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 msgr2=0x7f4150163a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f414800e990 tx=0x7f414800ee50 comp rx=0 tx=0).stop 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 shutdown_connections 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4130071680 0x7f4130073b40 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415015e5b0 0x7f4150163a80 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4150074240 0x7f4150154c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 --2- 172.21.15.78:0/849217310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415006fc50 0x7f415014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:31.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 >> 172.21.15.78:0/849217310 conn(0x7f415006ca80 msgr2=0x7f415015c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:31.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 shutdown_connections 2024-09-11T12:50:31.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:31.794+0000 7f4155200640 1 -- 172.21.15.78:0/849217310 wait complete. 2024-09-11T12:50:31.818 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-11T12:50:31.116340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236808,"kb_used_data":3912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737912,"statfs":{"total":773094113280,"available":772851621888,"internally_reserved":0,"allocated":4005888,"data_stored":2229312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807871"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.350863+0000","last_change":"2024-09-11T12:50:27.348834+0000","last_active":"2024-09-11T12:50:27.350863+0000","last_peered":"2024-09-11T12:50:27.350863+0000","last_clean":"2024-09-11T12:50:27.350863+0000","last_became_active":"2024-09-11T12:50:27.348574+0000","last_became_peered":"2024-09-11T12:50:27.348574+0000","last_unstale":"2024-09-11T12:50:27.350863+0000","last_undegraded":"2024-09-11T12:50:27.350863+0000","last_fullsized":"2024-09-11T12:50:27.350863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341400+0000","last_change":"2024-09-11T12:50:27.336993+0000","last_active":"2024-09-11T12:50:27.341400+0000","last_peered":"2024-09-11T12:50:27.341400+0000","last_clean":"2024-09-11T12:50:27.341400+0000","last_became_active":"2024-09-11T12:50:27.336588+0000","last_became_peered":"2024-09-11T12:50:27.336588+0000","last_unstale":"2024-09-11T12:50:27.341400+0000","last_undegraded":"2024-09-11T12:50:27.341400+0000","last_fullsized":"2024-09-11T12:50:27.341400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351364+0000","last_change":"2024-09-11T12:50:27.342867+0000","last_active":"2024-09-11T12:50:27.351364+0000","last_peered":"2024-09-11T12:50:27.351364+0000","last_clean":"2024-09-11T12:50:27.351364+0000","last_became_active":"2024-09-11T12:50:27.342576+0000","last_became_peered":"2024-09-11T12:50:27.342576+0000","last_unstale":"2024-09-11T12:50:27.351364+0000","last_undegraded":"2024-09-11T12:50:27.351364+0000","last_fullsized":"2024-09-11T12:50:27.351364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360719+0000","last_change":"2024-09-11T12:50:27.357904+0000","last_active":"2024-09-11T12:50:27.360719+0000","last_peered":"2024-09-11T12:50:27.360719+0000","last_clean":"2024-09-11T12:50:27.360719+0000","last_became_active":"2024-09-11T12:50:27.356836+0000","last_became_peered":"2024-09-11T12:50:27.356836+0000","last_unstale":"2024-09-11T12:50:27.360719+0000","last_undegraded":"2024-09-11T12:50:27.360719+0000","last_fullsized":"2024-09-11T12:50:27.360719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345686+0000","last_change":"2024-09-11T12:50:27.341619+0000","last_active":"2024-09-11T12:50:27.345686+0000","last_peered":"2024-09-11T12:50:27.345686+0000","last_clean":"2024-09-11T12:50:27.345686+0000","last_became_active":"2024-09-11T12:50:27.341345+0000","last_became_peered":"2024-09-11T12:50:27.341345+0000","last_unstale":"2024-09-11T12:50:27.345686+0000","last_undegraded":"2024-09-11T12:50:27.345686+0000","last_fullsized":"2024-09-11T12:50:27.345686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:44.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.339528+0000","last_change":"2024-09-11T12:50:27.335087+0000","last_active":"2024-09-11T12:50:27.339528+0000","last_peered":"2024-09-11T12:50:27.339528+0000","last_clean":"2024-09-11T12:50:27.339528+0000","last_became_active":"2024-09-11T12:50:27.334790+0000","last_became_peered":"2024-09-11T12:50:27.334790+0000","last_unstale":"2024-09-11T12:50:27.339528+0000","last_undegraded":"2024-09-11T12:50:27.339528+0000","last_fullsized":"2024-09-11T12:50:27.339528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364813+0000","last_change":"2024-09-11T12:50:27.355196+0000","last_active":"2024-09-11T12:50:27.364813+0000","last_peered":"2024-09-11T12:50:27.364813+0000","last_clean":"2024-09-11T12:50:27.364813+0000","last_became_active":"2024-09-11T12:50:27.353288+0000","last_became_peered":"2024-09-11T12:50:27.353288+0000","last_unstale":"2024-09-11T12:50:27.364813+0000","last_undegraded":"2024-09-11T12:50:27.364813+0000","last_fullsized":"2024-09-11T12:50:27.364813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364931+0000","last_change":"2024-09-11T12:50:27.358075+0000","last_active":"2024-09-11T12:50:27.364931+0000","last_peered":"2024-09-11T12:50:27.364931+0000","last_clean":"2024-09-11T12:50:27.364931+0000","last_became_active":"2024-09-11T12:50:27.357851+0000","last_became_peered":"2024-09-11T12:50:27.357851+0000","last_unstale":"2024-09-11T12:50:27.364931+0000","last_undegraded":"2024-09-11T12:50:27.364931+0000","last_fullsized":"2024-09-11T12:50:27.364931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352225+0000","last_change":"2024-09-11T12:50:27.343726+0000","last_active":"2024-09-11T12:50:27.352225+0000","last_peered":"2024-09-11T12:50:27.352225+0000","last_clean":"2024-09-11T12:50:27.352225+0000","last_became_active":"2024-09-11T12:50:27.343450+0000","last_became_peered":"2024-09-11T12:50:27.343450+0000","last_unstale":"2024-09-11T12:50:27.352225+0000","last_undegraded":"2024-09-11T12:50:27.352225+0000","last_fullsized":"2024-09-11T12:50:27.352225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342383+0000","last_change":"2024-09-11T12:50:27.339989+0000","last_active":"2024-09-11T12:50:27.342383+0000","last_peered":"2024-09-11T12:50:27.342383+0000","last_clean":"2024-09-11T12:50:27.342383+0000","last_became_active":"2024-09-11T12:50:27.339673+0000","last_became_peered":"2024-09-11T12:50:27.339673+0000","last_unstale":"2024-09-11T12:50:27.342383+0000","last_undegraded":"2024-09-11T12:50:27.342383+0000","last_fullsized":"2024-09-11T12:50:27.342383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365270+0000","last_change":"2024-09-11T12:50:27.363103+0000","last_active":"2024-09-11T12:50:27.365270+0000","last_peered":"2024-09-11T12:50:27.365270+0000","last_clean":"2024-09-11T12:50:27.365270+0000","last_became_active":"2024-09-11T12:50:27.362746+0000","last_became_peered":"2024-09-11T12:50:27.362746+0000","last_unstale":"2024-09-11T12:50:27.365270+0000","last_undegraded":"2024-09-11T12:50:27.365270+0000","last_fullsized":"2024-09-11T12:50:27.365270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347758+0000","last_change":"2024-09-11T12:50:27.329779+0000","last_active":"2024-09-11T12:50:27.347758+0000","last_peered":"2024-09-11T12:50:27.347758+0000","last_clean":"2024-09-11T12:50:27.347758+0000","last_became_active":"2024-09-11T12:50:27.327796+0000","last_became_peered":"2024-09-11T12:50:27.327796+0000","last_unstale":"2024-09-11T12:50:27.347758+0000","last_undegraded":"2024-09-11T12:50:27.347758+0000","last_fullsized":"2024-09-11T12:50:27.347758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352853+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.352853+0000","last_peered":"2024-09-11T12:50:27.352853+0000","last_clean":"2024-09-11T12:50:27.352853+0000","last_became_active":"2024-09-11T12:50:27.344834+0000","last_became_peered":"2024-09-11T12:50:27.344834+0000","last_unstale":"2024-09-11T12:50:27.352853+0000","last_undegraded":"2024-09-11T12:50:27.352853+0000","last_fullsized":"2024-09-11T12:50:27.352853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:48.794525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341535+0000","last_change":"2024-09-11T12:50:27.339363+0000","last_active":"2024-09-11T12:50:27.341535+0000","last_peered":"2024-09-11T12:50:27.341535+0000","last_clean":"2024-09-11T12:50:27.341535+0000","last_became_active":"2024-09-11T12:50:27.339131+0000","last_became_peered":"2024-09-11T12:50:27.339131+0000","last_unstale":"2024-09-11T12:50:27.341535+0000","last_undegraded":"2024-09-11T12:50:27.341535+0000","last_fullsized":"2024-09-11T12:50:27.341535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344599+0000","last_change":"2024-09-11T12:50:27.335582+0000","last_active":"2024-09-11T12:50:27.344599+0000","last_peered":"2024-09-11T12:50:27.344599+0000","last_clean":"2024-09-11T12:50:27.344599+0000","last_became_active":"2024-09-11T12:50:27.335002+0000","last_became_peered":"2024-09-11T12:50:27.335002+0000","last_unstale":"2024-09-11T12:50:27.344599+0000","last_undegraded":"2024-09-11T12:50:27.344599+0000","last_fullsized":"2024-09-11T12:50:27.344599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365195+0000","last_change":"2024-09-11T12:50:27.363062+0000","last_active":"2024-09-11T12:50:27.365195+0000","last_peered":"2024-09-11T12:50:27.365195+0000","last_clean":"2024-09-11T12:50:27.365195+0000","last_became_active":"2024-09-11T12:50:27.362788+0000","last_became_peered":"2024-09-11T12:50:27.362788+0000","last_unstale":"2024-09-11T12:50:27.365195+0000","last_undegraded":"2024-09-11T12:50:27.365195+0000","last_fullsized":"2024-09-11T12:50:27.365195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364886+0000","last_change":"2024-09-11T12:50:27.357790+0000","last_active":"2024-09-11T12:50:27.364886+0000","last_peered":"2024-09-11T12:50:27.364886+0000","last_clean":"2024-09-11T12:50:27.364886+0000","last_became_active":"2024-09-11T12:50:27.353639+0000","last_became_peered":"2024-09-11T12:50:27.353639+0000","last_unstale":"2024-09-11T12:50:27.364886+0000","last_undegraded":"2024-09-11T12:50:27.364886+0000","last_fullsized":"2024-09-11T12:50:27.364886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341377+0000","last_change":"2024-09-11T12:50:27.336825+0000","last_active":"2024-09-11T12:50:27.341377+0000","last_peered":"2024-09-11T12:50:27.341377+0000","last_clean":"2024-09-11T12:50:27.341377+0000","last_became_active":"2024-09-11T12:50:27.336241+0000","last_became_peered":"2024-09-11T12:50:27.336241+0000","last_unstale":"2024-09-11T12:50:27.341377+0000","last_undegraded":"2024-09-11T12:50:27.341377+0000","last_fullsized":"2024-09-11T12:50:27.341377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346581+0000","last_change":"2024-09-11T12:50:27.344954+0000","last_active":"2024-09-11T12:50:27.346581+0000","last_peered":"2024-09-11T12:50:27.346581+0000","last_clean":"2024-09-11T12:50:27.346581+0000","last_became_active":"2024-09-11T12:50:27.344681+0000","last_became_peered":"2024-09-11T12:50:27.344681+0000","last_unstale":"2024-09-11T12:50:27.346581+0000","last_undegraded":"2024-09-11T12:50:27.346581+0000","last_fullsized":"2024-09-11T12:50:27.346581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357472+0000","last_change":"2024-09-11T12:50:27.352798+0000","last_active":"2024-09-11T12:50:27.357472+0000","last_peered":"2024-09-11T12:50:27.357472+0000","last_clean":"2024-09-11T12:50:27.357472+0000","last_became_active":"2024-09-11T12:50:27.349194+0000","last_became_peered":"2024-09-11T12:50:27.349194+0000","last_unstale":"2024-09-11T12:50:27.357472+0000","last_undegraded":"2024-09-11T12:50:27.357472+0000","last_fullsized":"2024-09-11T12:50:27.357472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:32.390636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351801+0000","last_change":"2024-09-11T12:50:27.343151+0000","last_active":"2024-09-11T12:50:27.351801+0000","last_peered":"2024-09-11T12:50:27.351801+0000","last_clean":"2024-09-11T12:50:27.351801+0000","last_became_active":"2024-09-11T12:50:27.339404+0000","last_became_peered":"2024-09-11T12:50:27.339404+0000","last_unstale":"2024-09-11T12:50:27.351801+0000","last_undegraded":"2024-09-11T12:50:27.351801+0000","last_fullsized":"2024-09-11T12:50:27.351801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.336093+0000","last_change":"2024-09-11T12:50:27.330086+0000","last_active":"2024-09-11T12:50:27.336093+0000","last_peered":"2024-09-11T12:50:27.336093+0000","last_clean":"2024-09-11T12:50:27.336093+0000","last_became_active":"2024-09-11T12:50:27.329846+0000","last_became_peered":"2024-09-11T12:50:27.329846+0000","last_unstale":"2024-09-11T12:50:27.336093+0000","last_undegraded":"2024-09-11T12:50:27.336093+0000","last_fullsized":"2024-09-11T12:50:27.336093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:12.211705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324523+0000","last_change":"2024-09-11T12:50:26.324523+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324523+0000","last_undegraded":"2024-09-11T12:50:26.324523+0000","last_fullsized":"2024-09-11T12:50:26.324523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343833+0000","last_change":"2024-09-11T12:50:27.337469+0000","last_active":"2024-09-11T12:50:27.343833+0000","last_peered":"2024-09-11T12:50:27.343833+0000","last_clean":"2024-09-11T12:50:27.343833+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.343833+0000","last_undegraded":"2024-09-11T12:50:27.343833+0000","last_fullsized":"2024-09-11T12:50:27.343833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359468+0000","last_change":"2024-09-11T12:50:27.355625+0000","last_active":"2024-09-11T12:50:27.359468+0000","last_peered":"2024-09-11T12:50:27.359468+0000","last_clean":"2024-09-11T12:50:27.359468+0000","last_became_active":"2024-09-11T12:50:27.355274+0000","last_became_peered":"2024-09-11T12:50:27.355274+0000","last_unstale":"2024-09-11T12:50:27.359468+0000","last_undegraded":"2024-09-11T12:50:27.359468+0000","last_fullsized":"2024-09-11T12:50:27.359468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359556+0000","last_change":"2024-09-11T12:50:27.356361+0000","last_active":"2024-09-11T12:50:27.359556+0000","last_peered":"2024-09-11T12:50:27.359556+0000","last_clean":"2024-09-11T12:50:27.359556+0000","last_became_active":"2024-09-11T12:50:27.356074+0000","last_became_peered":"2024-09-11T12:50:27.356074+0000","last_unstale":"2024-09-11T12:50:27.359556+0000","last_undegraded":"2024-09-11T12:50:27.359556+0000","last_fullsized":"2024-09-11T12:50:27.359556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359308+0000","last_change":"2024-09-11T12:50:27.357026+0000","last_active":"2024-09-11T12:50:27.359308+0000","last_peered":"2024-09-11T12:50:27.359308+0000","last_clean":"2024-09-11T12:50:27.359308+0000","last_became_active":"2024-09-11T12:50:27.356785+0000","last_became_peered":"2024-09-11T12:50:27.356785+0000","last_unstale":"2024-09-11T12:50:27.359308+0000","last_undegraded":"2024-09-11T12:50:27.359308+0000","last_fullsized":"2024-09-11T12:50:27.359308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347829+0000","last_change":"2024-09-11T12:50:27.336803+0000","last_active":"2024-09-11T12:50:27.347829+0000","last_peered":"2024-09-11T12:50:27.347829+0000","last_clean":"2024-09-11T12:50:27.347829+0000","last_became_active":"2024-09-11T12:50:27.334739+0000","last_became_peered":"2024-09-11T12:50:27.334739+0000","last_unstale":"2024-09-11T12:50:27.347829+0000","last_undegraded":"2024-09-11T12:50:27.347829+0000","last_fullsized":"2024-09-11T12:50:27.347829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335097+0000","last_change":"2024-09-11T12:50:27.329208+0000","last_active":"2024-09-11T12:50:27.335097+0000","last_peered":"2024-09-11T12:50:27.335097+0000","last_clean":"2024-09-11T12:50:27.335097+0000","last_became_active":"2024-09-11T12:50:27.328939+0000","last_became_peered":"2024-09-11T12:50:27.328939+0000","last_unstale":"2024-09-11T12:50:27.335097+0000","last_undegraded":"2024-09-11T12:50:27.335097+0000","last_fullsized":"2024-09-11T12:50:27.335097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:25.059566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353911+0000","last_change":"2024-09-11T12:50:27.348968+0000","last_active":"2024-09-11T12:50:27.353911+0000","last_peered":"2024-09-11T12:50:27.353911+0000","last_clean":"2024-09-11T12:50:27.353911+0000","last_became_active":"2024-09-11T12:50:27.348693+0000","last_became_peered":"2024-09-11T12:50:27.348693+0000","last_unstale":"2024-09-11T12:50:27.353911+0000","last_undegraded":"2024-09-11T12:50:27.353911+0000","last_fullsized":"2024-09-11T12:50:27.353911+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345469+0000","last_change":"2024-09-11T12:50:27.341331+0000","last_active":"2024-09-11T12:50:27.345469+0000","last_peered":"2024-09-11T12:50:27.345469+0000","last_clean":"2024-09-11T12:50:27.345469+0000","last_became_active":"2024-09-11T12:50:27.341038+0000","last_became_peered":"2024-09-11T12:50:27.341038+0000","last_unstale":"2024-09-11T12:50:27.345469+0000","last_undegraded":"2024-09-11T12:50:27.345469+0000","last_fullsized":"2024-09-11T12:50:27.345469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364720+0000","last_change":"2024-09-11T12:50:27.356104+0000","last_active":"2024-09-11T12:50:27.364720+0000","last_peered":"2024-09-11T12:50:27.364720+0000","last_clean":"2024-09-11T12:50:27.364720+0000","last_became_active":"2024-09-11T12:50:27.355113+0000","last_became_peered":"2024-09-11T12:50:27.355113+0000","last_unstale":"2024-09-11T12:50:27.364720+0000","last_undegraded":"2024-09-11T12:50:27.364720+0000","last_fullsized":"2024-09-11T12:50:27.364720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324419+0000","last_change":"2024-09-11T12:50:26.324419+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324419+0000","last_undegraded":"2024-09-11T12:50:26.324419+0000","last_fullsized":"2024-09-11T12:50:26.324419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365295+0000","last_change":"2024-09-11T12:50:26.343869+0000","last_active":"2024-09-11T12:50:27.365295+0000","last_peered":"2024-09-11T12:50:27.365295+0000","last_clean":"2024-09-11T12:50:27.365295+0000","last_became_active":"2024-09-11T12:50:26.343588+0000","last_became_peered":"2024-09-11T12:50:26.343588+0000","last_unstale":"2024-09-11T12:50:27.365295+0000","last_undegraded":"2024-09-11T12:50:27.365295+0000","last_fullsized":"2024-09-11T12:50:27.365295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344316+0000","last_change":"2024-09-11T12:50:27.340558+0000","last_active":"2024-09-11T12:50:27.344316+0000","last_peered":"2024-09-11T12:50:27.344316+0000","last_clean":"2024-09-11T12:50:27.344316+0000","last_became_active":"2024-09-11T12:50:27.337140+0000","last_became_peered":"2024-09-11T12:50:27.337140+0000","last_unstale":"2024-09-11T12:50:27.344316+0000","last_undegraded":"2024-09-11T12:50:27.344316+0000","last_fullsized":"2024-09-11T12:50:27.344316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.341744+0000","last_change":"2024-09-11T12:50:26.335186+0000","last_active":"2024-09-11T12:50:26.341744+0000","last_peered":"2024-09-11T12:50:26.341744+0000","last_clean":"2024-09-11T12:50:26.341744+0000","last_became_active":"2024-09-11T12:50:26.334825+0000","last_became_peered":"2024-09-11T12:50:26.334825+0000","last_unstale":"2024-09-11T12:50:26.341744+0000","last_undegraded":"2024-09-11T12:50:26.341744+0000","last_fullsized":"2024-09-11T12:50:26.341744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:54.260958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335753+0000","last_change":"2024-09-11T12:50:27.328627+0000","last_active":"2024-09-11T12:50:27.335753+0000","last_peered":"2024-09-11T12:50:27.335753+0000","last_clean":"2024-09-11T12:50:27.335753+0000","last_became_active":"2024-09-11T12:50:27.328292+0000","last_became_peered":"2024-09-11T12:50:27.328292+0000","last_unstale":"2024-09-11T12:50:27.335753+0000","last_undegraded":"2024-09-11T12:50:27.335753+0000","last_fullsized":"2024-09-11T12:50:27.335753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:27.421789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360603+0000","last_change":"2024-09-11T12:50:26.342258+0000","last_active":"2024-09-11T12:50:27.360603+0000","last_peered":"2024-09-11T12:50:27.360603+0000","last_clean":"2024-09-11T12:50:27.360603+0000","last_became_active":"2024-09-11T12:50:26.341986+0000","last_became_peered":"2024-09-11T12:50:26.341986+0000","last_unstale":"2024-09-11T12:50:27.360603+0000","last_undegraded":"2024-09-11T12:50:27.360603+0000","last_fullsized":"2024-09-11T12:50:27.360603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:46.357747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359368+0000","last_change":"2024-09-11T12:50:27.357376+0000","last_active":"2024-09-11T12:50:27.359368+0000","last_peered":"2024-09-11T12:50:27.359368+0000","last_clean":"2024-09-11T12:50:27.359368+0000","last_became_active":"2024-09-11T12:50:27.357016+0000","last_became_peered":"2024-09-11T12:50:27.357016+0000","last_unstale":"2024-09-11T12:50:27.359368+0000","last_undegraded":"2024-09-11T12:50:27.359368+0000","last_fullsized":"2024-09-11T12:50:27.359368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360981+0000","last_change":"2024-09-11T12:50:26.343854+0000","last_active":"2024-09-11T12:50:27.360981+0000","last_peered":"2024-09-11T12:50:27.360981+0000","last_clean":"2024-09-11T12:50:27.360981+0000","last_became_active":"2024-09-11T12:50:26.341771+0000","last_became_peered":"2024-09-11T12:50:26.341771+0000","last_unstale":"2024-09-11T12:50:27.360981+0000","last_undegraded":"2024-09-11T12:50:27.360981+0000","last_fullsized":"2024-09-11T12:50:27.360981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:51.008327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364638+0000","last_change":"2024-09-11T12:50:27.352862+0000","last_active":"2024-09-11T12:50:27.364638+0000","last_peered":"2024-09-11T12:50:27.364638+0000","last_clean":"2024-09-11T12:50:27.364638+0000","last_became_active":"2024-09-11T12:50:27.352467+0000","last_became_peered":"2024-09-11T12:50:27.352467+0000","last_unstale":"2024-09-11T12:50:27.364638+0000","last_undegraded":"2024-09-11T12:50:27.364638+0000","last_fullsized":"2024-09-11T12:50:27.364638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363377+0000","last_change":"2024-09-11T12:50:26.342659+0000","last_active":"2024-09-11T12:50:27.363377+0000","last_peered":"2024-09-11T12:50:27.363377+0000","last_clean":"2024-09-11T12:50:27.363377+0000","last_became_active":"2024-09-11T12:50:26.342331+0000","last_became_peered":"2024-09-11T12:50:26.342331+0000","last_unstale":"2024-09-11T12:50:27.363377+0000","last_undegraded":"2024-09-11T12:50:27.363377+0000","last_fullsized":"2024-09-11T12:50:27.363377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:18.698135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352409+0000","last_change":"2024-09-11T12:50:27.347574+0000","last_active":"2024-09-11T12:50:27.352409+0000","last_peered":"2024-09-11T12:50:27.352409+0000","last_clean":"2024-09-11T12:50:27.352409+0000","last_became_active":"2024-09-11T12:50:27.347302+0000","last_became_peered":"2024-09-11T12:50:27.347302+0000","last_unstale":"2024-09-11T12:50:27.352409+0000","last_undegraded":"2024-09-11T12:50:27.352409+0000","last_fullsized":"2024-09-11T12:50:27.352409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.366007+0000","last_change":"2024-09-11T12:50:26.341601+0000","last_active":"2024-09-11T12:50:27.366007+0000","last_peered":"2024-09-11T12:50:27.366007+0000","last_clean":"2024-09-11T12:50:27.366007+0000","last_became_active":"2024-09-11T12:50:26.341238+0000","last_became_peered":"2024-09-11T12:50:26.341238+0000","last_unstale":"2024-09-11T12:50:27.366007+0000","last_undegraded":"2024-09-11T12:50:27.366007+0000","last_fullsized":"2024-09-11T12:50:27.366007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:46.542517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335996+0000","last_change":"2024-09-11T12:50:27.333846+0000","last_active":"2024-09-11T12:50:27.335996+0000","last_peered":"2024-09-11T12:50:27.335996+0000","last_clean":"2024-09-11T12:50:27.335996+0000","last_became_active":"2024-09-11T12:50:27.333500+0000","last_became_peered":"2024-09-11T12:50:27.333500+0000","last_unstale":"2024-09-11T12:50:27.335996+0000","last_undegraded":"2024-09-11T12:50:27.335996+0000","last_fullsized":"2024-09-11T12:50:27.335996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323107+0000","last_change":"2024-09-11T12:50:26.339433+0000","last_active":"2024-09-11T12:50:27.323107+0000","last_peered":"2024-09-11T12:50:27.323107+0000","last_clean":"2024-09-11T12:50:27.323107+0000","last_became_active":"2024-09-11T12:50:26.338991+0000","last_became_peered":"2024-09-11T12:50:26.338991+0000","last_unstale":"2024-09-11T12:50:27.323107+0000","last_undegraded":"2024-09-11T12:50:27.323107+0000","last_fullsized":"2024-09-11T12:50:27.323107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354941+0000","last_change":"2024-09-11T12:50:27.351561+0000","last_active":"2024-09-11T12:50:27.354941+0000","last_peered":"2024-09-11T12:50:27.354941+0000","last_clean":"2024-09-11T12:50:27.354941+0000","last_became_active":"2024-09-11T12:50:27.351277+0000","last_became_peered":"2024-09-11T12:50:27.351277+0000","last_unstale":"2024-09-11T12:50:27.354941+0000","last_undegraded":"2024-09-11T12:50:27.354941+0000","last_fullsized":"2024-09-11T12:50:27.354941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323830+0000","last_change":"2024-09-11T12:50:26.337847+0000","last_active":"2024-09-11T12:50:27.323830+0000","last_peered":"2024-09-11T12:50:27.323830+0000","last_clean":"2024-09-11T12:50:27.323830+0000","last_became_active":"2024-09-11T12:50:26.337533+0000","last_became_peered":"2024-09-11T12:50:26.337533+0000","last_unstale":"2024-09-11T12:50:27.323830+0000","last_undegraded":"2024-09-11T12:50:27.323830+0000","last_fullsized":"2024-09-11T12:50:27.323830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:06.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346859+0000","last_change":"2024-09-11T12:50:27.330135+0000","last_active":"2024-09-11T12:50:27.346859+0000","last_peered":"2024-09-11T12:50:27.346859+0000","last_clean":"2024-09-11T12:50:27.346859+0000","last_became_active":"2024-09-11T12:50:27.329324+0000","last_became_peered":"2024-09-11T12:50:27.329324+0000","last_unstale":"2024-09-11T12:50:27.346859+0000","last_undegraded":"2024-09-11T12:50:27.346859+0000","last_fullsized":"2024-09-11T12:50:27.346859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324111+0000","last_change":"2024-09-11T12:50:26.343684+0000","last_active":"2024-09-11T12:50:27.324111+0000","last_peered":"2024-09-11T12:50:27.324111+0000","last_clean":"2024-09-11T12:50:27.324111+0000","last_became_active":"2024-09-11T12:50:26.343386+0000","last_became_peered":"2024-09-11T12:50:26.343386+0000","last_unstale":"2024-09-11T12:50:27.324111+0000","last_undegraded":"2024-09-11T12:50:27.324111+0000","last_fullsized":"2024-09-11T12:50:27.324111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:06.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.348808+0000","last_change":"2024-09-11T12:50:27.346100+0000","last_active":"2024-09-11T12:50:27.348808+0000","last_peered":"2024-09-11T12:50:27.348808+0000","last_clean":"2024-09-11T12:50:27.348808+0000","last_became_active":"2024-09-11T12:50:27.345380+0000","last_became_peered":"2024-09-11T12:50:27.345380+0000","last_unstale":"2024-09-11T12:50:27.348808+0000","last_undegraded":"2024-09-11T12:50:27.348808+0000","last_fullsized":"2024-09-11T12:50:27.348808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345851+0000","last_change":"2024-09-11T12:50:26.340838+0000","last_active":"2024-09-11T12:50:26.345851+0000","last_peered":"2024-09-11T12:50:26.345851+0000","last_clean":"2024-09-11T12:50:26.345851+0000","last_became_active":"2024-09-11T12:50:26.340531+0000","last_became_peered":"2024-09-11T12:50:26.340531+0000","last_unstale":"2024-09-11T12:50:26.345851+0000","last_undegraded":"2024-09-11T12:50:26.345851+0000","last_fullsized":"2024-09-11T12:50:26.345851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:03.491437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359623+0000","last_change":"2024-09-11T12:50:27.356884+0000","last_active":"2024-09-11T12:50:27.359623+0000","last_peered":"2024-09-11T12:50:27.359623+0000","last_clean":"2024-09-11T12:50:27.359623+0000","last_became_active":"2024-09-11T12:50:27.355050+0000","last_became_peered":"2024-09-11T12:50:27.355050+0000","last_unstale":"2024-09-11T12:50:27.359623+0000","last_undegraded":"2024-09-11T12:50:27.359623+0000","last_fullsized":"2024-09-11T12:50:27.359623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365551+0000","last_change":"2024-09-11T12:50:26.334350+0000","last_active":"2024-09-11T12:50:27.365551+0000","last_peered":"2024-09-11T12:50:27.365551+0000","last_clean":"2024-09-11T12:50:27.365551+0000","last_became_active":"2024-09-11T12:50:26.334014+0000","last_became_peered":"2024-09-11T12:50:26.334014+0000","last_unstale":"2024-09-11T12:50:27.365551+0000","last_undegraded":"2024-09-11T12:50:27.365551+0000","last_fullsized":"2024-09-11T12:50:27.365551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351493+0000","last_change":"2024-09-11T12:50:27.350052+0000","last_active":"2024-09-11T12:50:27.351493+0000","last_peered":"2024-09-11T12:50:27.351493+0000","last_clean":"2024-09-11T12:50:27.351493+0000","last_became_active":"2024-09-11T12:50:27.349787+0000","last_became_peered":"2024-09-11T12:50:27.349787+0000","last_unstale":"2024-09-11T12:50:27.351493+0000","last_undegraded":"2024-09-11T12:50:27.351493+0000","last_fullsized":"2024-09-11T12:50:27.351493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.371174+0000","last_change":"2024-09-11T12:50:26.342451+0000","last_active":"2024-09-11T12:50:27.371174+0000","last_peered":"2024-09-11T12:50:27.371174+0000","last_clean":"2024-09-11T12:50:27.371174+0000","last_became_active":"2024-09-11T12:50:26.341973+0000","last_became_peered":"2024-09-11T12:50:26.341973+0000","last_unstale":"2024-09-11T12:50:27.371174+0000","last_undegraded":"2024-09-11T12:50:27.371174+0000","last_fullsized":"2024-09-11T12:50:27.371174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:54.047639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354826+0000","last_change":"2024-09-11T12:50:27.349605+0000","last_active":"2024-09-11T12:50:27.354826+0000","last_peered":"2024-09-11T12:50:27.354826+0000","last_clean":"2024-09-11T12:50:27.354826+0000","last_became_active":"2024-09-11T12:50:27.349304+0000","last_became_peered":"2024-09-11T12:50:27.349304+0000","last_unstale":"2024-09-11T12:50:27.354826+0000","last_undegraded":"2024-09-11T12:50:27.354826+0000","last_fullsized":"2024-09-11T12:50:27.354826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362391+0000","last_change":"2024-09-11T12:50:26.337760+0000","last_active":"2024-09-11T12:50:27.362391+0000","last_peered":"2024-09-11T12:50:27.362391+0000","last_clean":"2024-09-11T12:50:27.362391+0000","last_became_active":"2024-09-11T12:50:26.337496+0000","last_became_peered":"2024-09-11T12:50:26.337496+0000","last_unstale":"2024-09-11T12:50:27.362391+0000","last_undegraded":"2024-09-11T12:50:27.362391+0000","last_fullsized":"2024-09-11T12:50:27.362391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:09.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346498+0000","last_change":"2024-09-11T12:50:27.342946+0000","last_active":"2024-09-11T12:50:27.346498+0000","last_peered":"2024-09-11T12:50:27.346498+0000","last_clean":"2024-09-11T12:50:27.346498+0000","last_became_active":"2024-09-11T12:50:27.342692+0000","last_became_peered":"2024-09-11T12:50:27.342692+0000","last_unstale":"2024-09-11T12:50:27.346498+0000","last_undegraded":"2024-09-11T12:50:27.346498+0000","last_fullsized":"2024-09-11T12:50:27.346498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337180+0000","last_change":"2024-09-11T12:50:26.337698+0000","last_active":"2024-09-11T12:50:27.337180+0000","last_peered":"2024-09-11T12:50:27.337180+0000","last_clean":"2024-09-11T12:50:27.337180+0000","last_became_active":"2024-09-11T12:50:26.337145+0000","last_became_peered":"2024-09-11T12:50:26.337145+0000","last_unstale":"2024-09-11T12:50:27.337180+0000","last_undegraded":"2024-09-11T12:50:27.337180+0000","last_fullsized":"2024-09-11T12:50:27.337180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364743+0000","last_change":"2024-09-11T12:50:27.356536+0000","last_active":"2024-09-11T12:50:27.364743+0000","last_peered":"2024-09-11T12:50:27.364743+0000","last_clean":"2024-09-11T12:50:27.364743+0000","last_became_active":"2024-09-11T12:50:27.356235+0000","last_became_peered":"2024-09-11T12:50:27.356235+0000","last_unstale":"2024-09-11T12:50:27.364743+0000","last_undegraded":"2024-09-11T12:50:27.364743+0000","last_fullsized":"2024-09-11T12:50:27.364743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323110+0000","last_change":"2024-09-11T12:50:26.339598+0000","last_active":"2024-09-11T12:50:27.323110+0000","last_peered":"2024-09-11T12:50:27.323110+0000","last_clean":"2024-09-11T12:50:27.323110+0000","last_became_active":"2024-09-11T12:50:26.339322+0000","last_became_peered":"2024-09-11T12:50:26.339322+0000","last_unstale":"2024-09-11T12:50:27.323110+0000","last_undegraded":"2024-09-11T12:50:27.323110+0000","last_fullsized":"2024-09-11T12:50:27.323110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:45.625861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344441+0000","last_change":"2024-09-11T12:50:27.341021+0000","last_active":"2024-09-11T12:50:27.344441+0000","last_peered":"2024-09-11T12:50:27.344441+0000","last_clean":"2024-09-11T12:50:27.344441+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.344441+0000","last_undegraded":"2024-09-11T12:50:27.344441+0000","last_fullsized":"2024-09-11T12:50:27.344441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365719+0000","last_change":"2024-09-11T12:50:26.341481+0000","last_active":"2024-09-11T12:50:27.365719+0000","last_peered":"2024-09-11T12:50:27.365719+0000","last_clean":"2024-09-11T12:50:27.365719+0000","last_became_active":"2024-09-11T12:50:26.341148+0000","last_became_peered":"2024-09-11T12:50:26.341148+0000","last_unstale":"2024-09-11T12:50:27.365719+0000","last_undegraded":"2024-09-11T12:50:27.365719+0000","last_fullsized":"2024-09-11T12:50:27.365719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:41.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352075+0000","last_change":"2024-09-11T12:50:27.343792+0000","last_active":"2024-09-11T12:50:27.352075+0000","last_peered":"2024-09-11T12:50:27.352075+0000","last_clean":"2024-09-11T12:50:27.352075+0000","last_became_active":"2024-09-11T12:50:27.343522+0000","last_became_peered":"2024-09-11T12:50:27.343522+0000","last_unstale":"2024-09-11T12:50:27.352075+0000","last_undegraded":"2024-09-11T12:50:27.352075+0000","last_fullsized":"2024-09-11T12:50:27.352075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365752+0000","last_change":"2024-09-11T12:50:26.342891+0000","last_active":"2024-09-11T12:50:27.365752+0000","last_peered":"2024-09-11T12:50:27.365752+0000","last_clean":"2024-09-11T12:50:27.365752+0000","last_became_active":"2024-09-11T12:50:26.342639+0000","last_became_peered":"2024-09-11T12:50:26.342639+0000","last_unstale":"2024-09-11T12:50:27.365752+0000","last_undegraded":"2024-09-11T12:50:27.365752+0000","last_fullsized":"2024-09-11T12:50:27.365752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344203+0000","last_change":"2024-09-11T12:50:27.335239+0000","last_active":"2024-09-11T12:50:27.344203+0000","last_peered":"2024-09-11T12:50:27.344203+0000","last_clean":"2024-09-11T12:50:27.344203+0000","last_became_active":"2024-09-11T12:50:27.333463+0000","last_became_peered":"2024-09-11T12:50:27.333463+0000","last_unstale":"2024-09-11T12:50:27.344203+0000","last_undegraded":"2024-09-11T12:50:27.344203+0000","last_fullsized":"2024-09-11T12:50:27.344203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.361835+0000","last_change":"2024-09-11T12:50:26.337827+0000","last_active":"2024-09-11T12:50:27.361835+0000","last_peered":"2024-09-11T12:50:27.361835+0000","last_clean":"2024-09-11T12:50:27.361835+0000","last_became_active":"2024-09-11T12:50:26.337466+0000","last_became_peered":"2024-09-11T12:50:26.337466+0000","last_unstale":"2024-09-11T12:50:27.361835+0000","last_undegraded":"2024-09-11T12:50:27.361835+0000","last_fullsized":"2024-09-11T12:50:27.361835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:12.538604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342628+0000","last_change":"2024-09-11T12:50:27.338996+0000","last_active":"2024-09-11T12:50:27.342628+0000","last_peered":"2024-09-11T12:50:27.342628+0000","last_clean":"2024-09-11T12:50:27.342628+0000","last_became_active":"2024-09-11T12:50:27.338705+0000","last_became_peered":"2024-09-11T12:50:27.338705+0000","last_unstale":"2024-09-11T12:50:27.342628+0000","last_undegraded":"2024-09-11T12:50:27.342628+0000","last_fullsized":"2024-09-11T12:50:27.342628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363280+0000","last_change":"2024-09-11T12:50:26.339176+0000","last_active":"2024-09-11T12:50:27.363280+0000","last_peered":"2024-09-11T12:50:27.363280+0000","last_clean":"2024-09-11T12:50:27.363280+0000","last_became_active":"2024-09-11T12:50:26.338882+0000","last_became_peered":"2024-09-11T12:50:26.338882+0000","last_unstale":"2024-09-11T12:50:27.363280+0000","last_undegraded":"2024-09-11T12:50:27.363280+0000","last_fullsized":"2024-09-11T12:50:27.363280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:11.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.358426+0000","last_change":"2024-09-11T12:50:27.353963+0000","last_active":"2024-09-11T12:50:27.358426+0000","last_peered":"2024-09-11T12:50:27.358426+0000","last_clean":"2024-09-11T12:50:27.358426+0000","last_became_active":"2024-09-11T12:50:27.353684+0000","last_became_peered":"2024-09-11T12:50:27.353684+0000","last_unstale":"2024-09-11T12:50:27.358426+0000","last_undegraded":"2024-09-11T12:50:27.358426+0000","last_fullsized":"2024-09-11T12:50:27.358426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:07.300287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324319+0000","last_change":"2024-09-11T12:50:26.344464+0000","last_active":"2024-09-11T12:50:27.324319+0000","last_peered":"2024-09-11T12:50:27.324319+0000","last_clean":"2024-09-11T12:50:27.324319+0000","last_became_active":"2024-09-11T12:50:26.344093+0000","last_became_peered":"2024-09-11T12:50:26.344093+0000","last_unstale":"2024-09-11T12:50:27.324319+0000","last_undegraded":"2024-09-11T12:50:27.324319+0000","last_fullsized":"2024-09-11T12:50:27.324319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:47.700330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343140+0000","last_change":"2024-09-11T12:50:27.338133+0000","last_active":"2024-09-11T12:50:27.343140+0000","last_peered":"2024-09-11T12:50:27.343140+0000","last_clean":"2024-09-11T12:50:27.343140+0000","last_became_active":"2024-09-11T12:50:27.337816+0000","last_became_peered":"2024-09-11T12:50:27.337816+0000","last_unstale":"2024-09-11T12:50:27.343140+0000","last_undegraded":"2024-09-11T12:50:27.343140+0000","last_fullsized":"2024-09-11T12:50:27.343140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:36:21.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.344835+0000","last_change":"2024-09-11T12:50:26.339488+0000","last_active":"2024-09-11T12:50:26.344835+0000","last_peered":"2024-09-11T12:50:26.344835+0000","last_clean":"2024-09-11T12:50:26.344835+0000","last_became_active":"2024-09-11T12:50:26.339267+0000","last_became_peered":"2024-09-11T12:50:26.339267+0000","last_unstale":"2024-09-11T12:50:26.344835+0000","last_undegraded":"2024-09-11T12:50:26.344835+0000","last_fullsized":"2024-09-11T12:50:26.344835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:54:09.177646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353434+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.353434+0000","last_peered":"2024-09-11T12:50:27.353434+0000","last_clean":"2024-09-11T12:50:27.353434+0000","last_became_active":"2024-09-11T12:50:27.345261+0000","last_became_peered":"2024-09-11T12:50:27.345261+0000","last_unstale":"2024-09-11T12:50:27.353434+0000","last_undegraded":"2024-09-11T12:50:27.353434+0000","last_fullsized":"2024-09-11T12:50:27.353434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:58.388446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.330245+0000","last_change":"2024-09-11T12:50:26.346500+0000","last_active":"2024-09-11T12:50:27.330245+0000","last_peered":"2024-09-11T12:50:27.330245+0000","last_clean":"2024-09-11T12:50:27.330245+0000","last_became_active":"2024-09-11T12:50:26.346237+0000","last_became_peered":"2024-09-11T12:50:26.346237+0000","last_unstale":"2024-09-11T12:50:27.330245+0000","last_undegraded":"2024-09-11T12:50:27.330245+0000","last_fullsized":"2024-09-11T12:50:27.330245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346072+0000","last_change":"2024-09-11T12:50:27.342983+0000","last_active":"2024-09-11T12:50:27.346072+0000","last_peered":"2024-09-11T12:50:27.346072+0000","last_clean":"2024-09-11T12:50:27.346072+0000","last_became_active":"2024-09-11T12:50:27.342726+0000","last_became_peered":"2024-09-11T12:50:27.342726+0000","last_unstale":"2024-09-11T12:50:27.346072+0000","last_undegraded":"2024-09-11T12:50:27.346072+0000","last_fullsized":"2024-09-11T12:50:27.346072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:39.629583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325749+0000","last_change":"2024-09-11T12:50:26.339279+0000","last_active":"2024-09-11T12:50:27.325749+0000","last_peered":"2024-09-11T12:50:27.325749+0000","last_clean":"2024-09-11T12:50:27.325749+0000","last_became_active":"2024-09-11T12:50:26.338240+0000","last_became_peered":"2024-09-11T12:50:26.338240+0000","last_unstale":"2024-09-11T12:50:27.325749+0000","last_undegraded":"2024-09-11T12:50:27.325749+0000","last_fullsized":"2024-09-11T12:50:27.325749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:52.150201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357527+0000","last_change":"2024-09-11T12:50:27.349271+0000","last_active":"2024-09-11T12:50:27.357527+0000","last_peered":"2024-09-11T12:50:27.357527+0000","last_clean":"2024-09-11T12:50:27.357527+0000","last_became_active":"2024-09-11T12:50:27.348581+0000","last_became_peered":"2024-09-11T12:50:27.348581+0000","last_unstale":"2024-09-11T12:50:27.357527+0000","last_undegraded":"2024-09-11T12:50:27.357527+0000","last_fullsized":"2024-09-11T12:50:27.357527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324027+0000","last_change":"2024-09-11T12:50:26.336396+0000","last_active":"2024-09-11T12:50:27.324027+0000","last_peered":"2024-09-11T12:50:27.324027+0000","last_clean":"2024-09-11T12:50:27.324027+0000","last_became_active":"2024-09-11T12:50:26.335998+0000","last_became_peered":"2024-09-11T12:50:26.335998+0000","last_unstale":"2024-09-11T12:50:27.324027+0000","last_undegraded":"2024-09-11T12:50:27.324027+0000","last_fullsized":"2024-09-11T12:50:27.324027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357673+0000","last_change":"2024-09-11T12:50:27.349951+0000","last_active":"2024-09-11T12:50:27.357673+0000","last_peered":"2024-09-11T12:50:27.357673+0000","last_clean":"2024-09-11T12:50:27.357673+0000","last_became_active":"2024-09-11T12:50:27.349174+0000","last_became_peered":"2024-09-11T12:50:27.349174+0000","last_unstale":"2024-09-11T12:50:27.357673+0000","last_undegraded":"2024-09-11T12:50:27.357673+0000","last_fullsized":"2024-09-11T12:50:27.357673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:03.499603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364811+0000","last_change":"2024-09-11T12:50:26.337423+0000","last_active":"2024-09-11T12:50:27.364811+0000","last_peered":"2024-09-11T12:50:27.364811+0000","last_clean":"2024-09-11T12:50:27.364811+0000","last_became_active":"2024-09-11T12:50:26.337076+0000","last_became_peered":"2024-09-11T12:50:26.337076+0000","last_unstale":"2024-09-11T12:50:27.364811+0000","last_undegraded":"2024-09-11T12:50:27.364811+0000","last_fullsized":"2024-09-11T12:50:27.364811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351691+0000","last_change":"2024-09-11T12:50:27.349463+0000","last_active":"2024-09-11T12:50:27.351691+0000","last_peered":"2024-09-11T12:50:27.351691+0000","last_clean":"2024-09-11T12:50:27.351691+0000","last_became_active":"2024-09-11T12:50:27.349183+0000","last_became_peered":"2024-09-11T12:50:27.349183+0000","last_unstale":"2024-09-11T12:50:27.351691+0000","last_undegraded":"2024-09-11T12:50:27.351691+0000","last_fullsized":"2024-09-11T12:50:27.351691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345829+0000","last_change":"2024-09-11T12:50:26.340196+0000","last_active":"2024-09-11T12:50:26.345829+0000","last_peered":"2024-09-11T12:50:26.345829+0000","last_clean":"2024-09-11T12:50:26.345829+0000","last_became_active":"2024-09-11T12:50:26.339927+0000","last_became_peered":"2024-09-11T12:50:26.339927+0000","last_unstale":"2024-09-11T12:50:26.345829+0000","last_undegraded":"2024-09-11T12:50:26.345829+0000","last_fullsized":"2024-09-11T12:50:26.345829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:57.749644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.325394+0000","last_change":"2024-09-11T12:50:26.325394+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.325394+0000","last_undegraded":"2024-09-11T12:50:26.325394+0000","last_fullsized":"2024-09-11T12:50:26.325394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337747+0000","last_change":"2024-09-11T12:50:26.338822+0000","last_active":"2024-09-11T12:50:27.337747+0000","last_peered":"2024-09-11T12:50:27.337747+0000","last_clean":"2024-09-11T12:50:27.337747+0000","last_became_active":"2024-09-11T12:50:26.338459+0000","last_became_peered":"2024-09-11T12:50:26.338459+0000","last_unstale":"2024-09-11T12:50:27.337747+0000","last_undegraded":"2024-09-11T12:50:27.337747+0000","last_fullsized":"2024-09-11T12:50:27.337747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:01.412980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347818+0000","last_change":"2024-09-11T12:50:27.344966+0000","last_active":"2024-09-11T12:50:27.347818+0000","last_peered":"2024-09-11T12:50:27.347818+0000","last_clean":"2024-09-11T12:50:27.347818+0000","last_became_active":"2024-09-11T12:50:27.344634+0000","last_became_peered":"2024-09-11T12:50:27.344634+0000","last_unstale":"2024-09-11T12:50:27.347818+0000","last_undegraded":"2024-09-11T12:50:27.347818+0000","last_fullsized":"2024-09-11T12:50:27.347818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.337771+0000","last_change":"2024-09-11T12:50:26.330959+0000","last_active":"2024-09-11T12:50:26.337771+0000","last_peered":"2024-09-11T12:50:26.337771+0000","last_clean":"2024-09-11T12:50:26.337771+0000","last_became_active":"2024-09-11T12:50:26.330722+0000","last_became_peered":"2024-09-11T12:50:26.330722+0000","last_unstale":"2024-09-11T12:50:26.337771+0000","last_undegraded":"2024-09-11T12:50:26.337771+0000","last_fullsized":"2024-09-11T12:50:26.337771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:51.012754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324459+0000","last_change":"2024-09-11T12:50:26.324459+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324459+0000","last_undegraded":"2024-09-11T12:50:26.324459+0000","last_fullsized":"2024-09-11T12:50:26.324459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337362+0000","last_change":"2024-09-11T12:50:26.337961+0000","last_active":"2024-09-11T12:50:27.337362+0000","last_peered":"2024-09-11T12:50:27.337362+0000","last_clean":"2024-09-11T12:50:27.337362+0000","last_became_active":"2024-09-11T12:50:26.337461+0000","last_became_peered":"2024-09-11T12:50:26.337461+0000","last_unstale":"2024-09-11T12:50:27.337362+0000","last_undegraded":"2024-09-11T12:50:27.337362+0000","last_fullsized":"2024-09-11T12:50:27.337362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345998+0000","last_change":"2024-09-11T12:50:27.342392+0000","last_active":"2024-09-11T12:50:27.345998+0000","last_peered":"2024-09-11T12:50:27.345998+0000","last_clean":"2024-09-11T12:50:27.345998+0000","last_became_active":"2024-09-11T12:50:27.342116+0000","last_became_peered":"2024-09-11T12:50:27.342116+0000","last_unstale":"2024-09-11T12:50:27.345998+0000","last_undegraded":"2024-09-11T12:50:27.345998+0000","last_fullsized":"2024-09-11T12:50:27.345998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325924+0000","last_change":"2024-09-11T12:50:26.330978+0000","last_active":"2024-09-11T12:50:27.325924+0000","last_peered":"2024-09-11T12:50:27.325924+0000","last_clean":"2024-09-11T12:50:27.325924+0000","last_became_active":"2024-09-11T12:50:26.330681+0000","last_became_peered":"2024-09-11T12:50:26.330681+0000","last_unstale":"2024-09-11T12:50:27.325924+0000","last_undegraded":"2024-09-11T12:50:27.325924+0000","last_fullsized":"2024-09-11T12:50:27.325924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:43.803456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.334967+0000","last_change":"2024-09-11T12:50:27.328606+0000","last_active":"2024-09-11T12:50:27.334967+0000","last_peered":"2024-09-11T12:50:27.334967+0000","last_clean":"2024-09-11T12:50:27.334967+0000","last_became_active":"2024-09-11T12:50:27.328276+0000","last_became_peered":"2024-09-11T12:50:27.328276+0000","last_unstale":"2024-09-11T12:50:27.334967+0000","last_undegraded":"2024-09-11T12:50:27.334967+0000","last_fullsized":"2024-09-11T12:50:27.334967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:36.902883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323120+0000","last_change":"2024-09-11T12:50:26.337651+0000","last_active":"2024-09-11T12:50:27.323120+0000","last_peered":"2024-09-11T12:50:27.323120+0000","last_clean":"2024-09-11T12:50:27.323120+0000","last_became_active":"2024-09-11T12:50:26.337387+0000","last_became_peered":"2024-09-11T12:50:26.337387+0000","last_unstale":"2024-09-11T12:50:27.323120+0000","last_undegraded":"2024-09-11T12:50:27.323120+0000","last_fullsized":"2024-09-11T12:50:27.323120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:52.922609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362603+0000","last_change":"2024-09-11T12:50:27.351937+0000","last_active":"2024-09-11T12:50:27.362603+0000","last_peered":"2024-09-11T12:50:27.362603+0000","last_clean":"2024-09-11T12:50:27.362603+0000","last_became_active":"2024-09-11T12:50:27.350570+0000","last_became_peered":"2024-09-11T12:50:27.350570+0000","last_unstale":"2024-09-11T12:50:27.362603+0000","last_undegraded":"2024-09-11T12:50:27.362603+0000","last_fullsized":"2024-09-11T12:50:27.362603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.328966+0000","last_change":"2024-09-11T12:50:26.344815+0000","last_active":"2024-09-11T12:50:27.328966+0000","last_peered":"2024-09-11T12:50:27.328966+0000","last_clean":"2024-09-11T12:50:27.328966+0000","last_became_active":"2024-09-11T12:50:26.344518+0000","last_became_peered":"2024-09-11T12:50:26.344518+0000","last_unstale":"2024-09-11T12:50:27.328966+0000","last_undegraded":"2024-09-11T12:50:27.328966+0000","last_fullsized":"2024-09-11T12:50:27.328966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:24.053403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347970+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:50:27.347970+0000","last_peered":"2024-09-11T12:50:27.347970+0000","last_clean":"2024-09-11T12:50:27.347970+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:50:27.347970+0000","last_undegraded":"2024-09-11T12:50:27.347970+0000","last_fullsized":"2024-09-11T12:50:27.347970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:50:31.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- 172.21.15.78:0/90964455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80ccb20 msgr2=0x7f39c806ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 --2- 172.21.15.78:0/90964455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80ccb20 0x7f39c806ecf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f39c0009900 tx=0x7f39c001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- 172.21.15.78:0/90964455 shutdown_connections 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 --2- 172.21.15.78:0/90964455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80ccb20 0x7f39c806ecf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 --2- 172.21.15.78:0/90964455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- 172.21.15.78:0/90964455 >> 172.21.15.78:0/90964455 conn(0x7f39c806ca80 msgr2=0x7f39c80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- 172.21.15.78:0/90964455 shutdown_connections 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- 172.21.15.78:0/90964455 wait complete. 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 Processor -- start 2024-09-11T12:50:32.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.002+0000 7f39cfe00640 1 -- start start 2024-09-11T12:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c80ccb20 0x7f39c81fb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8168cf0 con 0x7f39c80b4780 2024-09-11T12:50:32.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39c8168b70 con 0x7f39c80ccb20 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39c8168e70 con 0x7f39c815bc60 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60494/0 (socket says 172.21.15.78:60494) 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 -- 172.21.15.78:0/3758302743 learned_addr learned my addr 172.21.15.78:0/3758302743 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cde00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cca00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c80ccb20 0x7f39c81fb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 -- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 msgr2=0x7f39c8152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 -- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c80ccb20 msgr2=0x7f39c81fb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c80ccb20 0x7f39c81fb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 -- 172.21.15.78:0/3758302743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c8154dd0 con 0x7f39c80b4780 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cde00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:32.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cd400640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c7a0 tx=0x7f39bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc017070 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39bc005070 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc005500 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c81550b0 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c8150870 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39c8150a70 con 0x7f39c80b4780 2024-09-11T12:50:32.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.006+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f39bc01a070 con 0x7f39c80b4780 2024-09-11T12:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.010+0000 7f39c5600640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 0x7f39a8073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.010+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f39bc091560 con 0x7f39c80b4780 2024-09-11T12:50:32.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.010+0000 7f39cca00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 0x7f39a8073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.010+0000 7f39cca00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 0x7f39a8073b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006fd0 tx=0x7f39b4008040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.014+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f39c8150a70 con 0x7f39c80b4780 2024-09-11T12:50:32.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.174+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f39c806ecf0 con 0x7f39a8071680 2024-09-11T12:50:32.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.182+0000 7f39c5600640 1 -- 172.21.15.78:0/3758302743 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251605 (secure 0 0 0) 0x7f39c806ecf0 con 0x7f39a8071680 2024-09-11T12:50:32.188 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:32.189 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.182+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 msgr2=0x7f39a8073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.182+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 0x7f39a8073b40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f39b4006fd0 tx=0x7f39b4008040 comp rx=0 tx=0).stop 2024-09-11T12:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.182+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 msgr2=0x7f39c8150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c7a0 tx=0x7f39bc00cc60 comp rx=0 tx=0).stop 2024-09-11T12:50:32.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 shutdown_connections 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f39a8071680 0x7f39a8073b40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39c815bc60 0x7f39c8152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39c80ccb20 0x7f39c81fb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 --2- 172.21.15.78:0/3758302743 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39c80b4780 0x7f39c8150d90 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 >> 172.21.15.78:0/3758302743 conn(0x7f39c806ca80 msgr2=0x7f39c81518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 shutdown_connections 2024-09-11T12:50:32.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.186+0000 7f39cfe00640 1 -- 172.21.15.78:0/3758302743 wait complete. 2024-09-11T12:50:32.207 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-11T12:50:31.116340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236808,"kb_used_data":3912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737912,"statfs":{"total":773094113280,"available":772851621888,"internally_reserved":0,"allocated":4005888,"data_stored":2229312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807871"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.350863+0000","last_change":"2024-09-11T12:50:27.348834+0000","last_active":"2024-09-11T12:50:27.350863+0000","last_peered":"2024-09-11T12:50:27.350863+0000","last_clean":"2024-09-11T12:50:27.350863+0000","last_became_active":"2024-09-11T12:50:27.348574+0000","last_became_peered":"2024-09-11T12:50:27.348574+0000","last_unstale":"2024-09-11T12:50:27.350863+0000","last_undegraded":"2024-09-11T12:50:27.350863+0000","last_fullsized":"2024-09-11T12:50:27.350863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341400+0000","last_change":"2024-09-11T12:50:27.336993+0000","last_active":"2024-09-11T12:50:27.341400+0000","last_peered":"2024-09-11T12:50:27.341400+0000","last_clean":"2024-09-11T12:50:27.341400+0000","last_became_active":"2024-09-11T12:50:27.336588+0000","last_became_peered":"2024-09-11T12:50:27.336588+0000","last_unstale":"2024-09-11T12:50:27.341400+0000","last_undegraded":"2024-09-11T12:50:27.341400+0000","last_fullsized":"2024-09-11T12:50:27.341400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351364+0000","last_change":"2024-09-11T12:50:27.342867+0000","last_active":"2024-09-11T12:50:27.351364+0000","last_peered":"2024-09-11T12:50:27.351364+0000","last_clean":"2024-09-11T12:50:27.351364+0000","last_became_active":"2024-09-11T12:50:27.342576+0000","last_became_peered":"2024-09-11T12:50:27.342576+0000","last_unstale":"2024-09-11T12:50:27.351364+0000","last_undegraded":"2024-09-11T12:50:27.351364+0000","last_fullsized":"2024-09-11T12:50:27.351364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360719+0000","last_change":"2024-09-11T12:50:27.357904+0000","last_active":"2024-09-11T12:50:27.360719+0000","last_peered":"2024-09-11T12:50:27.360719+0000","last_clean":"2024-09-11T12:50:27.360719+0000","last_became_active":"2024-09-11T12:50:27.356836+0000","last_became_peered":"2024-09-11T12:50:27.356836+0000","last_unstale":"2024-09-11T12:50:27.360719+0000","last_undegraded":"2024-09-11T12:50:27.360719+0000","last_fullsized":"2024-09-11T12:50:27.360719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345686+0000","last_change":"2024-09-11T12:50:27.341619+0000","last_active":"2024-09-11T12:50:27.345686+0000","last_peered":"2024-09-11T12:50:27.345686+0000","last_clean":"2024-09-11T12:50:27.345686+0000","last_became_active":"2024-09-11T12:50:27.341345+0000","last_became_peered":"2024-09-11T12:50:27.341345+0000","last_unstale":"2024-09-11T12:50:27.345686+0000","last_undegraded":"2024-09-11T12:50:27.345686+0000","last_fullsized":"2024-09-11T12:50:27.345686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:44.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.339528+0000","last_change":"2024-09-11T12:50:27.335087+0000","last_active":"2024-09-11T12:50:27.339528+0000","last_peered":"2024-09-11T12:50:27.339528+0000","last_clean":"2024-09-11T12:50:27.339528+0000","last_became_active":"2024-09-11T12:50:27.334790+0000","last_became_peered":"2024-09-11T12:50:27.334790+0000","last_unstale":"2024-09-11T12:50:27.339528+0000","last_undegraded":"2024-09-11T12:50:27.339528+0000","last_fullsized":"2024-09-11T12:50:27.339528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364813+0000","last_change":"2024-09-11T12:50:27.355196+0000","last_active":"2024-09-11T12:50:27.364813+0000","last_peered":"2024-09-11T12:50:27.364813+0000","last_clean":"2024-09-11T12:50:27.364813+0000","last_became_active":"2024-09-11T12:50:27.353288+0000","last_became_peered":"2024-09-11T12:50:27.353288+0000","last_unstale":"2024-09-11T12:50:27.364813+0000","last_undegraded":"2024-09-11T12:50:27.364813+0000","last_fullsized":"2024-09-11T12:50:27.364813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364931+0000","last_change":"2024-09-11T12:50:27.358075+0000","last_active":"2024-09-11T12:50:27.364931+0000","last_peered":"2024-09-11T12:50:27.364931+0000","last_clean":"2024-09-11T12:50:27.364931+0000","last_became_active":"2024-09-11T12:50:27.357851+0000","last_became_peered":"2024-09-11T12:50:27.357851+0000","last_unstale":"2024-09-11T12:50:27.364931+0000","last_undegraded":"2024-09-11T12:50:27.364931+0000","last_fullsized":"2024-09-11T12:50:27.364931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352225+0000","last_change":"2024-09-11T12:50:27.343726+0000","last_active":"2024-09-11T12:50:27.352225+0000","last_peered":"2024-09-11T12:50:27.352225+0000","last_clean":"2024-09-11T12:50:27.352225+0000","last_became_active":"2024-09-11T12:50:27.343450+0000","last_became_peered":"2024-09-11T12:50:27.343450+0000","last_unstale":"2024-09-11T12:50:27.352225+0000","last_undegraded":"2024-09-11T12:50:27.352225+0000","last_fullsized":"2024-09-11T12:50:27.352225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342383+0000","last_change":"2024-09-11T12:50:27.339989+0000","last_active":"2024-09-11T12:50:27.342383+0000","last_peered":"2024-09-11T12:50:27.342383+0000","last_clean":"2024-09-11T12:50:27.342383+0000","last_became_active":"2024-09-11T12:50:27.339673+0000","last_became_peered":"2024-09-11T12:50:27.339673+0000","last_unstale":"2024-09-11T12:50:27.342383+0000","last_undegraded":"2024-09-11T12:50:27.342383+0000","last_fullsized":"2024-09-11T12:50:27.342383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365270+0000","last_change":"2024-09-11T12:50:27.363103+0000","last_active":"2024-09-11T12:50:27.365270+0000","last_peered":"2024-09-11T12:50:27.365270+0000","last_clean":"2024-09-11T12:50:27.365270+0000","last_became_active":"2024-09-11T12:50:27.362746+0000","last_became_peered":"2024-09-11T12:50:27.362746+0000","last_unstale":"2024-09-11T12:50:27.365270+0000","last_undegraded":"2024-09-11T12:50:27.365270+0000","last_fullsized":"2024-09-11T12:50:27.365270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347758+0000","last_change":"2024-09-11T12:50:27.329779+0000","last_active":"2024-09-11T12:50:27.347758+0000","last_peered":"2024-09-11T12:50:27.347758+0000","last_clean":"2024-09-11T12:50:27.347758+0000","last_became_active":"2024-09-11T12:50:27.327796+0000","last_became_peered":"2024-09-11T12:50:27.327796+0000","last_unstale":"2024-09-11T12:50:27.347758+0000","last_undegraded":"2024-09-11T12:50:27.347758+0000","last_fullsized":"2024-09-11T12:50:27.347758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352853+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.352853+0000","last_peered":"2024-09-11T12:50:27.352853+0000","last_clean":"2024-09-11T12:50:27.352853+0000","last_became_active":"2024-09-11T12:50:27.344834+0000","last_became_peered":"2024-09-11T12:50:27.344834+0000","last_unstale":"2024-09-11T12:50:27.352853+0000","last_undegraded":"2024-09-11T12:50:27.352853+0000","last_fullsized":"2024-09-11T12:50:27.352853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:48.794525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341535+0000","last_change":"2024-09-11T12:50:27.339363+0000","last_active":"2024-09-11T12:50:27.341535+0000","last_peered":"2024-09-11T12:50:27.341535+0000","last_clean":"2024-09-11T12:50:27.341535+0000","last_became_active":"2024-09-11T12:50:27.339131+0000","last_became_peered":"2024-09-11T12:50:27.339131+0000","last_unstale":"2024-09-11T12:50:27.341535+0000","last_undegraded":"2024-09-11T12:50:27.341535+0000","last_fullsized":"2024-09-11T12:50:27.341535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344599+0000","last_change":"2024-09-11T12:50:27.335582+0000","last_active":"2024-09-11T12:50:27.344599+0000","last_peered":"2024-09-11T12:50:27.344599+0000","last_clean":"2024-09-11T12:50:27.344599+0000","last_became_active":"2024-09-11T12:50:27.335002+0000","last_became_peered":"2024-09-11T12:50:27.335002+0000","last_unstale":"2024-09-11T12:50:27.344599+0000","last_undegraded":"2024-09-11T12:50:27.344599+0000","last_fullsized":"2024-09-11T12:50:27.344599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365195+0000","last_change":"2024-09-11T12:50:27.363062+0000","last_active":"2024-09-11T12:50:27.365195+0000","last_peered":"2024-09-11T12:50:27.365195+0000","last_clean":"2024-09-11T12:50:27.365195+0000","last_became_active":"2024-09-11T12:50:27.362788+0000","last_became_peered":"2024-09-11T12:50:27.362788+0000","last_unstale":"2024-09-11T12:50:27.365195+0000","last_undegraded":"2024-09-11T12:50:27.365195+0000","last_fullsized":"2024-09-11T12:50:27.365195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364886+0000","last_change":"2024-09-11T12:50:27.357790+0000","last_active":"2024-09-11T12:50:27.364886+0000","last_peered":"2024-09-11T12:50:27.364886+0000","last_clean":"2024-09-11T12:50:27.364886+0000","last_became_active":"2024-09-11T12:50:27.353639+0000","last_became_peered":"2024-09-11T12:50:27.353639+0000","last_unstale":"2024-09-11T12:50:27.364886+0000","last_undegraded":"2024-09-11T12:50:27.364886+0000","last_fullsized":"2024-09-11T12:50:27.364886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341377+0000","last_change":"2024-09-11T12:50:27.336825+0000","last_active":"2024-09-11T12:50:27.341377+0000","last_peered":"2024-09-11T12:50:27.341377+0000","last_clean":"2024-09-11T12:50:27.341377+0000","last_became_active":"2024-09-11T12:50:27.336241+0000","last_became_peered":"2024-09-11T12:50:27.336241+0000","last_unstale":"2024-09-11T12:50:27.341377+0000","last_undegraded":"2024-09-11T12:50:27.341377+0000","last_fullsized":"2024-09-11T12:50:27.341377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346581+0000","last_change":"2024-09-11T12:50:27.344954+0000","last_active":"2024-09-11T12:50:27.346581+0000","last_peered":"2024-09-11T12:50:27.346581+0000","last_clean":"2024-09-11T12:50:27.346581+0000","last_became_active":"2024-09-11T12:50:27.344681+0000","last_became_peered":"2024-09-11T12:50:27.344681+0000","last_unstale":"2024-09-11T12:50:27.346581+0000","last_undegraded":"2024-09-11T12:50:27.346581+0000","last_fullsized":"2024-09-11T12:50:27.346581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357472+0000","last_change":"2024-09-11T12:50:27.352798+0000","last_active":"2024-09-11T12:50:27.357472+0000","last_peered":"2024-09-11T12:50:27.357472+0000","last_clean":"2024-09-11T12:50:27.357472+0000","last_became_active":"2024-09-11T12:50:27.349194+0000","last_became_peered":"2024-09-11T12:50:27.349194+0000","last_unstale":"2024-09-11T12:50:27.357472+0000","last_undegraded":"2024-09-11T12:50:27.357472+0000","last_fullsized":"2024-09-11T12:50:27.357472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:32.390636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351801+0000","last_change":"2024-09-11T12:50:27.343151+0000","last_active":"2024-09-11T12:50:27.351801+0000","last_peered":"2024-09-11T12:50:27.351801+0000","last_clean":"2024-09-11T12:50:27.351801+0000","last_became_active":"2024-09-11T12:50:27.339404+0000","last_became_peered":"2024-09-11T12:50:27.339404+0000","last_unstale":"2024-09-11T12:50:27.351801+0000","last_undegraded":"2024-09-11T12:50:27.351801+0000","last_fullsized":"2024-09-11T12:50:27.351801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.336093+0000","last_change":"2024-09-11T12:50:27.330086+0000","last_active":"2024-09-11T12:50:27.336093+0000","last_peered":"2024-09-11T12:50:27.336093+0000","last_clean":"2024-09-11T12:50:27.336093+0000","last_became_active":"2024-09-11T12:50:27.329846+0000","last_became_peered":"2024-09-11T12:50:27.329846+0000","last_unstale":"2024-09-11T12:50:27.336093+0000","last_undegraded":"2024-09-11T12:50:27.336093+0000","last_fullsized":"2024-09-11T12:50:27.336093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:12.211705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324523+0000","last_change":"2024-09-11T12:50:26.324523+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324523+0000","last_undegraded":"2024-09-11T12:50:26.324523+0000","last_fullsized":"2024-09-11T12:50:26.324523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343833+0000","last_change":"2024-09-11T12:50:27.337469+0000","last_active":"2024-09-11T12:50:27.343833+0000","last_peered":"2024-09-11T12:50:27.343833+0000","last_clean":"2024-09-11T12:50:27.343833+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.343833+0000","last_undegraded":"2024-09-11T12:50:27.343833+0000","last_fullsized":"2024-09-11T12:50:27.343833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359468+0000","last_change":"2024-09-11T12:50:27.355625+0000","last_active":"2024-09-11T12:50:27.359468+0000","last_peered":"2024-09-11T12:50:27.359468+0000","last_clean":"2024-09-11T12:50:27.359468+0000","last_became_active":"2024-09-11T12:50:27.355274+0000","last_became_peered":"2024-09-11T12:50:27.355274+0000","last_unstale":"2024-09-11T12:50:27.359468+0000","last_undegraded":"2024-09-11T12:50:27.359468+0000","last_fullsized":"2024-09-11T12:50:27.359468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359556+0000","last_change":"2024-09-11T12:50:27.356361+0000","last_active":"2024-09-11T12:50:27.359556+0000","last_peered":"2024-09-11T12:50:27.359556+0000","last_clean":"2024-09-11T12:50:27.359556+0000","last_became_active":"2024-09-11T12:50:27.356074+0000","last_became_peered":"2024-09-11T12:50:27.356074+0000","last_unstale":"2024-09-11T12:50:27.359556+0000","last_undegraded":"2024-09-11T12:50:27.359556+0000","last_fullsized":"2024-09-11T12:50:27.359556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359308+0000","last_change":"2024-09-11T12:50:27.357026+0000","last_active":"2024-09-11T12:50:27.359308+0000","last_peered":"2024-09-11T12:50:27.359308+0000","last_clean":"2024-09-11T12:50:27.359308+0000","last_became_active":"2024-09-11T12:50:27.356785+0000","last_became_peered":"2024-09-11T12:50:27.356785+0000","last_unstale":"2024-09-11T12:50:27.359308+0000","last_undegraded":"2024-09-11T12:50:27.359308+0000","last_fullsized":"2024-09-11T12:50:27.359308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347829+0000","last_change":"2024-09-11T12:50:27.336803+0000","last_active":"2024-09-11T12:50:27.347829+0000","last_peered":"2024-09-11T12:50:27.347829+0000","last_clean":"2024-09-11T12:50:27.347829+0000","last_became_active":"2024-09-11T12:50:27.334739+0000","last_became_peered":"2024-09-11T12:50:27.334739+0000","last_unstale":"2024-09-11T12:50:27.347829+0000","last_undegraded":"2024-09-11T12:50:27.347829+0000","last_fullsized":"2024-09-11T12:50:27.347829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335097+0000","last_change":"2024-09-11T12:50:27.329208+0000","last_active":"2024-09-11T12:50:27.335097+0000","last_peered":"2024-09-11T12:50:27.335097+0000","last_clean":"2024-09-11T12:50:27.335097+0000","last_became_active":"2024-09-11T12:50:27.328939+0000","last_became_peered":"2024-09-11T12:50:27.328939+0000","last_unstale":"2024-09-11T12:50:27.335097+0000","last_undegraded":"2024-09-11T12:50:27.335097+0000","last_fullsized":"2024-09-11T12:50:27.335097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:25.059566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353911+0000","last_change":"2024-09-11T12:50:27.348968+0000","last_active":"2024-09-11T12:50:27.353911+0000","last_peered":"2024-09-11T12:50:27.353911+0000","last_clean":"2024-09-11T12:50:27.353911+0000","last_became_active":"2024-09-11T12:50:27.348693+0000","last_became_peered":"2024-09-11T12:50:27.348693+0000","last_unstale":"2024-09-11T12:50:27.353911+0000","last_undegraded":"2024-09-11T12:50:27.353911+0000","last_fullsized":"2024-09-11T12:50:27.353911+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345469+0000","last_change":"2024-09-11T12:50:27.341331+0000","last_active":"2024-09-11T12:50:27.345469+0000","last_peered":"2024-09-11T12:50:27.345469+0000","last_clean":"2024-09-11T12:50:27.345469+0000","last_became_active":"2024-09-11T12:50:27.341038+0000","last_became_peered":"2024-09-11T12:50:27.341038+0000","last_unstale":"2024-09-11T12:50:27.345469+0000","last_undegraded":"2024-09-11T12:50:27.345469+0000","last_fullsized":"2024-09-11T12:50:27.345469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364720+0000","last_change":"2024-09-11T12:50:27.356104+0000","last_active":"2024-09-11T12:50:27.364720+0000","last_peered":"2024-09-11T12:50:27.364720+0000","last_clean":"2024-09-11T12:50:27.364720+0000","last_became_active":"2024-09-11T12:50:27.355113+0000","last_became_peered":"2024-09-11T12:50:27.355113+0000","last_unstale":"2024-09-11T12:50:27.364720+0000","last_undegraded":"2024-09-11T12:50:27.364720+0000","last_fullsized":"2024-09-11T12:50:27.364720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324419+0000","last_change":"2024-09-11T12:50:26.324419+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324419+0000","last_undegraded":"2024-09-11T12:50:26.324419+0000","last_fullsized":"2024-09-11T12:50:26.324419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365295+0000","last_change":"2024-09-11T12:50:26.343869+0000","last_active":"2024-09-11T12:50:27.365295+0000","last_peered":"2024-09-11T12:50:27.365295+0000","last_clean":"2024-09-11T12:50:27.365295+0000","last_became_active":"2024-09-11T12:50:26.343588+0000","last_became_peered":"2024-09-11T12:50:26.343588+0000","last_unstale":"2024-09-11T12:50:27.365295+0000","last_undegraded":"2024-09-11T12:50:27.365295+0000","last_fullsized":"2024-09-11T12:50:27.365295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344316+0000","last_change":"2024-09-11T12:50:27.340558+0000","last_active":"2024-09-11T12:50:27.344316+0000","last_peered":"2024-09-11T12:50:27.344316+0000","last_clean":"2024-09-11T12:50:27.344316+0000","last_became_active":"2024-09-11T12:50:27.337140+0000","last_became_peered":"2024-09-11T12:50:27.337140+0000","last_unstale":"2024-09-11T12:50:27.344316+0000","last_undegraded":"2024-09-11T12:50:27.344316+0000","last_fullsized":"2024-09-11T12:50:27.344316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.341744+0000","last_change":"2024-09-11T12:50:26.335186+0000","last_active":"2024-09-11T12:50:26.341744+0000","last_peered":"2024-09-11T12:50:26.341744+0000","last_clean":"2024-09-11T12:50:26.341744+0000","last_became_active":"2024-09-11T12:50:26.334825+0000","last_became_peered":"2024-09-11T12:50:26.334825+0000","last_unstale":"2024-09-11T12:50:26.341744+0000","last_undegraded":"2024-09-11T12:50:26.341744+0000","last_fullsized":"2024-09-11T12:50:26.341744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:54.260958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335753+0000","last_change":"2024-09-11T12:50:27.328627+0000","last_active":"2024-09-11T12:50:27.335753+0000","last_peered":"2024-09-11T12:50:27.335753+0000","last_clean":"2024-09-11T12:50:27.335753+0000","last_became_active":"2024-09-11T12:50:27.328292+0000","last_became_peered":"2024-09-11T12:50:27.328292+0000","last_unstale":"2024-09-11T12:50:27.335753+0000","last_undegraded":"2024-09-11T12:50:27.335753+0000","last_fullsized":"2024-09-11T12:50:27.335753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:27.421789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360603+0000","last_change":"2024-09-11T12:50:26.342258+0000","last_active":"2024-09-11T12:50:27.360603+0000","last_peered":"2024-09-11T12:50:27.360603+0000","last_clean":"2024-09-11T12:50:27.360603+0000","last_became_active":"2024-09-11T12:50:26.341986+0000","last_became_peered":"2024-09-11T12:50:26.341986+0000","last_unstale":"2024-09-11T12:50:27.360603+0000","last_undegraded":"2024-09-11T12:50:27.360603+0000","last_fullsized":"2024-09-11T12:50:27.360603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:46.357747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359368+0000","last_change":"2024-09-11T12:50:27.357376+0000","last_active":"2024-09-11T12:50:27.359368+0000","last_peered":"2024-09-11T12:50:27.359368+0000","last_clean":"2024-09-11T12:50:27.359368+0000","last_became_active":"2024-09-11T12:50:27.357016+0000","last_became_peered":"2024-09-11T12:50:27.357016+0000","last_unstale":"2024-09-11T12:50:27.359368+0000","last_undegraded":"2024-09-11T12:50:27.359368+0000","last_fullsized":"2024-09-11T12:50:27.359368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360981+0000","last_change":"2024-09-11T12:50:26.343854+0000","last_active":"2024-09-11T12:50:27.360981+0000","last_peered":"2024-09-11T12:50:27.360981+0000","last_clean":"2024-09-11T12:50:27.360981+0000","last_became_active":"2024-09-11T12:50:26.341771+0000","last_became_peered":"2024-09-11T12:50:26.341771+0000","last_unstale":"2024-09-11T12:50:27.360981+0000","last_undegraded":"2024-09-11T12:50:27.360981+0000","last_fullsized":"2024-09-11T12:50:27.360981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:51.008327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364638+0000","last_change":"2024-09-11T12:50:27.352862+0000","last_active":"2024-09-11T12:50:27.364638+0000","last_peered":"2024-09-11T12:50:27.364638+0000","last_clean":"2024-09-11T12:50:27.364638+0000","last_became_active":"2024-09-11T12:50:27.352467+0000","last_became_peered":"2024-09-11T12:50:27.352467+0000","last_unstale":"2024-09-11T12:50:27.364638+0000","last_undegraded":"2024-09-11T12:50:27.364638+0000","last_fullsized":"2024-09-11T12:50:27.364638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363377+0000","last_change":"2024-09-11T12:50:26.342659+0000","last_active":"2024-09-11T12:50:27.363377+0000","last_peered":"2024-09-11T12:50:27.363377+0000","last_clean":"2024-09-11T12:50:27.363377+0000","last_became_active":"2024-09-11T12:50:26.342331+0000","last_became_peered":"2024-09-11T12:50:26.342331+0000","last_unstale":"2024-09-11T12:50:27.363377+0000","last_undegraded":"2024-09-11T12:50:27.363377+0000","last_fullsized":"2024-09-11T12:50:27.363377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:18.698135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352409+0000","last_change":"2024-09-11T12:50:27.347574+0000","last_active":"2024-09-11T12:50:27.352409+0000","last_peered":"2024-09-11T12:50:27.352409+0000","last_clean":"2024-09-11T12:50:27.352409+0000","last_became_active":"2024-09-11T12:50:27.347302+0000","last_became_peered":"2024-09-11T12:50:27.347302+0000","last_unstale":"2024-09-11T12:50:27.352409+0000","last_undegraded":"2024-09-11T12:50:27.352409+0000","last_fullsized":"2024-09-11T12:50:27.352409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.366007+0000","last_change":"2024-09-11T12:50:26.341601+0000","last_active":"2024-09-11T12:50:27.366007+0000","last_peered":"2024-09-11T12:50:27.366007+0000","last_clean":"2024-09-11T12:50:27.366007+0000","last_became_active":"2024-09-11T12:50:26.341238+0000","last_became_peered":"2024-09-11T12:50:26.341238+0000","last_unstale":"2024-09-11T12:50:27.366007+0000","last_undegraded":"2024-09-11T12:50:27.366007+0000","last_fullsized":"2024-09-11T12:50:27.366007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:46.542517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335996+0000","last_change":"2024-09-11T12:50:27.333846+0000","last_active":"2024-09-11T12:50:27.335996+0000","last_peered":"2024-09-11T12:50:27.335996+0000","last_clean":"2024-09-11T12:50:27.335996+0000","last_became_active":"2024-09-11T12:50:27.333500+0000","last_became_peered":"2024-09-11T12:50:27.333500+0000","last_unstale":"2024-09-11T12:50:27.335996+0000","last_undegraded":"2024-09-11T12:50:27.335996+0000","last_fullsized":"2024-09-11T12:50:27.335996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323107+0000","last_change":"2024-09-11T12:50:26.339433+0000","last_active":"2024-09-11T12:50:27.323107+0000","last_peered":"2024-09-11T12:50:27.323107+0000","last_clean":"2024-09-11T12:50:27.323107+0000","last_became_active":"2024-09-11T12:50:26.338991+0000","last_became_peered":"2024-09-11T12:50:26.338991+0000","last_unstale":"2024-09-11T12:50:27.323107+0000","last_undegraded":"2024-09-11T12:50:27.323107+0000","last_fullsized":"2024-09-11T12:50:27.323107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354941+0000","last_change":"2024-09-11T12:50:27.351561+0000","last_active":"2024-09-11T12:50:27.354941+0000","last_peered":"2024-09-11T12:50:27.354941+0000","last_clean":"2024-09-11T12:50:27.354941+0000","last_became_active":"2024-09-11T12:50:27.351277+0000","last_became_peered":"2024-09-11T12:50:27.351277+0000","last_unstale":"2024-09-11T12:50:27.354941+0000","last_undegraded":"2024-09-11T12:50:27.354941+0000","last_fullsized":"2024-09-11T12:50:27.354941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323830+0000","last_change":"2024-09-11T12:50:26.337847+0000","last_active":"2024-09-11T12:50:27.323830+0000","last_peered":"2024-09-11T12:50:27.323830+0000","last_clean":"2024-09-11T12:50:27.323830+0000","last_became_active":"2024-09-11T12:50:26.337533+0000","last_became_peered":"2024-09-11T12:50:26.337533+0000","last_unstale":"2024-09-11T12:50:27.323830+0000","last_undegraded":"2024-09-11T12:50:27.323830+0000","last_fullsized":"2024-09-11T12:50:27.323830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:06.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346859+0000","last_change":"2024-09-11T12:50:27.330135+0000","last_active":"2024-09-11T12:50:27.346859+0000","last_peered":"2024-09-11T12:50:27.346859+0000","last_clean":"2024-09-11T12:50:27.346859+0000","last_became_active":"2024-09-11T12:50:27.329324+0000","last_became_peered":"2024-09-11T12:50:27.329324+0000","last_unstale":"2024-09-11T12:50:27.346859+0000","last_undegraded":"2024-09-11T12:50:27.346859+0000","last_fullsized":"2024-09-11T12:50:27.346859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324111+0000","last_change":"2024-09-11T12:50:26.343684+0000","last_active":"2024-09-11T12:50:27.324111+0000","last_peered":"2024-09-11T12:50:27.324111+0000","last_clean":"2024-09-11T12:50:27.324111+0000","last_became_active":"2024-09-11T12:50:26.343386+0000","last_became_peered":"2024-09-11T12:50:26.343386+0000","last_unstale":"2024-09-11T12:50:27.324111+0000","last_undegraded":"2024-09-11T12:50:27.324111+0000","last_fullsized":"2024-09-11T12:50:27.324111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:06.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.348808+0000","last_change":"2024-09-11T12:50:27.346100+0000","last_active":"2024-09-11T12:50:27.348808+0000","last_peered":"2024-09-11T12:50:27.348808+0000","last_clean":"2024-09-11T12:50:27.348808+0000","last_became_active":"2024-09-11T12:50:27.345380+0000","last_became_peered":"2024-09-11T12:50:27.345380+0000","last_unstale":"2024-09-11T12:50:27.348808+0000","last_undegraded":"2024-09-11T12:50:27.348808+0000","last_fullsized":"2024-09-11T12:50:27.348808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345851+0000","last_change":"2024-09-11T12:50:26.340838+0000","last_active":"2024-09-11T12:50:26.345851+0000","last_peered":"2024-09-11T12:50:26.345851+0000","last_clean":"2024-09-11T12:50:26.345851+0000","last_became_active":"2024-09-11T12:50:26.340531+0000","last_became_peered":"2024-09-11T12:50:26.340531+0000","last_unstale":"2024-09-11T12:50:26.345851+0000","last_undegraded":"2024-09-11T12:50:26.345851+0000","last_fullsized":"2024-09-11T12:50:26.345851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:03.491437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359623+0000","last_change":"2024-09-11T12:50:27.356884+0000","last_active":"2024-09-11T12:50:27.359623+0000","last_peered":"2024-09-11T12:50:27.359623+0000","last_clean":"2024-09-11T12:50:27.359623+0000","last_became_active":"2024-09-11T12:50:27.355050+0000","last_became_peered":"2024-09-11T12:50:27.355050+0000","last_unstale":"2024-09-11T12:50:27.359623+0000","last_undegraded":"2024-09-11T12:50:27.359623+0000","last_fullsized":"2024-09-11T12:50:27.359623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365551+0000","last_change":"2024-09-11T12:50:26.334350+0000","last_active":"2024-09-11T12:50:27.365551+0000","last_peered":"2024-09-11T12:50:27.365551+0000","last_clean":"2024-09-11T12:50:27.365551+0000","last_became_active":"2024-09-11T12:50:26.334014+0000","last_became_peered":"2024-09-11T12:50:26.334014+0000","last_unstale":"2024-09-11T12:50:27.365551+0000","last_undegraded":"2024-09-11T12:50:27.365551+0000","last_fullsized":"2024-09-11T12:50:27.365551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351493+0000","last_change":"2024-09-11T12:50:27.350052+0000","last_active":"2024-09-11T12:50:27.351493+0000","last_peered":"2024-09-11T12:50:27.351493+0000","last_clean":"2024-09-11T12:50:27.351493+0000","last_became_active":"2024-09-11T12:50:27.349787+0000","last_became_peered":"2024-09-11T12:50:27.349787+0000","last_unstale":"2024-09-11T12:50:27.351493+0000","last_undegraded":"2024-09-11T12:50:27.351493+0000","last_fullsized":"2024-09-11T12:50:27.351493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.371174+0000","last_change":"2024-09-11T12:50:26.342451+0000","last_active":"2024-09-11T12:50:27.371174+0000","last_peered":"2024-09-11T12:50:27.371174+0000","last_clean":"2024-09-11T12:50:27.371174+0000","last_became_active":"2024-09-11T12:50:26.341973+0000","last_became_peered":"2024-09-11T12:50:26.341973+0000","last_unstale":"2024-09-11T12:50:27.371174+0000","last_undegraded":"2024-09-11T12:50:27.371174+0000","last_fullsized":"2024-09-11T12:50:27.371174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:54.047639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354826+0000","last_change":"2024-09-11T12:50:27.349605+0000","last_active":"2024-09-11T12:50:27.354826+0000","last_peered":"2024-09-11T12:50:27.354826+0000","last_clean":"2024-09-11T12:50:27.354826+0000","last_became_active":"2024-09-11T12:50:27.349304+0000","last_became_peered":"2024-09-11T12:50:27.349304+0000","last_unstale":"2024-09-11T12:50:27.354826+0000","last_undegraded":"2024-09-11T12:50:27.354826+0000","last_fullsized":"2024-09-11T12:50:27.354826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362391+0000","last_change":"2024-09-11T12:50:26.337760+0000","last_active":"2024-09-11T12:50:27.362391+0000","last_peered":"2024-09-11T12:50:27.362391+0000","last_clean":"2024-09-11T12:50:27.362391+0000","last_became_active":"2024-09-11T12:50:26.337496+0000","last_became_peered":"2024-09-11T12:50:26.337496+0000","last_unstale":"2024-09-11T12:50:27.362391+0000","last_undegraded":"2024-09-11T12:50:27.362391+0000","last_fullsized":"2024-09-11T12:50:27.362391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:09.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346498+0000","last_change":"2024-09-11T12:50:27.342946+0000","last_active":"2024-09-11T12:50:27.346498+0000","last_peered":"2024-09-11T12:50:27.346498+0000","last_clean":"2024-09-11T12:50:27.346498+0000","last_became_active":"2024-09-11T12:50:27.342692+0000","last_became_peered":"2024-09-11T12:50:27.342692+0000","last_unstale":"2024-09-11T12:50:27.346498+0000","last_undegraded":"2024-09-11T12:50:27.346498+0000","last_fullsized":"2024-09-11T12:50:27.346498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337180+0000","last_change":"2024-09-11T12:50:26.337698+0000","last_active":"2024-09-11T12:50:27.337180+0000","last_peered":"2024-09-11T12:50:27.337180+0000","last_clean":"2024-09-11T12:50:27.337180+0000","last_became_active":"2024-09-11T12:50:26.337145+0000","last_became_peered":"2024-09-11T12:50:26.337145+0000","last_unstale":"2024-09-11T12:50:27.337180+0000","last_undegraded":"2024-09-11T12:50:27.337180+0000","last_fullsized":"2024-09-11T12:50:27.337180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364743+0000","last_change":"2024-09-11T12:50:27.356536+0000","last_active":"2024-09-11T12:50:27.364743+0000","last_peered":"2024-09-11T12:50:27.364743+0000","last_clean":"2024-09-11T12:50:27.364743+0000","last_became_active":"2024-09-11T12:50:27.356235+0000","last_became_peered":"2024-09-11T12:50:27.356235+0000","last_unstale":"2024-09-11T12:50:27.364743+0000","last_undegraded":"2024-09-11T12:50:27.364743+0000","last_fullsized":"2024-09-11T12:50:27.364743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323110+0000","last_change":"2024-09-11T12:50:26.339598+0000","last_active":"2024-09-11T12:50:27.323110+0000","last_peered":"2024-09-11T12:50:27.323110+0000","last_clean":"2024-09-11T12:50:27.323110+0000","last_became_active":"2024-09-11T12:50:26.339322+0000","last_became_peered":"2024-09-11T12:50:26.339322+0000","last_unstale":"2024-09-11T12:50:27.323110+0000","last_undegraded":"2024-09-11T12:50:27.323110+0000","last_fullsized":"2024-09-11T12:50:27.323110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:45.625861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344441+0000","last_change":"2024-09-11T12:50:27.341021+0000","last_active":"2024-09-11T12:50:27.344441+0000","last_peered":"2024-09-11T12:50:27.344441+0000","last_clean":"2024-09-11T12:50:27.344441+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.344441+0000","last_undegraded":"2024-09-11T12:50:27.344441+0000","last_fullsized":"2024-09-11T12:50:27.344441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365719+0000","last_change":"2024-09-11T12:50:26.341481+0000","last_active":"2024-09-11T12:50:27.365719+0000","last_peered":"2024-09-11T12:50:27.365719+0000","last_clean":"2024-09-11T12:50:27.365719+0000","last_became_active":"2024-09-11T12:50:26.341148+0000","last_became_peered":"2024-09-11T12:50:26.341148+0000","last_unstale":"2024-09-11T12:50:27.365719+0000","last_undegraded":"2024-09-11T12:50:27.365719+0000","last_fullsized":"2024-09-11T12:50:27.365719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:41.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352075+0000","last_change":"2024-09-11T12:50:27.343792+0000","last_active":"2024-09-11T12:50:27.352075+0000","last_peered":"2024-09-11T12:50:27.352075+0000","last_clean":"2024-09-11T12:50:27.352075+0000","last_became_active":"2024-09-11T12:50:27.343522+0000","last_became_peered":"2024-09-11T12:50:27.343522+0000","last_unstale":"2024-09-11T12:50:27.352075+0000","last_undegraded":"2024-09-11T12:50:27.352075+0000","last_fullsized":"2024-09-11T12:50:27.352075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365752+0000","last_change":"2024-09-11T12:50:26.342891+0000","last_active":"2024-09-11T12:50:27.365752+0000","last_peered":"2024-09-11T12:50:27.365752+0000","last_clean":"2024-09-11T12:50:27.365752+0000","last_became_active":"2024-09-11T12:50:26.342639+0000","last_became_peered":"2024-09-11T12:50:26.342639+0000","last_unstale":"2024-09-11T12:50:27.365752+0000","last_undegraded":"2024-09-11T12:50:27.365752+0000","last_fullsized":"2024-09-11T12:50:27.365752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344203+0000","last_change":"2024-09-11T12:50:27.335239+0000","last_active":"2024-09-11T12:50:27.344203+0000","last_peered":"2024-09-11T12:50:27.344203+0000","last_clean":"2024-09-11T12:50:27.344203+0000","last_became_active":"2024-09-11T12:50:27.333463+0000","last_became_peered":"2024-09-11T12:50:27.333463+0000","last_unstale":"2024-09-11T12:50:27.344203+0000","last_undegraded":"2024-09-11T12:50:27.344203+0000","last_fullsized":"2024-09-11T12:50:27.344203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.361835+0000","last_change":"2024-09-11T12:50:26.337827+0000","last_active":"2024-09-11T12:50:27.361835+0000","last_peered":"2024-09-11T12:50:27.361835+0000","last_clean":"2024-09-11T12:50:27.361835+0000","last_became_active":"2024-09-11T12:50:26.337466+0000","last_became_peered":"2024-09-11T12:50:26.337466+0000","last_unstale":"2024-09-11T12:50:27.361835+0000","last_undegraded":"2024-09-11T12:50:27.361835+0000","last_fullsized":"2024-09-11T12:50:27.361835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:12.538604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342628+0000","last_change":"2024-09-11T12:50:27.338996+0000","last_active":"2024-09-11T12:50:27.342628+0000","last_peered":"2024-09-11T12:50:27.342628+0000","last_clean":"2024-09-11T12:50:27.342628+0000","last_became_active":"2024-09-11T12:50:27.338705+0000","last_became_peered":"2024-09-11T12:50:27.338705+0000","last_unstale":"2024-09-11T12:50:27.342628+0000","last_undegraded":"2024-09-11T12:50:27.342628+0000","last_fullsized":"2024-09-11T12:50:27.342628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363280+0000","last_change":"2024-09-11T12:50:26.339176+0000","last_active":"2024-09-11T12:50:27.363280+0000","last_peered":"2024-09-11T12:50:27.363280+0000","last_clean":"2024-09-11T12:50:27.363280+0000","last_became_active":"2024-09-11T12:50:26.338882+0000","last_became_peered":"2024-09-11T12:50:26.338882+0000","last_unstale":"2024-09-11T12:50:27.363280+0000","last_undegraded":"2024-09-11T12:50:27.363280+0000","last_fullsized":"2024-09-11T12:50:27.363280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:11.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.358426+0000","last_change":"2024-09-11T12:50:27.353963+0000","last_active":"2024-09-11T12:50:27.358426+0000","last_peered":"2024-09-11T12:50:27.358426+0000","last_clean":"2024-09-11T12:50:27.358426+0000","last_became_active":"2024-09-11T12:50:27.353684+0000","last_became_peered":"2024-09-11T12:50:27.353684+0000","last_unstale":"2024-09-11T12:50:27.358426+0000","last_undegraded":"2024-09-11T12:50:27.358426+0000","last_fullsized":"2024-09-11T12:50:27.358426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:07.300287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324319+0000","last_change":"2024-09-11T12:50:26.344464+0000","last_active":"2024-09-11T12:50:27.324319+0000","last_peered":"2024-09-11T12:50:27.324319+0000","last_clean":"2024-09-11T12:50:27.324319+0000","last_became_active":"2024-09-11T12:50:26.344093+0000","last_became_peered":"2024-09-11T12:50:26.344093+0000","last_unstale":"2024-09-11T12:50:27.324319+0000","last_undegraded":"2024-09-11T12:50:27.324319+0000","last_fullsized":"2024-09-11T12:50:27.324319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:47.700330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343140+0000","last_change":"2024-09-11T12:50:27.338133+0000","last_active":"2024-09-11T12:50:27.343140+0000","last_peered":"2024-09-11T12:50:27.343140+0000","last_clean":"2024-09-11T12:50:27.343140+0000","last_became_active":"2024-09-11T12:50:27.337816+0000","last_became_peered":"2024-09-11T12:50:27.337816+0000","last_unstale":"2024-09-11T12:50:27.343140+0000","last_undegraded":"2024-09-11T12:50:27.343140+0000","last_fullsized":"2024-09-11T12:50:27.343140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:36:21.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.344835+0000","last_change":"2024-09-11T12:50:26.339488+0000","last_active":"2024-09-11T12:50:26.344835+0000","last_peered":"2024-09-11T12:50:26.344835+0000","last_clean":"2024-09-11T12:50:26.344835+0000","last_became_active":"2024-09-11T12:50:26.339267+0000","last_became_peered":"2024-09-11T12:50:26.339267+0000","last_unstale":"2024-09-11T12:50:26.344835+0000","last_undegraded":"2024-09-11T12:50:26.344835+0000","last_fullsized":"2024-09-11T12:50:26.344835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:54:09.177646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353434+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.353434+0000","last_peered":"2024-09-11T12:50:27.353434+0000","last_clean":"2024-09-11T12:50:27.353434+0000","last_became_active":"2024-09-11T12:50:27.345261+0000","last_became_peered":"2024-09-11T12:50:27.345261+0000","last_unstale":"2024-09-11T12:50:27.353434+0000","last_undegraded":"2024-09-11T12:50:27.353434+0000","last_fullsized":"2024-09-11T12:50:27.353434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:58.388446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.330245+0000","last_change":"2024-09-11T12:50:26.346500+0000","last_active":"2024-09-11T12:50:27.330245+0000","last_peered":"2024-09-11T12:50:27.330245+0000","last_clean":"2024-09-11T12:50:27.330245+0000","last_became_active":"2024-09-11T12:50:26.346237+0000","last_became_peered":"2024-09-11T12:50:26.346237+0000","last_unstale":"2024-09-11T12:50:27.330245+0000","last_undegraded":"2024-09-11T12:50:27.330245+0000","last_fullsized":"2024-09-11T12:50:27.330245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346072+0000","last_change":"2024-09-11T12:50:27.342983+0000","last_active":"2024-09-11T12:50:27.346072+0000","last_peered":"2024-09-11T12:50:27.346072+0000","last_clean":"2024-09-11T12:50:27.346072+0000","last_became_active":"2024-09-11T12:50:27.342726+0000","last_became_peered":"2024-09-11T12:50:27.342726+0000","last_unstale":"2024-09-11T12:50:27.346072+0000","last_undegraded":"2024-09-11T12:50:27.346072+0000","last_fullsized":"2024-09-11T12:50:27.346072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:39.629583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325749+0000","last_change":"2024-09-11T12:50:26.339279+0000","last_active":"2024-09-11T12:50:27.325749+0000","last_peered":"2024-09-11T12:50:27.325749+0000","last_clean":"2024-09-11T12:50:27.325749+0000","last_became_active":"2024-09-11T12:50:26.338240+0000","last_became_peered":"2024-09-11T12:50:26.338240+0000","last_unstale":"2024-09-11T12:50:27.325749+0000","last_undegraded":"2024-09-11T12:50:27.325749+0000","last_fullsized":"2024-09-11T12:50:27.325749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:52.150201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357527+0000","last_change":"2024-09-11T12:50:27.349271+0000","last_active":"2024-09-11T12:50:27.357527+0000","last_peered":"2024-09-11T12:50:27.357527+0000","last_clean":"2024-09-11T12:50:27.357527+0000","last_became_active":"2024-09-11T12:50:27.348581+0000","last_became_peered":"2024-09-11T12:50:27.348581+0000","last_unstale":"2024-09-11T12:50:27.357527+0000","last_undegraded":"2024-09-11T12:50:27.357527+0000","last_fullsized":"2024-09-11T12:50:27.357527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324027+0000","last_change":"2024-09-11T12:50:26.336396+0000","last_active":"2024-09-11T12:50:27.324027+0000","last_peered":"2024-09-11T12:50:27.324027+0000","last_clean":"2024-09-11T12:50:27.324027+0000","last_became_active":"2024-09-11T12:50:26.335998+0000","last_became_peered":"2024-09-11T12:50:26.335998+0000","last_unstale":"2024-09-11T12:50:27.324027+0000","last_undegraded":"2024-09-11T12:50:27.324027+0000","last_fullsized":"2024-09-11T12:50:27.324027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357673+0000","last_change":"2024-09-11T12:50:27.349951+0000","last_active":"2024-09-11T12:50:27.357673+0000","last_peered":"2024-09-11T12:50:27.357673+0000","last_clean":"2024-09-11T12:50:27.357673+0000","last_became_active":"2024-09-11T12:50:27.349174+0000","last_became_peered":"2024-09-11T12:50:27.349174+0000","last_unstale":"2024-09-11T12:50:27.357673+0000","last_undegraded":"2024-09-11T12:50:27.357673+0000","last_fullsized":"2024-09-11T12:50:27.357673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:03.499603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364811+0000","last_change":"2024-09-11T12:50:26.337423+0000","last_active":"2024-09-11T12:50:27.364811+0000","last_peered":"2024-09-11T12:50:27.364811+0000","last_clean":"2024-09-11T12:50:27.364811+0000","last_became_active":"2024-09-11T12:50:26.337076+0000","last_became_peered":"2024-09-11T12:50:26.337076+0000","last_unstale":"2024-09-11T12:50:27.364811+0000","last_undegraded":"2024-09-11T12:50:27.364811+0000","last_fullsized":"2024-09-11T12:50:27.364811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351691+0000","last_change":"2024-09-11T12:50:27.349463+0000","last_active":"2024-09-11T12:50:27.351691+0000","last_peered":"2024-09-11T12:50:27.351691+0000","last_clean":"2024-09-11T12:50:27.351691+0000","last_became_active":"2024-09-11T12:50:27.349183+0000","last_became_peered":"2024-09-11T12:50:27.349183+0000","last_unstale":"2024-09-11T12:50:27.351691+0000","last_undegraded":"2024-09-11T12:50:27.351691+0000","last_fullsized":"2024-09-11T12:50:27.351691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345829+0000","last_change":"2024-09-11T12:50:26.340196+0000","last_active":"2024-09-11T12:50:26.345829+0000","last_peered":"2024-09-11T12:50:26.345829+0000","last_clean":"2024-09-11T12:50:26.345829+0000","last_became_active":"2024-09-11T12:50:26.339927+0000","last_became_peered":"2024-09-11T12:50:26.339927+0000","last_unstale":"2024-09-11T12:50:26.345829+0000","last_undegraded":"2024-09-11T12:50:26.345829+0000","last_fullsized":"2024-09-11T12:50:26.345829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:57.749644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.325394+0000","last_change":"2024-09-11T12:50:26.325394+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.325394+0000","last_undegraded":"2024-09-11T12:50:26.325394+0000","last_fullsized":"2024-09-11T12:50:26.325394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337747+0000","last_change":"2024-09-11T12:50:26.338822+0000","last_active":"2024-09-11T12:50:27.337747+0000","last_peered":"2024-09-11T12:50:27.337747+0000","last_clean":"2024-09-11T12:50:27.337747+0000","last_became_active":"2024-09-11T12:50:26.338459+0000","last_became_peered":"2024-09-11T12:50:26.338459+0000","last_unstale":"2024-09-11T12:50:27.337747+0000","last_undegraded":"2024-09-11T12:50:27.337747+0000","last_fullsized":"2024-09-11T12:50:27.337747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:01.412980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347818+0000","last_change":"2024-09-11T12:50:27.344966+0000","last_active":"2024-09-11T12:50:27.347818+0000","last_peered":"2024-09-11T12:50:27.347818+0000","last_clean":"2024-09-11T12:50:27.347818+0000","last_became_active":"2024-09-11T12:50:27.344634+0000","last_became_peered":"2024-09-11T12:50:27.344634+0000","last_unstale":"2024-09-11T12:50:27.347818+0000","last_undegraded":"2024-09-11T12:50:27.347818+0000","last_fullsized":"2024-09-11T12:50:27.347818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.337771+0000","last_change":"2024-09-11T12:50:26.330959+0000","last_active":"2024-09-11T12:50:26.337771+0000","last_peered":"2024-09-11T12:50:26.337771+0000","last_clean":"2024-09-11T12:50:26.337771+0000","last_became_active":"2024-09-11T12:50:26.330722+0000","last_became_peered":"2024-09-11T12:50:26.330722+0000","last_unstale":"2024-09-11T12:50:26.337771+0000","last_undegraded":"2024-09-11T12:50:26.337771+0000","last_fullsized":"2024-09-11T12:50:26.337771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:51.012754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324459+0000","last_change":"2024-09-11T12:50:26.324459+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324459+0000","last_undegraded":"2024-09-11T12:50:26.324459+0000","last_fullsized":"2024-09-11T12:50:26.324459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337362+0000","last_change":"2024-09-11T12:50:26.337961+0000","last_active":"2024-09-11T12:50:27.337362+0000","last_peered":"2024-09-11T12:50:27.337362+0000","last_clean":"2024-09-11T12:50:27.337362+0000","last_became_active":"2024-09-11T12:50:26.337461+0000","last_became_peered":"2024-09-11T12:50:26.337461+0000","last_unstale":"2024-09-11T12:50:27.337362+0000","last_undegraded":"2024-09-11T12:50:27.337362+0000","last_fullsized":"2024-09-11T12:50:27.337362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345998+0000","last_change":"2024-09-11T12:50:27.342392+0000","last_active":"2024-09-11T12:50:27.345998+0000","last_peered":"2024-09-11T12:50:27.345998+0000","last_clean":"2024-09-11T12:50:27.345998+0000","last_became_active":"2024-09-11T12:50:27.342116+0000","last_became_peered":"2024-09-11T12:50:27.342116+0000","last_unstale":"2024-09-11T12:50:27.345998+0000","last_undegraded":"2024-09-11T12:50:27.345998+0000","last_fullsized":"2024-09-11T12:50:27.345998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325924+0000","last_change":"2024-09-11T12:50:26.330978+0000","last_active":"2024-09-11T12:50:27.325924+0000","last_peered":"2024-09-11T12:50:27.325924+0000","last_clean":"2024-09-11T12:50:27.325924+0000","last_became_active":"2024-09-11T12:50:26.330681+0000","last_became_peered":"2024-09-11T12:50:26.330681+0000","last_unstale":"2024-09-11T12:50:27.325924+0000","last_undegraded":"2024-09-11T12:50:27.325924+0000","last_fullsized":"2024-09-11T12:50:27.325924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:43.803456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.334967+0000","last_change":"2024-09-11T12:50:27.328606+0000","last_active":"2024-09-11T12:50:27.334967+0000","last_peered":"2024-09-11T12:50:27.334967+0000","last_clean":"2024-09-11T12:50:27.334967+0000","last_became_active":"2024-09-11T12:50:27.328276+0000","last_became_peered":"2024-09-11T12:50:27.328276+0000","last_unstale":"2024-09-11T12:50:27.334967+0000","last_undegraded":"2024-09-11T12:50:27.334967+0000","last_fullsized":"2024-09-11T12:50:27.334967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:36.902883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323120+0000","last_change":"2024-09-11T12:50:26.337651+0000","last_active":"2024-09-11T12:50:27.323120+0000","last_peered":"2024-09-11T12:50:27.323120+0000","last_clean":"2024-09-11T12:50:27.323120+0000","last_became_active":"2024-09-11T12:50:26.337387+0000","last_became_peered":"2024-09-11T12:50:26.337387+0000","last_unstale":"2024-09-11T12:50:27.323120+0000","last_undegraded":"2024-09-11T12:50:27.323120+0000","last_fullsized":"2024-09-11T12:50:27.323120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:52.922609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362603+0000","last_change":"2024-09-11T12:50:27.351937+0000","last_active":"2024-09-11T12:50:27.362603+0000","last_peered":"2024-09-11T12:50:27.362603+0000","last_clean":"2024-09-11T12:50:27.362603+0000","last_became_active":"2024-09-11T12:50:27.350570+0000","last_became_peered":"2024-09-11T12:50:27.350570+0000","last_unstale":"2024-09-11T12:50:27.362603+0000","last_undegraded":"2024-09-11T12:50:27.362603+0000","last_fullsized":"2024-09-11T12:50:27.362603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.328966+0000","last_change":"2024-09-11T12:50:26.344815+0000","last_active":"2024-09-11T12:50:27.328966+0000","last_peered":"2024-09-11T12:50:27.328966+0000","last_clean":"2024-09-11T12:50:27.328966+0000","last_became_active":"2024-09-11T12:50:26.344518+0000","last_became_peered":"2024-09-11T12:50:26.344518+0000","last_unstale":"2024-09-11T12:50:27.328966+0000","last_undegraded":"2024-09-11T12:50:27.328966+0000","last_fullsized":"2024-09-11T12:50:27.328966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:24.053403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347970+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:50:27.347970+0000","last_peered":"2024-09-11T12:50:27.347970+0000","last_clean":"2024-09-11T12:50:27.347970+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:50:27.347970+0000","last_undegraded":"2024-09-11T12:50:27.347970+0000","last_fullsized":"2024-09-11T12:50:27.347970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:PG 5.29 is not active+clean 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:50:26.324523+0000', 'last_change': '2024-09-11T12:50:26.324523+0000', 'last_active': '2024-09-11T12:50:26.307616+0000', 'last_peered': '2024-09-11T12:50:26.307616+0000', 'last_clean': '2024-09-11T12:50:26.307616+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:50:26.324523+0000', 'last_undegraded': '2024-09-11T12:50:26.324523+0000', 'last_fullsized': '2024-09-11T12:50:26.324523+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_clean_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1e is not active+clean 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:50:26.324419+0000', 'last_change': '2024-09-11T12:50:26.324419+0000', 'last_active': '2024-09-11T12:50:26.307616+0000', 'last_peered': '2024-09-11T12:50:26.307616+0000', 'last_clean': '2024-09-11T12:50:26.307616+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:50:26.324419+0000', 'last_undegraded': '2024-09-11T12:50:26.324419+0000', 'last_fullsized': '2024-09-11T12:50:26.324419+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_clean_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:PG 5.5 is not active+clean 2024-09-11T12:50:32.211 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:50:26.325394+0000', 'last_change': '2024-09-11T12:50:26.325394+0000', 'last_active': '2024-09-11T12:50:26.307616+0000', 'last_peered': '2024-09-11T12:50:26.307616+0000', 'last_clean': '2024-09-11T12:50:26.307616+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:50:26.325394+0000', 'last_undegraded': '2024-09-11T12:50:26.325394+0000', 'last_fullsized': '2024-09-11T12:50:26.325394+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_clean_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:50:32.212 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3 is not active+clean 2024-09-11T12:50:32.212 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:50:26.324459+0000', 'last_change': '2024-09-11T12:50:26.324459+0000', 'last_active': '2024-09-11T12:50:26.307616+0000', 'last_peered': '2024-09-11T12:50:26.307616+0000', 'last_clean': '2024-09-11T12:50:26.307616+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:50:26.324459+0000', 'last_undegraded': '2024-09-11T12:50:26.324459+0000', 'last_fullsized': '2024-09-11T12:50:26.324459+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'last_clean_scrub_stamp': '2024-09-11T12:50:26.307616+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:50:32.212 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T12:50:32.301 INFO:tasks.ceph.mds.b.smithi157.stderr:2024-09-11T12:50:32.301+0000 7fb738000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- 172.21.15.78:0/3201857276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 msgr2=0x7f520c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 --2- 172.21.15.78:0/3201857276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f520c00b9d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5220009b10 tx=0x7f522001d270 comp rx=0 tx=0).stop 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- 172.21.15.78:0/3201857276 shutdown_connections 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 --2- 172.21.15.78:0/3201857276 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f520c00b9d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- 172.21.15.78:0/3201857276 >> 172.21.15.78:0/3201857276 conn(0x7f522806ca80 msgr2=0x7f52280b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- 172.21.15.78:0/3201857276 shutdown_connections 2024-09-11T12:50:32.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- 172.21.15.78:0/3201857276 wait complete. 2024-09-11T12:50:32.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 Processor -- start 2024-09-11T12:50:32.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- start start 2024-09-11T12:50:32.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f522814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52280b1030 0x7f52280b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f522806eb40 con 0x7f522815dad0 2024-09-11T12:50:32.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5228157da0 con 0x7f520c0052f0 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f522e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f522806ecc0 con 0x7f52280b1030 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.386+0000 7f5226a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60504/0 (socket says 172.21.15.78:60504) 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 -- 172.21.15.78:0/3396988700 learned_addr learned my addr 172.21.15.78:0/3396988700 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5227400640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f522814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5227e00640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52280b1030 0x7f52280b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 -- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52280b1030 msgr2=0x7f52280b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52280b1030 0x7f52280b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 -- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 msgr2=0x7f522814ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f522814ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 -- 172.21.15.78:0/3396988700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5228150f00 con 0x7f522815dad0 2024-09-11T12:50:32.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5227400640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f522814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:32.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f5226a00640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5210003f60 tx=0x7f5210003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210011640 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5210016020 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52281511e0 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5210011a60 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52281517a0 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f522815af70 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.390+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f5210011c00 con 0x7f522815dad0 2024-09-11T12:50:32.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.394+0000 7f521f400640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 0x7f5204073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.394+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f5210092a40 con 0x7f522815dad0 2024-09-11T12:50:32.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.394+0000 7f5227400640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 0x7f5204073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.394+0000 7f5227400640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 0x7f5204073b70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5228072670 tx=0x7f5218006b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.394+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5210020020 con 0x7f522815dad0 2024-09-11T12:50:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f522806faf0 con 0x7f522815dad0 2024-09-11T12:50:32.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f521f400640 1 -- 172.21.15.78:0/3396988700 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+2753 (secure 0 0 0) 0x7f521005e8a0 con 0x7f522815dad0 2024-09-11T12:50:32.626 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:32.627 INFO:teuthology.orchestra.run.smithi078.stdout:{"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":34,"num_osds":8,"num_up_osds":8,"osd_up_since":1726058972,"num_in_osds":8,"osd_in_since":1726058961,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":93},{"state_name":"creating+peering","count":4}],"num_pgs":97,"num_pools":3,"num_objects":22,"data_bytes":592778,"bytes_used":242491392,"bytes_avail":772851621888,"bytes_total":773094113280,"inactive_pgs_ratio":0.041237112134695053,"write_bytes_sec":2468,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":16,"btime":"2024-09-11T12:50:30:342061+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"b","status":"up:active","gid":4885}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-09-11T12:50:25.115235+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 msgr2=0x7f5204073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 0x7f5204073b70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5228072670 tx=0x7f5218006b30 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 msgr2=0x7f522815df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5210003f60 tx=0x7f5210003f90 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 shutdown_connections 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f52040716b0 0x7f5204073b70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f52280b1030 0x7f52280b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f522815dad0 0x7f522815df50 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 --2- 172.21.15.78:0/3396988700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f520c0052f0 0x7f522814ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.622+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 >> 172.21.15.78:0/3396988700 conn(0x7f522806ca80 msgr2=0x7f522815c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.626+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 shutdown_connections 2024-09-11T12:50:32.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.626+0000 7f522e600640 1 -- 172.21.15.78:0/3396988700 wait complete. 2024-09-11T12:50:32.649 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-11T12:50:32.649 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:50:32.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- 172.21.15.78:0/2377232871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 msgr2=0x7f861415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 --2- 172.21.15.78:0/2377232871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f861415cf60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f860c009bc0 tx=0x7f860c01cba0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- 172.21.15.78:0/2377232871 shutdown_connections 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 --2- 172.21.15.78:0/2377232871 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f861415cf60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- 172.21.15.78:0/2377232871 >> 172.21.15.78:0/2377232871 conn(0x7f861406ca80 msgr2=0x7f86140b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- 172.21.15.78:0/2377232871 shutdown_connections 2024-09-11T12:50:32.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- 172.21.15.78:0/2377232871 wait complete. 2024-09-11T12:50:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 Processor -- start 2024-09-11T12:50:32.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 -- start start 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.786+0000 7f861a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86140aed20 0x7f8614154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861415cad0 0x7f86140ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86140cd460 con 0x7f86140b4c30 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86140cd2e0 con 0x7f861415cad0 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86140cd5e0 con 0x7f86140aed20 2024-09-11T12:50:32.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60526/0 (socket says 172.21.15.78:60526) 2024-09-11T12:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 -- 172.21.15.78:0/1913293966 learned_addr learned my addr 172.21.15.78:0/1913293966 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8613e00640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861415cad0 0x7f86140ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8613400640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86140aed20 0x7f8614154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 -- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86140aed20 msgr2=0x7f8614154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86140aed20 0x7f8614154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 -- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861415cad0 msgr2=0x7f86140ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861415cad0 0x7f86140ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 -- 172.21.15.78:0/1913293966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86141555f0 con 0x7f86140b4c30 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f8612a00640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f860000c420 tx=0x7f860000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600011790 con 0x7f86140b4c30 2024-09-11T12:50:32.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8600017070 con 0x7f86140b4c30 2024-09-11T12:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86141558d0 con 0x7f86140b4c30 2024-09-11T12:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600011ce0 con 0x7f86140b4c30 2024-09-11T12:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86141552c0 con 0x7f86140b4c30 2024-09-11T12:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86140cdb40 con 0x7f86140b4c30 2024-09-11T12:50:32.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.790+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f860001c460 con 0x7f86140b4c30 2024-09-11T12:50:32.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.794+0000 7f860b400640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 0x7f85f4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.794+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8600091b20 con 0x7f86140b4c30 2024-09-11T12:50:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.794+0000 7f8613400640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 0x7f85f4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:32.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.794+0000 7f8613400640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 0x7f85f4073b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8604007fd0 tx=0x7f8604009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:32.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.798+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f860005d330 con 0x7f86140b4c30 2024-09-11T12:50:32.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.962+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8614155df0 con 0x7f85f4071680 2024-09-11T12:50:32.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.966+0000 7f860b400640 1 -- 172.21.15.78:0/1913293966 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251605 (secure 0 0 0) 0x7f8614155df0 con 0x7f85f4071680 2024-09-11T12:50:32.972 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:32.974 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 msgr2=0x7f85f4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 0x7f85f4073b40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8604007fd0 tx=0x7f8604009040 comp rx=0 tx=0).stop 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 msgr2=0x7f8614140a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f860000c420 tx=0x7f860000c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 shutdown_connections 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f85f4071680 0x7f85f4073b40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f861415cad0 0x7f86140ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86140b4c30 0x7f8614140a70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 --2- 172.21.15.78:0/1913293966 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86140aed20 0x7f8614154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 >> 172.21.15.78:0/1913293966 conn(0x7f861406ca80 msgr2=0x7f861414e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 shutdown_connections 2024-09-11T12:50:32.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:32.970+0000 7f861a800640 1 -- 172.21.15.78:0/1913293966 wait complete. 2024-09-11T12:50:32.993 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-11T12:50:31.116340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":144,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107,"ondisk_log_size":107,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":25,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236808,"kb_used_data":3912,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737912,"statfs":{"total":773094113280,"available":772851621888,"internally_reserved":0,"allocated":4005888,"data_stored":2229312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.807871"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.350863+0000","last_change":"2024-09-11T12:50:27.348834+0000","last_active":"2024-09-11T12:50:27.350863+0000","last_peered":"2024-09-11T12:50:27.350863+0000","last_clean":"2024-09-11T12:50:27.350863+0000","last_became_active":"2024-09-11T12:50:27.348574+0000","last_became_peered":"2024-09-11T12:50:27.348574+0000","last_unstale":"2024-09-11T12:50:27.350863+0000","last_undegraded":"2024-09-11T12:50:27.350863+0000","last_fullsized":"2024-09-11T12:50:27.350863+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341400+0000","last_change":"2024-09-11T12:50:27.336993+0000","last_active":"2024-09-11T12:50:27.341400+0000","last_peered":"2024-09-11T12:50:27.341400+0000","last_clean":"2024-09-11T12:50:27.341400+0000","last_became_active":"2024-09-11T12:50:27.336588+0000","last_became_peered":"2024-09-11T12:50:27.336588+0000","last_unstale":"2024-09-11T12:50:27.341400+0000","last_undegraded":"2024-09-11T12:50:27.341400+0000","last_fullsized":"2024-09-11T12:50:27.341400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351364+0000","last_change":"2024-09-11T12:50:27.342867+0000","last_active":"2024-09-11T12:50:27.351364+0000","last_peered":"2024-09-11T12:50:27.351364+0000","last_clean":"2024-09-11T12:50:27.351364+0000","last_became_active":"2024-09-11T12:50:27.342576+0000","last_became_peered":"2024-09-11T12:50:27.342576+0000","last_unstale":"2024-09-11T12:50:27.351364+0000","last_undegraded":"2024-09-11T12:50:27.351364+0000","last_fullsized":"2024-09-11T12:50:27.351364+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360719+0000","last_change":"2024-09-11T12:50:27.357904+0000","last_active":"2024-09-11T12:50:27.360719+0000","last_peered":"2024-09-11T12:50:27.360719+0000","last_clean":"2024-09-11T12:50:27.360719+0000","last_became_active":"2024-09-11T12:50:27.356836+0000","last_became_peered":"2024-09-11T12:50:27.356836+0000","last_unstale":"2024-09-11T12:50:27.360719+0000","last_undegraded":"2024-09-11T12:50:27.360719+0000","last_fullsized":"2024-09-11T12:50:27.360719+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345686+0000","last_change":"2024-09-11T12:50:27.341619+0000","last_active":"2024-09-11T12:50:27.345686+0000","last_peered":"2024-09-11T12:50:27.345686+0000","last_clean":"2024-09-11T12:50:27.345686+0000","last_became_active":"2024-09-11T12:50:27.341345+0000","last_became_peered":"2024-09-11T12:50:27.341345+0000","last_unstale":"2024-09-11T12:50:27.345686+0000","last_undegraded":"2024-09-11T12:50:27.345686+0000","last_fullsized":"2024-09-11T12:50:27.345686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:12:44.021059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.339528+0000","last_change":"2024-09-11T12:50:27.335087+0000","last_active":"2024-09-11T12:50:27.339528+0000","last_peered":"2024-09-11T12:50:27.339528+0000","last_clean":"2024-09-11T12:50:27.339528+0000","last_became_active":"2024-09-11T12:50:27.334790+0000","last_became_peered":"2024-09-11T12:50:27.334790+0000","last_unstale":"2024-09-11T12:50:27.339528+0000","last_undegraded":"2024-09-11T12:50:27.339528+0000","last_fullsized":"2024-09-11T12:50:27.339528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364813+0000","last_change":"2024-09-11T12:50:27.355196+0000","last_active":"2024-09-11T12:50:27.364813+0000","last_peered":"2024-09-11T12:50:27.364813+0000","last_clean":"2024-09-11T12:50:27.364813+0000","last_became_active":"2024-09-11T12:50:27.353288+0000","last_became_peered":"2024-09-11T12:50:27.353288+0000","last_unstale":"2024-09-11T12:50:27.364813+0000","last_undegraded":"2024-09-11T12:50:27.364813+0000","last_fullsized":"2024-09-11T12:50:27.364813+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364931+0000","last_change":"2024-09-11T12:50:27.358075+0000","last_active":"2024-09-11T12:50:27.364931+0000","last_peered":"2024-09-11T12:50:27.364931+0000","last_clean":"2024-09-11T12:50:27.364931+0000","last_became_active":"2024-09-11T12:50:27.357851+0000","last_became_peered":"2024-09-11T12:50:27.357851+0000","last_unstale":"2024-09-11T12:50:27.364931+0000","last_undegraded":"2024-09-11T12:50:27.364931+0000","last_fullsized":"2024-09-11T12:50:27.364931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352225+0000","last_change":"2024-09-11T12:50:27.343726+0000","last_active":"2024-09-11T12:50:27.352225+0000","last_peered":"2024-09-11T12:50:27.352225+0000","last_clean":"2024-09-11T12:50:27.352225+0000","last_became_active":"2024-09-11T12:50:27.343450+0000","last_became_peered":"2024-09-11T12:50:27.343450+0000","last_unstale":"2024-09-11T12:50:27.352225+0000","last_undegraded":"2024-09-11T12:50:27.352225+0000","last_fullsized":"2024-09-11T12:50:27.352225+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342383+0000","last_change":"2024-09-11T12:50:27.339989+0000","last_active":"2024-09-11T12:50:27.342383+0000","last_peered":"2024-09-11T12:50:27.342383+0000","last_clean":"2024-09-11T12:50:27.342383+0000","last_became_active":"2024-09-11T12:50:27.339673+0000","last_became_peered":"2024-09-11T12:50:27.339673+0000","last_unstale":"2024-09-11T12:50:27.342383+0000","last_undegraded":"2024-09-11T12:50:27.342383+0000","last_fullsized":"2024-09-11T12:50:27.342383+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365270+0000","last_change":"2024-09-11T12:50:27.363103+0000","last_active":"2024-09-11T12:50:27.365270+0000","last_peered":"2024-09-11T12:50:27.365270+0000","last_clean":"2024-09-11T12:50:27.365270+0000","last_became_active":"2024-09-11T12:50:27.362746+0000","last_became_peered":"2024-09-11T12:50:27.362746+0000","last_unstale":"2024-09-11T12:50:27.365270+0000","last_undegraded":"2024-09-11T12:50:27.365270+0000","last_fullsized":"2024-09-11T12:50:27.365270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347758+0000","last_change":"2024-09-11T12:50:27.329779+0000","last_active":"2024-09-11T12:50:27.347758+0000","last_peered":"2024-09-11T12:50:27.347758+0000","last_clean":"2024-09-11T12:50:27.347758+0000","last_became_active":"2024-09-11T12:50:27.327796+0000","last_became_peered":"2024-09-11T12:50:27.327796+0000","last_unstale":"2024-09-11T12:50:27.347758+0000","last_undegraded":"2024-09-11T12:50:27.347758+0000","last_fullsized":"2024-09-11T12:50:27.347758+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352853+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.352853+0000","last_peered":"2024-09-11T12:50:27.352853+0000","last_clean":"2024-09-11T12:50:27.352853+0000","last_became_active":"2024-09-11T12:50:27.344834+0000","last_became_peered":"2024-09-11T12:50:27.344834+0000","last_unstale":"2024-09-11T12:50:27.352853+0000","last_undegraded":"2024-09-11T12:50:27.352853+0000","last_fullsized":"2024-09-11T12:50:27.352853+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:45:48.794525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341535+0000","last_change":"2024-09-11T12:50:27.339363+0000","last_active":"2024-09-11T12:50:27.341535+0000","last_peered":"2024-09-11T12:50:27.341535+0000","last_clean":"2024-09-11T12:50:27.341535+0000","last_became_active":"2024-09-11T12:50:27.339131+0000","last_became_peered":"2024-09-11T12:50:27.339131+0000","last_unstale":"2024-09-11T12:50:27.341535+0000","last_undegraded":"2024-09-11T12:50:27.341535+0000","last_fullsized":"2024-09-11T12:50:27.341535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344599+0000","last_change":"2024-09-11T12:50:27.335582+0000","last_active":"2024-09-11T12:50:27.344599+0000","last_peered":"2024-09-11T12:50:27.344599+0000","last_clean":"2024-09-11T12:50:27.344599+0000","last_became_active":"2024-09-11T12:50:27.335002+0000","last_became_peered":"2024-09-11T12:50:27.335002+0000","last_unstale":"2024-09-11T12:50:27.344599+0000","last_undegraded":"2024-09-11T12:50:27.344599+0000","last_fullsized":"2024-09-11T12:50:27.344599+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365195+0000","last_change":"2024-09-11T12:50:27.363062+0000","last_active":"2024-09-11T12:50:27.365195+0000","last_peered":"2024-09-11T12:50:27.365195+0000","last_clean":"2024-09-11T12:50:27.365195+0000","last_became_active":"2024-09-11T12:50:27.362788+0000","last_became_peered":"2024-09-11T12:50:27.362788+0000","last_unstale":"2024-09-11T12:50:27.365195+0000","last_undegraded":"2024-09-11T12:50:27.365195+0000","last_fullsized":"2024-09-11T12:50:27.365195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364886+0000","last_change":"2024-09-11T12:50:27.357790+0000","last_active":"2024-09-11T12:50:27.364886+0000","last_peered":"2024-09-11T12:50:27.364886+0000","last_clean":"2024-09-11T12:50:27.364886+0000","last_became_active":"2024-09-11T12:50:27.353639+0000","last_became_peered":"2024-09-11T12:50:27.353639+0000","last_unstale":"2024-09-11T12:50:27.364886+0000","last_undegraded":"2024-09-11T12:50:27.364886+0000","last_fullsized":"2024-09-11T12:50:27.364886+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.341377+0000","last_change":"2024-09-11T12:50:27.336825+0000","last_active":"2024-09-11T12:50:27.341377+0000","last_peered":"2024-09-11T12:50:27.341377+0000","last_clean":"2024-09-11T12:50:27.341377+0000","last_became_active":"2024-09-11T12:50:27.336241+0000","last_became_peered":"2024-09-11T12:50:27.336241+0000","last_unstale":"2024-09-11T12:50:27.341377+0000","last_undegraded":"2024-09-11T12:50:27.341377+0000","last_fullsized":"2024-09-11T12:50:27.341377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346581+0000","last_change":"2024-09-11T12:50:27.344954+0000","last_active":"2024-09-11T12:50:27.346581+0000","last_peered":"2024-09-11T12:50:27.346581+0000","last_clean":"2024-09-11T12:50:27.346581+0000","last_became_active":"2024-09-11T12:50:27.344681+0000","last_became_peered":"2024-09-11T12:50:27.344681+0000","last_unstale":"2024-09-11T12:50:27.346581+0000","last_undegraded":"2024-09-11T12:50:27.346581+0000","last_fullsized":"2024-09-11T12:50:27.346581+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357472+0000","last_change":"2024-09-11T12:50:27.352798+0000","last_active":"2024-09-11T12:50:27.357472+0000","last_peered":"2024-09-11T12:50:27.357472+0000","last_clean":"2024-09-11T12:50:27.357472+0000","last_became_active":"2024-09-11T12:50:27.349194+0000","last_became_peered":"2024-09-11T12:50:27.349194+0000","last_unstale":"2024-09-11T12:50:27.357472+0000","last_undegraded":"2024-09-11T12:50:27.357472+0000","last_fullsized":"2024-09-11T12:50:27.357472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:03:32.390636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351801+0000","last_change":"2024-09-11T12:50:27.343151+0000","last_active":"2024-09-11T12:50:27.351801+0000","last_peered":"2024-09-11T12:50:27.351801+0000","last_clean":"2024-09-11T12:50:27.351801+0000","last_became_active":"2024-09-11T12:50:27.339404+0000","last_became_peered":"2024-09-11T12:50:27.339404+0000","last_unstale":"2024-09-11T12:50:27.351801+0000","last_undegraded":"2024-09-11T12:50:27.351801+0000","last_fullsized":"2024-09-11T12:50:27.351801+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.336093+0000","last_change":"2024-09-11T12:50:27.330086+0000","last_active":"2024-09-11T12:50:27.336093+0000","last_peered":"2024-09-11T12:50:27.336093+0000","last_clean":"2024-09-11T12:50:27.336093+0000","last_became_active":"2024-09-11T12:50:27.329846+0000","last_became_peered":"2024-09-11T12:50:27.329846+0000","last_unstale":"2024-09-11T12:50:27.336093+0000","last_undegraded":"2024-09-11T12:50:27.336093+0000","last_fullsized":"2024-09-11T12:50:27.336093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:12.211705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324523+0000","last_change":"2024-09-11T12:50:26.324523+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324523+0000","last_undegraded":"2024-09-11T12:50:26.324523+0000","last_fullsized":"2024-09-11T12:50:26.324523+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343833+0000","last_change":"2024-09-11T12:50:27.337469+0000","last_active":"2024-09-11T12:50:27.343833+0000","last_peered":"2024-09-11T12:50:27.343833+0000","last_clean":"2024-09-11T12:50:27.343833+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.343833+0000","last_undegraded":"2024-09-11T12:50:27.343833+0000","last_fullsized":"2024-09-11T12:50:27.343833+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359468+0000","last_change":"2024-09-11T12:50:27.355625+0000","last_active":"2024-09-11T12:50:27.359468+0000","last_peered":"2024-09-11T12:50:27.359468+0000","last_clean":"2024-09-11T12:50:27.359468+0000","last_became_active":"2024-09-11T12:50:27.355274+0000","last_became_peered":"2024-09-11T12:50:27.355274+0000","last_unstale":"2024-09-11T12:50:27.359468+0000","last_undegraded":"2024-09-11T12:50:27.359468+0000","last_fullsized":"2024-09-11T12:50:27.359468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359556+0000","last_change":"2024-09-11T12:50:27.356361+0000","last_active":"2024-09-11T12:50:27.359556+0000","last_peered":"2024-09-11T12:50:27.359556+0000","last_clean":"2024-09-11T12:50:27.359556+0000","last_became_active":"2024-09-11T12:50:27.356074+0000","last_became_peered":"2024-09-11T12:50:27.356074+0000","last_unstale":"2024-09-11T12:50:27.359556+0000","last_undegraded":"2024-09-11T12:50:27.359556+0000","last_fullsized":"2024-09-11T12:50:27.359556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359308+0000","last_change":"2024-09-11T12:50:27.357026+0000","last_active":"2024-09-11T12:50:27.359308+0000","last_peered":"2024-09-11T12:50:27.359308+0000","last_clean":"2024-09-11T12:50:27.359308+0000","last_became_active":"2024-09-11T12:50:27.356785+0000","last_became_peered":"2024-09-11T12:50:27.356785+0000","last_unstale":"2024-09-11T12:50:27.359308+0000","last_undegraded":"2024-09-11T12:50:27.359308+0000","last_fullsized":"2024-09-11T12:50:27.359308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347829+0000","last_change":"2024-09-11T12:50:27.336803+0000","last_active":"2024-09-11T12:50:27.347829+0000","last_peered":"2024-09-11T12:50:27.347829+0000","last_clean":"2024-09-11T12:50:27.347829+0000","last_became_active":"2024-09-11T12:50:27.334739+0000","last_became_peered":"2024-09-11T12:50:27.334739+0000","last_unstale":"2024-09-11T12:50:27.347829+0000","last_undegraded":"2024-09-11T12:50:27.347829+0000","last_fullsized":"2024-09-11T12:50:27.347829+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335097+0000","last_change":"2024-09-11T12:50:27.329208+0000","last_active":"2024-09-11T12:50:27.335097+0000","last_peered":"2024-09-11T12:50:27.335097+0000","last_clean":"2024-09-11T12:50:27.335097+0000","last_became_active":"2024-09-11T12:50:27.328939+0000","last_became_peered":"2024-09-11T12:50:27.328939+0000","last_unstale":"2024-09-11T12:50:27.335097+0000","last_undegraded":"2024-09-11T12:50:27.335097+0000","last_fullsized":"2024-09-11T12:50:27.335097+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:25.059566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353911+0000","last_change":"2024-09-11T12:50:27.348968+0000","last_active":"2024-09-11T12:50:27.353911+0000","last_peered":"2024-09-11T12:50:27.353911+0000","last_clean":"2024-09-11T12:50:27.353911+0000","last_became_active":"2024-09-11T12:50:27.348693+0000","last_became_peered":"2024-09-11T12:50:27.348693+0000","last_unstale":"2024-09-11T12:50:27.353911+0000","last_undegraded":"2024-09-11T12:50:27.353911+0000","last_fullsized":"2024-09-11T12:50:27.353911+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345469+0000","last_change":"2024-09-11T12:50:27.341331+0000","last_active":"2024-09-11T12:50:27.345469+0000","last_peered":"2024-09-11T12:50:27.345469+0000","last_clean":"2024-09-11T12:50:27.345469+0000","last_became_active":"2024-09-11T12:50:27.341038+0000","last_became_peered":"2024-09-11T12:50:27.341038+0000","last_unstale":"2024-09-11T12:50:27.345469+0000","last_undegraded":"2024-09-11T12:50:27.345469+0000","last_fullsized":"2024-09-11T12:50:27.345469+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364720+0000","last_change":"2024-09-11T12:50:27.356104+0000","last_active":"2024-09-11T12:50:27.364720+0000","last_peered":"2024-09-11T12:50:27.364720+0000","last_clean":"2024-09-11T12:50:27.364720+0000","last_became_active":"2024-09-11T12:50:27.355113+0000","last_became_peered":"2024-09-11T12:50:27.355113+0000","last_unstale":"2024-09-11T12:50:27.364720+0000","last_undegraded":"2024-09-11T12:50:27.364720+0000","last_fullsized":"2024-09-11T12:50:27.364720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324419+0000","last_change":"2024-09-11T12:50:26.324419+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324419+0000","last_undegraded":"2024-09-11T12:50:26.324419+0000","last_fullsized":"2024-09-11T12:50:26.324419+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365295+0000","last_change":"2024-09-11T12:50:26.343869+0000","last_active":"2024-09-11T12:50:27.365295+0000","last_peered":"2024-09-11T12:50:27.365295+0000","last_clean":"2024-09-11T12:50:27.365295+0000","last_became_active":"2024-09-11T12:50:26.343588+0000","last_became_peered":"2024-09-11T12:50:26.343588+0000","last_unstale":"2024-09-11T12:50:27.365295+0000","last_undegraded":"2024-09-11T12:50:27.365295+0000","last_fullsized":"2024-09-11T12:50:27.365295+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344316+0000","last_change":"2024-09-11T12:50:27.340558+0000","last_active":"2024-09-11T12:50:27.344316+0000","last_peered":"2024-09-11T12:50:27.344316+0000","last_clean":"2024-09-11T12:50:27.344316+0000","last_became_active":"2024-09-11T12:50:27.337140+0000","last_became_peered":"2024-09-11T12:50:27.337140+0000","last_unstale":"2024-09-11T12:50:27.344316+0000","last_undegraded":"2024-09-11T12:50:27.344316+0000","last_fullsized":"2024-09-11T12:50:27.344316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.341744+0000","last_change":"2024-09-11T12:50:26.335186+0000","last_active":"2024-09-11T12:50:26.341744+0000","last_peered":"2024-09-11T12:50:26.341744+0000","last_clean":"2024-09-11T12:50:26.341744+0000","last_became_active":"2024-09-11T12:50:26.334825+0000","last_became_peered":"2024-09-11T12:50:26.334825+0000","last_unstale":"2024-09-11T12:50:26.341744+0000","last_undegraded":"2024-09-11T12:50:26.341744+0000","last_fullsized":"2024-09-11T12:50:26.341744+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:50:54.260958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335753+0000","last_change":"2024-09-11T12:50:27.328627+0000","last_active":"2024-09-11T12:50:27.335753+0000","last_peered":"2024-09-11T12:50:27.335753+0000","last_clean":"2024-09-11T12:50:27.335753+0000","last_became_active":"2024-09-11T12:50:27.328292+0000","last_became_peered":"2024-09-11T12:50:27.328292+0000","last_unstale":"2024-09-11T12:50:27.335753+0000","last_undegraded":"2024-09-11T12:50:27.335753+0000","last_fullsized":"2024-09-11T12:50:27.335753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:08:27.421789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360603+0000","last_change":"2024-09-11T12:50:26.342258+0000","last_active":"2024-09-11T12:50:27.360603+0000","last_peered":"2024-09-11T12:50:27.360603+0000","last_clean":"2024-09-11T12:50:27.360603+0000","last_became_active":"2024-09-11T12:50:26.341986+0000","last_became_peered":"2024-09-11T12:50:26.341986+0000","last_unstale":"2024-09-11T12:50:27.360603+0000","last_undegraded":"2024-09-11T12:50:27.360603+0000","last_fullsized":"2024-09-11T12:50:27.360603+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:46.357747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359368+0000","last_change":"2024-09-11T12:50:27.357376+0000","last_active":"2024-09-11T12:50:27.359368+0000","last_peered":"2024-09-11T12:50:27.359368+0000","last_clean":"2024-09-11T12:50:27.359368+0000","last_became_active":"2024-09-11T12:50:27.357016+0000","last_became_peered":"2024-09-11T12:50:27.357016+0000","last_unstale":"2024-09-11T12:50:27.359368+0000","last_undegraded":"2024-09-11T12:50:27.359368+0000","last_fullsized":"2024-09-11T12:50:27.359368+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.360981+0000","last_change":"2024-09-11T12:50:26.343854+0000","last_active":"2024-09-11T12:50:27.360981+0000","last_peered":"2024-09-11T12:50:27.360981+0000","last_clean":"2024-09-11T12:50:27.360981+0000","last_became_active":"2024-09-11T12:50:26.341771+0000","last_became_peered":"2024-09-11T12:50:26.341771+0000","last_unstale":"2024-09-11T12:50:27.360981+0000","last_undegraded":"2024-09-11T12:50:27.360981+0000","last_fullsized":"2024-09-11T12:50:27.360981+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:51.008327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364638+0000","last_change":"2024-09-11T12:50:27.352862+0000","last_active":"2024-09-11T12:50:27.364638+0000","last_peered":"2024-09-11T12:50:27.364638+0000","last_clean":"2024-09-11T12:50:27.364638+0000","last_became_active":"2024-09-11T12:50:27.352467+0000","last_became_peered":"2024-09-11T12:50:27.352467+0000","last_unstale":"2024-09-11T12:50:27.364638+0000","last_undegraded":"2024-09-11T12:50:27.364638+0000","last_fullsized":"2024-09-11T12:50:27.364638+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363377+0000","last_change":"2024-09-11T12:50:26.342659+0000","last_active":"2024-09-11T12:50:27.363377+0000","last_peered":"2024-09-11T12:50:27.363377+0000","last_clean":"2024-09-11T12:50:27.363377+0000","last_became_active":"2024-09-11T12:50:26.342331+0000","last_became_peered":"2024-09-11T12:50:26.342331+0000","last_unstale":"2024-09-11T12:50:27.363377+0000","last_undegraded":"2024-09-11T12:50:27.363377+0000","last_fullsized":"2024-09-11T12:50:27.363377+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:18.698135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352409+0000","last_change":"2024-09-11T12:50:27.347574+0000","last_active":"2024-09-11T12:50:27.352409+0000","last_peered":"2024-09-11T12:50:27.352409+0000","last_clean":"2024-09-11T12:50:27.352409+0000","last_became_active":"2024-09-11T12:50:27.347302+0000","last_became_peered":"2024-09-11T12:50:27.347302+0000","last_unstale":"2024-09-11T12:50:27.352409+0000","last_undegraded":"2024-09-11T12:50:27.352409+0000","last_fullsized":"2024-09-11T12:50:27.352409+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.366007+0000","last_change":"2024-09-11T12:50:26.341601+0000","last_active":"2024-09-11T12:50:27.366007+0000","last_peered":"2024-09-11T12:50:27.366007+0000","last_clean":"2024-09-11T12:50:27.366007+0000","last_became_active":"2024-09-11T12:50:26.341238+0000","last_became_peered":"2024-09-11T12:50:26.341238+0000","last_unstale":"2024-09-11T12:50:27.366007+0000","last_undegraded":"2024-09-11T12:50:27.366007+0000","last_fullsized":"2024-09-11T12:50:27.366007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:46.542517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.335996+0000","last_change":"2024-09-11T12:50:27.333846+0000","last_active":"2024-09-11T12:50:27.335996+0000","last_peered":"2024-09-11T12:50:27.335996+0000","last_clean":"2024-09-11T12:50:27.335996+0000","last_became_active":"2024-09-11T12:50:27.333500+0000","last_became_peered":"2024-09-11T12:50:27.333500+0000","last_unstale":"2024-09-11T12:50:27.335996+0000","last_undegraded":"2024-09-11T12:50:27.335996+0000","last_fullsized":"2024-09-11T12:50:27.335996+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:38.067062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323107+0000","last_change":"2024-09-11T12:50:26.339433+0000","last_active":"2024-09-11T12:50:27.323107+0000","last_peered":"2024-09-11T12:50:27.323107+0000","last_clean":"2024-09-11T12:50:27.323107+0000","last_became_active":"2024-09-11T12:50:26.338991+0000","last_became_peered":"2024-09-11T12:50:26.338991+0000","last_unstale":"2024-09-11T12:50:27.323107+0000","last_undegraded":"2024-09-11T12:50:27.323107+0000","last_fullsized":"2024-09-11T12:50:27.323107+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354941+0000","last_change":"2024-09-11T12:50:27.351561+0000","last_active":"2024-09-11T12:50:27.354941+0000","last_peered":"2024-09-11T12:50:27.354941+0000","last_clean":"2024-09-11T12:50:27.354941+0000","last_became_active":"2024-09-11T12:50:27.351277+0000","last_became_peered":"2024-09-11T12:50:27.351277+0000","last_unstale":"2024-09-11T12:50:27.354941+0000","last_undegraded":"2024-09-11T12:50:27.354941+0000","last_fullsized":"2024-09-11T12:50:27.354941+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323830+0000","last_change":"2024-09-11T12:50:26.337847+0000","last_active":"2024-09-11T12:50:27.323830+0000","last_peered":"2024-09-11T12:50:27.323830+0000","last_clean":"2024-09-11T12:50:27.323830+0000","last_became_active":"2024-09-11T12:50:26.337533+0000","last_became_peered":"2024-09-11T12:50:26.337533+0000","last_unstale":"2024-09-11T12:50:27.323830+0000","last_undegraded":"2024-09-11T12:50:27.323830+0000","last_fullsized":"2024-09-11T12:50:27.323830+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:06.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346859+0000","last_change":"2024-09-11T12:50:27.330135+0000","last_active":"2024-09-11T12:50:27.346859+0000","last_peered":"2024-09-11T12:50:27.346859+0000","last_clean":"2024-09-11T12:50:27.346859+0000","last_became_active":"2024-09-11T12:50:27.329324+0000","last_became_peered":"2024-09-11T12:50:27.329324+0000","last_unstale":"2024-09-11T12:50:27.346859+0000","last_undegraded":"2024-09-11T12:50:27.346859+0000","last_fullsized":"2024-09-11T12:50:27.346859+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324111+0000","last_change":"2024-09-11T12:50:26.343684+0000","last_active":"2024-09-11T12:50:27.324111+0000","last_peered":"2024-09-11T12:50:27.324111+0000","last_clean":"2024-09-11T12:50:27.324111+0000","last_became_active":"2024-09-11T12:50:26.343386+0000","last_became_peered":"2024-09-11T12:50:26.343386+0000","last_unstale":"2024-09-11T12:50:27.324111+0000","last_undegraded":"2024-09-11T12:50:27.324111+0000","last_fullsized":"2024-09-11T12:50:27.324111+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:06.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.348808+0000","last_change":"2024-09-11T12:50:27.346100+0000","last_active":"2024-09-11T12:50:27.348808+0000","last_peered":"2024-09-11T12:50:27.348808+0000","last_clean":"2024-09-11T12:50:27.348808+0000","last_became_active":"2024-09-11T12:50:27.345380+0000","last_became_peered":"2024-09-11T12:50:27.345380+0000","last_unstale":"2024-09-11T12:50:27.348808+0000","last_undegraded":"2024-09-11T12:50:27.348808+0000","last_fullsized":"2024-09-11T12:50:27.348808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345851+0000","last_change":"2024-09-11T12:50:26.340838+0000","last_active":"2024-09-11T12:50:26.345851+0000","last_peered":"2024-09-11T12:50:26.345851+0000","last_clean":"2024-09-11T12:50:26.345851+0000","last_became_active":"2024-09-11T12:50:26.340531+0000","last_became_peered":"2024-09-11T12:50:26.340531+0000","last_unstale":"2024-09-11T12:50:26.345851+0000","last_undegraded":"2024-09-11T12:50:26.345851+0000","last_fullsized":"2024-09-11T12:50:26.345851+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:03.491437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.359623+0000","last_change":"2024-09-11T12:50:27.356884+0000","last_active":"2024-09-11T12:50:27.359623+0000","last_peered":"2024-09-11T12:50:27.359623+0000","last_clean":"2024-09-11T12:50:27.359623+0000","last_became_active":"2024-09-11T12:50:27.355050+0000","last_became_peered":"2024-09-11T12:50:27.355050+0000","last_unstale":"2024-09-11T12:50:27.359623+0000","last_undegraded":"2024-09-11T12:50:27.359623+0000","last_fullsized":"2024-09-11T12:50:27.359623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365551+0000","last_change":"2024-09-11T12:50:26.334350+0000","last_active":"2024-09-11T12:50:27.365551+0000","last_peered":"2024-09-11T12:50:27.365551+0000","last_clean":"2024-09-11T12:50:27.365551+0000","last_became_active":"2024-09-11T12:50:26.334014+0000","last_became_peered":"2024-09-11T12:50:26.334014+0000","last_unstale":"2024-09-11T12:50:27.365551+0000","last_undegraded":"2024-09-11T12:50:27.365551+0000","last_fullsized":"2024-09-11T12:50:27.365551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351493+0000","last_change":"2024-09-11T12:50:27.350052+0000","last_active":"2024-09-11T12:50:27.351493+0000","last_peered":"2024-09-11T12:50:27.351493+0000","last_clean":"2024-09-11T12:50:27.351493+0000","last_became_active":"2024-09-11T12:50:27.349787+0000","last_became_peered":"2024-09-11T12:50:27.349787+0000","last_unstale":"2024-09-11T12:50:27.351493+0000","last_undegraded":"2024-09-11T12:50:27.351493+0000","last_fullsized":"2024-09-11T12:50:27.351493+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":19,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.371174+0000","last_change":"2024-09-11T12:50:26.342451+0000","last_active":"2024-09-11T12:50:27.371174+0000","last_peered":"2024-09-11T12:50:27.371174+0000","last_clean":"2024-09-11T12:50:27.371174+0000","last_became_active":"2024-09-11T12:50:26.341973+0000","last_became_peered":"2024-09-11T12:50:26.341973+0000","last_unstale":"2024-09-11T12:50:27.371174+0000","last_undegraded":"2024-09-11T12:50:27.371174+0000","last_fullsized":"2024-09-11T12:50:27.371174+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:54.047639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.354826+0000","last_change":"2024-09-11T12:50:27.349605+0000","last_active":"2024-09-11T12:50:27.354826+0000","last_peered":"2024-09-11T12:50:27.354826+0000","last_clean":"2024-09-11T12:50:27.354826+0000","last_became_active":"2024-09-11T12:50:27.349304+0000","last_became_peered":"2024-09-11T12:50:27.349304+0000","last_unstale":"2024-09-11T12:50:27.354826+0000","last_undegraded":"2024-09-11T12:50:27.354826+0000","last_fullsized":"2024-09-11T12:50:27.354826+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362391+0000","last_change":"2024-09-11T12:50:26.337760+0000","last_active":"2024-09-11T12:50:27.362391+0000","last_peered":"2024-09-11T12:50:27.362391+0000","last_clean":"2024-09-11T12:50:27.362391+0000","last_became_active":"2024-09-11T12:50:26.337496+0000","last_became_peered":"2024-09-11T12:50:26.337496+0000","last_unstale":"2024-09-11T12:50:27.362391+0000","last_undegraded":"2024-09-11T12:50:27.362391+0000","last_fullsized":"2024-09-11T12:50:27.362391+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:09.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346498+0000","last_change":"2024-09-11T12:50:27.342946+0000","last_active":"2024-09-11T12:50:27.346498+0000","last_peered":"2024-09-11T12:50:27.346498+0000","last_clean":"2024-09-11T12:50:27.346498+0000","last_became_active":"2024-09-11T12:50:27.342692+0000","last_became_peered":"2024-09-11T12:50:27.342692+0000","last_unstale":"2024-09-11T12:50:27.346498+0000","last_undegraded":"2024-09-11T12:50:27.346498+0000","last_fullsized":"2024-09-11T12:50:27.346498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337180+0000","last_change":"2024-09-11T12:50:26.337698+0000","last_active":"2024-09-11T12:50:27.337180+0000","last_peered":"2024-09-11T12:50:27.337180+0000","last_clean":"2024-09-11T12:50:27.337180+0000","last_became_active":"2024-09-11T12:50:26.337145+0000","last_became_peered":"2024-09-11T12:50:26.337145+0000","last_unstale":"2024-09-11T12:50:27.337180+0000","last_undegraded":"2024-09-11T12:50:27.337180+0000","last_fullsized":"2024-09-11T12:50:27.337180+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364743+0000","last_change":"2024-09-11T12:50:27.356536+0000","last_active":"2024-09-11T12:50:27.364743+0000","last_peered":"2024-09-11T12:50:27.364743+0000","last_clean":"2024-09-11T12:50:27.364743+0000","last_became_active":"2024-09-11T12:50:27.356235+0000","last_became_peered":"2024-09-11T12:50:27.356235+0000","last_unstale":"2024-09-11T12:50:27.364743+0000","last_undegraded":"2024-09-11T12:50:27.364743+0000","last_fullsized":"2024-09-11T12:50:27.364743+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323110+0000","last_change":"2024-09-11T12:50:26.339598+0000","last_active":"2024-09-11T12:50:27.323110+0000","last_peered":"2024-09-11T12:50:27.323110+0000","last_clean":"2024-09-11T12:50:27.323110+0000","last_became_active":"2024-09-11T12:50:26.339322+0000","last_became_peered":"2024-09-11T12:50:26.339322+0000","last_unstale":"2024-09-11T12:50:27.323110+0000","last_undegraded":"2024-09-11T12:50:27.323110+0000","last_fullsized":"2024-09-11T12:50:27.323110+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:45.625861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344441+0000","last_change":"2024-09-11T12:50:27.341021+0000","last_active":"2024-09-11T12:50:27.344441+0000","last_peered":"2024-09-11T12:50:27.344441+0000","last_clean":"2024-09-11T12:50:27.344441+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:27.344441+0000","last_undegraded":"2024-09-11T12:50:27.344441+0000","last_fullsized":"2024-09-11T12:50:27.344441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365719+0000","last_change":"2024-09-11T12:50:26.341481+0000","last_active":"2024-09-11T12:50:27.365719+0000","last_peered":"2024-09-11T12:50:27.365719+0000","last_clean":"2024-09-11T12:50:27.365719+0000","last_became_active":"2024-09-11T12:50:26.341148+0000","last_became_peered":"2024-09-11T12:50:26.341148+0000","last_unstale":"2024-09-11T12:50:27.365719+0000","last_undegraded":"2024-09-11T12:50:27.365719+0000","last_fullsized":"2024-09-11T12:50:27.365719+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:41.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.352075+0000","last_change":"2024-09-11T12:50:27.343792+0000","last_active":"2024-09-11T12:50:27.352075+0000","last_peered":"2024-09-11T12:50:27.352075+0000","last_clean":"2024-09-11T12:50:27.352075+0000","last_became_active":"2024-09-11T12:50:27.343522+0000","last_became_peered":"2024-09-11T12:50:27.343522+0000","last_unstale":"2024-09-11T12:50:27.352075+0000","last_undegraded":"2024-09-11T12:50:27.352075+0000","last_fullsized":"2024-09-11T12:50:27.352075+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:11.279790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.365752+0000","last_change":"2024-09-11T12:50:26.342891+0000","last_active":"2024-09-11T12:50:27.365752+0000","last_peered":"2024-09-11T12:50:27.365752+0000","last_clean":"2024-09-11T12:50:27.365752+0000","last_became_active":"2024-09-11T12:50:26.342639+0000","last_became_peered":"2024-09-11T12:50:26.342639+0000","last_unstale":"2024-09-11T12:50:27.365752+0000","last_undegraded":"2024-09-11T12:50:27.365752+0000","last_fullsized":"2024-09-11T12:50:27.365752+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.344203+0000","last_change":"2024-09-11T12:50:27.335239+0000","last_active":"2024-09-11T12:50:27.344203+0000","last_peered":"2024-09-11T12:50:27.344203+0000","last_clean":"2024-09-11T12:50:27.344203+0000","last_became_active":"2024-09-11T12:50:27.333463+0000","last_became_peered":"2024-09-11T12:50:27.333463+0000","last_unstale":"2024-09-11T12:50:27.344203+0000","last_undegraded":"2024-09-11T12:50:27.344203+0000","last_fullsized":"2024-09-11T12:50:27.344203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.361835+0000","last_change":"2024-09-11T12:50:26.337827+0000","last_active":"2024-09-11T12:50:27.361835+0000","last_peered":"2024-09-11T12:50:27.361835+0000","last_clean":"2024-09-11T12:50:27.361835+0000","last_became_active":"2024-09-11T12:50:26.337466+0000","last_became_peered":"2024-09-11T12:50:26.337466+0000","last_unstale":"2024-09-11T12:50:27.361835+0000","last_undegraded":"2024-09-11T12:50:27.361835+0000","last_fullsized":"2024-09-11T12:50:27.361835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:12.538604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.342628+0000","last_change":"2024-09-11T12:50:27.338996+0000","last_active":"2024-09-11T12:50:27.342628+0000","last_peered":"2024-09-11T12:50:27.342628+0000","last_clean":"2024-09-11T12:50:27.342628+0000","last_became_active":"2024-09-11T12:50:27.338705+0000","last_became_peered":"2024-09-11T12:50:27.338705+0000","last_unstale":"2024-09-11T12:50:27.342628+0000","last_undegraded":"2024-09-11T12:50:27.342628+0000","last_fullsized":"2024-09-11T12:50:27.342628+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.363280+0000","last_change":"2024-09-11T12:50:26.339176+0000","last_active":"2024-09-11T12:50:27.363280+0000","last_peered":"2024-09-11T12:50:27.363280+0000","last_clean":"2024-09-11T12:50:27.363280+0000","last_became_active":"2024-09-11T12:50:26.338882+0000","last_became_peered":"2024-09-11T12:50:26.338882+0000","last_unstale":"2024-09-11T12:50:27.363280+0000","last_undegraded":"2024-09-11T12:50:27.363280+0000","last_fullsized":"2024-09-11T12:50:27.363280+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:11.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.358426+0000","last_change":"2024-09-11T12:50:27.353963+0000","last_active":"2024-09-11T12:50:27.358426+0000","last_peered":"2024-09-11T12:50:27.358426+0000","last_clean":"2024-09-11T12:50:27.358426+0000","last_became_active":"2024-09-11T12:50:27.353684+0000","last_became_peered":"2024-09-11T12:50:27.353684+0000","last_unstale":"2024-09-11T12:50:27.358426+0000","last_undegraded":"2024-09-11T12:50:27.358426+0000","last_fullsized":"2024-09-11T12:50:27.358426+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:07.300287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324319+0000","last_change":"2024-09-11T12:50:26.344464+0000","last_active":"2024-09-11T12:50:27.324319+0000","last_peered":"2024-09-11T12:50:27.324319+0000","last_clean":"2024-09-11T12:50:27.324319+0000","last_became_active":"2024-09-11T12:50:26.344093+0000","last_became_peered":"2024-09-11T12:50:26.344093+0000","last_unstale":"2024-09-11T12:50:27.324319+0000","last_undegraded":"2024-09-11T12:50:27.324319+0000","last_fullsized":"2024-09-11T12:50:27.324319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:47.700330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.343140+0000","last_change":"2024-09-11T12:50:27.338133+0000","last_active":"2024-09-11T12:50:27.343140+0000","last_peered":"2024-09-11T12:50:27.343140+0000","last_clean":"2024-09-11T12:50:27.343140+0000","last_became_active":"2024-09-11T12:50:27.337816+0000","last_became_peered":"2024-09-11T12:50:27.337816+0000","last_unstale":"2024-09-11T12:50:27.343140+0000","last_undegraded":"2024-09-11T12:50:27.343140+0000","last_fullsized":"2024-09-11T12:50:27.343140+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:36:21.709506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.344835+0000","last_change":"2024-09-11T12:50:26.339488+0000","last_active":"2024-09-11T12:50:26.344835+0000","last_peered":"2024-09-11T12:50:26.344835+0000","last_clean":"2024-09-11T12:50:26.344835+0000","last_became_active":"2024-09-11T12:50:26.339267+0000","last_became_peered":"2024-09-11T12:50:26.339267+0000","last_unstale":"2024-09-11T12:50:26.344835+0000","last_undegraded":"2024-09-11T12:50:26.344835+0000","last_fullsized":"2024-09-11T12:50:26.344835+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:54:09.177646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.353434+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:27.353434+0000","last_peered":"2024-09-11T12:50:27.353434+0000","last_clean":"2024-09-11T12:50:27.353434+0000","last_became_active":"2024-09-11T12:50:27.345261+0000","last_became_peered":"2024-09-11T12:50:27.345261+0000","last_unstale":"2024-09-11T12:50:27.353434+0000","last_undegraded":"2024-09-11T12:50:27.353434+0000","last_fullsized":"2024-09-11T12:50:27.353434+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:58.388446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.330245+0000","last_change":"2024-09-11T12:50:26.346500+0000","last_active":"2024-09-11T12:50:27.330245+0000","last_peered":"2024-09-11T12:50:27.330245+0000","last_clean":"2024-09-11T12:50:27.330245+0000","last_became_active":"2024-09-11T12:50:26.346237+0000","last_became_peered":"2024-09-11T12:50:26.346237+0000","last_unstale":"2024-09-11T12:50:27.330245+0000","last_undegraded":"2024-09-11T12:50:27.330245+0000","last_fullsized":"2024-09-11T12:50:27.330245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.346072+0000","last_change":"2024-09-11T12:50:27.342983+0000","last_active":"2024-09-11T12:50:27.346072+0000","last_peered":"2024-09-11T12:50:27.346072+0000","last_clean":"2024-09-11T12:50:27.346072+0000","last_became_active":"2024-09-11T12:50:27.342726+0000","last_became_peered":"2024-09-11T12:50:27.342726+0000","last_unstale":"2024-09-11T12:50:27.346072+0000","last_undegraded":"2024-09-11T12:50:27.346072+0000","last_fullsized":"2024-09-11T12:50:27.346072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:39.629583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325749+0000","last_change":"2024-09-11T12:50:26.339279+0000","last_active":"2024-09-11T12:50:27.325749+0000","last_peered":"2024-09-11T12:50:27.325749+0000","last_clean":"2024-09-11T12:50:27.325749+0000","last_became_active":"2024-09-11T12:50:26.338240+0000","last_became_peered":"2024-09-11T12:50:26.338240+0000","last_unstale":"2024-09-11T12:50:27.325749+0000","last_undegraded":"2024-09-11T12:50:27.325749+0000","last_fullsized":"2024-09-11T12:50:27.325749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:52.150201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357527+0000","last_change":"2024-09-11T12:50:27.349271+0000","last_active":"2024-09-11T12:50:27.357527+0000","last_peered":"2024-09-11T12:50:27.357527+0000","last_clean":"2024-09-11T12:50:27.357527+0000","last_became_active":"2024-09-11T12:50:27.348581+0000","last_became_peered":"2024-09-11T12:50:27.348581+0000","last_unstale":"2024-09-11T12:50:27.357527+0000","last_undegraded":"2024-09-11T12:50:27.357527+0000","last_fullsized":"2024-09-11T12:50:27.357527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:02.419143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.324027+0000","last_change":"2024-09-11T12:50:26.336396+0000","last_active":"2024-09-11T12:50:27.324027+0000","last_peered":"2024-09-11T12:50:27.324027+0000","last_clean":"2024-09-11T12:50:27.324027+0000","last_became_active":"2024-09-11T12:50:26.335998+0000","last_became_peered":"2024-09-11T12:50:26.335998+0000","last_unstale":"2024-09-11T12:50:27.324027+0000","last_undegraded":"2024-09-11T12:50:27.324027+0000","last_fullsized":"2024-09-11T12:50:27.324027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.357673+0000","last_change":"2024-09-11T12:50:27.349951+0000","last_active":"2024-09-11T12:50:27.357673+0000","last_peered":"2024-09-11T12:50:27.357673+0000","last_clean":"2024-09-11T12:50:27.357673+0000","last_became_active":"2024-09-11T12:50:27.349174+0000","last_became_peered":"2024-09-11T12:50:27.349174+0000","last_unstale":"2024-09-11T12:50:27.357673+0000","last_undegraded":"2024-09-11T12:50:27.357673+0000","last_fullsized":"2024-09-11T12:50:27.357673+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:48:03.499603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.364811+0000","last_change":"2024-09-11T12:50:26.337423+0000","last_active":"2024-09-11T12:50:27.364811+0000","last_peered":"2024-09-11T12:50:27.364811+0000","last_clean":"2024-09-11T12:50:27.364811+0000","last_became_active":"2024-09-11T12:50:26.337076+0000","last_became_peered":"2024-09-11T12:50:26.337076+0000","last_unstale":"2024-09-11T12:50:27.364811+0000","last_undegraded":"2024-09-11T12:50:27.364811+0000","last_fullsized":"2024-09-11T12:50:27.364811+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.351691+0000","last_change":"2024-09-11T12:50:27.349463+0000","last_active":"2024-09-11T12:50:27.351691+0000","last_peered":"2024-09-11T12:50:27.351691+0000","last_clean":"2024-09-11T12:50:27.351691+0000","last_became_active":"2024-09-11T12:50:27.349183+0000","last_became_peered":"2024-09-11T12:50:27.349183+0000","last_unstale":"2024-09-11T12:50:27.351691+0000","last_undegraded":"2024-09-11T12:50:27.351691+0000","last_fullsized":"2024-09-11T12:50:27.351691+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.345829+0000","last_change":"2024-09-11T12:50:26.340196+0000","last_active":"2024-09-11T12:50:26.345829+0000","last_peered":"2024-09-11T12:50:26.345829+0000","last_clean":"2024-09-11T12:50:26.345829+0000","last_became_active":"2024-09-11T12:50:26.339927+0000","last_became_peered":"2024-09-11T12:50:26.339927+0000","last_unstale":"2024-09-11T12:50:26.345829+0000","last_undegraded":"2024-09-11T12:50:26.345829+0000","last_fullsized":"2024-09-11T12:50:26.345829+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:57.749644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.325394+0000","last_change":"2024-09-11T12:50:26.325394+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.325394+0000","last_undegraded":"2024-09-11T12:50:26.325394+0000","last_fullsized":"2024-09-11T12:50:26.325394+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337747+0000","last_change":"2024-09-11T12:50:26.338822+0000","last_active":"2024-09-11T12:50:27.337747+0000","last_peered":"2024-09-11T12:50:27.337747+0000","last_clean":"2024-09-11T12:50:27.337747+0000","last_became_active":"2024-09-11T12:50:26.338459+0000","last_became_peered":"2024-09-11T12:50:26.338459+0000","last_unstale":"2024-09-11T12:50:27.337747+0000","last_undegraded":"2024-09-11T12:50:27.337747+0000","last_fullsized":"2024-09-11T12:50:27.337747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:01.412980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347818+0000","last_change":"2024-09-11T12:50:27.344966+0000","last_active":"2024-09-11T12:50:27.347818+0000","last_peered":"2024-09-11T12:50:27.347818+0000","last_clean":"2024-09-11T12:50:27.347818+0000","last_became_active":"2024-09-11T12:50:27.344634+0000","last_became_peered":"2024-09-11T12:50:27.344634+0000","last_unstale":"2024-09-11T12:50:27.347818+0000","last_undegraded":"2024-09-11T12:50:27.347818+0000","last_fullsized":"2024-09-11T12:50:27.347818+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":10,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-11T12:50:26.337771+0000","last_change":"2024-09-11T12:50:26.330959+0000","last_active":"2024-09-11T12:50:26.337771+0000","last_peered":"2024-09-11T12:50:26.337771+0000","last_clean":"2024-09-11T12:50:26.337771+0000","last_became_active":"2024-09-11T12:50:26.330722+0000","last_became_peered":"2024-09-11T12:50:26.330722+0000","last_unstale":"2024-09-11T12:50:26.337771+0000","last_undegraded":"2024-09-11T12:50:26.337771+0000","last_fullsized":"2024-09-11T12:50:26.337771+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:51.012754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2024-09-11T12:50:26.324459+0000","last_change":"2024-09-11T12:50:26.324459+0000","last_active":"2024-09-11T12:50:26.307616+0000","last_peered":"2024-09-11T12:50:26.307616+0000","last_clean":"2024-09-11T12:50:26.307616+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:50:26.324459+0000","last_undegraded":"2024-09-11T12:50:26.324459+0000","last_fullsized":"2024-09-11T12:50:26.324459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.337362+0000","last_change":"2024-09-11T12:50:26.337961+0000","last_active":"2024-09-11T12:50:27.337362+0000","last_peered":"2024-09-11T12:50:27.337362+0000","last_clean":"2024-09-11T12:50:27.337362+0000","last_became_active":"2024-09-11T12:50:26.337461+0000","last_became_peered":"2024-09-11T12:50:26.337461+0000","last_unstale":"2024-09-11T12:50:27.337362+0000","last_undegraded":"2024-09-11T12:50:27.337362+0000","last_fullsized":"2024-09-11T12:50:27.337362+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.345998+0000","last_change":"2024-09-11T12:50:27.342392+0000","last_active":"2024-09-11T12:50:27.345998+0000","last_peered":"2024-09-11T12:50:27.345998+0000","last_clean":"2024-09-11T12:50:27.345998+0000","last_became_active":"2024-09-11T12:50:27.342116+0000","last_became_peered":"2024-09-11T12:50:27.342116+0000","last_unstale":"2024-09-11T12:50:27.345998+0000","last_undegraded":"2024-09-11T12:50:27.345998+0000","last_fullsized":"2024-09-11T12:50:27.345998+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.325924+0000","last_change":"2024-09-11T12:50:26.330978+0000","last_active":"2024-09-11T12:50:27.325924+0000","last_peered":"2024-09-11T12:50:27.325924+0000","last_clean":"2024-09-11T12:50:27.325924+0000","last_became_active":"2024-09-11T12:50:26.330681+0000","last_became_peered":"2024-09-11T12:50:26.330681+0000","last_unstale":"2024-09-11T12:50:27.325924+0000","last_undegraded":"2024-09-11T12:50:27.325924+0000","last_fullsized":"2024-09-11T12:50:27.325924+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:43.803456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.334967+0000","last_change":"2024-09-11T12:50:27.328606+0000","last_active":"2024-09-11T12:50:27.334967+0000","last_peered":"2024-09-11T12:50:27.334967+0000","last_clean":"2024-09-11T12:50:27.334967+0000","last_became_active":"2024-09-11T12:50:27.328276+0000","last_became_peered":"2024-09-11T12:50:27.328276+0000","last_unstale":"2024-09-11T12:50:27.334967+0000","last_undegraded":"2024-09-11T12:50:27.334967+0000","last_fullsized":"2024-09-11T12:50:27.334967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:36.902883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.323120+0000","last_change":"2024-09-11T12:50:26.337651+0000","last_active":"2024-09-11T12:50:27.323120+0000","last_peered":"2024-09-11T12:50:27.323120+0000","last_clean":"2024-09-11T12:50:27.323120+0000","last_became_active":"2024-09-11T12:50:26.337387+0000","last_became_peered":"2024-09-11T12:50:26.337387+0000","last_unstale":"2024-09-11T12:50:27.323120+0000","last_undegraded":"2024-09-11T12:50:27.323120+0000","last_fullsized":"2024-09-11T12:50:27.323120+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:52.922609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.362603+0000","last_change":"2024-09-11T12:50:27.351937+0000","last_active":"2024-09-11T12:50:27.362603+0000","last_peered":"2024-09-11T12:50:27.362603+0000","last_clean":"2024-09-11T12:50:27.362603+0000","last_became_active":"2024-09-11T12:50:27.350570+0000","last_became_peered":"2024-09-11T12:50:27.350570+0000","last_unstale":"2024-09-11T12:50:27.362603+0000","last_undegraded":"2024-09-11T12:50:27.362603+0000","last_fullsized":"2024-09-11T12:50:27.362603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:45.432199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.328966+0000","last_change":"2024-09-11T12:50:26.344815+0000","last_active":"2024-09-11T12:50:27.328966+0000","last_peered":"2024-09-11T12:50:27.328966+0000","last_clean":"2024-09-11T12:50:27.328966+0000","last_became_active":"2024-09-11T12:50:26.344518+0000","last_became_peered":"2024-09-11T12:50:26.344518+0000","last_unstale":"2024-09-11T12:50:27.328966+0000","last_undegraded":"2024-09-11T12:50:27.328966+0000","last_fullsized":"2024-09-11T12:50:27.328966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:24.053403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-11T12:50:27.347970+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:50:27.347970+0000","last_peered":"2024-09-11T12:50:27.347970+0000","last_clean":"2024-09-11T12:50:27.347970+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:50:27.347970+0000","last_undegraded":"2024-09-11T12:50:27.347970+0000","last_fullsized":"2024-09-11T12:50:27.347970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":20,"num_object_clones":0,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35,"ondisk_log_size":35,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509454,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32116,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339724,"statfs":{"total":96636764160,"available":96603877376,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509454,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509454,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509454,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":344,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":352256,"data_stored":131072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343884,"statfs":{"total":96636764160,"available":96608137216,"internally_reserved":0,"allocated":946176,"data_stored":721440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:50:35.997 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 -- 172.21.15.78:0/284843794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b2c004810 msgr2=0x7f3b2c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/284843794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b2c004810 0x7f3b2c004c90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3b340016f0 tx=0x7f3b3401c750 comp rx=0 tx=0).stop 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 -- 172.21.15.78:0/284843794 shutdown_connections 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/284843794 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b2c004810 0x7f3b2c004c90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 -- 172.21.15.78:0/284843794 >> 172.21.15.78:0/284843794 conn(0x7f3b480cda10 msgr2=0x7f3b480cde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 -- 172.21.15.78:0/284843794 shutdown_connections 2024-09-11T12:50:36.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.142+0000 7f3b4fa00640 1 -- 172.21.15.78:0/284843794 wait complete. 2024-09-11T12:50:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 Processor -- start 2024-09-11T12:50:36.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 -- start start 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c004810 0x7f3b48155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b48153630 0x7f3b48153ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b480c6680 con 0x7f3b481576e0 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b480c6500 con 0x7f3b2c004810 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3b48073770 con 0x7f3b48153630 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32974/0 (socket says 172.21.15.78:32974) 2024-09-11T12:50:36.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 -- 172.21.15.78:0/2486456541 learned_addr learned my addr 172.21.15.78:0/2486456541 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4d000640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c004810 0x7f3b48155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4da00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b48153630 0x7f3b48153ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 -- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b48153630 msgr2=0x7f3b48153ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b48153630 0x7f3b48153ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 -- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c004810 msgr2=0x7f3b48155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c004810 0x7f3b48155510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 -- 172.21.15.78:0/2486456541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b48155af0 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b47e00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000fe0 tx=0x7f3b34006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b34003890 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b34003b20 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b340a55b0 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.146+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b48155dd0 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b48156390 con 0x7f3b481576e0 2024-09-11T12:50:36.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b4815cdb0 con 0x7f3b481576e0 2024-09-11T12:50:36.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f3b340a5750 con 0x7f3b481576e0 2024-09-11T12:50:36.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b44c00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 0x7f3b28073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3b3412b7d0 con 0x7f3b481576e0 2024-09-11T12:50:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b4d000640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 0x7f3b28073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.150+0000 7f3b4d000640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 0x7f3b28073b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b48052540 tx=0x7f3b380091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.154+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3b340b3070 con 0x7f3b481576e0 2024-09-11T12:50:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.314+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3b4815cad0 con 0x7f3b280716b0 2024-09-11T12:50:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.322+0000 7f3b44c00640 1 -- 172.21.15.78:0/2486456541 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251940 (secure 0 0 0) 0x7f3b3805e840 con 0x7f3b280716b0 2024-09-11T12:50:36.328 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:36.330 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:50:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 msgr2=0x7f3b28073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 0x7f3b28073b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3b48052540 tx=0x7f3b380091e0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 msgr2=0x7f3b4814a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3b34000fe0 tx=0x7f3b34006950 comp rx=0 tx=0).stop 2024-09-11T12:50:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 shutdown_connections 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3b280716b0 0x7f3b28073b70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b48153630 0x7f3b48153ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b481576e0 0x7f3b4814a650 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 --2- 172.21.15.78:0/2486456541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b2c004810 0x7f3b48155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 >> 172.21.15.78:0/2486456541 conn(0x7f3b480cda10 msgr2=0x7f3b480c4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 shutdown_connections 2024-09-11T12:50:36.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.326+0000 7f3b4fa00640 1 -- 172.21.15.78:0/2486456541 wait complete. 2024-09-11T12:50:36.349 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-11T12:50:35.117290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237036,"kb_used_data":4168,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737684,"statfs":{"total":773094113280,"available":772851388416,"internally_reserved":0,"allocated":4268032,"data_stored":2338732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001733"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381527+0000","last_change":"2024-09-11T12:50:27.348834+0000","last_active":"2024-09-11T12:50:29.381527+0000","last_peered":"2024-09-11T12:50:29.381527+0000","last_clean":"2024-09-11T12:50:29.381527+0000","last_became_active":"2024-09-11T12:50:27.348574+0000","last_became_peered":"2024-09-11T12:50:27.348574+0000","last_unstale":"2024-09-11T12:50:29.381527+0000","last_undegraded":"2024-09-11T12:50:29.381527+0000","last_fullsized":"2024-09-11T12:50:29.381527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381459+0000","last_change":"2024-09-11T12:50:27.336993+0000","last_active":"2024-09-11T12:50:29.381459+0000","last_peered":"2024-09-11T12:50:29.381459+0000","last_clean":"2024-09-11T12:50:29.381459+0000","last_became_active":"2024-09-11T12:50:27.336588+0000","last_became_peered":"2024-09-11T12:50:27.336588+0000","last_unstale":"2024-09-11T12:50:29.381459+0000","last_undegraded":"2024-09-11T12:50:29.381459+0000","last_fullsized":"2024-09-11T12:50:29.381459+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:20.234111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334509+0000","last_change":"2024-09-11T12:50:27.342867+0000","last_active":"2024-09-11T12:50:28.334509+0000","last_peered":"2024-09-11T12:50:28.334509+0000","last_clean":"2024-09-11T12:50:28.334509+0000","last_became_active":"2024-09-11T12:50:27.342576+0000","last_became_peered":"2024-09-11T12:50:27.342576+0000","last_unstale":"2024-09-11T12:50:28.334509+0000","last_undegraded":"2024-09-11T12:50:28.334509+0000","last_fullsized":"2024-09-11T12:50:28.334509+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:08.273227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346925+0000","last_change":"2024-09-11T12:50:27.357904+0000","last_active":"2024-09-11T12:50:28.346925+0000","last_peered":"2024-09-11T12:50:28.346925+0000","last_clean":"2024-09-11T12:50:28.346925+0000","last_became_active":"2024-09-11T12:50:27.356836+0000","last_became_peered":"2024-09-11T12:50:27.356836+0000","last_unstale":"2024-09-11T12:50:28.346925+0000","last_undegraded":"2024-09-11T12:50:28.346925+0000","last_fullsized":"2024-09-11T12:50:28.346925+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771162+0000","last_change":"2024-09-11T12:50:27.341619+0000","last_active":"2024-09-11T12:50:28.771162+0000","last_peered":"2024-09-11T12:50:28.771162+0000","last_clean":"2024-09-11T12:50:28.771162+0000","last_became_active":"2024-09-11T12:50:27.341345+0000","last_became_peered":"2024-09-11T12:50:27.341345+0000","last_unstale":"2024-09-11T12:50:28.771162+0000","last_undegraded":"2024-09-11T12:50:28.771162+0000","last_fullsized":"2024-09-11T12:50:28.771162+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:48.706494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380918+0000","last_change":"2024-09-11T12:50:27.335087+0000","last_active":"2024-09-11T12:50:29.380918+0000","last_peered":"2024-09-11T12:50:29.380918+0000","last_clean":"2024-09-11T12:50:29.380918+0000","last_became_active":"2024-09-11T12:50:27.334790+0000","last_became_peered":"2024-09-11T12:50:27.334790+0000","last_unstale":"2024-09-11T12:50:29.380918+0000","last_undegraded":"2024-09-11T12:50:29.380918+0000","last_fullsized":"2024-09-11T12:50:29.380918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:42.771708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946432+0000","last_change":"2024-09-11T12:50:27.355196+0000","last_active":"2024-09-11T12:50:28.946432+0000","last_peered":"2024-09-11T12:50:28.946432+0000","last_clean":"2024-09-11T12:50:28.946432+0000","last_became_active":"2024-09-11T12:50:27.353288+0000","last_became_peered":"2024-09-11T12:50:27.353288+0000","last_unstale":"2024-09-11T12:50:28.946432+0000","last_undegraded":"2024-09-11T12:50:28.946432+0000","last_fullsized":"2024-09-11T12:50:28.946432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:41.424479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.945862+0000","last_change":"2024-09-11T12:50:27.358075+0000","last_active":"2024-09-11T12:50:28.945862+0000","last_peered":"2024-09-11T12:50:28.945862+0000","last_clean":"2024-09-11T12:50:28.945862+0000","last_became_active":"2024-09-11T12:50:27.357851+0000","last_became_peered":"2024-09-11T12:50:27.357851+0000","last_unstale":"2024-09-11T12:50:28.945862+0000","last_undegraded":"2024-09-11T12:50:28.945862+0000","last_fullsized":"2024-09-11T12:50:28.945862+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.335310+0000","last_change":"2024-09-11T12:50:27.343726+0000","last_active":"2024-09-11T12:50:28.335310+0000","last_peered":"2024-09-11T12:50:28.335310+0000","last_clean":"2024-09-11T12:50:28.335310+0000","last_became_active":"2024-09-11T12:50:27.343450+0000","last_became_peered":"2024-09-11T12:50:27.343450+0000","last_unstale":"2024-09-11T12:50:28.335310+0000","last_undegraded":"2024-09-11T12:50:28.335310+0000","last_fullsized":"2024-09-11T12:50:28.335310+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:50.455693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347002+0000","last_change":"2024-09-11T12:50:27.339989+0000","last_active":"2024-09-11T12:50:28.347002+0000","last_peered":"2024-09-11T12:50:28.347002+0000","last_clean":"2024-09-11T12:50:28.347002+0000","last_became_active":"2024-09-11T12:50:27.339673+0000","last_became_peered":"2024-09-11T12:50:27.339673+0000","last_unstale":"2024-09-11T12:50:28.347002+0000","last_undegraded":"2024-09-11T12:50:28.347002+0000","last_fullsized":"2024-09-11T12:50:28.347002+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946264+0000","last_change":"2024-09-11T12:50:27.363103+0000","last_active":"2024-09-11T12:50:28.946264+0000","last_peered":"2024-09-11T12:50:28.946264+0000","last_clean":"2024-09-11T12:50:28.946264+0000","last_became_active":"2024-09-11T12:50:27.362746+0000","last_became_peered":"2024-09-11T12:50:27.362746+0000","last_unstale":"2024-09-11T12:50:28.946264+0000","last_undegraded":"2024-09-11T12:50:28.946264+0000","last_fullsized":"2024-09-11T12:50:28.946264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333723+0000","last_change":"2024-09-11T12:50:27.329779+0000","last_active":"2024-09-11T12:50:28.333723+0000","last_peered":"2024-09-11T12:50:28.333723+0000","last_clean":"2024-09-11T12:50:28.333723+0000","last_became_active":"2024-09-11T12:50:27.327796+0000","last_became_peered":"2024-09-11T12:50:27.327796+0000","last_unstale":"2024-09-11T12:50:28.333723+0000","last_undegraded":"2024-09-11T12:50:28.333723+0000","last_fullsized":"2024-09-11T12:50:28.333723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:19.025601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771276+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:28.771276+0000","last_peered":"2024-09-11T12:50:28.771276+0000","last_clean":"2024-09-11T12:50:28.771276+0000","last_became_active":"2024-09-11T12:50:27.344834+0000","last_became_peered":"2024-09-11T12:50:27.344834+0000","last_unstale":"2024-09-11T12:50:28.771276+0000","last_undegraded":"2024-09-11T12:50:28.771276+0000","last_fullsized":"2024-09-11T12:50:28.771276+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:13.544768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380903+0000","last_change":"2024-09-11T12:50:27.339363+0000","last_active":"2024-09-11T12:50:29.380903+0000","last_peered":"2024-09-11T12:50:29.380903+0000","last_clean":"2024-09-11T12:50:29.380903+0000","last_became_active":"2024-09-11T12:50:27.339131+0000","last_became_peered":"2024-09-11T12:50:27.339131+0000","last_unstale":"2024-09-11T12:50:29.380903+0000","last_undegraded":"2024-09-11T12:50:29.380903+0000","last_fullsized":"2024-09-11T12:50:29.380903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:25.420948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333529+0000","last_change":"2024-09-11T12:50:27.335582+0000","last_active":"2024-09-11T12:50:28.333529+0000","last_peered":"2024-09-11T12:50:28.333529+0000","last_clean":"2024-09-11T12:50:28.333529+0000","last_became_active":"2024-09-11T12:50:27.335002+0000","last_became_peered":"2024-09-11T12:50:27.335002+0000","last_unstale":"2024-09-11T12:50:28.333529+0000","last_undegraded":"2024-09-11T12:50:28.333529+0000","last_fullsized":"2024-09-11T12:50:28.333529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946456+0000","last_change":"2024-09-11T12:50:27.363062+0000","last_active":"2024-09-11T12:50:28.946456+0000","last_peered":"2024-09-11T12:50:28.946456+0000","last_clean":"2024-09-11T12:50:28.946456+0000","last_became_active":"2024-09-11T12:50:27.362788+0000","last_became_peered":"2024-09-11T12:50:27.362788+0000","last_unstale":"2024-09-11T12:50:28.946456+0000","last_undegraded":"2024-09-11T12:50:28.946456+0000","last_fullsized":"2024-09-11T12:50:28.946456+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:25.420948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946301+0000","last_change":"2024-09-11T12:50:27.357790+0000","last_active":"2024-09-11T12:50:28.946301+0000","last_peered":"2024-09-11T12:50:28.946301+0000","last_clean":"2024-09-11T12:50:28.946301+0000","last_became_active":"2024-09-11T12:50:27.353639+0000","last_became_peered":"2024-09-11T12:50:27.353639+0000","last_unstale":"2024-09-11T12:50:28.946301+0000","last_undegraded":"2024-09-11T12:50:28.946301+0000","last_fullsized":"2024-09-11T12:50:28.946301+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381453+0000","last_change":"2024-09-11T12:50:27.336825+0000","last_active":"2024-09-11T12:50:29.381453+0000","last_peered":"2024-09-11T12:50:29.381453+0000","last_clean":"2024-09-11T12:50:29.381453+0000","last_became_active":"2024-09-11T12:50:27.336241+0000","last_became_peered":"2024-09-11T12:50:27.336241+0000","last_unstale":"2024-09-11T12:50:29.381453+0000","last_undegraded":"2024-09-11T12:50:29.381453+0000","last_fullsized":"2024-09-11T12:50:29.381453+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:02.387884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381252+0000","last_change":"2024-09-11T12:50:27.344954+0000","last_active":"2024-09-11T12:50:29.381252+0000","last_peered":"2024-09-11T12:50:29.381252+0000","last_clean":"2024-09-11T12:50:29.381252+0000","last_became_active":"2024-09-11T12:50:27.344681+0000","last_became_peered":"2024-09-11T12:50:27.344681+0000","last_unstale":"2024-09-11T12:50:29.381252+0000","last_undegraded":"2024-09-11T12:50:29.381252+0000","last_fullsized":"2024-09-11T12:50:29.381252+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:24:01.273711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771395+0000","last_change":"2024-09-11T12:50:27.352798+0000","last_active":"2024-09-11T12:50:28.771395+0000","last_peered":"2024-09-11T12:50:28.771395+0000","last_clean":"2024-09-11T12:50:28.771395+0000","last_became_active":"2024-09-11T12:50:27.349194+0000","last_became_peered":"2024-09-11T12:50:27.349194+0000","last_unstale":"2024-09-11T12:50:28.771395+0000","last_undegraded":"2024-09-11T12:50:28.771395+0000","last_fullsized":"2024-09-11T12:50:28.771395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:44.809620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334700+0000","last_change":"2024-09-11T12:50:27.343151+0000","last_active":"2024-09-11T12:50:28.334700+0000","last_peered":"2024-09-11T12:50:28.334700+0000","last_clean":"2024-09-11T12:50:28.334700+0000","last_became_active":"2024-09-11T12:50:27.339404+0000","last_became_peered":"2024-09-11T12:50:27.339404+0000","last_unstale":"2024-09-11T12:50:28.334700+0000","last_undegraded":"2024-09-11T12:50:28.334700+0000","last_fullsized":"2024-09-11T12:50:28.334700+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334649+0000","last_change":"2024-09-11T12:50:27.330086+0000","last_active":"2024-09-11T12:50:28.334649+0000","last_peered":"2024-09-11T12:50:28.334649+0000","last_clean":"2024-09-11T12:50:28.334649+0000","last_became_active":"2024-09-11T12:50:27.329846+0000","last_became_peered":"2024-09-11T12:50:27.329846+0000","last_unstale":"2024-09-11T12:50:28.334649+0000","last_undegraded":"2024-09-11T12:50:28.334649+0000","last_fullsized":"2024-09-11T12:50:28.334649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331510+0000","last_change":"2024-09-11T12:50:27.352562+0000","last_active":"2024-09-11T12:50:28.331510+0000","last_peered":"2024-09-11T12:50:28.331510+0000","last_clean":"2024-09-11T12:50:28.331510+0000","last_became_active":"2024-09-11T12:50:27.352288+0000","last_became_peered":"2024-09-11T12:50:27.352288+0000","last_unstale":"2024-09-11T12:50:28.331510+0000","last_undegraded":"2024-09-11T12:50:28.331510+0000","last_fullsized":"2024-09-11T12:50:28.331510+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:53.123507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333389+0000","last_change":"2024-09-11T12:50:27.337469+0000","last_active":"2024-09-11T12:50:28.333389+0000","last_peered":"2024-09-11T12:50:28.333389+0000","last_clean":"2024-09-11T12:50:28.333389+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:28.333389+0000","last_undegraded":"2024-09-11T12:50:28.333389+0000","last_fullsized":"2024-09-11T12:50:28.333389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347120+0000","last_change":"2024-09-11T12:50:27.355625+0000","last_active":"2024-09-11T12:50:28.347120+0000","last_peered":"2024-09-11T12:50:28.347120+0000","last_clean":"2024-09-11T12:50:28.347120+0000","last_became_active":"2024-09-11T12:50:27.355274+0000","last_became_peered":"2024-09-11T12:50:27.355274+0000","last_unstale":"2024-09-11T12:50:28.347120+0000","last_undegraded":"2024-09-11T12:50:28.347120+0000","last_fullsized":"2024-09-11T12:50:28.347120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381634+0000","last_change":"2024-09-11T12:50:27.356361+0000","last_active":"2024-09-11T12:50:29.381634+0000","last_peered":"2024-09-11T12:50:29.381634+0000","last_clean":"2024-09-11T12:50:29.381634+0000","last_became_active":"2024-09-11T12:50:27.356074+0000","last_became_peered":"2024-09-11T12:50:27.356074+0000","last_unstale":"2024-09-11T12:50:29.381634+0000","last_undegraded":"2024-09-11T12:50:29.381634+0000","last_fullsized":"2024-09-11T12:50:29.381634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:28.266222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347161+0000","last_change":"2024-09-11T12:50:27.357026+0000","last_active":"2024-09-11T12:50:28.347161+0000","last_peered":"2024-09-11T12:50:28.347161+0000","last_clean":"2024-09-11T12:50:28.347161+0000","last_became_active":"2024-09-11T12:50:27.356785+0000","last_became_peered":"2024-09-11T12:50:27.356785+0000","last_unstale":"2024-09-11T12:50:28.347161+0000","last_undegraded":"2024-09-11T12:50:28.347161+0000","last_fullsized":"2024-09-11T12:50:28.347161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333807+0000","last_change":"2024-09-11T12:50:27.336803+0000","last_active":"2024-09-11T12:50:28.333807+0000","last_peered":"2024-09-11T12:50:28.333807+0000","last_clean":"2024-09-11T12:50:28.333807+0000","last_became_active":"2024-09-11T12:50:27.334739+0000","last_became_peered":"2024-09-11T12:50:27.334739+0000","last_unstale":"2024-09-11T12:50:28.333807+0000","last_undegraded":"2024-09-11T12:50:28.333807+0000","last_fullsized":"2024-09-11T12:50:28.333807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:13:38.183816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334774+0000","last_change":"2024-09-11T12:50:27.329208+0000","last_active":"2024-09-11T12:50:28.334774+0000","last_peered":"2024-09-11T12:50:28.334774+0000","last_clean":"2024-09-11T12:50:28.334774+0000","last_became_active":"2024-09-11T12:50:27.328939+0000","last_became_peered":"2024-09-11T12:50:27.328939+0000","last_unstale":"2024-09-11T12:50:28.334774+0000","last_undegraded":"2024-09-11T12:50:28.334774+0000","last_fullsized":"2024-09-11T12:50:28.334774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946003+0000","last_change":"2024-09-11T12:50:27.348968+0000","last_active":"2024-09-11T12:50:28.946003+0000","last_peered":"2024-09-11T12:50:28.946003+0000","last_clean":"2024-09-11T12:50:28.946003+0000","last_became_active":"2024-09-11T12:50:27.348693+0000","last_became_peered":"2024-09-11T12:50:27.348693+0000","last_unstale":"2024-09-11T12:50:28.946003+0000","last_undegraded":"2024-09-11T12:50:28.946003+0000","last_fullsized":"2024-09-11T12:50:28.946003+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381244+0000","last_change":"2024-09-11T12:50:27.341331+0000","last_active":"2024-09-11T12:50:29.381244+0000","last_peered":"2024-09-11T12:50:29.381244+0000","last_clean":"2024-09-11T12:50:29.381244+0000","last_became_active":"2024-09-11T12:50:27.341038+0000","last_became_peered":"2024-09-11T12:50:27.341038+0000","last_unstale":"2024-09-11T12:50:29.381244+0000","last_undegraded":"2024-09-11T12:50:29.381244+0000","last_fullsized":"2024-09-11T12:50:29.381244+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:42.394300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.945914+0000","last_change":"2024-09-11T12:50:27.356104+0000","last_active":"2024-09-11T12:50:28.945914+0000","last_peered":"2024-09-11T12:50:28.945914+0000","last_clean":"2024-09-11T12:50:28.945914+0000","last_became_active":"2024-09-11T12:50:27.355113+0000","last_became_peered":"2024-09-11T12:50:27.355113+0000","last_unstale":"2024-09-11T12:50:28.945914+0000","last_undegraded":"2024-09-11T12:50:28.945914+0000","last_fullsized":"2024-09-11T12:50:28.945914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:25.110316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331772+0000","last_change":"2024-09-11T12:50:27.350757+0000","last_active":"2024-09-11T12:50:28.331772+0000","last_peered":"2024-09-11T12:50:28.331772+0000","last_clean":"2024-09-11T12:50:28.331772+0000","last_became_active":"2024-09-11T12:50:27.350476+0000","last_became_peered":"2024-09-11T12:50:27.350476+0000","last_unstale":"2024-09-11T12:50:28.331772+0000","last_undegraded":"2024-09-11T12:50:28.331772+0000","last_fullsized":"2024-09-11T12:50:28.331772+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:37:44.034586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381654+0000","last_change":"2024-09-11T12:50:26.343869+0000","last_active":"2024-09-11T12:50:29.381654+0000","last_peered":"2024-09-11T12:50:29.381654+0000","last_clean":"2024-09-11T12:50:29.381654+0000","last_became_active":"2024-09-11T12:50:26.343588+0000","last_became_peered":"2024-09-11T12:50:26.343588+0000","last_unstale":"2024-09-11T12:50:29.381654+0000","last_undegraded":"2024-09-11T12:50:29.381654+0000","last_fullsized":"2024-09-11T12:50:29.381654+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381578+0000","last_change":"2024-09-11T12:50:27.340558+0000","last_active":"2024-09-11T12:50:29.381578+0000","last_peered":"2024-09-11T12:50:29.381578+0000","last_clean":"2024-09-11T12:50:29.381578+0000","last_became_active":"2024-09-11T12:50:27.337140+0000","last_became_peered":"2024-09-11T12:50:27.337140+0000","last_unstale":"2024-09-11T12:50:29.381578+0000","last_undegraded":"2024-09-11T12:50:29.381578+0000","last_fullsized":"2024-09-11T12:50:29.381578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331866+0000","last_change":"2024-09-11T12:50:26.335186+0000","last_active":"2024-09-11T12:50:28.331866+0000","last_peered":"2024-09-11T12:50:28.331866+0000","last_clean":"2024-09-11T12:50:28.331866+0000","last_became_active":"2024-09-11T12:50:26.334825+0000","last_became_peered":"2024-09-11T12:50:26.334825+0000","last_unstale":"2024-09-11T12:50:28.331866+0000","last_undegraded":"2024-09-11T12:50:28.331866+0000","last_fullsized":"2024-09-11T12:50:28.331866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:06:41.207480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334889+0000","last_change":"2024-09-11T12:50:27.328627+0000","last_active":"2024-09-11T12:50:28.334889+0000","last_peered":"2024-09-11T12:50:28.334889+0000","last_clean":"2024-09-11T12:50:28.334889+0000","last_became_active":"2024-09-11T12:50:27.328292+0000","last_became_peered":"2024-09-11T12:50:27.328292+0000","last_unstale":"2024-09-11T12:50:28.334889+0000","last_undegraded":"2024-09-11T12:50:28.334889+0000","last_fullsized":"2024-09-11T12:50:28.334889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:07.396893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381227+0000","last_change":"2024-09-11T12:50:26.342258+0000","last_active":"2024-09-11T12:50:29.381227+0000","last_peered":"2024-09-11T12:50:29.381227+0000","last_clean":"2024-09-11T12:50:29.381227+0000","last_became_active":"2024-09-11T12:50:26.341986+0000","last_became_peered":"2024-09-11T12:50:26.341986+0000","last_unstale":"2024-09-11T12:50:29.381227+0000","last_undegraded":"2024-09-11T12:50:29.381227+0000","last_fullsized":"2024-09-11T12:50:29.381227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:46.357747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347194+0000","last_change":"2024-09-11T12:50:27.357376+0000","last_active":"2024-09-11T12:50:28.347194+0000","last_peered":"2024-09-11T12:50:28.347194+0000","last_clean":"2024-09-11T12:50:28.347194+0000","last_became_active":"2024-09-11T12:50:27.357016+0000","last_became_peered":"2024-09-11T12:50:27.357016+0000","last_unstale":"2024-09-11T12:50:28.347194+0000","last_undegraded":"2024-09-11T12:50:28.347194+0000","last_fullsized":"2024-09-11T12:50:28.347194+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:41.424479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771440+0000","last_change":"2024-09-11T12:50:26.343854+0000","last_active":"2024-09-11T12:50:28.771440+0000","last_peered":"2024-09-11T12:50:28.771440+0000","last_clean":"2024-09-11T12:50:28.771440+0000","last_became_active":"2024-09-11T12:50:26.341771+0000","last_became_peered":"2024-09-11T12:50:26.341771+0000","last_unstale":"2024-09-11T12:50:28.771440+0000","last_undegraded":"2024-09-11T12:50:28.771440+0000","last_fullsized":"2024-09-11T12:50:28.771440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:19:51.008327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946342+0000","last_change":"2024-09-11T12:50:27.352862+0000","last_active":"2024-09-11T12:50:28.946342+0000","last_peered":"2024-09-11T12:50:28.946342+0000","last_clean":"2024-09-11T12:50:28.946342+0000","last_became_active":"2024-09-11T12:50:27.352467+0000","last_became_peered":"2024-09-11T12:50:27.352467+0000","last_unstale":"2024-09-11T12:50:28.946342+0000","last_undegraded":"2024-09-11T12:50:28.946342+0000","last_fullsized":"2024-09-11T12:50:28.946342+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334860+0000","last_change":"2024-09-11T12:50:26.342659+0000","last_active":"2024-09-11T12:50:28.334860+0000","last_peered":"2024-09-11T12:50:28.334860+0000","last_clean":"2024-09-11T12:50:28.334860+0000","last_became_active":"2024-09-11T12:50:26.342331+0000","last_became_peered":"2024-09-11T12:50:26.342331+0000","last_unstale":"2024-09-11T12:50:28.334860+0000","last_undegraded":"2024-09-11T12:50:28.334860+0000","last_fullsized":"2024-09-11T12:50:28.334860+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:18.698135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333695+0000","last_change":"2024-09-11T12:50:27.347574+0000","last_active":"2024-09-11T12:50:28.333695+0000","last_peered":"2024-09-11T12:50:28.333695+0000","last_clean":"2024-09-11T12:50:28.333695+0000","last_became_active":"2024-09-11T12:50:27.347302+0000","last_became_peered":"2024-09-11T12:50:27.347302+0000","last_unstale":"2024-09-11T12:50:28.333695+0000","last_undegraded":"2024-09-11T12:50:28.333695+0000","last_fullsized":"2024-09-11T12:50:28.333695+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334565+0000","last_change":"2024-09-11T12:50:26.341601+0000","last_active":"2024-09-11T12:50:28.334565+0000","last_peered":"2024-09-11T12:50:28.334565+0000","last_clean":"2024-09-11T12:50:28.334565+0000","last_became_active":"2024-09-11T12:50:26.341238+0000","last_became_peered":"2024-09-11T12:50:26.341238+0000","last_unstale":"2024-09-11T12:50:28.334565+0000","last_undegraded":"2024-09-11T12:50:28.334565+0000","last_fullsized":"2024-09-11T12:50:28.334565+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:46.542517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333765+0000","last_change":"2024-09-11T12:50:27.333846+0000","last_active":"2024-09-11T12:50:28.333765+0000","last_peered":"2024-09-11T12:50:28.333765+0000","last_clean":"2024-09-11T12:50:28.333765+0000","last_became_active":"2024-09-11T12:50:27.333500+0000","last_became_peered":"2024-09-11T12:50:27.333500+0000","last_unstale":"2024-09-11T12:50:28.333765+0000","last_undegraded":"2024-09-11T12:50:28.333765+0000","last_fullsized":"2024-09-11T12:50:28.333765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:46.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380735+0000","last_change":"2024-09-11T12:50:26.339433+0000","last_active":"2024-09-11T12:50:29.380735+0000","last_peered":"2024-09-11T12:50:29.380735+0000","last_clean":"2024-09-11T12:50:29.380735+0000","last_became_active":"2024-09-11T12:50:26.338991+0000","last_became_peered":"2024-09-11T12:50:26.338991+0000","last_unstale":"2024-09-11T12:50:29.380735+0000","last_undegraded":"2024-09-11T12:50:29.380735+0000","last_fullsized":"2024-09-11T12:50:29.380735+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346617+0000","last_change":"2024-09-11T12:50:27.351561+0000","last_active":"2024-09-11T12:50:28.346617+0000","last_peered":"2024-09-11T12:50:28.346617+0000","last_clean":"2024-09-11T12:50:28.346617+0000","last_became_active":"2024-09-11T12:50:27.351277+0000","last_became_peered":"2024-09-11T12:50:27.351277+0000","last_unstale":"2024-09-11T12:50:28.346617+0000","last_undegraded":"2024-09-11T12:50:28.346617+0000","last_fullsized":"2024-09-11T12:50:28.346617+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333991+0000","last_change":"2024-09-11T12:50:26.337847+0000","last_active":"2024-09-11T12:50:28.333991+0000","last_peered":"2024-09-11T12:50:28.333991+0000","last_clean":"2024-09-11T12:50:28.333991+0000","last_became_active":"2024-09-11T12:50:26.337533+0000","last_became_peered":"2024-09-11T12:50:26.337533+0000","last_unstale":"2024-09-11T12:50:28.333991+0000","last_undegraded":"2024-09-11T12:50:28.333991+0000","last_fullsized":"2024-09-11T12:50:28.333991+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:06.294123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334249+0000","last_change":"2024-09-11T12:50:27.330135+0000","last_active":"2024-09-11T12:50:28.334249+0000","last_peered":"2024-09-11T12:50:28.334249+0000","last_clean":"2024-09-11T12:50:28.334249+0000","last_became_active":"2024-09-11T12:50:27.329324+0000","last_became_peered":"2024-09-11T12:50:27.329324+0000","last_unstale":"2024-09-11T12:50:28.334249+0000","last_undegraded":"2024-09-11T12:50:28.334249+0000","last_fullsized":"2024-09-11T12:50:28.334249+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:41.424479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381481+0000","last_change":"2024-09-11T12:50:26.343684+0000","last_active":"2024-09-11T12:50:29.381481+0000","last_peered":"2024-09-11T12:50:29.381481+0000","last_clean":"2024-09-11T12:50:29.381481+0000","last_became_active":"2024-09-11T12:50:26.343386+0000","last_became_peered":"2024-09-11T12:50:26.343386+0000","last_unstale":"2024-09-11T12:50:29.381481+0000","last_undegraded":"2024-09-11T12:50:29.381481+0000","last_fullsized":"2024-09-11T12:50:29.381481+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:31:06.390729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381468+0000","last_change":"2024-09-11T12:50:27.346100+0000","last_active":"2024-09-11T12:50:29.381468+0000","last_peered":"2024-09-11T12:50:29.381468+0000","last_clean":"2024-09-11T12:50:29.381468+0000","last_became_active":"2024-09-11T12:50:27.345380+0000","last_became_peered":"2024-09-11T12:50:27.345380+0000","last_unstale":"2024-09-11T12:50:29.381468+0000","last_undegraded":"2024-09-11T12:50:29.381468+0000","last_fullsized":"2024-09-11T12:50:29.381468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:29.514292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331941+0000","last_change":"2024-09-11T12:50:26.340838+0000","last_active":"2024-09-11T12:50:28.331941+0000","last_peered":"2024-09-11T12:50:28.331941+0000","last_clean":"2024-09-11T12:50:28.331941+0000","last_became_active":"2024-09-11T12:50:26.340531+0000","last_became_peered":"2024-09-11T12:50:26.340531+0000","last_unstale":"2024-09-11T12:50:28.331941+0000","last_undegraded":"2024-09-11T12:50:28.331941+0000","last_fullsized":"2024-09-11T12:50:28.331941+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:04.925393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381199+0000","last_change":"2024-09-11T12:50:27.356884+0000","last_active":"2024-09-11T12:50:29.381199+0000","last_peered":"2024-09-11T12:50:29.381199+0000","last_clean":"2024-09-11T12:50:29.381199+0000","last_became_active":"2024-09-11T12:50:27.355050+0000","last_became_peered":"2024-09-11T12:50:27.355050+0000","last_unstale":"2024-09-11T12:50:29.381199+0000","last_undegraded":"2024-09-11T12:50:29.381199+0000","last_fullsized":"2024-09-11T12:50:29.381199+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:09.638844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":18,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334549+0000","last_change":"2024-09-11T12:50:26.334350+0000","last_active":"2024-09-11T12:50:28.334549+0000","last_peered":"2024-09-11T12:50:28.334549+0000","last_clean":"2024-09-11T12:50:28.334549+0000","last_became_active":"2024-09-11T12:50:26.334014+0000","last_became_peered":"2024-09-11T12:50:26.334014+0000","last_unstale":"2024-09-11T12:50:28.334549+0000","last_undegraded":"2024-09-11T12:50:28.334549+0000","last_fullsized":"2024-09-11T12:50:28.334549+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334388+0000","last_change":"2024-09-11T12:50:27.350052+0000","last_active":"2024-09-11T12:50:28.334388+0000","last_peered":"2024-09-11T12:50:28.334388+0000","last_clean":"2024-09-11T12:50:28.334388+0000","last_became_active":"2024-09-11T12:50:27.349787+0000","last_became_peered":"2024-09-11T12:50:27.349787+0000","last_unstale":"2024-09-11T12:50:28.334388+0000","last_undegraded":"2024-09-11T12:50:28.334388+0000","last_fullsized":"2024-09-11T12:50:28.334388+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:42.771708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":21,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381185+0000","last_change":"2024-09-11T12:50:26.342451+0000","last_active":"2024-09-11T12:50:29.381185+0000","last_peered":"2024-09-11T12:50:29.381185+0000","last_clean":"2024-09-11T12:50:29.381185+0000","last_became_active":"2024-09-11T12:50:26.341973+0000","last_became_peered":"2024-09-11T12:50:26.341973+0000","last_unstale":"2024-09-11T12:50:29.381185+0000","last_undegraded":"2024-09-11T12:50:29.381185+0000","last_fullsized":"2024-09-11T12:50:29.381185+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:54.047639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346663+0000","last_change":"2024-09-11T12:50:27.349605+0000","last_active":"2024-09-11T12:50:28.346663+0000","last_peered":"2024-09-11T12:50:28.346663+0000","last_clean":"2024-09-11T12:50:28.346663+0000","last_became_active":"2024-09-11T12:50:27.349304+0000","last_became_peered":"2024-09-11T12:50:27.349304+0000","last_unstale":"2024-09-11T12:50:28.346663+0000","last_undegraded":"2024-09-11T12:50:28.346663+0000","last_fullsized":"2024-09-11T12:50:28.346663+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:05:12.942602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334910+0000","last_change":"2024-09-11T12:50:26.337760+0000","last_active":"2024-09-11T12:50:28.334910+0000","last_peered":"2024-09-11T12:50:28.334910+0000","last_clean":"2024-09-11T12:50:28.334910+0000","last_became_active":"2024-09-11T12:50:26.337496+0000","last_became_peered":"2024-09-11T12:50:26.337496+0000","last_unstale":"2024-09-11T12:50:28.334910+0000","last_undegraded":"2024-09-11T12:50:28.334910+0000","last_fullsized":"2024-09-11T12:50:28.334910+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:09.519349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381053+0000","last_change":"2024-09-11T12:50:27.342946+0000","last_active":"2024-09-11T12:50:29.381053+0000","last_peered":"2024-09-11T12:50:29.381053+0000","last_clean":"2024-09-11T12:50:29.381053+0000","last_became_active":"2024-09-11T12:50:27.342692+0000","last_became_peered":"2024-09-11T12:50:27.342692+0000","last_unstale":"2024-09-11T12:50:29.381053+0000","last_undegraded":"2024-09-11T12:50:29.381053+0000","last_fullsized":"2024-09-11T12:50:29.381053+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:08.273227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346644+0000","last_change":"2024-09-11T12:50:26.337698+0000","last_active":"2024-09-11T12:50:28.346644+0000","last_peered":"2024-09-11T12:50:28.346644+0000","last_clean":"2024-09-11T12:50:28.346644+0000","last_became_active":"2024-09-11T12:50:26.337145+0000","last_became_peered":"2024-09-11T12:50:26.337145+0000","last_unstale":"2024-09-11T12:50:28.346644+0000","last_undegraded":"2024-09-11T12:50:28.346644+0000","last_fullsized":"2024-09-11T12:50:28.346644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946465+0000","last_change":"2024-09-11T12:50:27.356536+0000","last_active":"2024-09-11T12:50:28.946465+0000","last_peered":"2024-09-11T12:50:28.946465+0000","last_clean":"2024-09-11T12:50:28.946465+0000","last_became_active":"2024-09-11T12:50:27.356235+0000","last_became_peered":"2024-09-11T12:50:27.356235+0000","last_unstale":"2024-09-11T12:50:28.946465+0000","last_undegraded":"2024-09-11T12:50:28.946465+0000","last_fullsized":"2024-09-11T12:50:28.946465+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:42.771708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380674+0000","last_change":"2024-09-11T12:50:26.339598+0000","last_active":"2024-09-11T12:50:29.380674+0000","last_peered":"2024-09-11T12:50:29.380674+0000","last_clean":"2024-09-11T12:50:29.380674+0000","last_became_active":"2024-09-11T12:50:26.339322+0000","last_became_peered":"2024-09-11T12:50:26.339322+0000","last_unstale":"2024-09-11T12:50:29.380674+0000","last_undegraded":"2024-09-11T12:50:29.380674+0000","last_fullsized":"2024-09-11T12:50:29.380674+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:45.625861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381317+0000","last_change":"2024-09-11T12:50:27.341021+0000","last_active":"2024-09-11T12:50:29.381317+0000","last_peered":"2024-09-11T12:50:29.381317+0000","last_clean":"2024-09-11T12:50:29.381317+0000","last_became_active":"2024-09-11T12:50:27.337197+0000","last_became_peered":"2024-09-11T12:50:27.337197+0000","last_unstale":"2024-09-11T12:50:29.381317+0000","last_undegraded":"2024-09-11T12:50:29.381317+0000","last_fullsized":"2024-09-11T12:50:29.381317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:23:55.053803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":23,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380796+0000","last_change":"2024-09-11T12:50:26.341481+0000","last_active":"2024-09-11T12:50:29.380796+0000","last_peered":"2024-09-11T12:50:29.380796+0000","last_clean":"2024-09-11T12:50:29.380796+0000","last_became_active":"2024-09-11T12:50:26.341148+0000","last_became_peered":"2024-09-11T12:50:26.341148+0000","last_unstale":"2024-09-11T12:50:29.380796+0000","last_undegraded":"2024-09-11T12:50:29.380796+0000","last_fullsized":"2024-09-11T12:50:29.380796+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:21:41.388136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.335161+0000","last_change":"2024-09-11T12:50:27.343792+0000","last_active":"2024-09-11T12:50:28.335161+0000","last_peered":"2024-09-11T12:50:28.335161+0000","last_clean":"2024-09-11T12:50:28.335161+0000","last_became_active":"2024-09-11T12:50:27.343522+0000","last_became_peered":"2024-09-11T12:50:27.343522+0000","last_unstale":"2024-09-11T12:50:28.335161+0000","last_undegraded":"2024-09-11T12:50:28.335161+0000","last_fullsized":"2024-09-11T12:50:28.335161+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:13.070784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347246+0000","last_change":"2024-09-11T12:50:26.342891+0000","last_active":"2024-09-11T12:50:28.347246+0000","last_peered":"2024-09-11T12:50:28.347246+0000","last_clean":"2024-09-11T12:50:28.347246+0000","last_became_active":"2024-09-11T12:50:26.342639+0000","last_became_peered":"2024-09-11T12:50:26.342639+0000","last_unstale":"2024-09-11T12:50:28.347246+0000","last_undegraded":"2024-09-11T12:50:28.347246+0000","last_fullsized":"2024-09-11T12:50:28.347246+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334243+0000","last_change":"2024-09-11T12:50:27.335239+0000","last_active":"2024-09-11T12:50:28.334243+0000","last_peered":"2024-09-11T12:50:28.334243+0000","last_clean":"2024-09-11T12:50:28.334243+0000","last_became_active":"2024-09-11T12:50:27.333463+0000","last_became_peered":"2024-09-11T12:50:27.333463+0000","last_unstale":"2024-09-11T12:50:28.334243+0000","last_undegraded":"2024-09-11T12:50:28.334243+0000","last_fullsized":"2024-09-11T12:50:28.334243+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:29:49.375025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.335272+0000","last_change":"2024-09-11T12:50:26.337827+0000","last_active":"2024-09-11T12:50:28.335272+0000","last_peered":"2024-09-11T12:50:28.335272+0000","last_clean":"2024-09-11T12:50:28.335272+0000","last_became_active":"2024-09-11T12:50:26.337466+0000","last_became_peered":"2024-09-11T12:50:26.337466+0000","last_unstale":"2024-09-11T12:50:28.335272+0000","last_undegraded":"2024-09-11T12:50:28.335272+0000","last_fullsized":"2024-09-11T12:50:28.335272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:12.538604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346918+0000","last_change":"2024-09-11T12:50:27.338996+0000","last_active":"2024-09-11T12:50:28.346918+0000","last_peered":"2024-09-11T12:50:28.346918+0000","last_clean":"2024-09-11T12:50:28.346918+0000","last_became_active":"2024-09-11T12:50:27.338705+0000","last_became_peered":"2024-09-11T12:50:27.338705+0000","last_unstale":"2024-09-11T12:50:28.346918+0000","last_undegraded":"2024-09-11T12:50:28.346918+0000","last_fullsized":"2024-09-11T12:50:28.346918+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:57.191146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.335196+0000","last_change":"2024-09-11T12:50:26.339176+0000","last_active":"2024-09-11T12:50:28.335196+0000","last_peered":"2024-09-11T12:50:28.335196+0000","last_clean":"2024-09-11T12:50:28.335196+0000","last_became_active":"2024-09-11T12:50:26.338882+0000","last_became_peered":"2024-09-11T12:50:26.338882+0000","last_unstale":"2024-09-11T12:50:28.335196+0000","last_undegraded":"2024-09-11T12:50:28.335196+0000","last_fullsized":"2024-09-11T12:50:28.335196+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:49:11.091780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771306+0000","last_change":"2024-09-11T12:50:27.353963+0000","last_active":"2024-09-11T12:50:28.771306+0000","last_peered":"2024-09-11T12:50:28.771306+0000","last_clean":"2024-09-11T12:50:28.771306+0000","last_became_active":"2024-09-11T12:50:27.353684+0000","last_became_peered":"2024-09-11T12:50:27.353684+0000","last_unstale":"2024-09-11T12:50:28.771306+0000","last_undegraded":"2024-09-11T12:50:28.771306+0000","last_fullsized":"2024-09-11T12:50:28.771306+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:19.704299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334916+0000","last_change":"2024-09-11T12:50:26.344464+0000","last_active":"2024-09-11T12:50:28.334916+0000","last_peered":"2024-09-11T12:50:28.334916+0000","last_clean":"2024-09-11T12:50:28.334916+0000","last_became_active":"2024-09-11T12:50:26.344093+0000","last_became_peered":"2024-09-11T12:50:26.344093+0000","last_unstale":"2024-09-11T12:50:28.334916+0000","last_undegraded":"2024-09-11T12:50:28.334916+0000","last_fullsized":"2024-09-11T12:50:28.334916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:47.700330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771391+0000","last_change":"2024-09-11T12:50:27.338133+0000","last_active":"2024-09-11T12:50:28.771391+0000","last_peered":"2024-09-11T12:50:28.771391+0000","last_clean":"2024-09-11T12:50:28.771391+0000","last_became_active":"2024-09-11T12:50:27.337816+0000","last_became_peered":"2024-09-11T12:50:27.337816+0000","last_unstale":"2024-09-11T12:50:28.771391+0000","last_undegraded":"2024-09-11T12:50:28.771391+0000","last_fullsized":"2024-09-11T12:50:28.771391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:55:10.525512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331708+0000","last_change":"2024-09-11T12:50:26.339488+0000","last_active":"2024-09-11T12:50:28.331708+0000","last_peered":"2024-09-11T12:50:28.331708+0000","last_clean":"2024-09-11T12:50:28.331708+0000","last_became_active":"2024-09-11T12:50:26.339267+0000","last_became_peered":"2024-09-11T12:50:26.339267+0000","last_unstale":"2024-09-11T12:50:28.331708+0000","last_undegraded":"2024-09-11T12:50:28.331708+0000","last_fullsized":"2024-09-11T12:50:28.331708+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:15.102447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771596+0000","last_change":"2024-09-11T12:50:27.345520+0000","last_active":"2024-09-11T12:50:28.771596+0000","last_peered":"2024-09-11T12:50:28.771596+0000","last_clean":"2024-09-11T12:50:28.771596+0000","last_became_active":"2024-09-11T12:50:27.345261+0000","last_became_peered":"2024-09-11T12:50:27.345261+0000","last_unstale":"2024-09-11T12:50:28.771596+0000","last_undegraded":"2024-09-11T12:50:28.771596+0000","last_fullsized":"2024-09-11T12:50:28.771596+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:36.902883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381210+0000","last_change":"2024-09-11T12:50:26.346500+0000","last_active":"2024-09-11T12:50:29.381210+0000","last_peered":"2024-09-11T12:50:29.381210+0000","last_clean":"2024-09-11T12:50:29.381210+0000","last_became_active":"2024-09-11T12:50:26.346237+0000","last_became_peered":"2024-09-11T12:50:26.346237+0000","last_unstale":"2024-09-11T12:50:29.381210+0000","last_undegraded":"2024-09-11T12:50:29.381210+0000","last_fullsized":"2024-09-11T12:50:29.381210+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:11.205541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771329+0000","last_change":"2024-09-11T12:50:27.342983+0000","last_active":"2024-09-11T12:50:28.771329+0000","last_peered":"2024-09-11T12:50:28.771329+0000","last_clean":"2024-09-11T12:50:28.771329+0000","last_became_active":"2024-09-11T12:50:27.342726+0000","last_became_peered":"2024-09-11T12:50:27.342726+0000","last_unstale":"2024-09-11T12:50:28.771329+0000","last_undegraded":"2024-09-11T12:50:28.771329+0000","last_fullsized":"2024-09-11T12:50:28.771329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:53.156365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334376+0000","last_change":"2024-09-11T12:50:26.339279+0000","last_active":"2024-09-11T12:50:28.334376+0000","last_peered":"2024-09-11T12:50:28.334376+0000","last_clean":"2024-09-11T12:50:28.334376+0000","last_became_active":"2024-09-11T12:50:26.338240+0000","last_became_peered":"2024-09-11T12:50:26.338240+0000","last_unstale":"2024-09-11T12:50:28.334376+0000","last_undegraded":"2024-09-11T12:50:28.334376+0000","last_fullsized":"2024-09-11T12:50:28.334376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:37:52.150201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946195+0000","last_change":"2024-09-11T12:50:27.349271+0000","last_active":"2024-09-11T12:50:28.946195+0000","last_peered":"2024-09-11T12:50:28.946195+0000","last_clean":"2024-09-11T12:50:28.946195+0000","last_became_active":"2024-09-11T12:50:27.348581+0000","last_became_peered":"2024-09-11T12:50:27.348581+0000","last_unstale":"2024-09-11T12:50:28.946195+0000","last_undegraded":"2024-09-11T12:50:28.946195+0000","last_fullsized":"2024-09-11T12:50:28.946195+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:03.584698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.333524+0000","last_change":"2024-09-11T12:50:26.336396+0000","last_active":"2024-09-11T12:50:28.333524+0000","last_peered":"2024-09-11T12:50:28.333524+0000","last_clean":"2024-09-11T12:50:28.333524+0000","last_became_active":"2024-09-11T12:50:26.335998+0000","last_became_peered":"2024-09-11T12:50:26.335998+0000","last_unstale":"2024-09-11T12:50:28.333524+0000","last_undegraded":"2024-09-11T12:50:28.333524+0000","last_fullsized":"2024-09-11T12:50:28.333524+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:50:44.426035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.771429+0000","last_change":"2024-09-11T12:50:27.349951+0000","last_active":"2024-09-11T12:50:28.771429+0000","last_peered":"2024-09-11T12:50:28.771429+0000","last_clean":"2024-09-11T12:50:28.771429+0000","last_became_active":"2024-09-11T12:50:27.349174+0000","last_became_peered":"2024-09-11T12:50:27.349174+0000","last_unstale":"2024-09-11T12:50:28.771429+0000","last_undegraded":"2024-09-11T12:50:28.771429+0000","last_fullsized":"2024-09-11T12:50:28.771429+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:38:47.548681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347253+0000","last_change":"2024-09-11T12:50:26.337423+0000","last_active":"2024-09-11T12:50:28.347253+0000","last_peered":"2024-09-11T12:50:28.347253+0000","last_clean":"2024-09-11T12:50:28.347253+0000","last_became_active":"2024-09-11T12:50:26.337076+0000","last_became_peered":"2024-09-11T12:50:26.337076+0000","last_unstale":"2024-09-11T12:50:28.347253+0000","last_undegraded":"2024-09-11T12:50:28.347253+0000","last_fullsized":"2024-09-11T12:50:28.347253+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:10.273626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334281+0000","last_change":"2024-09-11T12:50:27.349463+0000","last_active":"2024-09-11T12:50:28.334281+0000","last_peered":"2024-09-11T12:50:28.334281+0000","last_clean":"2024-09-11T12:50:28.334281+0000","last_became_active":"2024-09-11T12:50:27.349183+0000","last_became_peered":"2024-09-11T12:50:27.349183+0000","last_unstale":"2024-09-11T12:50:28.334281+0000","last_undegraded":"2024-09-11T12:50:28.334281+0000","last_fullsized":"2024-09-11T12:50:28.334281+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:25.420948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":16,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331987+0000","last_change":"2024-09-11T12:50:26.340196+0000","last_active":"2024-09-11T12:50:28.331987+0000","last_peered":"2024-09-11T12:50:28.331987+0000","last_clean":"2024-09-11T12:50:28.331987+0000","last_became_active":"2024-09-11T12:50:26.339927+0000","last_became_peered":"2024-09-11T12:50:26.339927+0000","last_unstale":"2024-09-11T12:50:28.331987+0000","last_undegraded":"2024-09-11T12:50:28.331987+0000","last_fullsized":"2024-09-11T12:50:28.331987+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:43.000801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331625+0000","last_change":"2024-09-11T12:50:27.343198+0000","last_active":"2024-09-11T12:50:28.331625+0000","last_peered":"2024-09-11T12:50:28.331625+0000","last_clean":"2024-09-11T12:50:28.331625+0000","last_became_active":"2024-09-11T12:50:27.342935+0000","last_became_peered":"2024-09-11T12:50:27.342935+0000","last_unstale":"2024-09-11T12:50:28.331625+0000","last_undegraded":"2024-09-11T12:50:28.331625+0000","last_fullsized":"2024-09-11T12:50:28.331625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:51.052663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.347040+0000","last_change":"2024-09-11T12:50:26.338822+0000","last_active":"2024-09-11T12:50:28.347040+0000","last_peered":"2024-09-11T12:50:28.347040+0000","last_clean":"2024-09-11T12:50:28.347040+0000","last_became_active":"2024-09-11T12:50:26.338459+0000","last_became_peered":"2024-09-11T12:50:26.338459+0000","last_unstale":"2024-09-11T12:50:28.347040+0000","last_undegraded":"2024-09-11T12:50:28.347040+0000","last_fullsized":"2024-09-11T12:50:28.347040+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:01.412980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381253+0000","last_change":"2024-09-11T12:50:27.344966+0000","last_active":"2024-09-11T12:50:29.381253+0000","last_peered":"2024-09-11T12:50:29.381253+0000","last_clean":"2024-09-11T12:50:29.381253+0000","last_became_active":"2024-09-11T12:50:27.344634+0000","last_became_peered":"2024-09-11T12:50:27.344634+0000","last_unstale":"2024-09-11T12:50:29.381253+0000","last_undegraded":"2024-09-11T12:50:29.381253+0000","last_fullsized":"2024-09-11T12:50:29.381253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:45:47.363910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331827+0000","last_change":"2024-09-11T12:50:26.330959+0000","last_active":"2024-09-11T12:50:28.331827+0000","last_peered":"2024-09-11T12:50:28.331827+0000","last_clean":"2024-09-11T12:50:28.331827+0000","last_became_active":"2024-09-11T12:50:26.330722+0000","last_became_peered":"2024-09-11T12:50:26.330722+0000","last_unstale":"2024-09-11T12:50:28.331827+0000","last_undegraded":"2024-09-11T12:50:28.331827+0000","last_fullsized":"2024-09-11T12:50:28.331827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:10.662930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.331718+0000","last_change":"2024-09-11T12:50:27.348982+0000","last_active":"2024-09-11T12:50:28.331718+0000","last_peered":"2024-09-11T12:50:28.331718+0000","last_clean":"2024-09-11T12:50:28.331718+0000","last_became_active":"2024-09-11T12:50:27.348742+0000","last_became_peered":"2024-09-11T12:50:27.348742+0000","last_unstale":"2024-09-11T12:50:28.331718+0000","last_undegraded":"2024-09-11T12:50:28.331718+0000","last_fullsized":"2024-09-11T12:50:28.331718+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:22:45.405245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.346756+0000","last_change":"2024-09-11T12:50:26.337961+0000","last_active":"2024-09-11T12:50:28.346756+0000","last_peered":"2024-09-11T12:50:28.346756+0000","last_clean":"2024-09-11T12:50:28.346756+0000","last_became_active":"2024-09-11T12:50:26.337461+0000","last_became_peered":"2024-09-11T12:50:26.337461+0000","last_unstale":"2024-09-11T12:50:28.346756+0000","last_undegraded":"2024-09-11T12:50:28.346756+0000","last_fullsized":"2024-09-11T12:50:28.346756+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:37.060898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380669+0000","last_change":"2024-09-11T12:50:27.342392+0000","last_active":"2024-09-11T12:50:29.380669+0000","last_peered":"2024-09-11T12:50:29.380669+0000","last_clean":"2024-09-11T12:50:29.380669+0000","last_became_active":"2024-09-11T12:50:27.342116+0000","last_became_peered":"2024-09-11T12:50:27.342116+0000","last_unstale":"2024-09-11T12:50:29.380669+0000","last_undegraded":"2024-09-11T12:50:29.380669+0000","last_fullsized":"2024-09-11T12:50:29.380669+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:41.424479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334413+0000","last_change":"2024-09-11T12:50:26.330978+0000","last_active":"2024-09-11T12:50:28.334413+0000","last_peered":"2024-09-11T12:50:28.334413+0000","last_clean":"2024-09-11T12:50:28.334413+0000","last_became_active":"2024-09-11T12:50:26.330681+0000","last_became_peered":"2024-09-11T12:50:26.330681+0000","last_unstale":"2024-09-11T12:50:28.334413+0000","last_undegraded":"2024-09-11T12:50:28.334413+0000","last_fullsized":"2024-09-11T12:50:28.334413+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:32:43.803456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.334188+0000","last_change":"2024-09-11T12:50:27.328606+0000","last_active":"2024-09-11T12:50:28.334188+0000","last_peered":"2024-09-11T12:50:28.334188+0000","last_clean":"2024-09-11T12:50:28.334188+0000","last_became_active":"2024-09-11T12:50:27.328276+0000","last_became_peered":"2024-09-11T12:50:27.328276+0000","last_unstale":"2024-09-11T12:50:28.334188+0000","last_undegraded":"2024-09-11T12:50:28.334188+0000","last_fullsized":"2024-09-11T12:50:28.334188+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:53.928772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.380699+0000","last_change":"2024-09-11T12:50:26.337651+0000","last_active":"2024-09-11T12:50:29.380699+0000","last_peered":"2024-09-11T12:50:29.380699+0000","last_clean":"2024-09-11T12:50:29.380699+0000","last_became_active":"2024-09-11T12:50:26.337387+0000","last_became_peered":"2024-09-11T12:50:26.337387+0000","last_unstale":"2024-09-11T12:50:29.380699+0000","last_undegraded":"2024-09-11T12:50:29.380699+0000","last_fullsized":"2024-09-11T12:50:29.380699+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:56:52.922609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.945968+0000","last_change":"2024-09-11T12:50:27.351937+0000","last_active":"2024-09-11T12:50:28.945968+0000","last_peered":"2024-09-11T12:50:28.945968+0000","last_clean":"2024-09-11T12:50:28.945968+0000","last_became_active":"2024-09-11T12:50:27.350570+0000","last_became_peered":"2024-09-11T12:50:27.350570+0000","last_unstale":"2024-09-11T12:50:28.945968+0000","last_undegraded":"2024-09-11T12:50:28.945968+0000","last_fullsized":"2024-09-11T12:50:28.945968+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:26.307616+0000","last_clean_scrub_stamp":"2024-09-11T12:50:26.307616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:33:26.392731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:29.381220+0000","last_change":"2024-09-11T12:50:26.344815+0000","last_active":"2024-09-11T12:50:29.381220+0000","last_peered":"2024-09-11T12:50:29.381220+0000","last_clean":"2024-09-11T12:50:29.381220+0000","last_became_active":"2024-09-11T12:50:26.344518+0000","last_became_peered":"2024-09-11T12:50:26.344518+0000","last_unstale":"2024-09-11T12:50:29.381220+0000","last_undegraded":"2024-09-11T12:50:29.381220+0000","last_fullsized":"2024-09-11T12:50:29.381220+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:25.301453+0000","last_clean_scrub_stamp":"2024-09-11T12:50:25.301453+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:24.053403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":131,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-11T12:50:28.946543+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:50:28.946543+0000","last_peered":"2024-09-11T12:50:28.946543+0000","last_clean":"2024-09-11T12:50:28.946543+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:50:28.946543+0000","last_undegraded":"2024-09-11T12:50:28.946543+0000","last_fullsized":"2024-09-11T12:50:28.946543+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32140,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339700,"statfs":{"total":96636764160,"available":96603852800,"internally_reserved":0,"allocated":974848,"data_stored":734605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31568,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340272,"statfs":{"total":96636764160,"available":96604438528,"internally_reserved":0,"allocated":385024,"data_stored":144763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509455,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":385024,"data_stored":144819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31564,"kb_used_data":376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340276,"statfs":{"total":96636764160,"available":96604442624,"internally_reserved":0,"allocated":385024,"data_stored":144251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509455,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27400,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344440,"statfs":{"total":96636764160,"available":96608706560,"internally_reserved":0,"allocated":380928,"data_stored":144217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509455,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":384,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":393216,"data_stored":145781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27976,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343864,"statfs":{"total":96636764160,"available":96608116736,"internally_reserved":0,"allocated":970752,"data_stored":734515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:50:36.352 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:50:36.353 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 -- 172.21.15.78:0/4048118318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e84003c90 msgr2=0x7f3e84004090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/4048118318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e84003c90 0x7f3e84004090 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3e90001f50 tx=0x7f3e9001cbe0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 -- 172.21.15.78:0/4048118318 shutdown_connections 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/4048118318 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e84003c90 0x7f3e84004090 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 -- 172.21.15.78:0/4048118318 >> 172.21.15.78:0/4048118318 conn(0x7f3ea00b2ec0 msgr2=0x7f3ea00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 -- 172.21.15.78:0/4048118318 shutdown_connections 2024-09-11T12:50:36.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.538+0000 7f3ea6e00640 1 -- 172.21.15.78:0/4048118318 wait complete. 2024-09-11T12:50:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 Processor -- start 2024-09-11T12:50:36.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- start start 2024-09-11T12:50:36.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 0x7f3ea01f6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea013ca60 0x7f3ea013a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea0073a40 con 0x7f3ea01385b0 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea00738c0 con 0x7f3ea013ca60 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3ea0073bc0 con 0x7f3e84003c90 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32982/0 (socket says 172.21.15.78:32982) 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 -- 172.21.15.78:0/3531209229 learned_addr learned my addr 172.21.15.78:0/3531209229 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:36.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea4e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea013ca60 0x7f3ea013a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9fe00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 0x7f3ea01f6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 -- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 msgr2=0x7f3ea01f6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 0x7f3ea01f6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 -- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea013ca60 msgr2=0x7f3ea013a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea013ca60 0x7f3ea013a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 -- 172.21.15.78:0/3531209229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea01f6650 con 0x7f3ea01385b0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9fe00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 0x7f3ea01f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e9f400640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000fc0 tx=0x7f3e8c00ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c015020 con 0x7f3ea01385b0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e8c010980 con 0x7f3ea01385b0 2024-09-11T12:50:36.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c010dd0 con 0x7f3ea01385b0 2024-09-11T12:50:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea013e480 con 0x7f3ea01385b0 2024-09-11T12:50:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.542+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea013ea40 con 0x7f3ea01385b0 2024-09-11T12:50:36.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ea006fe80 con 0x7f3ea01385b0 2024-09-11T12:50:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f3e8c018020 con 0x7f3ea01385b0 2024-09-11T12:50:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3e97e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 0x7f3e80073b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3e8c092270 con 0x7f3ea01385b0 2024-09-11T12:50:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3e9fe00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 0x7f3e80073b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.546+0000 7f3e9fe00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 0x7f3e80073b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a9000 tx=0x7f3e90019a20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.550+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3e8c01f020 con 0x7f3ea01385b0 2024-09-11T12:50:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.774+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f3ea0070440 con 0x7f3ea01385b0 2024-09-11T12:50:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.778+0000 7f3e97e00640 1 -- 172.21.15.78:0/3531209229 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) ==== 190+0+147 (secure 0 0 0) 0x7f3ea01f6650 con 0x7f3ea01385b0 2024-09-11T12:50:36.783 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:50:36.784 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:50:36.784 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:50:36.784 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:50:36.784 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:50:36.784 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 msgr2=0x7f3e80073b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 0x7f3e80073b70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e900a9000 tx=0x7f3e90019a20 comp rx=0 tx=0).stop 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 msgr2=0x7f3ea013c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c000fc0 tx=0x7f3e8c00ea20 comp rx=0 tx=0).stop 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 shutdown_connections 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3e800716b0 0x7f3e80073b70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3ea013ca60 0x7f3ea013a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ea01385b0 0x7f3ea013c520 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 --2- 172.21.15.78:0/3531209229 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e84003c90 0x7f3ea01f6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 >> 172.21.15.78:0/3531209229 conn(0x7f3ea00b2ec0 msgr2=0x7f3ea01e5b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:36.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 shutdown_connections 2024-09-11T12:50:36.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.782+0000 7f3ea6e00640 1 -- 172.21.15.78:0/3531209229 wait complete. 2024-09-11T12:50:36.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:36.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- 172.21.15.78:0/4026290729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0b54c0 msgr2=0x7fdc2c0736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 --2- 172.21.15.78:0/4026290729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0b54c0 0x7fdc2c0736d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fdc2400acd0 tx=0x7fdc2401de60 comp rx=0 tx=0).stop 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- 172.21.15.78:0/4026290729 shutdown_connections 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 --2- 172.21.15.78:0/4026290729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c0b54c0 0x7fdc2c0736d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- 172.21.15.78:0/4026290729 >> 172.21.15.78:0/4026290729 conn(0x7fdc2c06ca80 msgr2=0x7fdc2c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- 172.21.15.78:0/4026290729 shutdown_connections 2024-09-11T12:50:36.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- 172.21.15.78:0/4026290729 wait complete. 2024-09-11T12:50:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 Processor -- start 2024-09-11T12:50:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.942+0000 7fdc34600640 1 -- start start 2024-09-11T12:50:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2c070a00 0x7fdc2c151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 0x7fdc2c1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c167f20 con 0x7fdc2c157ae0 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c06f760 con 0x7fdc2c070a00 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc2c1680a0 con 0x7fdc2c0b54c0 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:32996/0 (socket says 172.21.15.78:32996) 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 -- 172.21.15.78:0/435596399 learned_addr learned my addr 172.21.15.78:0/435596399 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc31c00640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2c070a00 0x7fdc2c151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc31200640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 0x7fdc2c1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 -- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 msgr2=0x7fdc2c1491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 0x7fdc2c1491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 -- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2c070a00 msgr2=0x7fdc2c151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:36.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2c070a00 0x7fdc2c151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 -- 172.21.15.78:0/435596399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c151d20 con 0x7fdc2c157ae0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc31200640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 0x7fdc2c1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc32600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdc2401aa70 tx=0x7fdc24004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc240b3040 con 0x7fdc2c157ae0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc24004b40 con 0x7fdc2c157ae0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24004460 con 0x7fdc2c157ae0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c151a80 con 0x7fdc2c157ae0 2024-09-11T12:50:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1f7df0 con 0x7fdc2c157ae0 2024-09-11T12:50:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c0736d0 con 0x7fdc2c157ae0 2024-09-11T12:50:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.946+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7fdc24004600 con 0x7fdc2c157ae0 2024-09-11T12:50:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.950+0000 7fdc29600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 0x7fdc0c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.950+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fdc2412c890 con 0x7fdc2c157ae0 2024-09-11T12:50:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.950+0000 7fdc31c00640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 0x7fdc0c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.950+0000 7fdc31c00640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 0x7fdc0c073b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc20001870 tx=0x7fdc20001380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:36.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:36.954+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdc24017ae0 con 0x7fdc2c157ae0 2024-09-11T12:50:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.154+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc2c15a270 con 0x7fdc2c157ae0 2024-09-11T12:50:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.154+0000 7fdc29600640 1 -- 172.21.15.78:0/435596399 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) ==== 94+0+5223 (secure 0 0 0) 0x7fdc240f8790 con 0x7fdc2c157ae0 2024-09-11T12:50:37.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:37.161 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":16,"btime":"2024-09-11T12:50:30:342061+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 16 2024-09-11T12:50:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 msgr2=0x7fdc0c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 0x7fdc0c073b40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdc20001870 tx=0x7fdc20001380 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 msgr2=0x7fdc2c14ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdc2401aa70 tx=0x7fdc24004270 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 shutdown_connections 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdc0c071680 0x7fdc0c073b40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2c157ae0 0x7fdc2c14ca70 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2c0b54c0 0x7fdc2c1491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 --2- 172.21.15.78:0/435596399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2c070a00 0x7fdc2c151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 >> 172.21.15.78:0/435596399 conn(0x7fdc2c06ca80 msgr2=0x7fdc2c0aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 shutdown_connections 2024-09-11T12:50:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.158+0000 7fdc34600640 1 -- 172.21.15.78:0/435596399 wait complete. 2024-09-11T12:50:37.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:50:26.715189+0000', 'modified': '2024-09-11T12:50:29.729751+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4885': {'gid': 4885, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6836/2884175884', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6835', 'nonce': 2884175884}, {'type': 'v1', 'addr': '172.21.15.157:6836', 'nonce': 2884175884}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4885, 'qdb_cluster': [4885]} 2024-09-11T12:50:37.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:50:37.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 -- 172.21.15.78:0/628874111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 msgr2=0x7f7764003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 --2- 172.21.15.78:0/628874111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f7764003e80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7770009b40 tx=0x7f777001d250 comp rx=0 tx=0).stop 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7781800640 1 -- 172.21.15.78:0/628874111 reap_dead start 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 -- 172.21.15.78:0/628874111 shutdown_connections 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 -- 172.21.15.78:0/628874111 >> 172.21.15.78:0/628874111 conn(0x7f777c074b90 msgr2=0x7f777c0d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 -- 172.21.15.78:0/628874111 shutdown_connections 2024-09-11T12:50:37.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.722+0000 7f7783800640 1 -- 172.21.15.78:0/628874111 wait complete. 2024-09-11T12:50:37.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 Processor -- start 2024-09-11T12:50:37.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- start start 2024-09-11T12:50:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 0x7f777c1397c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0b1be0 0x7f777c0b2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f777c166ab0 con 0x7f7764003a60 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f777c166930 con 0x7f777c151800 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f777c166c30 con 0x7f777c0b1be0 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33020/0 (socket says 172.21.15.78:33020) 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 -- 172.21.15.78:0/3664072824 learned_addr learned my addr 172.21.15.78:0/3664072824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 -- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0b1be0 msgr2=0x7f777c0b2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0b1be0 0x7f777c0b2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 -- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 msgr2=0x7f777c1397c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f777be00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 0x7f777c1397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 0x7f777c1397c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 -- 172.21.15.78:0/3664072824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c153b60 con 0x7f7764003a60 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f777be00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 0x7f777c1397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7780e00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77700ae040 tx=0x7f77700097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7778c00640 1 -- 172.21.15.78:0/3664072824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700bb030 con 0x7f7764003a60 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7778c00640 1 -- 172.21.15.78:0/3664072824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77700047c0 con 0x7f7764003a60 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c153de0 con 0x7f7764003a60 2024-09-11T12:50:37.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7778c00640 1 -- 172.21.15.78:0/3664072824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7770004bf0 con 0x7f7764003a60 2024-09-11T12:50:37.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.726+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c1543a0 con 0x7f7764003a60 2024-09-11T12:50:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7778c00640 1 -- 172.21.15.78:0/3664072824 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f77700be030 con 0x7f7764003a60 2024-09-11T12:50:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7778c00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 0x7f775c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7778c00640 1 -- 172.21.15.78:0/3664072824 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f777012d530 con 0x7f7764003a60 2024-09-11T12:50:37.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 Processor -- start 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f777be00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 0x7f775c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 -- start start 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f777be00640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 0x7f775c073b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7768000fc0 tx=0x7f7768006730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0e8ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0e8d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556b0cea270 con 0x5556b0ed9aa0 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5556b0cd9770 con 0x5556b0eda3c0 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.730+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5556b0cdd7b0 con 0x5556b0e8d340 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33022/0 (socket says 172.21.15.78:33022) 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 -- 172.21.15.78:0/3038093591 learned_addr learned my addr 172.21.15.78:0/3038093591 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f777be00640 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0e8ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7781800640 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0e8d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 -- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 msgr2=0x5556b0e8d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0e8d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 -- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 msgr2=0x5556b0e8ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0e8ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 -- 172.21.15.78:0/3038093591 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b0c78f90 con 0x5556b0ed9aa0 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7780e00640 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7770005890 tx=0x7f77700f8710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e021378fc925579 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f776d600640 1 -- 172.21.15.78:0/3038093591 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700c5040 con 0x5556b0ed9aa0 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f776d600640 1 -- 172.21.15.78:0/3038093591 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77700f8c00 con 0x5556b0ed9aa0 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f776d600640 1 -- 172.21.15.78:0/3038093591 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77700fd200 con 0x5556b0ed9aa0 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- 172.21.15.78:0/3038093591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 msgr2=0x5556b0ed9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7770005890 tx=0x7f77700f8710 comp rx=0 tx=0).stop 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- 172.21.15.78:0/3038093591 shutdown_connections 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0e8d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0e8ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- 172.21.15.78:0/3038093591 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ed9e80 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- 172.21.15.78:0/3038093591 >> 172.21.15.78:0/3038093591 conn(0x5556b0ebab60 msgr2=0x5556b0ebcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- 172.21.15.78:0/3038093591 shutdown_connections 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- 172.21.15.78:0/3038093591 wait complete. 2024-09-11T12:50:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- start start 2024-09-11T12:50:37.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 Processor -- start 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0e8d340 0x5556b0ee5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0eda3c0 0x5556b0ee62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556b0da9290 con 0x5556b0ed9aa0 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5556b0c85ab0 con 0x5556b0e8d340 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5556b0c83170 con 0x5556b0eda3c0 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f7786c8b000 10 client.0 _ll_register_callbacks cb 0x5556b0e7a830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33026/0 (socket says 172.21.15.78:33026) 2024-09-11T12:50:37.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.734+0000 7f777be00640 1 -- 172.21.15.78:0/2951849337 learned_addr learned my addr 172.21.15.78:0/2951849337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7780e00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0e8d340 0x5556b0ee5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7781800640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0eda3c0 0x5556b0ee62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 -- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0eda3c0 msgr2=0x5556b0ee62d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0eda3c0 0x5556b0ee62d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 -- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0e8d340 msgr2=0x5556b0ee5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0e8d340 0x5556b0ee5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 -- 172.21.15.78:0/2951849337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b0d5e6f0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f777be00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7768003b80 tx=0x7f777c15d1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd58fbc1e496138 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768016070 con 0x5556b0ed9aa0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556b0c72f10 con 0x5556b0ed9aa0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f776801c9d0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f776801cbd0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 10 client.5040 fetch_fsmap learned FSMap version 16 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5556b0d35c40 con 0x5556b0ed9aa0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5556b0c72f10 con 0x5556b0ed9aa0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f7768032070 con 0x5556b0ed9aa0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 10 client.5040 fetch_fsmap finished waiting for FSMap version 16 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 10 client.5040 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 20 client.5040 populate_metadata read hostname 'smithi078' 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0eedc20 0x5556b0ef0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 4 client.5040 mds_command: new command op to 4885 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 --> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556b0cec1c0 con 0x5556b0eedc20 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7780e00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0eedc20 0x5556b0ef0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7780e00640 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0eedc20 0x5556b0ef0010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b29737e701a8c3 server_cookie=120d9511ab355fba in_seq=0 out_seq=0 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 10 client.5040 ms_handle_connect on v2:172.21.15.157:6835/2884175884 2024-09-11T12:50:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 1 -- 172.21.15.78:0/2951849337 <== mds.0 v2:172.21.15.157:6835/2884175884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556b0cec1c0 con 0x5556b0eedc20 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7762000640 10 client.5040 handle_command_reply: tid=0 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 client.5040 shutdown 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 msgr2=0x5556b0ee5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.738+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7768003b80 tx=0x7f777c15d1d0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 shutdown_connections 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0eedc20 0x5556b0ef0010 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0eda3c0 0x5556b0ee62d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ee5d90 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2951849337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0e8d340 0x5556b0ee5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7762000640 0 client.5040 ms_handle_reset on v2:172.21.15.157:6835/2884175884 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 >> 172.21.15.78:0/2951849337 conn(0x5556b0ebab60 msgr2=0x5556b0e8c500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 shutdown_connections 2024-09-11T12:50:37.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 1 -- 172.21.15.78:0/2951849337 wait complete. 2024-09-11T12:50:37.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.742+0000 7f7786c8b000 20 client.5040 trim_cache size 0 max 16384 2024-09-11T12:50:37.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 Processor -- start 2024-09-11T12:50:37.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 -- start start 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0eb93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f777c06da50 con 0x5556b0ed9aa0 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f777c06fb60 con 0x5556b0eda3c0 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f777c0701b0 con 0x5556b0e8d340 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33036/0 (socket says 172.21.15.78:33036) 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 -- 172.21.15.78:0/13223163 learned_addr learned my addr 172.21.15.78:0/13223163 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7780e00640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7781800640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0eb93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 -- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 msgr2=0x5556b0ef3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 -- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 msgr2=0x5556b0eb93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0eb93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f777be00640 1 -- 172.21.15.78:0/13223163 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556b0c8ee70 con 0x5556b0ed9aa0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.758+0000 7f7781800640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0eb93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7768000c10 tx=0x7f7768012950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fd80613ffabfe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f776cc00640 1 -- 172.21.15.78:0/13223163 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768016040 con 0x5556b0ed9aa0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f776cc00640 1 -- 172.21.15.78:0/13223163 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7768023030 con 0x5556b0ed9aa0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f776cc00640 1 -- 172.21.15.78:0/13223163 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776801c3a0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- 172.21.15.78:0/13223163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 msgr2=0x5556b0eb8e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7768000c10 tx=0x7f7768012950 comp rx=0 tx=0).stop 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- 172.21.15.78:0/13223163 shutdown_connections 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0eb93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0eb8e60 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- 172.21.15.78:0/13223163 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- 172.21.15.78:0/13223163 >> 172.21.15.78:0/13223163 conn(0x5556b0ebab60 msgr2=0x5556b0e91950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- 172.21.15.78:0/13223163 shutdown_connections 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- 172.21.15.78:0/13223163 wait complete. 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- start start 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 Processor -- start 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0f0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x5556b0ce1de0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x5556b0cdcac0 con 0x5556b0eda3c0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x5556b0cecea0 con 0x5556b0e8d340 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7786c8b000 10 client.0 _ll_register_callbacks cb 0x7f7768003600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7780e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33050/0 (socket says 172.21.15.78:33050) 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 -- 172.21.15.78:0/2587878967 learned_addr learned my addr 172.21.15.78:0/2587878967 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f7781800640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0f0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 -- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 msgr2=0x5556b0ef27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 -- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 msgr2=0x5556b0f0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0f0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.762+0000 7f777be00640 1 -- 172.21.15.78:0/2587878967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556b0c63bf0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f777be00640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7768003b80 tx=0x7f77680017f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3ac60bda5834d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7768042060 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f776801c3a0 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7768023030 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) ==== 24+0+0 (secure 0 0 0) 0x7f7768048050 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 10 client.5046 fetch_fsmap learned FSMap version 16 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) -- 0x5556b0c7fc60 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7768023030 con 0x5556b0ed9aa0 2024-09-11T12:50:37.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 16) ==== 3502+0+0 (secure 0 0 0) 0x7f776804b020 con 0x5556b0ed9aa0 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 10 client.5046 fetch_fsmap finished waiting for FSMap version 16 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 10 client.5046 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 20 client.5046 populate_metadata read hostname 'smithi078' 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0f0ecc0 0x5556b0f51b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 4 client.5046 mds_command: new command op to 4885 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 --> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] -- command(tid 0: {"prefix": "status"}) -- 0x5556b0ce7c60 con 0x5556b0f0ecc0 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7780e00640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0f0ecc0 0x5556b0f51b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 10 client.5046 ms_handle_connect on v2:172.21.15.157:6835/2884175884 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7780e00640 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0f0ecc0 0x5556b0f51b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc0af71db7a33adf server_cookie=4b2d8bd909bbcb70 in_seq=0 out_seq=0 2024-09-11T12:50:37.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 1 -- 172.21.15.78:0/2587878967 <== mds.0 v2:172.21.15.157:6835/2884175884 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5556b0ce7c60 con 0x5556b0f0ecc0 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 10 client.5046 handle_command_reply: tid=0 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 client.5046 shutdown 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 msgr2=0x5556b0ef2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7768003b80 tx=0x7f77680017f0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 shutdown_connections 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:6835/2884175884,v1:172.21.15.157:6836/2884175884] conn(0x5556b0f0ecc0 0x5556b0f51b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x5556b0eda3c0 0x5556b0f0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x5556b0ed9aa0 0x5556b0ef2cf0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7762000640 0 client.5046 ms_handle_reset on v2:172.21.15.157:6835/2884175884 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 --2- 172.21.15.78:0/2587878967 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x5556b0e8d340 0x5556b0ef27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 >> 172.21.15.78:0/2587878967 conn(0x5556b0ebab60 msgr2=0x5556b0e99d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 shutdown_connections 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 1 -- 172.21.15.78:0/2587878967 wait complete. 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7786c8b000 20 client.5046 trim_cache size 0 max 16384 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:50:37.780 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 4885, 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 10.438871457999999, 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 16, 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 34, 2024-09-11T12:50:37.781 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 34, 2024-09-11T12:50:37.782 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 17.319378245999999 2024-09-11T12:50:37.782 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:50:37.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 msgr2=0x7f775c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 0x7f775c073b40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7768000fc0 tx=0x7f7768006730 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 msgr2=0x7f777c0b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f77700ae040 tx=0x7f77700097a0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 shutdown_connections 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f775c071680 0x7f775c073b40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f777c0b1be0 0x7f777c0b2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f777c151800 0x7f777c1397c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 --2- 172.21.15.78:0/3664072824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764003a60 0x7f777c0b3570 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 >> 172.21.15.78:0/3664072824 conn(0x7f777c074b90 msgr2=0x7f777c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 shutdown_connections 2024-09-11T12:50:37.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.766+0000 7f7783800640 1 -- 172.21.15.78:0/3664072824 wait complete. 2024-09-11T12:50:37.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 -- 172.21.15.78:0/474459410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0072310 msgr2=0x7f3da015bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 --2- 172.21.15.78:0/474459410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0072310 0x7f3da015bed0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009bc0 tx=0x7f3d9801cb10 comp rx=0 tx=0).stop 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 -- 172.21.15.78:0/474459410 shutdown_connections 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 --2- 172.21.15.78:0/474459410 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0072310 0x7f3da015bed0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 --2- 172.21.15.78:0/474459410 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0152b20 0x7f3da0071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 -- 172.21.15.78:0/474459410 >> 172.21.15.78:0/474459410 conn(0x7f3da00757e0 msgr2=0x7f3da006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 -- 172.21.15.78:0/474459410 shutdown_connections 2024-09-11T12:50:37.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.930+0000 7f3da6600640 1 -- 172.21.15.78:0/474459410 wait complete. 2024-09-11T12:50:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 Processor -- start 2024-09-11T12:50:37.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- start start 2024-09-11T12:50:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 0x7f3da014ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 0x7f3da0143d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0168650 con 0x7f3da0152b20 2024-09-11T12:50:37.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3da015ce30 con 0x7f3da0152740 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3da01687d0 con 0x7f3da0072310 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33070/0 (socket says 172.21.15.78:33070) 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 -- 172.21.15.78:0/1253032571 learned_addr learned my addr 172.21.15.78:0/1253032571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 -- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 msgr2=0x7f3da014ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5200640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 0x7f3da014ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3d9fe00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 0x7f3da0143d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 0x7f3da014ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 -- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 msgr2=0x7f3da0143d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 0x7f3da0143d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 -- 172.21.15.78:0/1253032571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01fa5c0 con 0x7f3da0152b20 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5200640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 0x7f3da014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:37.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3d9fe00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 0x7f3da0143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da5c00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009bc0 tx=0x7f3d98005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d980b2040 con 0x7f3da0152b20 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d98004040 con 0x7f3da0152b20 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da014c130 con 0x7f3da0152b20 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98004430 con 0x7f3da0152b20 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da01fa860 con 0x7f3da0152b20 2024-09-11T12:50:37.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.934+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da01faa30 con 0x7f3da0152b20 2024-09-11T12:50:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.938+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f3d980bd030 con 0x7f3da0152b20 2024-09-11T12:50:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.938+0000 7f3d9cc00640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 0x7f3d80073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.938+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f3d9812bee0 con 0x7f3da0152b20 2024-09-11T12:50:37.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.938+0000 7f3da5200640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 0x7f3d80073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:37.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.938+0000 7f3da5200640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 0x7f3d80073b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d90009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:37.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:37.942+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3d980c4060 con 0x7f3da0152b20 2024-09-11T12:50:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3da0156230 con 0x7f3da0152b20 2024-09-11T12:50:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3d9cc00640 1 -- 172.21.15.78:0/1253032571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7f3d98016790 con 0x7f3da0152b20 2024-09-11T12:50:38.147 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 msgr2=0x7f3d80073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 0x7f3d80073b40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3d90007fd0 tx=0x7f3d90009040 comp rx=0 tx=0).stop 2024-09-11T12:50:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 msgr2=0x7f3da01fad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.142+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009bc0 tx=0x7f3d98005b00 comp rx=0 tx=0).stop 2024-09-11T12:50:38.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 shutdown_connections 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3d80071680 0x7f3d80073b40 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3da0152b20 0x7f3da01fad80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3da0152740 0x7f3da0143d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 --2- 172.21.15.78:0/1253032571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3da0072310 0x7f3da014ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 >> 172.21.15.78:0/1253032571 conn(0x7f3da00757e0 msgr2=0x7f3da014dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 shutdown_connections 2024-09-11T12:50:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38.146+0000 7f3da6600640 1 -- 172.21.15.78:0/1253032571 wait complete. 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:38.168 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:50:38.169 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:50:38.169 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:38.169 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:50:38.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.169 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.179 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42396sec preferred_lft 42396sec 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:38.180 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:50:38.181 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:38.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.182 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:38.241 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:50:38.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.241 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:38.297 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:38.298 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:38.298 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:38.298 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:38.298 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:50:38.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:38.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.434 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:50:38.442 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:50:38.442 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:50:38.443 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:50:38.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.443 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:50:38.508 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:38.508 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:38.556 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:50:38.556 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:38.557 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:38.607 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:50:38.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:38.696 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:38.696 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:38.709 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:38.709 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:38.709 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:38.709 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:38.709 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:38.713 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:38.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:38.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:38.734 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:38.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:38.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:38.846 INFO:teuthology.orchestra.run.smithi078.stdout:client4778 2024-09-11T12:50:38.846 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:38.847 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:38.855 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 -- 172.21.15.78:0/324068393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 msgr2=0x7ff50c0cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 --2- 172.21.15.78:0/324068393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c0cbed0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff4f8009840 tx=0x7ff4f801c760 comp rx=0 tx=0).stop 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 -- 172.21.15.78:0/324068393 shutdown_connections 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 --2- 172.21.15.78:0/324068393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c0cbed0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 --2- 172.21.15.78:0/324068393 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50c0b4780 0x7ff50c15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 -- 172.21.15.78:0/324068393 >> 172.21.15.78:0/324068393 conn(0x7ff50c06ca80 msgr2=0x7ff50c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 -- 172.21.15.78:0/324068393 shutdown_connections 2024-09-11T12:50:39.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.030+0000 7ff513e00640 1 -- 172.21.15.78:0/324068393 wait complete. 2024-09-11T12:50:39.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 Processor -- start 2024-09-11T12:50:39.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 -- start start 2024-09-11T12:50:39.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c0b4780 0x7ff50c140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:39.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50c13d250 0x7ff50c13bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c13ebe0 con 0x7ff50c15a1c0 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff50c13ea60 con 0x7ff50c0b4780 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff50c13ed60 con 0x7ff50c13d250 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33084/0 (socket says 172.21.15.78:33084) 2024-09-11T12:50:39.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 -- 172.21.15.78:0/4162776624 learned_addr learned my addr 172.21.15.78:0/4162776624 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:39.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 -- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50c13d250 msgr2=0x7ff50c13bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:39.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50c13d250 0x7ff50c13bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 -- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c0b4780 msgr2=0x7ff50c140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:39.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c0b4780 0x7ff50c140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 -- 172.21.15.78:0/4162776624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50c1427d0 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff511e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c780 tx=0x7ff50400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff504017070 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff504005070 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5040054c0 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.034+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50c142ab0 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff50c143070 con 0x7ff50c15a1c0 2024-09-11T12:50:39.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50c13f270 con 0x7ff50c15a1c0 2024-09-11T12:50:39.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7ff50401a070 con 0x7ff50c15a1c0 2024-09-11T12:50:39.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff509600640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 0x7ff4ec073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:39.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff5040915b0 con 0x7ff50c15a1c0 2024-09-11T12:50:39.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.038+0000 7ff511400640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 0x7ff4ec073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:39.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.042+0000 7ff511400640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 0x7ff4ec073b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff50000a660 tx=0x7ff50000c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:39.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.042+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff50c15e700 con 0x7ff50c15a1c0 2024-09-11T12:50:39.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff50c0cc570 con 0x7ff50c15a1c0 2024-09-11T12:50:39.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff509600640 1 -- 172.21.15.78:0/4162776624 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) ==== 53+0+73 (secure 0 0 0) 0x7ff50405d380 con 0x7ff50c15a1c0 2024-09-11T12:50:39.250 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:50:39.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 msgr2=0x7ff4ec073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:39.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 0x7ff4ec073b40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff50000a660 tx=0x7ff50000c040 comp rx=0 tx=0).stop 2024-09-11T12:50:39.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 msgr2=0x7ff50c158c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7ff50400c780 tx=0x7ff50400cc40 comp rx=0 tx=0).stop 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 shutdown_connections 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff4ec071680 0x7ff4ec073b40 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff50c15a1c0 0x7ff50c158c90 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff50c13d250 0x7ff50c13bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 --2- 172.21.15.78:0/4162776624 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff50c0b4780 0x7ff50c140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 >> 172.21.15.78:0/4162776624 conn(0x7ff50c06ca80 msgr2=0x7ff50c15c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 shutdown_connections 2024-09-11T12:50:39.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:39.246+0000 7ff513e00640 1 -- 172.21.15.78:0/4162776624 wait complete. 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:50:39.272 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:50:39.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.273 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:39.283 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42404sec preferred_lft 42404sec 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:50:39.284 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:50:39.285 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:50:39.286 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:50:39.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.287 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:50:39.348 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:50:39.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.350 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:50:39.405 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:50:39.406 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:50:39.407 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:50:39.499 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:50:39.541 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:39.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.549 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:50:39.557 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:50:39.558 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:50:39.558 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:50:39.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.558 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:50:39.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:39.620 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:39.668 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:50:39.668 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:50:39.668 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:39.723 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:50:39.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.724 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:39.813 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:39.814 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:39.827 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:50:39.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:50:39.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:50:39.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:50:39.827 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:50:39.831 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:50:39.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:39.832 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:39.850 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:50:39.942 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:39.945 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:50:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:39.952 INFO:teuthology.orchestra.run.smithi157.stdout:client4784 2024-09-11T12:50:39.952 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:39.952 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:39.960 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:50:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 -- 172.21.15.78:0/2382152928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440b5c40 msgr2=0x7f2b440cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 --2- 172.21.15.78:0/2382152928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440b5c40 0x7f2b440cf960 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c009840 tx=0x7f2b3c01c820 comp rx=0 tx=0).stop 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 -- 172.21.15.78:0/2382152928 shutdown_connections 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 --2- 172.21.15.78:0/2382152928 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440b5c40 0x7f2b440cf960 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 --2- 172.21.15.78:0/2382152928 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b44152990 0x7f2b4415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 -- 172.21.15.78:0/2382152928 >> 172.21.15.78:0/2382152928 conn(0x7f2b440b3050 msgr2=0x7f2b440b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 -- 172.21.15.78:0/2382152928 shutdown_connections 2024-09-11T12:50:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.102+0000 7f2b4c000640 1 -- 172.21.15.78:0/2382152928 wait complete. 2024-09-11T12:50:40.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 Processor -- start 2024-09-11T12:50:40.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- start start 2024-09-11T12:50:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 0x7f2b440700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 0x7f2b4414e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44168a80 con 0x7f2b440cf520 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2b44168900 con 0x7f2b44152990 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2b44168c00 con 0x7f2b440b5c40 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33098/0 (socket says 172.21.15.78:33098) 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 -- 172.21.15.78:0/741315562 learned_addr learned my addr 172.21.15.78:0/741315562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b49600640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 0x7f2b440700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4a000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 0x7f2b4414e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 -- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 msgr2=0x7f2b440700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 0x7f2b440700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 -- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 msgr2=0x7f2b4414e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 0x7f2b4414e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 -- 172.21.15.78:0/741315562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b441fed50 con 0x7f2b440cf520 2024-09-11T12:50:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4a000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 0x7f2b4414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b49600640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 0x7f2b440700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:40.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b48c00640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c480 tx=0x7f2b3000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30017070 con 0x7f2b440cf520 2024-09-11T12:50:40.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b30003600 con 0x7f2b440cf520 2024-09-11T12:50:40.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30003a50 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b441ff010 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b441ff5d0 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.106+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b440cf960 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.110+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f2b3001a070 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.110+0000 7f2b41600640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 0x7f2b24073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.110+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2b30091220 con 0x7f2b440cf520 2024-09-11T12:50:40.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.110+0000 7f2b49600640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 0x7f2b24073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.110+0000 7f2b49600640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 0x7f2b24073b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e70 tx=0x7f2b38009850 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.114+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2b3005d140 con 0x7f2b440cf520 2024-09-11T12:50:40.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.314+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2b4415aa40 con 0x7f2b440cf520 2024-09-11T12:50:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.370+0000 7f2b41600640 1 -- 172.21.15.78:0/741315562 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v17) ==== 136+0+0 (secure 0 0 0) 0x7f2b3005ca40 con 0x7f2b440cf520 2024-09-11T12:50:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 msgr2=0x7f2b24073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 0x7f2b24073b40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2b38009e70 tx=0x7f2b38009850 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 msgr2=0x7f2b4414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c480 tx=0x7f2b3000c940 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 shutdown_connections 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2b24071680 0x7f2b24073b40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2b44152990 0x7f2b4414e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2b440cf520 0x7f2b4414e2b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 --2- 172.21.15.78:0/741315562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2b440b5c40 0x7f2b440700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 >> 172.21.15.78:0/741315562 conn(0x7f2b440b3050 msgr2=0x7f2b4415af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 shutdown_connections 2024-09-11T12:50:40.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.374+0000 7f2b4c000640 1 -- 172.21.15.78:0/741315562 wait complete. 2024-09-11T12:50:40.397 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:50:40.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 -- 172.21.15.78:0/1633151503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 msgr2=0x7f7ae4157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 --2- 172.21.15.78:0/1633151503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae4157ae0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7ae415dad0 tx=0x7f7ad001c830 comp rx=0 tx=0).stop 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 -- 172.21.15.78:0/1633151503 shutdown_connections 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 --2- 172.21.15.78:0/1633151503 >> v2:172.21.15.157:3300/0 conn(0x7f7ae406eb40 0x7f7ae406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 --2- 172.21.15.78:0/1633151503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae4157ae0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 -- 172.21.15.78:0/1633151503 >> 172.21.15.78:0/1633151503 conn(0x7f7ae406ca80 msgr2=0x7f7ae40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 -- 172.21.15.78:0/1633151503 shutdown_connections 2024-09-11T12:50:40.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.534+0000 7f7ae9000640 1 -- 172.21.15.78:0/1633151503 wait complete. 2024-09-11T12:50:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 Processor -- start 2024-09-11T12:50:40.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- start start 2024-09-11T12:50:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 0x7f7ae4165490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 0x7f7ae415cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae415f940 con 0x7f7ae40736d0 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae415f7c0 con 0x7f7ae4070a00 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae415fac0 con 0x7f7ae406eb40 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33122/0 (socket says 172.21.15.78:33122) 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 -- 172.21.15.78:0/2860960556 learned_addr learned my addr 172.21.15.78:0/2860960556 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae1e00640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 0x7f7ae4165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae1400640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 0x7f7ae415cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 -- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 msgr2=0x7f7ae4165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 0x7f7ae4165490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 -- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 msgr2=0x7f7ae415cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 0x7f7ae415cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 -- 172.21.15.78:0/2860960556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae415d790 con 0x7f7ae40736d0 2024-09-11T12:50:40.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae1e00640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 0x7f7ae4165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:40.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae1400640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 0x7f7ae415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae2800640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ac50 tx=0x7f7adc007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc00ae60 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7adc003980 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae41fba80 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc003d70 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae41f7a30 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.538+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae406efb0 con 0x7f7ae40736d0 2024-09-11T12:50:40.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.542+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f7adc003f10 con 0x7f7ae40736d0 2024-09-11T12:50:40.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.542+0000 7f7ada000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 0x7f7ac4073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.542+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f7adc0923c0 con 0x7f7ae40736d0 2024-09-11T12:50:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.542+0000 7f7ae1e00640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 0x7f7ac4073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.542+0000 7f7ae1e00640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 0x7f7ac4073b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4001750 tx=0x7f7ad4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.546+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7adc05e310 con 0x7f7ae40736d0 2024-09-11T12:50:40.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.730+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7ae40723c0 con 0x7f7ac4071680 2024-09-11T12:50:40.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.734+0000 7f7ada000640 1 -- 172.21.15.78:0/2860960556 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7ae40723c0 con 0x7f7ac4071680 2024-09-11T12:50:40.739 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:50:40.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:40.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:50:40.739 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:40.740 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:50:40.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 msgr2=0x7f7ac4073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 0x7f7ac4073b40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4001750 tx=0x7f7ad4009040 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 msgr2=0x7f7ae415d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7adc00ac50 tx=0x7f7adc007590 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 shutdown_connections 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7ac4071680 0x7f7ac4073b40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae40736d0 0x7f7ae415d250 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae4070a00 0x7f7ae415cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 --2- 172.21.15.78:0/2860960556 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae406eb40 0x7f7ae4165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 >> 172.21.15.78:0/2860960556 conn(0x7f7ae406ca80 msgr2=0x7f7ae41535f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 shutdown_connections 2024-09-11T12:50:40.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.738+0000 7f7ae9000640 1 -- 172.21.15.78:0/2860960556 wait complete. 2024-09-11T12:50:40.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:50:40.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- 172.21.15.78:0/1915004033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c005170 msgr2=0x7ff40c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 --2- 172.21.15.78:0/1915004033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40c005170 0x7ff40c005920 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff414006f70 tx=0x7ff41401cea0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42d600640 1 -- 172.21.15.78:0/1915004033 reap_dead start 2024-09-11T12:50:40.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- 172.21.15.78:0/1915004033 shutdown_connections 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- 172.21.15.78:0/1915004033 >> 172.21.15.78:0/1915004033 conn(0x7ff428071dd0 msgr2=0x7ff4280721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- 172.21.15.78:0/1915004033 shutdown_connections 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- 172.21.15.78:0/1915004033 wait complete. 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 Processor -- start 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- start start 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c005170 0x7ff428157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff428154d30 0x7ff4281424e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280aee80 con 0x7ff4281fb4d0 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4280aed00 con 0x7ff40c005170 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4280af000 con 0x7ff428154d30 2024-09-11T12:50:40.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33136/0 (socket says 172.21.15.78:33136) 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42d600640 1 -- 172.21.15.78:0/1677782877 learned_addr learned my addr 172.21.15.78:0/1677782877 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff42cc00640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c005170 0x7ff428157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.902+0000 7ff427e00640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff428154d30 0x7ff4281424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 -- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff428154d30 msgr2=0x7ff4281424e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff428154d30 0x7ff4281424e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 -- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c005170 msgr2=0x7ff428157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c005170 0x7ff428157c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 -- 172.21.15.78:0/1677782877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff428072660 con 0x7ff4281fb4d0 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42d600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff420000f90 tx=0x7ff4200019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420015020 con 0x7ff4281fb4d0 2024-09-11T12:50:40.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff420011960 con 0x7ff4281fb4d0 2024-09-11T12:50:40.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420011d50 con 0x7ff4281fb4d0 2024-09-11T12:50:40.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff428142a20 con 0x7ff4281fb4d0 2024-09-11T12:50:40.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff428155a50 con 0x7ff4281fb4d0 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff42815a700 con 0x7ff4281fb4d0 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7ff420018020 con 0x7ff4281fb4d0 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 0x7ff408073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.906+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7ff4200922e0 con 0x7ff4281fb4d0 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.910+0000 7ff42cc00640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 0x7ff408073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:40.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.910+0000 7ff42cc00640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 0x7ff408073b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff418009c60 tx=0x7ff4180091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:40.910+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff42001f030 con 0x7ff4281fb4d0 2024-09-11T12:50:41.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7ff4280722f0 con 0x7ff4281fb4d0 2024-09-11T12:50:41.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff424c00640 1 -- 172.21.15.78:0/1677782877 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7ff42005e140 con 0x7ff4281fb4d0 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 msgr2=0x7ff408073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 0x7ff408073b40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff418009c60 tx=0x7ff4180091e0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 msgr2=0x7ff428158530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff420000f90 tx=0x7ff4200019c0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 shutdown_connections 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ff408071680 0x7ff408073b40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4281fb4d0 0x7ff428158530 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff428154d30 0x7ff4281424e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 --2- 172.21.15.78:0/1677782877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff40c005170 0x7ff428157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 >> 172.21.15.78:0/1677782877 conn(0x7ff428071dd0 msgr2=0x7ff42815bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:41.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 shutdown_connections 2024-09-11T12:50:41.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.078+0000 7ff42f600640 1 -- 172.21.15.78:0/1677782877 wait complete. 2024-09-11T12:50:41.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:41.103 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp58940/subvol89170) 2024-09-11T12:50:41.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:41.131 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:50:41.183 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:50:41.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:41.183 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp58940/subvol89170) 2024-09-11T12:50:41.256 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:41.257 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp58940/subvol89170/.meta 2024-09-11T12:50:41.325 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol89170 subvol_grp58940 2024-09-11T12:50:41.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:50:41.506 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:50:41.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 -- 172.21.15.78:0/2377689686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc070e50 msgr2=0x7f1abc06e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 --2- 172.21.15.78:0/2377689686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc070e50 0x7f1abc06e960 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1abc073100 tx=0x7f1ab401cb80 comp rx=0 tx=0).stop 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 -- 172.21.15.78:0/2377689686 shutdown_connections 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 --2- 172.21.15.78:0/2377689686 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc070e50 0x7f1abc06e960 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 --2- 172.21.15.78:0/2377689686 >> v2:172.21.15.157:3300/0 conn(0x7f1abc0b7950 0x7f1abc0b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 -- 172.21.15.78:0/2377689686 >> 172.21.15.78:0/2377689686 conn(0x7f1abc0ce980 msgr2=0x7f1abc0ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 -- 172.21.15.78:0/2377689686 shutdown_connections 2024-09-11T12:50:41.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.510+0000 7f1ac2200640 1 -- 172.21.15.78:0/2377689686 wait complete. 2024-09-11T12:50:41.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 Processor -- start 2024-09-11T12:50:41.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 -- start start 2024-09-11T12:50:41.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc070e50 0x7f1abc14bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:41.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 0x7f1abc14db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:41.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc168d50 con 0x7f1abc0b7950 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc168bd0 con 0x7f1abc070e50 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1ac2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc168ed0 con 0x7f1abc0b4520 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33152/0 (socket says 172.21.15.78:33152) 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 -- 172.21.15.78:0/949444458 learned_addr learned my addr 172.21.15.78:0/949444458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abb400640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc070e50 0x7f1abc14bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abaa00640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 0x7f1abc14db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 -- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 msgr2=0x7f1abc14db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 0x7f1abc14db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 -- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc070e50 msgr2=0x7f1abc14bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc070e50 0x7f1abc14bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 -- 172.21.15.78:0/949444458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc14ebe0 con 0x7f1abc0b7950 2024-09-11T12:50:41.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abaa00640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 0x7f1abc14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:41.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.514+0000 7f1abbe00640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4098660 tx=0x7f1ab4005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:41.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab40b2040 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ab40bb070 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4004330 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc14eec0 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc156870 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc156a70 con 0x7f1abc0b7950 2024-09-11T12:50:41.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.518+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f1ab40044d0 con 0x7f1abc0b7950 2024-09-11T12:50:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.522+0000 7f1ab3400640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 0x7f1a9c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.522+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f1ab412c660 con 0x7f1abc0b7950 2024-09-11T12:50:41.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.522+0000 7f1abb400640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 0x7f1a9c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:41.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.522+0000 7f1abb400640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 0x7f1a9c073b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009b40 tx=0x7f1aac0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:41.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.526+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1abc156a70 con 0x7f1abc0b7950 2024-09-11T12:50:41.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.714+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol89170", "group_name": "subvol_grp58940", "target": ["mon-mgr", ""]}) -- 0x7f1abc14f3e0 con 0x7f1a9c071680 2024-09-11T12:50:41.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:41.724+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:41.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:41.724+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:41.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:41.724+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:41.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:41.724+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:41.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:41.724+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:41.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.866+0000 7f1ab3400640 1 -- 172.21.15.78:0/949444458 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1aac011be0 con 0x7f1a9c071680 2024-09-11T12:50:41.873 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp58940/subvol89170 2024-09-11T12:50:41.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 msgr2=0x7f1a9c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 0x7f1a9c073b40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009b40 tx=0x7f1aac0091e0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 msgr2=0x7f1abc14e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4098660 tx=0x7f1ab4005a80 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 shutdown_connections 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1a9c071680 0x7f1a9c073b40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1abc0b7950 0x7f1abc14e080 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1abc0b4520 0x7f1abc14db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 --2- 172.21.15.78:0/949444458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1abc070e50 0x7f1abc14bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 >> 172.21.15.78:0/949444458 conn(0x7f1abc0ce980 msgr2=0x7f1abc152280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 shutdown_connections 2024-09-11T12:50:41.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:41.870+0000 7f1ac2200640 1 -- 172.21.15.78:0/949444458 wait complete. 2024-09-11T12:50:41.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol89170 subvol_grp58940 2024-09-11T12:50:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 -- 172.21.15.78:0/3821967024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c073e90 msgr2=0x7f019c071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 --2- 172.21.15.78:0/3821967024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c073e90 0x7f019c071e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f0188009900 tx=0x7f018801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 -- 172.21.15.78:0/3821967024 shutdown_connections 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 --2- 172.21.15.78:0/3821967024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c073e90 0x7f019c071e10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 --2- 172.21.15.78:0/3821967024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c151050 0x7f019c073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 -- 172.21.15.78:0/3821967024 >> 172.21.15.78:0/3821967024 conn(0x7f019c0b3050 msgr2=0x7f019c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 -- 172.21.15.78:0/3821967024 shutdown_connections 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 -- 172.21.15.78:0/3821967024 wait complete. 2024-09-11T12:50:42.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.034+0000 7f01a3c00640 1 Processor -- start 2024-09-11T12:50:42.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- start start 2024-09-11T12:50:42.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 0x7f019c1f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f019c151050 0x7f019c1f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f019c15e970 con 0x7f019c072350 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f019c15e7f0 con 0x7f019c151050 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f019c15eaf0 con 0x7f019c073e90 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33168/0 (socket says 172.21.15.78:33168) 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 -- 172.21.15.78:0/1215241675 learned_addr learned my addr 172.21.15.78:0/1215241675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:42.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f019c151050 0x7f019c1f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f019be00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 0x7f019c1f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 -- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 msgr2=0x7f019c1f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 0x7f019c1f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 -- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f019c151050 msgr2=0x7f019c1f8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f019c151050 0x7f019c1f8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 -- 172.21.15.78:0/1215241675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c15afe0 con 0x7f019c072350 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f019be00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 0x7f019c1f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:42.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a1200640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f019000e7c0 tx=0x7f019000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000ccc0 con 0x7f019c072350 2024-09-11T12:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f019000ce80 con 0x7f019c072350 2024-09-11T12:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190005280 con 0x7f019c072350 2024-09-11T12:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c15d770 con 0x7f019c072350 2024-09-11T12:50:42.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c15b590 con 0x7f019c072350 2024-09-11T12:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.038+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c071e10 con 0x7f019c072350 2024-09-11T12:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.042+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f0190005420 con 0x7f019c072350 2024-09-11T12:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.042+0000 7f0198c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 0x7f017c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.042+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f0190092760 con 0x7f019c072350 2024-09-11T12:50:42.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.042+0000 7f019be00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 0x7f017c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.042+0000 7f019be00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 0x7f017c073b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01880af000 tx=0x7f018801b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.046+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f019005e5c0 con 0x7f019c072350 2024-09-11T12:50:42.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.238+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol89170", "group_name": "subvol_grp58940", "target": ["mon-mgr", ""]}) -- 0x7f019c0cc080 con 0x7f017c071680 2024-09-11T12:50:42.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.242+0000 7f0198c00640 1 -- 172.21.15.78:0/1215241675 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f019c0cc080 con 0x7f017c071680 2024-09-11T12:50:42.247 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp58940/subvol89170 2024-09-11T12:50:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 msgr2=0x7f017c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 0x7f017c073b40 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f01880af000 tx=0x7f018801b0f0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 msgr2=0x7f019c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f019000e7c0 tx=0x7f019000ec80 comp rx=0 tx=0).stop 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 shutdown_connections 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f017c071680 0x7f017c073b40 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f019c151050 0x7f019c1f8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f019c073e90 0x7f019c1f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 --2- 172.21.15.78:0/1215241675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f019c072350 0x7f019c15d230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 >> 172.21.15.78:0/1215241675 conn(0x7f019c0b3050 msgr2=0x7f019c0ce300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 shutdown_connections 2024-09-11T12:50:42.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.246+0000 7f01a3c00640 1 -- 172.21.15.78:0/1215241675 wait complete. 2024-09-11T12:50:42.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:42.269 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/3ea922d3e11759cc191a58869820a534.meta) 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/subvol_grp58940/subvol89170 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:50:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:42.296 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol89170 subvol_grp58940 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 -- 172.21.15.78:0/948205363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 msgr2=0x7f816c15a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 --2- 172.21.15.78:0/948205363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c15a380 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f815c009900 tx=0x7f815c01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 -- 172.21.15.78:0/948205363 shutdown_connections 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 --2- 172.21.15.78:0/948205363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0cf3c0 0x7f816c070180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 --2- 172.21.15.78:0/948205363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c15a380 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 -- 172.21.15.78:0/948205363 >> 172.21.15.78:0/948205363 conn(0x7f816c0b2c70 msgr2=0x7f816c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:42.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 -- 172.21.15.78:0/948205363 shutdown_connections 2024-09-11T12:50:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.482+0000 7f8171e00640 1 -- 172.21.15.78:0/948205363 wait complete. 2024-09-11T12:50:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 Processor -- start 2024-09-11T12:50:42.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- start start 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0706c0 0x7f816c151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f816c0cf3c0 0x7f816c1f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f816c1670a0 con 0x7f816c1433c0 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f816c166f20 con 0x7f816c0cf3c0 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f816c167220 con 0x7f816c0706c0 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33190/0 (socket says 172.21.15.78:33190) 2024-09-11T12:50:42.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 -- 172.21.15.78:0/3231994887 learned_addr learned my addr 172.21.15.78:0/3231994887 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8170a00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0706c0 0x7f816c151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f816be00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f816c0cf3c0 0x7f816c1f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 -- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0706c0 msgr2=0x7f816c151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0706c0 0x7f816c151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 -- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f816c0cf3c0 msgr2=0x7f816c1f7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f816c0cf3c0 0x7f816c1f7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 -- 172.21.15.78:0/3231994887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f816c153070 con 0x7f816c1433c0 2024-09-11T12:50:42.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171400640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f816400c780 tx=0x7f816400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164004b70 con 0x7f816c1433c0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8164004e00 con 0x7f816c1433c0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f816400aa90 con 0x7f816c1433c0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f816c1514b0 con 0x7f816c1433c0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f816c151a70 con 0x7f816c1433c0 2024-09-11T12:50:42.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.486+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f816c151c70 con 0x7f816c1433c0 2024-09-11T12:50:42.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.490+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f816400ac30 con 0x7f816c1433c0 2024-09-11T12:50:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.490+0000 7f8168c00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 0x7f814c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.490+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8164092010 con 0x7f816c1433c0 2024-09-11T12:50:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.490+0000 7f8170a00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 0x7f814c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.490+0000 7f8170a00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 0x7f814c073b40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f815c0af000 tx=0x7f815c01b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.494+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f816c151c70 con 0x7f816c1433c0 2024-09-11T12:50:42.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.682+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol89170", "group_name": "subvol_grp58940", "target": ["mon-mgr", ""]}) -- 0x7f816c15a380 con 0x7f814c071680 2024-09-11T12:50:42.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.718+0000 7f8168c00640 1 -- 172.21.15.78:0/3231994887 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f816c15a380 con 0x7f814c071680 2024-09-11T12:50:42.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 msgr2=0x7f814c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 0x7f814c073b40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f815c0af000 tx=0x7f815c01b0f0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 msgr2=0x7f816c1f8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f816400c780 tx=0x7f816400cc40 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 shutdown_connections 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f814c071680 0x7f814c073b40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f816c1433c0 0x7f816c1f8010 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f816c0cf3c0 0x7f816c1f7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 --2- 172.21.15.78:0/3231994887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f816c0706c0 0x7f816c151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 >> 172.21.15.78:0/3231994887 conn(0x7f816c0b2c70 msgr2=0x7f816c153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 shutdown_connections 2024-09-11T12:50:42.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.722+0000 7f8171e00640 1 -- 172.21.15.78:0/3231994887 wait complete. 2024-09-11T12:50:42.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.724+0000 7fa58de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.724+0000 7fa58de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.724+0000 7fa58de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.724+0000 7fa58de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.728 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.724+0000 7fa58de00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.747 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:50:42.747 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.744+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.748 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.744+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.748 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.744+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.748 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.744+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.748 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:50:42.744+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:50:42.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:50:42.822 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:50:42.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:50:42.829 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp58940 2024-09-11T12:50:42.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- 172.21.15.78:0/2650113498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15cf90 msgr2=0x7f238c0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 --2- 172.21.15.78:0/2650113498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15cf90 0x7f238c0cf6e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f238400acd0 tx=0x7f238401def0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- 172.21.15.78:0/2650113498 shutdown_connections 2024-09-11T12:50:42.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 --2- 172.21.15.78:0/2650113498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c15cf90 0x7f238c0cf6e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- 172.21.15.78:0/2650113498 >> 172.21.15.78:0/2650113498 conn(0x7f238c071dd0 msgr2=0x7f238c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:42.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- 172.21.15.78:0/2650113498 shutdown_connections 2024-09-11T12:50:42.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- 172.21.15.78:0/2650113498 wait complete. 2024-09-11T12:50:42.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 Processor -- start 2024-09-11T12:50:42.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- start start 2024-09-11T12:50:42.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c0b3040 0x7f238c159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 0x7f238c138330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0aefc0 con 0x7f238c0b3420 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f238c0aee40 con 0x7f238c0b3040 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f2391e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f238c0af140 con 0x7f238c15cf90 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33208/0 (socket says 172.21.15.78:33208) 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 -- 172.21.15.78:0/2205661758 learned_addr learned my addr 172.21.15.78:0/2205661758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238aa00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c0b3040 0x7f238c159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238b400640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 0x7f238c138330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 -- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 msgr2=0x7f238c138330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 0x7f238c138330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 -- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c0b3040 msgr2=0x7f238c159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:42.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c0b3040 0x7f238c159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:42.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238a000640 1 -- 172.21.15.78:0/2205661758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238c140010 con 0x7f238c0b3420 2024-09-11T12:50:42.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.970+0000 7f238b400640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 0x7f238c138330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:42.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f238a000640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f237800c500 tx=0x7f237800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378017070 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23780053c0 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238c1402f0 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378012890 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f238c1590f0 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f238c0cf6e0 con 0x7f238c0b3420 2024-09-11T12:50:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 5) ==== 94932+0+0 (secure 0 0 0) 0x7f237801a070 con 0x7f238c0b3420 2024-09-11T12:50:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 0x7f236c073b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2378091050 con 0x7f238c0b3420 2024-09-11T12:50:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.974+0000 7f238aa00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 0x7f236c073b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:42.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.978+0000 7f238aa00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 0x7f236c073b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f237c001910 tx=0x7f237c001420 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:42.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:42.978+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f237805c880 con 0x7f238c0b3420 2024-09-11T12:50:43.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.166+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp58940", "target": ["mon-mgr", ""]}) -- 0x7f238c13fdc0 con 0x7f236c071680 2024-09-11T12:50:43.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.170+0000 7f2383400640 1 -- 172.21.15.78:0/2205661758 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f238c13fdc0 con 0x7f236c071680 2024-09-11T12:50:43.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 msgr2=0x7f236c073b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 0x7f236c073b40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f237c001910 tx=0x7f237c001420 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 msgr2=0x7f238c158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f237800c500 tx=0x7f237800c9c0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 shutdown_connections 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f236c071680 0x7f236c073b40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f238c15cf90 0x7f238c138330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f238c0b3420 0x7f238c158bb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 --2- 172.21.15.78:0/2205661758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f238c0b3040 0x7f238c159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 >> 172.21.15.78:0/2205661758 conn(0x7f238c071dd0 msgr2=0x7f238c0708f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 shutdown_connections 2024-09-11T12:50:43.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.174+0000 7f2391e00640 1 -- 172.21.15.78:0/2205661758 wait complete. 2024-09-11T12:50:43.198 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:50:43.198 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:43.275 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:43.363 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:50:43.363 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:50:43.363 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:43.364 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:43.373 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:43.373 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:43.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:43.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 -- 172.21.15.78:0/2184270446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc290003a10 msgr2=0x7fc290003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/2184270446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc290003a10 0x7fc290003e30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00c640 tx=0x7fc29c01d2d0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2ab400640 1 -- 172.21.15.78:0/2184270446 reap_dead start 2024-09-11T12:50:43.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 -- 172.21.15.78:0/2184270446 shutdown_connections 2024-09-11T12:50:43.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 -- 172.21.15.78:0/2184270446 >> 172.21.15.78:0/2184270446 conn(0x7fc2ac06ca80 msgr2=0x7fc2ac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:43.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 -- 172.21.15.78:0/2184270446 shutdown_connections 2024-09-11T12:50:43.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.606+0000 7fc2b1e00640 1 -- 172.21.15.78:0/2184270446 wait complete. 2024-09-11T12:50:43.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 Processor -- start 2024-09-11T12:50:43.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- start start 2024-09-11T12:50:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 0x7fc2ac150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 0x7fc2ac15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac166a00 con 0x7fc2ac1f8500 2024-09-11T12:50:43.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac06ffd0 con 0x7fc290003a10 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac166b80 con 0x7fc2ac1f8080 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:33222/0 (socket says 172.21.15.78:33222) 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 -- 172.21.15.78:0/1716398614 learned_addr learned my addr 172.21.15.78:0/1716398614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2aaa00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 0x7fc2ac150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2aa000640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 0x7fc2ac15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 -- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 msgr2=0x7fc2ac15a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 0x7fc2ac15a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 -- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 msgr2=0x7fc2ac150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 0x7fc2ac150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 -- 172.21.15.78:0/1716398614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1647e0 con 0x7fc2ac1f8500 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2aaa00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 0x7fc2ac150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:43.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2aa000640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 0x7fc2ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2ab400640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac06eaf0 tx=0x7fc2a4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4004440 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2a400f920 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac072390 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400fd70 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac072950 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.610+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac06ecf0 con 0x7fc2ac1f8500 2024-09-11T12:50:43.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.614+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fc2a4003600 con 0x7fc2ac1f8500 2024-09-11T12:50:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.614+0000 7fc2a3400640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 0x7fc28c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.614+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fc2a40928c0 con 0x7fc2ac1f8500 2024-09-11T12:50:43.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.614+0000 7fc2aaa00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 0x7fc28c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:43.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.614+0000 7fc2aaa00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 0x7fc28c073cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0ac000 tx=0x7fc29c01a110 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:43.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.618+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc2a405e430 con 0x7fc2ac1f8500 2024-09-11T12:50:43.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.794+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2ac162480 con 0x7fc2ac1f8500 2024-09-11T12:50:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.794+0000 7fc2a3400640 1 -- 172.21.15.78:0/1716398614 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fc2a405dd30 con 0x7fc2ac1f8500 2024-09-11T12:50:43.801 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 msgr2=0x7fc28c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 0x7fc28c073cf0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0ac000 tx=0x7fc29c01a110 comp rx=0 tx=0).stop 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 msgr2=0x7fc2ac073150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac06eaf0 tx=0x7fc2a4011c40 comp rx=0 tx=0).stop 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 shutdown_connections 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc28c071830 0x7fc28c073cf0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ac1f8500 0x7fc2ac073150 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ac1f8080 0x7fc2ac15a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 --2- 172.21.15.78:0/1716398614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc290003a10 0x7fc2ac150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 >> 172.21.15.78:0/1716398614 conn(0x7fc2ac06ca80 msgr2=0x7fc2ac161990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 shutdown_connections 2024-09-11T12:50:43.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:43.798+0000 7fc2b1e00640 1 -- 172.21.15.78:0/1716398614 wait complete. 2024-09-11T12:50:43.821 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:50:43.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:43.822 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:50:43.870 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:50:43.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:43.871 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:50:43.925 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:50:43.925 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:43.925 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:43.935 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:43.935 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:43.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:50:44.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- 172.21.15.78:0/1698706415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6500cf760 msgr2=0x7fd650074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- 172.21.15.78:0/1698706415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6500cf760 0x7fd650074260 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd640009900 tx=0x7fd64001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- 172.21.15.78:0/1698706415 shutdown_connections 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- 172.21.15.78:0/1698706415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- 172.21.15.78:0/1698706415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd6500cf760 0x7fd650074260 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- 172.21.15.78:0/1698706415 >> 172.21.15.78:0/1698706415 conn(0x7fd65006ca80 msgr2=0x7fd6500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- 172.21.15.78:0/1698706415 shutdown_connections 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- 172.21.15.78:0/1698706415 wait complete. 2024-09-11T12:50:44.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 Processor -- start 2024-09-11T12:50:44.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- start start 2024-09-11T12:50:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65015aec0 0x7fd650159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd650151010 con 0x7fd6500cf760 2024-09-11T12:50:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd65015f2d0 con 0x7fd65015aec0 2024-09-11T12:50:44.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.122+0000 7fd656000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd650151190 con 0x7fd65006fc50 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55486/0 (socket says 172.21.15.78:55486) 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 -- 172.21.15.78:0/1619264643 learned_addr learned my addr 172.21.15.78:0/1619264643 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64f400640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64fe00640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65015aec0 0x7fd650159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 -- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 msgr2=0x7fd6500b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500b14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 -- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65015aec0 msgr2=0x7fd650159b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65015aec0 0x7fd650159b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 -- 172.21.15.78:0/1619264643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6501fae90 con 0x7fd6500cf760 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64f400640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd64ea00640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c3c0 tx=0x7fd63c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c017070 con 0x7fd6500cf760 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd63c0053c0 con 0x7fd6500cf760 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd63c003860 con 0x7fd6500cf760 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6501fb130 con 0x7fd6500cf760 2024-09-11T12:50:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65014f2a0 con 0x7fd6500cf760 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6501fb370 con 0x7fd6500cf760 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.126+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fd63c01a070 con 0x7fd6500cf760 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.130+0000 7fd647400640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 0x7fd630073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.130+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd63c091410 con 0x7fd6500cf760 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.130+0000 7fd64f400640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 0x7fd630073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.130+0000 7fd64f400640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 0x7fd630073cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6400af000 tx=0x7fd64001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.134+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd6501fb370 con 0x7fd6500cf760 2024-09-11T12:50:44.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.306+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd6500700e0 con 0x7fd6500cf760 2024-09-11T12:50:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd647400640 1 -- 172.21.15.78:0/1619264643 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v34) ==== 80+0+0 (secure 0 0 0) 0x7fd63c05cff0 con 0x7fd6500cf760 2024-09-11T12:50:44.313 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:50:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 msgr2=0x7fd630073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 0x7fd630073cf0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6400af000 tx=0x7fd64001b120 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 msgr2=0x7fd65014f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd63c00c3c0 tx=0x7fd63c00c880 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 shutdown_connections 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd630071830 0x7fd630073cf0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd65015aec0 0x7fd650159b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd6500cf760 0x7fd65014f7f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 --2- 172.21.15.78:0/1619264643 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd65006fc50 0x7fd6500b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 >> 172.21.15.78:0/1619264643 conn(0x7fd65006ca80 msgr2=0x7fd6500ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 shutdown_connections 2024-09-11T12:50:44.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.310+0000 7fd656000640 1 -- 172.21.15.78:0/1619264643 wait complete. 2024-09-11T12:50:44.334 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:50:44.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:44.335 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:50:44.364 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:50:44.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:50:44.364 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:50:44.421 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:50:44.422 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:44.422 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:44.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:50:44.431 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:50:44.431 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:44.431 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:44.472 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:50:44.472 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:44.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- 172.21.15.78:0/1519685394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 msgr2=0x7fd84415bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- 172.21.15.78:0/1519685394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd84415bfa0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd84406dae0 tx=0x7fd83c01caf0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- 172.21.15.78:0/1519685394 shutdown_connections 2024-09-11T12:50:44.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- 172.21.15.78:0/1519685394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd84415bfa0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- 172.21.15.78:0/1519685394 >> v2:172.21.15.157:3300/0 conn(0x7fd8440aebc0 0x7fd8440b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- 172.21.15.78:0/1519685394 >> 172.21.15.78:0/1519685394 conn(0x7fd8440ccd10 msgr2=0x7fd8440cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:44.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- 172.21.15.78:0/1519685394 shutdown_connections 2024-09-11T12:50:44.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- 172.21.15.78:0/1519685394 wait complete. 2024-09-11T12:50:44.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 Processor -- start 2024-09-11T12:50:44.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- start start 2024-09-11T12:50:44.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd84406c8b0 0x7fd8440c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8440aebc0 0x7fd8440afe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd844072890 con 0x7fd84406cc90 2024-09-11T12:50:44.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd844072710 con 0x7fd8440aebc0 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.610+0000 7fd84b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd844072a10 con 0x7fd84406c8b0 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55500/0 (socket says 172.21.15.78:55500) 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 -- 172.21.15.78:0/552430240 learned_addr learned my addr 172.21.15.78:0/552430240 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843e00640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd84406c8b0 0x7fd8440c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd849000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8440aebc0 0x7fd8440afe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 -- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd84406c8b0 msgr2=0x7fd8440c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd84406c8b0 0x7fd8440c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 -- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8440aebc0 msgr2=0x7fd8440afe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8440aebc0 0x7fd8440afe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 -- 172.21.15.78:0/552430240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8440c5b10 con 0x7fd84406cc90 2024-09-11T12:50:44.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd843400640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd83000c9f0 tx=0x7fd83000ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd830007b50 con 0x7fd84406cc90 2024-09-11T12:50:44.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd830007de0 con 0x7fd84406cc90 2024-09-11T12:50:44.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83000f6b0 con 0x7fd84406cc90 2024-09-11T12:50:44.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8440c5df0 con 0x7fd84406cc90 2024-09-11T12:50:44.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8440c5770 con 0x7fd84406cc90 2024-09-11T12:50:44.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd84413cef0 con 0x7fd84406cc90 2024-09-11T12:50:44.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.614+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fd83000f850 con 0x7fd84406cc90 2024-09-11T12:50:44.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.618+0000 7fd83be00640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 0x7fd824073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.618+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fd830092850 con 0x7fd84406cc90 2024-09-11T12:50:44.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.618+0000 7fd843e00640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 0x7fd824073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.618+0000 7fd843e00640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 0x7fd824073cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8340016e0 tx=0x7fd834009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.622+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd830014030 con 0x7fd84406cc90 2024-09-11T12:50:44.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.818+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd844073170 con 0x7fd84406cc90 2024-09-11T12:50:44.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.818+0000 7fd83be00640 1 -- 172.21.15.78:0/552430240 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7fd83005e320 con 0x7fd84406cc90 2024-09-11T12:50:44.824 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:44.824 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"btime":"2024-09-11T12:50:40:372090+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:44.824 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 msgr2=0x7fd824073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 0x7fd824073cf0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd8340016e0 tx=0x7fd834009040 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 msgr2=0x7fd8441f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd83000c9f0 tx=0x7fd83000ceb0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 shutdown_connections 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd824071830 0x7fd824073cf0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8440aebc0 0x7fd8440afe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd84406cc90 0x7fd8441f68c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 --2- 172.21.15.78:0/552430240 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd84406c8b0 0x7fd8440c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 >> 172.21.15.78:0/552430240 conn(0x7fd8440ccd10 msgr2=0x7fd844150d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 shutdown_connections 2024-09-11T12:50:44.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.822+0000 7fd84b000640 1 -- 172.21.15.78:0/552430240 wait complete. 2024-09-11T12:50:44.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 -- 172.21.15.78:0/1429151417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c0cfeb0 msgr2=0x7f287c073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 --2- 172.21.15.78:0/1429151417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c0cfeb0 0x7f287c073c70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2868009840 tx=0x7f286801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 -- 172.21.15.78:0/1429151417 shutdown_connections 2024-09-11T12:50:44.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 --2- 172.21.15.78:0/1429151417 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c15ae40 0x7f287c15c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 --2- 172.21.15.78:0/1429151417 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c0cfeb0 0x7f287c073c70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 -- 172.21.15.78:0/1429151417 >> 172.21.15.78:0/1429151417 conn(0x7f287c06f3a0 msgr2=0x7f287c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 -- 172.21.15.78:0/1429151417 shutdown_connections 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.982+0000 7f2881a00640 1 -- 172.21.15.78:0/1429151417 wait complete. 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 Processor -- start 2024-09-11T12:50:44.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- start start 2024-09-11T12:50:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c0cfeb0 0x7f287c14b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c1518f0 0x7f287c1f7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f287c1675c0 con 0x7f287c15ae40 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f287c15d980 con 0x7f287c1518f0 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f287c167740 con 0x7f287c0cfeb0 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55508/0 (socket says 172.21.15.78:55508) 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 -- 172.21.15.78:0/803462441 learned_addr learned my addr 172.21.15.78:0/803462441 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287aa00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c0cfeb0 0x7f287c14b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287a000640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c1518f0 0x7f287c1f7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 -- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c0cfeb0 msgr2=0x7f287c14b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c0cfeb0 0x7f287c14b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 -- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c1518f0 msgr2=0x7f287c1f7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c1518f0 0x7f287c1f7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 -- 172.21.15.78:0/803462441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f287c1f8050 con 0x7f287c15ae40 2024-09-11T12:50:44.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f287b400640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f287400ed30 tx=0x7f287400c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2874004d30 con 0x7f287c15ae40 2024-09-11T12:50:44.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f287400b570 con 0x7f287c15ae40 2024-09-11T12:50:44.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f287c149320 con 0x7f287c15ae40 2024-09-11T12:50:44.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f287400b9a0 con 0x7f287c15ae40 2024-09-11T12:50:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.986+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f287c149b80 con 0x7f287c15ae40 2024-09-11T12:50:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287c073c70 con 0x7f287c15ae40 2024-09-11T12:50:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f287400bb40 con 0x7f287c15ae40 2024-09-11T12:50:44.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f2872a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 0x7f285c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f2874092be0 con 0x7f287c15ae40 2024-09-11T12:50:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f287aa00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 0x7f285c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:44.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.990+0000 7f287aa00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 0x7f285c073cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f286c000f70 tx=0x7f286c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:44.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:44.994+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f287c1f8050 con 0x7f287c15ae40 2024-09-11T12:50:45.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.194+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f287c150a90 con 0x7f287c15ae40 2024-09-11T12:50:45.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.194+0000 7f2872a00640 1 -- 172.21.15.78:0/803462441 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f287400ee20 con 0x7f287c15ae40 2024-09-11T12:50:45.199 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:45.199 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"btime":"2024-09-11T12:50:40:372090+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:45.200 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-09-11T12:50:45.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 msgr2=0x7f285c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 0x7f285c073cf0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f286c000f70 tx=0x7f286c009040 comp rx=0 tx=0).stop 2024-09-11T12:50:45.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 msgr2=0x7f287c14f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f287400ed30 tx=0x7f287400c740 comp rx=0 tx=0).stop 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 shutdown_connections 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f285c071830 0x7f285c073cf0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f287c15ae40 0x7f287c14f9f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f287c1518f0 0x7f287c1f7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 --2- 172.21.15.78:0/803462441 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f287c0cfeb0 0x7f287c14b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 >> 172.21.15.78:0/803462441 conn(0x7f287c06f3a0 msgr2=0x7f287c148090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 shutdown_connections 2024-09-11T12:50:45.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.198+0000 7f2881a00640 1 -- 172.21.15.78:0/803462441 wait complete. 2024-09-11T12:50:45.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 -- 172.21.15.78:0/1738582069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc400cd910 msgr2=0x7efc400cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 --2- 172.21.15.78:0/1738582069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc400cd910 0x7efc400cdd80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efc2c009840 tx=0x7efc2c01c820 comp rx=0 tx=0).stop 2024-09-11T12:50:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 -- 172.21.15.78:0/1738582069 shutdown_connections 2024-09-11T12:50:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 --2- 172.21.15.78:0/1738582069 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc400cd910 0x7efc400cdd80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 -- 172.21.15.78:0/1738582069 >> 172.21.15.78:0/1738582069 conn(0x7efc4006ca80 msgr2=0x7efc400b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 -- 172.21.15.78:0/1738582069 shutdown_connections 2024-09-11T12:50:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.358+0000 7efc45800640 1 -- 172.21.15.78:0/1738582069 wait complete. 2024-09-11T12:50:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 Processor -- start 2024-09-11T12:50:45.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- start start 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc400cd910 0x7efc40155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc40159fc0 0x7efc40145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc401689d0 con 0x7efc4006f200 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc40168850 con 0x7efc40159fc0 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc40168b50 con 0x7efc400cd910 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55512/0 (socket says 172.21.15.78:55512) 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 -- 172.21.15.78:0/2405214411 learned_addr learned my addr 172.21.15.78:0/2405214411 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3e000640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc400cd910 0x7efc40155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3f400640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc40159fc0 0x7efc40145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 -- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc400cd910 msgr2=0x7efc40155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc400cd910 0x7efc40155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 -- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc40159fc0 msgr2=0x7efc40145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc40159fc0 0x7efc40145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 -- 172.21.15.78:0/2405214411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc400af1e0 con 0x7efc4006f200 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc3ea00640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc3000c460 tx=0x7efc3000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30016020 con 0x7efc4006f200 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc30012960 con 0x7efc4006f200 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc30012db0 con 0x7efc4006f200 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc400aec20 con 0x7efc4006f200 2024-09-11T12:50:45.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.362+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc401553f0 con 0x7efc4006f200 2024-09-11T12:50:45.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.366+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc400cdd80 con 0x7efc4006f200 2024-09-11T12:50:45.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.366+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7efc30019020 con 0x7efc4006f200 2024-09-11T12:50:45.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.366+0000 7efc36a00640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 0x7efc20073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.366+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7efc30091c80 con 0x7efc4006f200 2024-09-11T12:50:45.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.366+0000 7efc3e000640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 0x7efc20073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.370+0000 7efc3e000640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 0x7efc20073cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0af000 tx=0x7efc2c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:45.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.370+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efc30020030 con 0x7efc4006f200 2024-09-11T12:50:45.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.542+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc40156130 con 0x7efc4006f200 2024-09-11T12:50:45.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.542+0000 7efc36a00640 1 -- 172.21.15.78:0/2405214411 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7efc3005d180 con 0x7efc4006f200 2024-09-11T12:50:45.547 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:45.547 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:28.327657+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:24.824997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:25.670550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 msgr2=0x7efc20073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 0x7efc20073cf0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7efc2c0af000 tx=0x7efc2c01b060 comp rx=0 tx=0).stop 2024-09-11T12:50:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 msgr2=0x7efc4015d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7efc3000c460 tx=0x7efc3000c920 comp rx=0 tx=0).stop 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 shutdown_connections 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7efc20071830 0x7efc20073cf0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc40159fc0 0x7efc40145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc400cd910 0x7efc40155bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 --2- 172.21.15.78:0/2405214411 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc4006f200 0x7efc4015d2a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 >> 172.21.15.78:0/2405214411 conn(0x7efc4006ca80 msgr2=0x7efc40153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 shutdown_connections 2024-09-11T12:50:45.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.546+0000 7efc45800640 1 -- 172.21.15.78:0/2405214411 wait complete. 2024-09-11T12:50:45.570 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:50:45.570 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 -- 172.21.15.78:0/997257262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc15e5b0 msgr2=0x7f27dc15ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 --2- 172.21.15.78:0/997257262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15ea20 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0b5790 tx=0x7f27c801c890 comp rx=0 tx=0).stop 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 -- 172.21.15.78:0/997257262 shutdown_connections 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 --2- 172.21.15.78:0/997257262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15ea20 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 --2- 172.21.15.78:0/997257262 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 0x7f27dc0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 --2- 172.21.15.78:0/997257262 >> v2:172.21.15.157:3300/0 conn(0x7f27dc074240 0x7f27dc15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 -- 172.21.15.78:0/997257262 >> 172.21.15.78:0/997257262 conn(0x7f27dc06ca80 msgr2=0x7f27dc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:45.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 -- 172.21.15.78:0/997257262 shutdown_connections 2024-09-11T12:50:45.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.706+0000 7f27e1200640 1 -- 172.21.15.78:0/997257262 wait complete. 2024-09-11T12:50:45.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 Processor -- start 2024-09-11T12:50:45.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- start start 2024-09-11T12:50:45.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 0x7f27dc14d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc072900 con 0x7f27dc074240 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27dc072780 con 0x7f27dc15e5b0 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27dc0b4520 con 0x7f27dc06fc50 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55534/0 (socket says 172.21.15.78:55534) 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 -- 172.21.15.78:0/3216996437 learned_addr learned my addr 172.21.15.78:0/3216996437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27da800640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9e00640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 0x7f27dc14d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 -- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 msgr2=0x7f27dc14d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 0x7f27dc14d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 -- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 msgr2=0x7f27dc15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 -- 172.21.15.78:0/3216996437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27dc0b17a0 con 0x7f27dc074240 2024-09-11T12:50:45.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27da800640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d9400640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f27c80b0040 tx=0x7f27c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c80b2040 con 0x7f27dc074240 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27c8004430 con 0x7f27dc074240 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27dc153260 con 0x7f27dc074240 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8005290 con 0x7f27dc074240 2024-09-11T12:50:45.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.710+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27dc153820 con 0x7f27dc074240 2024-09-11T12:50:45.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27dc15ba80 con 0x7f27dc074240 2024-09-11T12:50:45.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f27c8005430 con 0x7f27dc074240 2024-09-11T12:50:45.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27d2000640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 0x7f27bc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:45.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f27c812bc30 con 0x7f27dc074240 2024-09-11T12:50:45.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27d9e00640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 0x7f27bc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:45.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.714+0000 7f27d9e00640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 0x7f27bc073d20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009fc0 tx=0x7f27cc0096d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:45.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.718+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f27c80164b0 con 0x7f27dc074240 2024-09-11T12:50:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.918+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27dc15d320 con 0x7f27dc074240 2024-09-11T12:50:45.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.918+0000 7f27d2000640 1 -- 172.21.15.78:0/3216996437 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7f27c80f7810 con 0x7f27dc074240 2024-09-11T12:50:45.923 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:45.924 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"btime":"2024-09-11T12:50:40:372090+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:45.924 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-09-11T12:50:45.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 msgr2=0x7f27bc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 0x7f27bc073d20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27cc009fc0 tx=0x7f27cc0096d0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 msgr2=0x7f27dc0b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f27c80b0040 tx=0x7f27c8004040 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 shutdown_connections 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27bc071860 0x7f27bc073d20 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27dc15e5b0 0x7f27dc15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27dc074240 0x7f27dc0b0d00 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 --2- 172.21.15.78:0/3216996437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27dc06fc50 0x7f27dc14d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 >> 172.21.15.78:0/3216996437 conn(0x7f27dc06ca80 msgr2=0x7f27dc151880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 shutdown_connections 2024-09-11T12:50:45.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:45.922+0000 7f27e1200640 1 -- 172.21.15.78:0/3216996437 wait complete. 2024-09-11T12:50:45.945 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:46.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- 172.21.15.78:0/1333175197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 msgr2=0x7fb72c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 --2- 172.21.15.78:0/1333175197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c073f70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb71c009870 tx=0x7fb71c01c820 comp rx=0 tx=0).stop 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- 172.21.15.78:0/1333175197 shutdown_connections 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 --2- 172.21.15.78:0/1333175197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c06ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 --2- 172.21.15.78:0/1333175197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c073f70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- 172.21.15.78:0/1333175197 >> 172.21.15.78:0/1333175197 conn(0x7fb72c06ca80 msgr2=0x7fb72c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- 172.21.15.78:0/1333175197 shutdown_connections 2024-09-11T12:50:46.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- 172.21.15.78:0/1333175197 wait complete. 2024-09-11T12:50:46.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 Processor -- start 2024-09-11T12:50:46.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.082+0000 7fb730a00640 1 -- start start 2024-09-11T12:50:46.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c1433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c1f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c168be0 con 0x7fb72c0ccb20 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb72c168a60 con 0x7fb72c0b4780 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb72c168d60 con 0x7fb72c15bc60 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb72a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c1f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c1433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55546/0 (socket says 172.21.15.78:55546) 2024-09-11T12:50:46.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729400640 1 -- 172.21.15.78:0/201461367 learned_addr learned my addr 172.21.15.78:0/201461367 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:46.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 -- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 msgr2=0x7fb72c1f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c1f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 -- 172.21.15.78:0/201461367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 msgr2=0x7fb72c072fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 -- 172.21.15.78:0/201461367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72c0735d0 con 0x7fb72c0b4780 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb72a800640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c1f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729400640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb729e00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c1433c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0ac000 tx=0x7fb71c0a5650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0b2040 con 0x7fb72c0b4780 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb71c004470 con 0x7fb72c0b4780 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb72c1fafb0 con 0x7fb72c0b4780 2024-09-11T12:50:46.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c005280 con 0x7fb72c0b4780 2024-09-11T12:50:46.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb72c1fb570 con 0x7fb72c0b4780 2024-09-11T12:50:46.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.086+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72c073f70 con 0x7fb72c0b4780 2024-09-11T12:50:46.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.090+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fb71c005420 con 0x7fb72c0b4780 2024-09-11T12:50:46.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.090+0000 7fb722000640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 0x7fb70c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.090+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fb71c12ba50 con 0x7fb72c0b4780 2024-09-11T12:50:46.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.090+0000 7fb729400640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 0x7fb70c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.090+0000 7fb729400640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 0x7fb70c073d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb72c158f20 tx=0x7fb718009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.094+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb71c0bc060 con 0x7fb72c0b4780 2024-09-11T12:50:46.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.294+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72c151c20 con 0x7fb72c0b4780 2024-09-11T12:50:46.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.294+0000 7fb722000640 1 -- 172.21.15.78:0/201461367 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5223 (secure 0 0 0) 0x7fb71c016410 con 0x7fb72c0b4780 2024-09-11T12:50:46.299 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:46.299 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":17,"btime":"2024-09-11T12:50:40:372090+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:26.715189+0000","modified":"2024-09-11T12:50:29.729751+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4885":{"gid":4885,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6836/2884175884","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2884175884},{"type":"v1","addr":"172.21.15.157:6836","nonce":2884175884}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4885,"qdb_cluster":[4885]},"id":2}]} 2024-09-11T12:50:46.300 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 17 2024-09-11T12:50:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 msgr2=0x7fb70c073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 0x7fb70c073d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb72c158f20 tx=0x7fb718009040 comp rx=0 tx=0).stop 2024-09-11T12:50:46.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 msgr2=0x7fb72c1433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c1433c0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb71c0ac000 tx=0x7fb71c0a5650 comp rx=0 tx=0).stop 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 shutdown_connections 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb70c071860 0x7fb70c073d20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb72c15bc60 0x7fb72c1f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb72c0ccb20 0x7fb72c072fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 --2- 172.21.15.78:0/201461367 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb72c0b4780 0x7fb72c1433c0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 >> 172.21.15.78:0/201461367 conn(0x7fb72c06ca80 msgr2=0x7fb72c0701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 shutdown_connections 2024-09-11T12:50:46.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.298+0000 7fb730a00640 1 -- 172.21.15.78:0/201461367 wait complete. 2024-09-11T12:50:46.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:46.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- 172.21.15.78:0/3746908963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f303c004d50 msgr2=0x7f303c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 --2- 172.21.15.78:0/3746908963 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f303c004d50 0x7f303c0051d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f30540b2f90 tx=0x7f304c01f5b0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305a400640 1 -- 172.21.15.78:0/3746908963 reap_dead start 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- 172.21.15.78:0/3746908963 shutdown_connections 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- 172.21.15.78:0/3746908963 >> 172.21.15.78:0/3746908963 conn(0x7f305406ca80 msgr2=0x7f30540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- 172.21.15.78:0/3746908963 shutdown_connections 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- 172.21.15.78:0/3746908963 wait complete. 2024-09-11T12:50:46.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 Processor -- start 2024-09-11T12:50:46.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 -- start start 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f303c004d50 0x7f30541e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 0x7f30541433d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.462+0000 7f305c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f305415bc00 con 0x7f30540b1e20 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f305415ba80 con 0x7f30540b19a0 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f305415bd80 con 0x7f303c004d50 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55564/0 (socket says 172.21.15.78:55564) 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 -- 172.21.15.78:0/2005396647 learned_addr learned my addr 172.21.15.78:0/2005396647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3059a00640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f303c004d50 0x7f30541e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3059000640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 0x7f30541433d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 -- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f303c004d50 msgr2=0x7f30541e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f303c004d50 0x7f30541e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 -- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 msgr2=0x7f30541433d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 0x7f30541433d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 -- 172.21.15.78:0/2005396647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30541e7150 con 0x7f30540b1e20 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3059000640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 0x7f30541433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305a400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c10 tx=0x7f304c006bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c001c20 con 0x7f30540b1e20 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f304c006290 con 0x7f30540b1e20 2024-09-11T12:50:46.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f304c003600 con 0x7f30540b1e20 2024-09-11T12:50:46.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30541e7430 con 0x7f30540b1e20 2024-09-11T12:50:46.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30541e79f0 con 0x7f30540b1e20 2024-09-11T12:50:46.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f305415ed70 con 0x7f30540b1e20 2024-09-11T12:50:46.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.466+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f304c0037a0 con 0x7f30540b1e20 2024-09-11T12:50:46.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.470+0000 7f3051600640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 0x7f3034073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.470+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f304c12f2b0 con 0x7f30540b1e20 2024-09-11T12:50:46.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.470+0000 7f3059a00640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 0x7f3034073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.470+0000 7f3059a00640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 0x7f3034073d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3048009f80 tx=0x7f3048009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.474+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f304c019360 con 0x7f30540b1e20 2024-09-11T12:50:46.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.638+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3054071f70 con 0x7f30540b1e20 2024-09-11T12:50:46.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f3051600640 1 -- 172.21.15.78:0/2005396647 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16729 (secure 0 0 0) 0x7f304c0fae90 con 0x7f30540b1e20 2024-09-11T12:50:46.646 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:46.646 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":34,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:28.327657+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:24.824997+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:25.670550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 msgr2=0x7f3034073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 0x7f3034073d20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f3048009f80 tx=0x7f3048009500 comp rx=0 tx=0).stop 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 msgr2=0x7f3054143910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f304c000c10 tx=0x7f304c006bb0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 shutdown_connections 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3034071860 0x7f3034073d20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f30540b1e20 0x7f3054143910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f30540b19a0 0x7f30541433d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 --2- 172.21.15.78:0/2005396647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f303c004d50 0x7f30541e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 >> 172.21.15.78:0/2005396647 conn(0x7f305406ca80 msgr2=0x7f30541516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:46.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 shutdown_connections 2024-09-11T12:50:46.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.642+0000 7f305c400640 1 -- 172.21.15.78:0/2005396647 wait complete. 2024-09-11T12:50:46.668 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:50:46.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- 172.21.15.78:0/2805680662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40ccb20 msgr2=0x7fa1b415b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/2805680662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40ccb20 0x7fa1b415b550 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009900 tx=0x7fa1a001c8a0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- 172.21.15.78:0/2805680662 shutdown_connections 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/2805680662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40ccb20 0x7fa1b415b550 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/2805680662 >> v2:172.21.15.157:3300/0 conn(0x7fa1b40b4780 0x7fa1b4073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- 172.21.15.78:0/2805680662 >> 172.21.15.78:0/2805680662 conn(0x7fa1b406ca80 msgr2=0x7fa1b40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- 172.21.15.78:0/2805680662 shutdown_connections 2024-09-11T12:50:46.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- 172.21.15.78:0/2805680662 wait complete. 2024-09-11T12:50:46.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 Processor -- start 2024-09-11T12:50:46.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- start start 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b40b4b60 0x7fa1b415d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 0x7fa1b40b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4168fc0 con 0x7fa1b40b4780 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b4168e40 con 0x7fa1b40b4b60 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.806+0000 7fa1b8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1b4169140 con 0x7fa1b40ccb20 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55584/0 (socket says 172.21.15.78:55584) 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 -- 172.21.15.78:0/114358206 learned_addr learned my addr 172.21.15.78:0/114358206 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1400640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b40b4b60 0x7fa1b415d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b2800640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 0x7fa1b40b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 -- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 msgr2=0x7fa1b40b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 0x7fa1b40b16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 -- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b40b4b60 msgr2=0x7fa1b415d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b40b4b60 0x7fa1b415d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 -- 172.21.15.78:0/114358206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b41511c0 con 0x7fa1b40b4780 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b2800640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 0x7fa1b40b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b1e00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400eba0 tx=0x7fa1a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400eea0 con 0x7fa1b40b4780 2024-09-11T12:50:46.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1a4004d50 con 0x7fa1b40b4780 2024-09-11T12:50:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005280 con 0x7fa1b40b4780 2024-09-11T12:50:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b4150820 con 0x7fa1b40b4780 2024-09-11T12:50:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b4155230 con 0x7fa1b40b4780 2024-09-11T12:50:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b4073f70 con 0x7fa1b40b4780 2024-09-11T12:50:46.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.810+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fa1a4005420 con 0x7fa1b40b4780 2024-09-11T12:50:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.814+0000 7fa1aa000640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 0x7fa194073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.814+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fa1a4092e90 con 0x7fa1b40b4780 2024-09-11T12:50:46.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.814+0000 7fa1b1400640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 0x7fa194073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:46.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.814+0000 7fa1b1400640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 0x7fa194073cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00af000 tx=0x7fa1a001b0e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:46.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:46.814+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa1a405e960 con 0x7fa1b40b4780 2024-09-11T12:50:47.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.014+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa1b40cc280 con 0x7fa1b40b4780 2024-09-11T12:50:47.021 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:50:47.018+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:50:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.018+0000 7fa1aa000640 1 -- 172.21.15.78:0/114358206 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7fa1a4094090 con 0x7fa1b40b4780 2024-09-11T12:50:47.023 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:50:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 msgr2=0x7fa194073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 0x7fa194073cf0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00af000 tx=0x7fa1a001b0e0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 msgr2=0x7fa1b41502e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400eba0 tx=0x7fa1a400c6a0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 shutdown_connections 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa194071830 0x7fa194073cf0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1b40ccb20 0x7fa1b40b16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1b40b4b60 0x7fa1b415d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 --2- 172.21.15.78:0/114358206 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1b40b4780 0x7fa1b41502e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 >> 172.21.15.78:0/114358206 conn(0x7fa1b406ca80 msgr2=0x7fa1b40731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 shutdown_connections 2024-09-11T12:50:47.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.022+0000 7fa1b8c00640 1 -- 172.21.15.78:0/114358206 wait complete. 2024-09-11T12:50:47.049 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:50:47.094 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:50:47.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- 172.21.15.78:0/1144645039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90159350 msgr2=0x7fea9015c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:47.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- 172.21.15.78:0/1144645039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90159350 0x7fea9015c990 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fea80009840 tx=0x7fea8001c700 comp rx=0 tx=0).stop 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- 172.21.15.78:0/1144645039 shutdown_connections 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- 172.21.15.78:0/1144645039 >> v2:172.21.15.78:3300/0 conn(0x7fea900cfc90 0x7fea9006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- 172.21.15.78:0/1144645039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90159350 0x7fea9015c990 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- 172.21.15.78:0/1144645039 >> 172.21.15.78:0/1144645039 conn(0x7fea900ccd10 msgr2=0x7fea900cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- 172.21.15.78:0/1144645039 shutdown_connections 2024-09-11T12:50:47.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- 172.21.15.78:0/1144645039 wait complete. 2024-09-11T12:50:47.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 Processor -- start 2024-09-11T12:50:47.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- start start 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 0x7fea901474b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea90159350 0x7fea90151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea901686c0 con 0x7fea900cfc90 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea90168540 con 0x7fea90159350 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea90168840 con 0x7fea90075610 2024-09-11T12:50:47.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55602/0 (socket says 172.21.15.78:55602) 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 -- 172.21.15.78:0/4158576647 learned_addr learned my addr 172.21.15.78:0/4158576647 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8f400640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea90159350 0x7fea90151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8ea00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 0x7fea901474b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 -- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 msgr2=0x7fea901474b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 0x7fea901474b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 -- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea90159350 msgr2=0x7fea90151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea90159350 0x7fea90151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 -- 172.21.15.78:0/4158576647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea90147c30 con 0x7fea900cfc90 2024-09-11T12:50:47.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8ea00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 0x7fea901474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.190+0000 7fea8e000640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c430 tx=0x7fea7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c017070 con 0x7fea900cfc90 2024-09-11T12:50:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea7c003600 con 0x7fea900cfc90 2024-09-11T12:50:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea90147f10 con 0x7fea900cfc90 2024-09-11T12:50:47.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c003a50 con 0x7fea900cfc90 2024-09-11T12:50:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901484d0 con 0x7fea900cfc90 2024-09-11T12:50:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea901486d0 con 0x7fea900cfc90 2024-09-11T12:50:47.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fea7c01a070 con 0x7fea900cfc90 2024-09-11T12:50:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 0x7fea70073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.194+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7fea7c091450 con 0x7fea900cfc90 2024-09-11T12:50:47.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.198+0000 7fea8ea00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 0x7fea70073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:47.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.198+0000 7fea8ea00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 0x7fea70073cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea800af000 tx=0x7fea8001af40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:47.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.198+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fea901486d0 con 0x7fea900cfc90 2024-09-11T12:50:47.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:47.402+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fea900cca00 con 0x7fea900cfc90 2024-09-11T12:50:48.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.022+0000 7fea86a00640 1 -- 172.21.15.78:0/4158576647 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fea7c05c8f0 con 0x7fea900cfc90 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 msgr2=0x7fea70073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 0x7fea70073cf0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fea800af000 tx=0x7fea8001af40 comp rx=0 tx=0).stop 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 msgr2=0x7fea901489f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fea7c00c430 tx=0x7fea7c00c8f0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 shutdown_connections 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea70071830 0x7fea70073cf0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea90159350 0x7fea90151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea900cfc90 0x7fea901489f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 --2- 172.21.15.78:0/4158576647 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea90075610 0x7fea901474b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 >> 172.21.15.78:0/4158576647 conn(0x7fea900ccd10 msgr2=0x7fea900b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 shutdown_connections 2024-09-11T12:50:48.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.026+0000 7fea8fe00640 1 -- 172.21.15.78:0/4158576647 wait complete. 2024-09-11T12:50:48.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:50:48.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- 172.21.15.78:0/3051240000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 msgr2=0x7f21d8073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:48.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- 172.21.15.78:0/3051240000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8073950 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21d8074f00 tx=0x7f21cc01c890 comp rx=0 tx=0).stop 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- 172.21.15.78:0/3051240000 shutdown_connections 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- 172.21.15.78:0/3051240000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8072350 0x7f21d815dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- 172.21.15.78:0/3051240000 >> v2:172.21.15.157:3300/0 conn(0x7f21d8073e90 0x7f21d8071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- 172.21.15.78:0/3051240000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8073950 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- 172.21.15.78:0/3051240000 >> 172.21.15.78:0/3051240000 conn(0x7f21d80b3050 msgr2=0x7f21d80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- 172.21.15.78:0/3051240000 shutdown_connections 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- 172.21.15.78:0/3051240000 wait complete. 2024-09-11T12:50:48.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 Processor -- start 2024-09-11T12:50:48.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- start start 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 0x7f21d815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 0x7f21d815bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f21d815e7f0 con 0x7f21d8151050 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f21d815e670 con 0x7f21d8072350 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21e0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f21d815e970 con 0x7f21d8073e90 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55614/0 (socket says 172.21.15.78:55614) 2024-09-11T12:50:48.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 -- 172.21.15.78:0/4257847274 learned_addr learned my addr 172.21.15.78:0/4257847274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21ddc00640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 0x7f21d815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21dd200640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 0x7f21d815bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 -- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 msgr2=0x7f21d815bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 0x7f21d815bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 -- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 msgr2=0x7f21d815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 0x7f21d815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 -- 172.21.15.78:0/4257847274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d8147a30 con 0x7f21d8151050 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21dd200640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 0x7f21d815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21ddc00640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 0x7f21d815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21de600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21d000e9c0 tx=0x7f21d000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:48.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.194+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d000cd80 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0004b30 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0005020 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d8147d10 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d81482d0 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21d8071e10 con 0x7f21d8151050 2024-09-11T12:50:48.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f21d00051c0 con 0x7f21d8151050 2024-09-11T12:50:48.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21d5600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 0x7f21b8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:48.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.198+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5795+0+0 (secure 0 0 0) 0x7f21d0092b70 con 0x7f21d8151050 2024-09-11T12:50:48.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.202+0000 7f21ddc00640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 0x7f21b8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:48.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.202+0000 7f21ddc00640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 0x7f21b8073cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b0d0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:48.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.202+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f21d0016030 con 0x7f21d8151050 2024-09-11T12:50:48.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:48.370+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f21d8070ce0 con 0x7f21d8151050 2024-09-11T12:50:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.026+0000 7f21d5600640 1 -- 172.21.15.78:0/4257847274 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) ==== 181+0+0 (secure 0 0 0) 0x7f21d005e550 con 0x7f21d8151050 2024-09-11T12:50:49.031 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:50:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 msgr2=0x7f21b8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 0x7f21b8073cf0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b0d0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 msgr2=0x7f21d8139790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f21d000e9c0 tx=0x7f21d000ee80 comp rx=0 tx=0).stop 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 shutdown_connections 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f21b8071830 0x7f21b8073cf0 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b0d0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f21d8151050 0x7f21d8139790 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f21d8073e90 0x7f21d815bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 --2- 172.21.15.78:0/4257847274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f21d8072350 0x7f21d815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 >> 172.21.15.78:0/4257847274 conn(0x7f21d80b3050 msgr2=0x7f21d806f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:49.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 shutdown_connections 2024-09-11T12:50:49.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.030+0000 7f21e0600640 1 -- 172.21.15.78:0/4257847274 wait complete. 2024-09-11T12:50:49.061 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:50:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 -- 172.21.15.78:0/49611937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26580b5790 msgr2=0x7f26580723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 --2- 172.21.15.78:0/49611937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26580b5790 0x7f26580723f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2648009840 tx=0x7f264801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:49.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 -- 172.21.15.78:0/49611937 shutdown_connections 2024-09-11T12:50:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 --2- 172.21.15.78:0/49611937 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 0x7f2658072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 --2- 172.21.15.78:0/49611937 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26580b5790 0x7f26580723f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 -- 172.21.15.78:0/49611937 >> 172.21.15.78:0/49611937 conn(0x7f26580b3050 msgr2=0x7f26580b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 -- 172.21.15.78:0/49611937 shutdown_connections 2024-09-11T12:50:49.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.202+0000 7f265cc00640 1 -- 172.21.15.78:0/49611937 wait complete. 2024-09-11T12:50:49.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 Processor -- start 2024-09-11T12:50:49.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- start start 2024-09-11T12:50:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 0x7f26581fb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 0x7f2658159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2658167ad0 con 0x7f2658151340 2024-09-11T12:50:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2658167950 con 0x7f26580b5790 2024-09-11T12:50:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2658167c50 con 0x7f26580b5b70 2024-09-11T12:50:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55636/0 (socket says 172.21.15.78:55636) 2024-09-11T12:50:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 -- 172.21.15.78:0/135571031 learned_addr learned my addr 172.21.15.78:0/135571031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:49.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2655e00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 0x7f26581fb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f264fe00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 0x7f2658159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 -- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 msgr2=0x7f2658159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 0x7f2658159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 -- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 msgr2=0x7f26581fb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:49.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 0x7f26581fb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 -- 172.21.15.78:0/135571031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26581455a0 con 0x7f2658151340 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2655e00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 0x7f26581fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f2656800640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb10 tx=0x7f265000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2650016020 con 0x7f2658151340 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f265000ce50 con 0x7f2658151340 2024-09-11T12:50:49.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f265000ae00 con 0x7f2658151340 2024-09-11T12:50:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26581fba80 con 0x7f2658151340 2024-09-11T12:50:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.206+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26581f7a10 con 0x7f2658151340 2024-09-11T12:50:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26581517d0 con 0x7f2658151340 2024-09-11T12:50:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f2650019020 con 0x7f2658151340 2024-09-11T12:50:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f264e000640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 0x7f2634073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5357+0+0 (secure 0 0 0) 0x7f2650092160 con 0x7f2658151340 2024-09-11T12:50:49.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f2655e00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 0x7f2634073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.210+0000 7f2655e00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 0x7f2634073cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26480af000 tx=0x7f264801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.214+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2650020020 con 0x7f2658151340 2024-09-11T12:50:49.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:49.382+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2658072010 con 0x7f2658151340 2024-09-11T12:50:50.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.030+0000 7f264e000640 1 -- 172.21.15.78:0/135571031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) ==== 169+0+0 (secure 0 0 0) 0x7f26581fba80 con 0x7f2658151340 2024-09-11T12:50:50.037 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 msgr2=0x7f2634073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 0x7f2634073cf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f26480af000 tx=0x7f264801b060 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 msgr2=0x7f2658153420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f265000cb10 tx=0x7f265000cc00 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 shutdown_connections 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2634071830 0x7f2634073cf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2658151340 0x7f2658153420 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26580b5b70 0x7f2658159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 --2- 172.21.15.78:0/135571031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26580b5790 0x7f26581fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 >> 172.21.15.78:0/135571031 conn(0x7f26580b3050 msgr2=0x7f2658154260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:50.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 shutdown_connections 2024-09-11T12:50:50.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.034+0000 7f265cc00640 1 -- 172.21.15.78:0/135571031 wait complete. 2024-09-11T12:50:50.069 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:50:50.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1498505593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 msgr2=0x7f09f0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1498505593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f09f0004810 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f09f400c290 tx=0x7f09f401f6c0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0da00640 1 -- 172.21.15.78:0/1498505593 reap_dead start 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1498505593 shutdown_connections 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1498505593 >> 172.21.15.78:0/1498505593 conn(0x7f0a08074b90 msgr2=0x7f0a080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1498505593 shutdown_connections 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1498505593 wait complete. 2024-09-11T12:50:50.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 Processor -- start 2024-09-11T12:50:50.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- start start 2024-09-11T12:50:50.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a0815b6c0 0x7f0a081f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081f86f0 0x7f0a08155530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0815c5d0 con 0x7f09f0003fe0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0815c450 con 0x7f0a0815b6c0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0815c750 con 0x7f0a081f86f0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55656/0 (socket says 172.21.15.78:55656) 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 -- 172.21.15.78:0/1404924630 learned_addr learned my addr 172.21.15.78:0/1404924630 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a07e00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a0815b6c0 0x7f0a081f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0da00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081f86f0 0x7f0a08155530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 -- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081f86f0 msgr2=0x7f0a08155530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081f86f0 0x7f0a08155530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 -- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a0815b6c0 msgr2=0x7f0a081f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a0815b6c0 0x7f0a081f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.210+0000 7f0a0d000640 1 -- 172.21.15.78:0/1404924630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a081f4700 con 0x7f09f0003fe0 2024-09-11T12:50:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a0d000640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09fc011b10 tx=0x7f09fc011f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:50.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01e070 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09fc00fbf0 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a081f49e0 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc019410 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a081f4fa0 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a080d10a0 con 0x7f09f0003fe0 2024-09-11T12:50:50.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f09fc021070 con 0x7f09f0003fe0 2024-09-11T12:50:50.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 0x7f09e8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.214+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f09fc092490 con 0x7f09f0003fe0 2024-09-11T12:50:50.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.218+0000 7f0a07e00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 0x7f09e8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.218+0000 7f0a07e00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 0x7f09e8073d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09f40b0000 tx=0x7f09f401c500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:50.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.218+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f09fc05dcc0 con 0x7f09f0003fe0 2024-09-11T12:50:50.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.390+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f0a080b3700 con 0x7f09f0003fe0 2024-09-11T12:50:50.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a04c00640 1 -- 172.21.15.78:0/1404924630 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f09fc062210 con 0x7f09f0003fe0 2024-09-11T12:50:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 msgr2=0x7f09e8073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 0x7f09e8073d20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f09f40b0000 tx=0x7f09f401c500 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 msgr2=0x7f0a0815b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f09fc011b10 tx=0x7f09fc011f10 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 shutdown_connections 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f09e8071860 0x7f09e8073d20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a081f86f0 0x7f0a08155530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a0815b6c0 0x7f0a081f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 --2- 172.21.15.78:0/1404924630 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09f0003fe0 0x7f0a0815b180 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 >> 172.21.15.78:0/1404924630 conn(0x7f0a08074b90 msgr2=0x7f0a080b1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 shutdown_connections 2024-09-11T12:50:50.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.394+0000 7f0a0fa00640 1 -- 172.21.15.78:0/1404924630 wait complete. 2024-09-11T12:50:50.421 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- 172.21.15.78:0/3808438092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3015b360 msgr2=0x7f6d300b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 --2- 172.21.15.78:0/3808438092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3015b360 0x7f6d300b4db0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6d28009900 tx=0x7f6d2801c820 comp rx=0 tx=0).stop 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- 172.21.15.78:0/3808438092 shutdown_connections 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 --2- 172.21.15.78:0/3808438092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d3015b360 0x7f6d300b4db0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 --2- 172.21.15.78:0/3808438092 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30073710 0x7f6d3015ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- 172.21.15.78:0/3808438092 >> 172.21.15.78:0/3808438092 conn(0x7f6d3006ca80 msgr2=0x7f6d300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- 172.21.15.78:0/3808438092 shutdown_connections 2024-09-11T12:50:50.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- 172.21.15.78:0/3808438092 wait complete. 2024-09-11T12:50:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 Processor -- start 2024-09-11T12:50:50.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.562+0000 7f6d37200640 1 -- start start 2024-09-11T12:50:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30157a50 0x7f6d301509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d30152eb0 0x7f6d3015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3006f540 con 0x7f6d30152a30 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d3006f3c0 con 0x7f6d30152eb0 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d30164db0 con 0x7f6d30157a50 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55678/0 (socket says 172.21.15.78:55678) 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 -- 172.21.15.78:0/2740748217 learned_addr learned my addr 172.21.15.78:0/2740748217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d35200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d30152eb0 0x7f6d3015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2fe00640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30157a50 0x7f6d301509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 -- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30157a50 msgr2=0x7f6d301509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30157a50 0x7f6d301509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 -- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d30152eb0 msgr2=0x7f6d3015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d30152eb0 0x7f6d3015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:50.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 -- 172.21.15.78:0/2740748217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d30153480 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d2f400640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00d980 tx=0x7f6d1c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c017070 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d1c009d60 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c005280 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d301faff0 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d301fb5b0 con 0x7f6d30152a30 2024-09-11T12:50:50.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.566+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d3015ae20 con 0x7f6d30152a30 2024-09-11T12:50:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.570+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f6d1c005420 con 0x7f6d30152a30 2024-09-11T12:50:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.570+0000 7f6d27e00640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 0x7f6d10073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.570+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f6d1c0920f0 con 0x7f6d30152a30 2024-09-11T12:50:50.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.570+0000 7f6d2fe00640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 0x7f6d10073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.570+0000 7f6d2fe00640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 0x7f6d10073cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005530 tx=0x7f6d20009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:50.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.574+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6d1c05de30 con 0x7f6d30152a30 2024-09-11T12:50:50.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:50.798+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f6d301545c0 con 0x7f6d30152a30 2024-09-11T12:50:51.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d27e00640 1 -- 172.21.15.78:0/2740748217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v86) ==== 158+0+0 (secure 0 0 0) 0x7f6d1c066070 con 0x7f6d30152a30 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 msgr2=0x7f6d10073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 0x7f6d10073cf0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6d20005530 tx=0x7f6d20009040 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 msgr2=0x7f6d3015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00d980 tx=0x7f6d1c00de40 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 shutdown_connections 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6d10071830 0x7f6d10073cf0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d30152eb0 0x7f6d3015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d30152a30 0x7f6d3015d070 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 --2- 172.21.15.78:0/2740748217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d30157a50 0x7f6d301509c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.034+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 >> 172.21.15.78:0/2740748217 conn(0x7f6d3006ca80 msgr2=0x7f6d300ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.038+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 shutdown_connections 2024-09-11T12:50:51.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.038+0000 7f6d37200640 1 -- 172.21.15.78:0/2740748217 wait complete. 2024-09-11T12:50:51.060 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-11T12:50:51.060 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:50:51.061 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 -- 172.21.15.78:0/2180126910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 msgr2=0x7fbd94004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 --2- 172.21.15.78:0/2180126910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbd94004810 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbd98009b70 tx=0x7fbd9801d250 comp rx=0 tx=0).stop 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb2400640 1 -- 172.21.15.78:0/2180126910 reap_dead start 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 -- 172.21.15.78:0/2180126910 shutdown_connections 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 -- 172.21.15.78:0/2180126910 >> 172.21.15.78:0/2180126910 conn(0x7fbdac06ca80 msgr2=0x7fbdac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 -- 172.21.15.78:0/2180126910 shutdown_connections 2024-09-11T12:50:51.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.198+0000 7fbdb4400640 1 -- 172.21.15.78:0/2180126910 wait complete. 2024-09-11T12:50:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 Processor -- start 2024-09-11T12:50:51.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- start start 2024-09-11T12:50:51.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdac1f5740 0x7fbdac1f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdac0acdb0 0x7fbdac0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac15eb70 con 0x7fbd94003fe0 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbdac15e9f0 con 0x7fbdac1f5740 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbdac15ecf0 con 0x7fbdac0acdb0 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55688/0 (socket says 172.21.15.78:55688) 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 -- 172.21.15.78:0/4273054891 learned_addr learned my addr 172.21.15.78:0/4273054891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1000640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdac1f5740 0x7fbdac1f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb2400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdac0acdb0 0x7fbdac0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 -- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdac0acdb0 msgr2=0x7fbdac0ad230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdac0acdb0 0x7fbdac0ad230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 -- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdac1f5740 msgr2=0x7fbdac1f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdac1f5740 0x7fbdac1f5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 -- 172.21.15.78:0/4273054891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdac1f3260 con 0x7fbd94003fe0 2024-09-11T12:50:51.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb1a00640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbda0010cf0 tx=0x7fbda000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda0011640 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbda00118f0 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbda0011d40 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdac0adc90 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdac1f37b0 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.202+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdac070340 con 0x7fbd94003fe0 2024-09-11T12:50:51.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.206+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fbda001b420 con 0x7fbd94003fe0 2024-09-11T12:50:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.206+0000 7fbda9600640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 0x7fbd8c078100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.206+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fbda0092dc0 con 0x7fbd94003fe0 2024-09-11T12:50:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.206+0000 7fbdb1000640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 0x7fbd8c078100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.206+0000 7fbdb1000640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 0x7fbd8c078100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbd980ad000 tx=0x7fbd98019ff0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.210+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbda005ed40 con 0x7fbd94003fe0 2024-09-11T12:50:51.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.378+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdac072a10 con 0x7fbd94003fe0 2024-09-11T12:50:51.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.378+0000 7fbda9600640 1 -- 172.21.15.78:0/4273054891 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v37) ==== 74+0+13034 (secure 0 0 0) 0x7fbda005e640 con 0x7fbd94003fe0 2024-09-11T12:50:51.384 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:51.384 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:50.031771+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6836/2884175884":"2024-09-12T12:50:47.022051+0000","172.21.15.157:6835/2884175884":"2024-09-12T12:50:47.022051+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:51.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 msgr2=0x7fbd8c078100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 0x7fbd8c078100 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fbd980ad000 tx=0x7fbd98019ff0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 msgr2=0x7fbdac0cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbda0010cf0 tx=0x7fbda000e720 comp rx=0 tx=0).stop 2024-09-11T12:50:51.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 shutdown_connections 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fbd8c075c40 0x7fbd8c078100 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbdac0acdb0 0x7fbdac0ad230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbdac1f5740 0x7fbdac1f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 --2- 172.21.15.78:0/4273054891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd94003fe0 0x7fbdac0cb950 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 >> 172.21.15.78:0/4273054891 conn(0x7fbdac06ca80 msgr2=0x7fbdac071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 shutdown_connections 2024-09-11T12:50:51.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.382+0000 7fbdb4400640 1 -- 172.21.15.78:0/4273054891 wait complete. 2024-09-11T12:50:51.405 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:50:51.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:50:51.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- 172.21.15.78:0/924314956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48500731a0 msgr2=0x7f4850073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- 172.21.15.78:0/924314956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48500731a0 0x7f4850073610 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f485006c870 tx=0x7f483c01dbf0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- 172.21.15.78:0/924314956 shutdown_connections 2024-09-11T12:50:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- 172.21.15.78:0/924314956 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48500731a0 0x7f4850073610 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- 172.21.15.78:0/924314956 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f485015db00 0x7f485015df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- 172.21.15.78:0/924314956 >> v2:172.21.15.157:3300/0 conn(0x7f4850140190 0x7f48500cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- 172.21.15.78:0/924314956 >> 172.21.15.78:0/924314956 conn(0x7f4850075c10 msgr2=0x7f48500b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- 172.21.15.78:0/924314956 shutdown_connections 2024-09-11T12:50:51.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- 172.21.15.78:0/924314956 wait complete. 2024-09-11T12:50:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 Processor -- start 2024-09-11T12:50:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- start start 2024-09-11T12:50:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48500731a0 0x7f48501596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4850140190 0x7f48500aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f485006f010 con 0x7f485015db00 2024-09-11T12:50:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f485006ee90 con 0x7f48500731a0 2024-09-11T12:50:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f485006f190 con 0x7f4850140190 2024-09-11T12:50:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55710/0 (socket says 172.21.15.78:55710) 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f4855200640 1 -- 172.21.15.78:0/1822850073 learned_addr learned my addr 172.21.15.78:0/1822850073 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f484fe00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48500731a0 0x7f48501596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.546+0000 7f484f400640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4850140190 0x7f48500aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 -- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4850140190 msgr2=0x7f48500aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4850140190 0x7f48500aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 -- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48500731a0 msgr2=0x7f48501596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48500731a0 0x7f48501596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 -- 172.21.15.78:0/1822850073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4850073900 con 0x7f485015db00 2024-09-11T12:50:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855200640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f484800c9a0 tx=0x7f484800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848007bb0 con 0x7f485015db00 2024-09-11T12:50:51.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f484800f450 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4848007e60 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48501f85a0 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48501f77e0 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4850164350 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f484800f8a0 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 0x7f4830073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.550+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4848092570 con 0x7f485015db00 2024-09-11T12:50:51.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.554+0000 7f484fe00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 0x7f4830073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.554+0000 7f484fe00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 0x7f4830073cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48500737a0 tx=0x7f484000a580 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.554+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4848014030 con 0x7f485015db00 2024-09-11T12:50:51.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.722+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4850163d90 con 0x7f485015db00 2024-09-11T12:50:51.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.722+0000 7f4847e00640 1 -- 172.21.15.78:0/1822850073 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v37) ==== 93+0+10 (secure 0 0 0) 0x7f484805dc60 con 0x7f485015db00 2024-09-11T12:50:51.728 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:50:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 msgr2=0x7f4830073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 0x7f4830073cf0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48500737a0 tx=0x7f484000a580 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 msgr2=0x7f48500af100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f484800c9a0 tx=0x7f484800ce60 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 shutdown_connections 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4830071830 0x7f4830073cf0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f485015db00 0x7f48500af100 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4850140190 0x7f48500aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 --2- 172.21.15.78:0/1822850073 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f48500731a0 0x7f48501596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 >> 172.21.15.78:0/1822850073 conn(0x7f4850075c10 msgr2=0x7f4850163000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 shutdown_connections 2024-09-11T12:50:51.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.726+0000 7f4855c00640 1 -- 172.21.15.78:0/1822850073 wait complete. 2024-09-11T12:50:51.752 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- 172.21.15.78:0/2463756510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 msgr2=0x7f27a40051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 --2- 172.21.15.78:0/2463756510 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27a40051b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f27b800ac50 tx=0x7f27b801e070 comp rx=0 tx=0).stop 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27bfe00640 1 -- 172.21.15.78:0/2463756510 reap_dead start 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- 172.21.15.78:0/2463756510 shutdown_connections 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- 172.21.15.78:0/2463756510 >> 172.21.15.78:0/2463756510 conn(0x7f27c006ca80 msgr2=0x7f27c00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- 172.21.15.78:0/2463756510 shutdown_connections 2024-09-11T12:50:51.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- 172.21.15.78:0/2463756510 wait complete. 2024-09-11T12:50:51.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 Processor -- start 2024-09-11T12:50:51.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.890+0000 7f27c6000640 1 -- start start 2024-09-11T12:50:51.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 0x7f27c0152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 0x7f27c01f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27c0073a20 con 0x7f27a4004d50 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27c00b4520 con 0x7f27c0153270 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27c01f0530 con 0x7f27c014c540 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55742/0 (socket says 172.21.15.78:55742) 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 -- 172.21.15.78:0/2608292356 learned_addr learned my addr 172.21.15.78:0/2608292356 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bfe00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 0x7f27c01f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bea00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 0x7f27c0152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 -- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 msgr2=0x7f27c0152d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 0x7f27c0152d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 -- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 msgr2=0x7f27c01f09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 0x7f27c01f09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 -- 172.21.15.78:0/2608292356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27c01f0220 con 0x7f27a4004d50 2024-09-11T12:50:51.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bea00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 0x7f27c0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bfe00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 0x7f27c01f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27bf400640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27b0001340 tx=0x7f27b0012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0010ab0 con 0x7f27a4004d50 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27b0017030 con 0x7f27a4004d50 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27c015cd20 con 0x7f27a4004d50 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0012e60 con 0x7f27a4004d50 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27c014c160 con 0x7f27a4004d50 2024-09-11T12:50:51.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27c00ce7e0 con 0x7f27a4004d50 2024-09-11T12:50:51.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.894+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f27b0013450 con 0x7f27a4004d50 2024-09-11T12:50:51.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.898+0000 7f27b7400640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 0x7f27a0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:51.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.898+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f27b0095e70 con 0x7f27a4004d50 2024-09-11T12:50:51.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.898+0000 7f27bea00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 0x7f27a0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:51.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.898+0000 7f27bea00640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 0x7f27a0073d20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27c00ce7b0 tx=0x7f27ac006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:51.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:51.898+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f27c01f0220 con 0x7f27a4004d50 2024-09-11T12:50:52.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:52.126+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f27c00ce220 con 0x7f27a4004d50 2024-09-11T12:50:53.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.042+0000 7f27b7400640 1 -- 172.21.15.78:0/2608292356 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v88) ==== 143+0+0 (secure 0 0 0) 0x7f27b0061ed0 con 0x7f27a4004d50 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 msgr2=0x7f27a0073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 0x7f27a0073d20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27c00ce7b0 tx=0x7f27ac006b80 comp rx=0 tx=0).stop 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 msgr2=0x7f27c014bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f27b0001340 tx=0x7f27b0012b20 comp rx=0 tx=0).stop 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 shutdown_connections 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27a0071860 0x7f27a0073d20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27c0153270 0x7f27c01f09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27c014c540 0x7f27c0152d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 --2- 172.21.15.78:0/2608292356 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27a4004d50 0x7f27c014bc20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 >> 172.21.15.78:0/2608292356 conn(0x7f27c006ca80 msgr2=0x7f27c00cef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:53.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 shutdown_connections 2024-09-11T12:50:53.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.046+0000 7f27c6000640 1 -- 172.21.15.78:0/2608292356 wait complete. 2024-09-11T12:50:53.073 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 -- 172.21.15.78:0/830012577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98815a1c0 msgr2=0x7fe9880cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 --2- 172.21.15.78:0/830012577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98815a1c0 0x7fe9880cbed0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe98806cf90 tx=0x7fe97c01c800 comp rx=0 tx=0).stop 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 -- 172.21.15.78:0/830012577 shutdown_connections 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 --2- 172.21.15.78:0/830012577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98815a1c0 0x7fe9880cbed0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 --2- 172.21.15.78:0/830012577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe98813d250 0x7fe9880cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 --2- 172.21.15.78:0/830012577 >> v2:172.21.15.157:3300/0 conn(0x7fe9880b4780 0x7fe98815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 -- 172.21.15.78:0/830012577 >> 172.21.15.78:0/830012577 conn(0x7fe98806ca80 msgr2=0x7fe9880b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 -- 172.21.15.78:0/830012577 shutdown_connections 2024-09-11T12:50:53.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.214+0000 7fe98fe00640 1 -- 172.21.15.78:0/830012577 wait complete. 2024-09-11T12:50:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 Processor -- start 2024-09-11T12:50:53.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- start start 2024-09-11T12:50:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9880b4780 0x7fe98813c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98815a1c0 0x7fe98813bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe98806eed0 con 0x7fe98813d250 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe98806ed50 con 0x7fe98815a1c0 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe98806f050 con 0x7fe9880b4780 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55764/0 (socket says 172.21.15.78:55764) 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 -- 172.21.15.78:0/1867380763 learned_addr learned my addr 172.21.15.78:0/1867380763 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 -- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9880b4780 msgr2=0x7fe98813c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9880b4780 0x7fe98813c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 -- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98815a1c0 msgr2=0x7fe98813bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98815a1c0 0x7fe98813bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 -- 172.21.15.78:0/1867380763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe98813c450 con 0x7fe98813d250 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98ca00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0b0040 tx=0x7fe97c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:53.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c0b2040 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe97c003600 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe98815c9e0 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97c003a30 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe98815cfa0 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.218+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe98815e1c0 con 0x7fe98813d250 2024-09-11T12:50:53.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.222+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fe97c003bd0 con 0x7fe98813d250 2024-09-11T12:50:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.222+0000 7fe985600640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 0x7fe96c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.222+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fe97c12bc70 con 0x7fe98813d250 2024-09-11T12:50:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.222+0000 7fe98d400640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 0x7fe96c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.222+0000 7fe98d400640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 0x7fe96c073cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe97800a9b0 tx=0x7fe978005c80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:53.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.226+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe97c0bd060 con 0x7fe98813d250 2024-09-11T12:50:53.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.390+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe9880b3960 con 0x7fe98813d250 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe985600640 1 -- 172.21.15.78:0/1867380763 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7fe97c0164b0 con 0x7fe98813d250 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:50:53.397 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:50:53.398 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:50:53.399 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 msgr2=0x7fe96c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 0x7fe96c073cf0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe97800a9b0 tx=0x7fe978005c80 comp rx=0 tx=0).stop 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 msgr2=0x7fe98813b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe97c0b0040 tx=0x7fe97c004040 comp rx=0 tx=0).stop 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 shutdown_connections 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe96c071830 0x7fe96c073cf0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe98815a1c0 0x7fe98813bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe98813d250 0x7fe98813b7e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 --2- 172.21.15.78:0/1867380763 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9880b4780 0x7fe98813c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 >> 172.21.15.78:0/1867380763 conn(0x7fe98806ca80 msgr2=0x7fe988159830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 shutdown_connections 2024-09-11T12:50:53.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.394+0000 7fe98fe00640 1 -- 172.21.15.78:0/1867380763 wait complete. 2024-09-11T12:50:53.418 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:50:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- 172.21.15.78:0/3143813502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e940ce150 msgr2=0x7f2e9415e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 --2- 172.21.15.78:0/3143813502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e940ce150 0x7f2e9415e4d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2e9406d280 tx=0x7f2e8c01c5e0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- 172.21.15.78:0/3143813502 shutdown_connections 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 --2- 172.21.15.78:0/3143813502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e940ce150 0x7f2e9415e4d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 --2- 172.21.15.78:0/3143813502 >> v2:172.21.15.157:3300/0 conn(0x7f2e940b1f50 0x7f2e940cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 --2- 172.21.15.78:0/3143813502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e941518b0 0x7f2e940b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- 172.21.15.78:0/3143813502 >> 172.21.15.78:0/3143813502 conn(0x7f2e94071dd0 msgr2=0x7f2e940721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- 172.21.15.78:0/3143813502 shutdown_connections 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- 172.21.15.78:0/3143813502 wait complete. 2024-09-11T12:50:53.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 Processor -- start 2024-09-11T12:50:53.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.558+0000 7f2e9b600640 1 -- start start 2024-09-11T12:50:53.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e940b1f50 0x7f2e94147b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 0x7f2e94154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940b33e0 con 0x7f2e941518b0 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e940b3260 con 0x7f2e940b1f50 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e940b3560 con 0x7f2e940ce150 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55780/0 (socket says 172.21.15.78:55780) 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 -- 172.21.15.78:0/1215848995 learned_addr learned my addr 172.21.15.78:0/1215848995 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e98c00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e940b1f50 0x7f2e94147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e93e00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 0x7f2e94154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 -- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 msgr2=0x7f2e94154c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 0x7f2e94154c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 -- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e940b1f50 msgr2=0x7f2e94147b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e940b1f50 0x7f2e94147b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 -- 172.21.15.78:0/1215848995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e941f7470 con 0x7f2e941518b0 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e93e00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 0x7f2e94154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:53.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e99600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c01ce80 tx=0x7f2e8c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c0ad040 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e8c0a9bf0 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8c003600 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e941f7750 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e941551b0 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e9415e4d0 con 0x7f2e941518b0 2024-09-11T12:50:53.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.562+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f2e8c0b0070 con 0x7f2e941518b0 2024-09-11T12:50:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.566+0000 7f2e90c00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 0x7f2e74073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:53.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.566+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f2e8c12c2a0 con 0x7f2e941518b0 2024-09-11T12:50:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.566+0000 7f2e98c00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 0x7f2e74073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:53.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.566+0000 7f2e98c00640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 0x7f2e74073cf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009e60 tx=0x7f2e840095b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:53.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.570+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2e8c0c3030 con 0x7f2e941518b0 2024-09-11T12:50:53.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.738+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e94076290 con 0x7f2e941518b0 2024-09-11T12:50:53.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e90c00640 1 -- 172.21.15.78:0/1215848995 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f2e8c016490 con 0x7f2e941518b0 2024-09-11T12:50:53.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 msgr2=0x7f2e74073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 0x7f2e74073cf0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2e84009e60 tx=0x7f2e840095b0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 msgr2=0x7f2e94145220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c01ce80 tx=0x7f2e8c004620 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 shutdown_connections 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e74071830 0x7f2e74073cf0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e941518b0 0x7f2e94145220 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e940ce150 0x7f2e94154c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 --2- 172.21.15.78:0/1215848995 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e940b1f50 0x7f2e94147b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 >> 172.21.15.78:0/1215848995 conn(0x7f2e94071dd0 msgr2=0x7f2e9415c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 shutdown_connections 2024-09-11T12:50:53.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:53.742+0000 7f2e9b600640 1 -- 172.21.15.78:0/1215848995 wait complete. 2024-09-11T12:50:53.767 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:50:53.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:53.863 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:50:53.953 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:50:53.953 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:53.953 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:53.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:50:53.962 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:53.962 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:53.971 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:50:53.971 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:50:53.971 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:54.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:50:54.016 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:50:54.016 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:50:54.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:50:54.025 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:50:54.025 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:50:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 -- 172.21.15.78:0/2110915789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea415cad0 msgr2=0x7f4ea415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 --2- 172.21.15.78:0/2110915789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea415cad0 0x7f4ea415cf60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f4e90009840 tx=0x7f4e9001c740 comp rx=0 tx=0).stop 2024-09-11T12:50:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 -- 172.21.15.78:0/2110915789 shutdown_connections 2024-09-11T12:50:54.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 --2- 172.21.15.78:0/2110915789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea415cad0 0x7f4ea415cf60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 -- 172.21.15.78:0/2110915789 >> 172.21.15.78:0/2110915789 conn(0x7f4ea406ca80 msgr2=0x7f4ea40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 -- 172.21.15.78:0/2110915789 shutdown_connections 2024-09-11T12:50:54.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.198+0000 7f4ea9400640 1 -- 172.21.15.78:0/2110915789 wait complete. 2024-09-11T12:50:54.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 Processor -- start 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 -- start start 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 0x7f4ea4148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea415cad0 0x7f4ea4150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40cd460 con 0x7f4ea40b4520 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ea40cd2e0 con 0x7f4ea40aed20 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ea40cd5e0 con 0x7f4ea415cad0 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 0x7f4ea4148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43008/0 (socket says 172.21.15.78:43008) 2024-09-11T12:50:54.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea1e00640 1 -- 172.21.15.78:0/1207463997 learned_addr learned my addr 172.21.15.78:0/1207463997 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea3200640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea415cad0 0x7f4ea4150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 -- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea415cad0 msgr2=0x7f4ea4150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea415cad0 0x7f4ea4150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 -- 172.21.15.78:0/1207463997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 msgr2=0x7f4ea4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 -- 172.21.15.78:0/1207463997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ea4151b00 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea1e00640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea2800640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 0x7f4ea4148720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c7a0 tx=0x7f4e9400cc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e94003b80 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e94003d20 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ea4151de0 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.202+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e9400aab0 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ea414fb40 con 0x7f4ea40aed20 2024-09-11T12:50:54.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ea40cdb70 con 0x7f4ea40aed20 2024-09-11T12:50:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f4e9400ac50 con 0x7f4ea40aed20 2024-09-11T12:50:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4e9aa00640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 0x7f4e84073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7f4e94091c40 con 0x7f4ea40aed20 2024-09-11T12:50:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.206+0000 7f4ea1e00640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 0x7f4e84073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.210+0000 7f4ea1e00640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 0x7f4e84073cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e9001a370 tx=0x7f4e900b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:54.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.210+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4e9405d550 con 0x7f4ea40aed20 2024-09-11T12:50:54.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.410+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ea40b0a40 con 0x7f4ea40aed20 2024-09-11T12:50:54.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.410+0000 7f4e9aa00640 1 -- 172.21.15.78:0/1207463997 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3911 (secure 0 0 0) 0x7f4e94061aa0 con 0x7f4ea40aed20 2024-09-11T12:50:54.416 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:54.416 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":19,"btime":"2024-09-11T12:50:48:024487+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4839,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/1229501410","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1229501410},{"type":"v1","addr":"172.21.15.78:6833","nonce":1229501410}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":4842,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/428247846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":428247846},{"type":"v1","addr":"172.21.15.78:6835","nonce":428247846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4880,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/3519565726","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":3519565726},{"type":"v1","addr":"172.21.15.157:6836","nonce":3519565726}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19},{"gid":4882,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/539804984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":539804984},{"type":"v1","addr":"172.21.15.157:6837","nonce":539804984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[]} 2024-09-11T12:50:54.417 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 19 2024-09-11T12:50:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 msgr2=0x7f4e84073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 0x7f4e84073cf0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f4e9001a370 tx=0x7f4e900b2040 comp rx=0 tx=0).stop 2024-09-11T12:50:54.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 msgr2=0x7f4ea4148720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 0x7f4ea4148720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4e9400c7a0 tx=0x7f4e9400cc60 comp rx=0 tx=0).stop 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 shutdown_connections 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4e84071830 0x7f4e84073cf0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ea415cad0 0x7f4ea4150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ea40b4520 0x7f4ea4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 --2- 172.21.15.78:0/1207463997 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ea40aed20 0x7f4ea4148720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 >> 172.21.15.78:0/1207463997 conn(0x7f4ea406ca80 msgr2=0x7f4ea415a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 shutdown_connections 2024-09-11T12:50:54.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.414+0000 7f4ea9400640 1 -- 172.21.15.78:0/1207463997 wait complete. 2024-09-11T12:50:54.437 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:50:54.437 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:50:54.438 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:50:54.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:54.438 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:50:54.438 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:50:54.438 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:50:54.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:54.439 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:50:54.439 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:50:54.439 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:50:54.439 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:54.439 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:50:54.439 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:50:54.439 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:50:54.439 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:50:54.450 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:50:54.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:50:54.451 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:50:54.451 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:50:54.459 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:50:54.460 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:50:54.460 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:50:54.461 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:50:54.495 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:50:54.495 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:54.496 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:50:54.496 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:54.507 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:50:54.507 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:54.508 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:50:54.509 DEBUG:teuthology.parallel:result is None 2024-09-11T12:50:54.509 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:50:54.581 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:50:54.581 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:50:54.601 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:50:54.601 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:50:54.603 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-09-11T12:50:54.600+0000 7f031e200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-09-11T12:50:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.654+0000 7fccf8200640 1 -- 172.21.15.78:0/1119430889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf0070c30 msgr2=0x7fccf00cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.654+0000 7fccf8200640 1 --2- 172.21.15.78:0/1119430889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf0070c30 0x7fccf00cd910 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009840 tx=0x7fccdc01c820 comp rx=0 tx=0).stop 2024-09-11T12:50:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- 172.21.15.78:0/1119430889 shutdown_connections 2024-09-11T12:50:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 --2- 172.21.15.78:0/1119430889 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf0070c30 0x7fccf00cd910 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 --2- 172.21.15.78:0/1119430889 >> v2:172.21.15.157:3300/0 conn(0x7fccf0159fc0 0x7fccf00706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- 172.21.15.78:0/1119430889 >> 172.21.15.78:0/1119430889 conn(0x7fccf006ca80 msgr2=0x7fccf00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:54.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- 172.21.15.78:0/1119430889 shutdown_connections 2024-09-11T12:50:54.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- 172.21.15.78:0/1119430889 wait complete. 2024-09-11T12:50:54.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 Processor -- start 2024-09-11T12:50:54.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- start start 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf00cde50 0x7fccf015b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf014f390 0x7fccf00ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fccf006f750 con 0x7fccf0159fc0 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fccf006f5d0 con 0x7fccf014f390 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf8200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fccf006f8d0 con 0x7fccf00cde50 2024-09-11T12:50:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43052/0 (socket says 172.21.15.78:43052) 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 -- 172.21.15.78:0/3822286539 learned_addr learned my addr 172.21.15.78:0/3822286539 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf5800640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf00cde50 0x7fccf015b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf6200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf014f390 0x7fccf00ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 -- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf00cde50 msgr2=0x7fccf015b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf00cde50 0x7fccf015b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 -- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf014f390 msgr2=0x7fccf00ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf014f390 0x7fccf00ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 -- 172.21.15.78:0/3822286539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccf0158dc0 con 0x7fccf0159fc0 2024-09-11T12:50:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.658+0000 7fccf4e00640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009790 tx=0x7fccdc0a55f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0b2040 con 0x7fccf0159fc0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccdc0043b0 con 0x7fccf0159fc0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccdc005280 con 0x7fccf0159fc0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccf01590a0 con 0x7fccf0159fc0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccf015bf20 con 0x7fccf0159fc0 2024-09-11T12:50:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccf00cd910 con 0x7fccf0159fc0 2024-09-11T12:50:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fccdc0be060 con 0x7fccf0159fc0 2024-09-11T12:50:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 0x7fccd0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 4943+0+0 (secure 0 0 0) 0x7fccdc12b530 con 0x7fccf0159fc0 2024-09-11T12:50:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fccf5800640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 0x7fccd0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.662+0000 7fccf5800640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 0x7fccd0073cf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009ec0 tx=0x7fcce4009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.666+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fccdc016410 con 0x7fccf0159fc0 2024-09-11T12:50:54.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:54.842+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fccf00b1e30 con 0x7fccf0159fc0 2024-09-11T12:50:55.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.058+0000 7fcced600640 1 -- 172.21.15.78:0/3822286539 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v39) ==== 97+0+0 (secure 0 0 0) 0x7fccdc0f6dc0 con 0x7fccf0159fc0 2024-09-11T12:50:55.065 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 msgr2=0x7fccd0073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 0x7fccd0073cf0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcce4009ec0 tx=0x7fcce4009500 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 msgr2=0x7fccf014ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fccdc009790 tx=0x7fccdc0a55f0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 shutdown_connections 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fccd0071830 0x7fccd0073cf0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fccf014f390 0x7fccf00ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fccf0159fc0 0x7fccf014ee50 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 --2- 172.21.15.78:0/3822286539 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fccf00cde50 0x7fccf015b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 >> 172.21.15.78:0/3822286539 conn(0x7fccf006ca80 msgr2=0x7fccf0152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 shutdown_connections 2024-09-11T12:50:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.062+0000 7fccf8200640 1 -- 172.21.15.78:0/3822286539 wait complete. 2024-09-11T12:50:55.090 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:50:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 -- 172.21.15.78:0/2421896376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 msgr2=0x7facb400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 --2- 172.21.15.78:0/2421896376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facb400b9d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7facd00ccf00 tx=0x7facc801d630 comp rx=0 tx=0).stop 2024-09-11T12:50:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd5000640 1 -- 172.21.15.78:0/2421896376 reap_dead start 2024-09-11T12:50:55.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 -- 172.21.15.78:0/2421896376 shutdown_connections 2024-09-11T12:50:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 -- 172.21.15.78:0/2421896376 >> 172.21.15.78:0/2421896376 conn(0x7facd006ca80 msgr2=0x7facd00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 -- 172.21.15.78:0/2421896376 shutdown_connections 2024-09-11T12:50:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 -- 172.21.15.78:0/2421896376 wait complete. 2024-09-11T12:50:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.230+0000 7facd7000640 1 Processor -- start 2024-09-11T12:50:55.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- start start 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 0x7facd0155ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 0x7facd015dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7facd00743c0 con 0x7facb40051d0 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7facd0074240 con 0x7facd0158cd0 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7facd006f400 con 0x7facd015d750 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43078/0 (socket says 172.21.15.78:43078) 2024-09-11T12:50:55.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 -- 172.21.15.78:0/4245213127 learned_addr learned my addr 172.21.15.78:0/4245213127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd5000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 0x7facd015dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 -- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 msgr2=0x7facd015dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccf400640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 0x7facd0155ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 0x7facd015dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 -- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 msgr2=0x7facd0155ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 0x7facd0155ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 -- 172.21.15.78:0/4245213127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facd00c53a0 con 0x7facb40051d0 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd5000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 0x7facd015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccf400640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 0x7facd0155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7faccfe00640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7facc0003e20 tx=0x7facc0001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0011700 con 0x7facb40051d0 2024-09-11T12:50:55.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc0016020 con 0x7facb40051d0 2024-09-11T12:50:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facd01f7810 con 0x7facb40051d0 2024-09-11T12:50:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc0011b80 con 0x7facb40051d0 2024-09-11T12:50:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facd01f7dd0 con 0x7facb40051d0 2024-09-11T12:50:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.234+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facd00ccfd0 con 0x7facb40051d0 2024-09-11T12:50:55.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.238+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7facc001b410 con 0x7facb40051d0 2024-09-11T12:50:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.238+0000 7facc7e00640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 0x7facb0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.238+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7facc0092b60 con 0x7facb40051d0 2024-09-11T12:50:55.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.238+0000 7faccf400640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 0x7facb0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.238+0000 7faccf400640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 0x7facb0073d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7facbc0046e0 tx=0x7facbc000ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.242+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7facc0020020 con 0x7facb40051d0 2024-09-11T12:50:55.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7facd0073700 con 0x7facb40051d0 2024-09-11T12:50:55.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facc7e00640 1 -- 172.21.15.78:0/4245213127 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) ==== 80+0+5304 (secure 0 0 0) 0x7facc005e980 con 0x7facb40051d0 2024-09-11T12:50:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:55.470 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:50:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:50:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:50:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:50:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:50:55.485 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:50:55.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 msgr2=0x7facb0073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 0x7facb0073d20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7facbc0046e0 tx=0x7facbc000ee0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 msgr2=0x7facd00c4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7facc0003e20 tx=0x7facc0001810 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 shutdown_connections 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7facb0071860 0x7facb0073d20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7facd015d750 0x7facd015dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7facd0158cd0 0x7facd0155ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 --2- 172.21.15.78:0/4245213127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7facb40051d0 0x7facd00c4e60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 >> 172.21.15.78:0/4245213127 conn(0x7facd006ca80 msgr2=0x7facd00728c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 shutdown_connections 2024-09-11T12:50:55.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.466+0000 7facd7000640 1 -- 172.21.15.78:0/4245213127 wait complete. 2024-09-11T12:50:55.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:50:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- 172.21.15.78:0/1771507508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de8070c30 msgr2=0x7f5de80cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 --2- 172.21.15.78:0/1771507508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de8070c30 0x7f5de80cd910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009900 tx=0x7f5dd401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- 172.21.15.78:0/1771507508 shutdown_connections 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 --2- 172.21.15.78:0/1771507508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de8070c30 0x7f5de80cd910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 --2- 172.21.15.78:0/1771507508 >> v2:172.21.15.157:3300/0 conn(0x7f5de8159fc0 0x7f5de80706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- 172.21.15.78:0/1771507508 >> 172.21.15.78:0/1771507508 conn(0x7f5de806ca80 msgr2=0x7f5de80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- 172.21.15.78:0/1771507508 shutdown_connections 2024-09-11T12:50:55.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- 172.21.15.78:0/1771507508 wait complete. 2024-09-11T12:50:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 Processor -- start 2024-09-11T12:50:55.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.630+0000 7f5def600640 1 -- start start 2024-09-11T12:50:55.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de80cde50 0x7f5de81471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 0x7f5de81fb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8168040 con 0x7f5de8159fc0 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5de80b5330 con 0x7f5de80cde50 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5de81681c0 con 0x7f5de80addb0 2024-09-11T12:50:55.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43094/0 (socket says 172.21.15.78:43094) 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 -- 172.21.15.78:0/3907622385 learned_addr learned my addr 172.21.15.78:0/3907622385 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5decc00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de80cde50 0x7f5de81471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5ded600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 0x7f5de81fb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 -- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 msgr2=0x7f5de81fb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 0x7f5de81fb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 -- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de80cde50 msgr2=0x7f5de81471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de80cde50 0x7f5de81471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 -- 172.21.15.78:0/3907622385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de80ae230 con 0x7f5de8159fc0 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5ded600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 0x7f5de81fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:55.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de7e00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009850 tx=0x7f5dd40a6b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40b2040 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dd40043f0 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de80ae510 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4005290 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de80aead0 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.634+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5de80cd910 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.638+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f5dd40bd060 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.638+0000 7f5de4c00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 0x7f5dc8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.638+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f5dd412b4c0 con 0x7f5de8159fc0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.638+0000 7f5decc00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 0x7f5dc8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.638+0000 7f5decc00640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 0x7f5dc8073cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5de81597a0 tx=0x7f5dd8009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.642+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5dd40164d0 con 0x7f5de8159fc0 2024-09-11T12:50:55.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.810+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5de8151250 con 0x7f5de8159fc0 2024-09-11T12:50:55.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.810+0000 7f5de4c00640 1 -- 172.21.15.78:0/3907622385 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v41) ==== 84+0+65 (secure 0 0 0) 0x7f5dd40f73f0 con 0x7f5de8159fc0 2024-09-11T12:50:55.816 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:55.816 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:50:55.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:50:55.817 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:50:55.817 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:50:55.817 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:50:55.817 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:50:55.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.810+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 msgr2=0x7f5dc8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.810+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 0x7f5dc8073cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5de81597a0 tx=0x7f5dd8009500 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 msgr2=0x7f5de80ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009850 tx=0x7f5dd40a6b60 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 shutdown_connections 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5dc8071830 0x7f5dc8073cf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5de80addb0 0x7f5de81fb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5de8159fc0 0x7f5de80ad870 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 --2- 172.21.15.78:0/3907622385 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5de80cde50 0x7f5de81471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 >> 172.21.15.78:0/3907622385 conn(0x7f5de806ca80 msgr2=0x7f5de8153de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 shutdown_connections 2024-09-11T12:50:55.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.814+0000 7f5def600640 1 -- 172.21.15.78:0/3907622385 wait complete. 2024-09-11T12:50:55.836 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:50:55.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- 172.21.15.78:0/429306887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7710073cb0 msgr2=0x7f77100c6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- 172.21.15.78:0/429306887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7710073cb0 0x7f77100c6310 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7708009900 tx=0x7f770801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- 172.21.15.78:0/429306887 shutdown_connections 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- 172.21.15.78:0/429306887 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7710073cb0 0x7f77100c6310 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- 172.21.15.78:0/429306887 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f771015b660 0x7f77100c5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- 172.21.15.78:0/429306887 >> 172.21.15.78:0/429306887 conn(0x7f77100cd190 msgr2=0x7f77100cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:55.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- 172.21.15.78:0/429306887 shutdown_connections 2024-09-11T12:50:55.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- 172.21.15.78:0/429306887 wait complete. 2024-09-11T12:50:55.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 Processor -- start 2024-09-11T12:50:55.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- start start 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77100738d0 0x7f771013a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7710073cb0 0x7f77100caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7710167830 con 0x7f771015b660 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f77101676b0 con 0x7f7710073cb0 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7716e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f77101679b0 con 0x7f77100738d0 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7714e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7714e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43106/0 (socket says 172.21.15.78:43106) 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f7714e00640 1 -- 172.21.15.78:0/2396867703 learned_addr learned my addr 172.21.15.78:0/2396867703 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f770f400640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7710073cb0 0x7f77100caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.978+0000 7f770fe00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77100738d0 0x7f771013a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 -- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77100738d0 msgr2=0x7f771013a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77100738d0 0x7f771013a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 -- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7710073cb0 msgr2=0x7f77100caf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7710073cb0 0x7f77100caf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:55.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 -- 172.21.15.78:0/2396867703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7710137480 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7714e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7708009230 tx=0x7f77080a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77080b2040 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7708004230 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77101f89f0 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7708003600 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77101f8fb0 con 0x7f771015b660 2024-09-11T12:50:55.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771015af50 con 0x7f771015b660 2024-09-11T12:50:55.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.982+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f77080bf020 con 0x7f771015b660 2024-09-11T12:50:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.986+0000 7f7707e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 0x7f76f0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.986+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4857+0+0 (secure 0 0 0) 0x7f770812c260 con 0x7f771015b660 2024-09-11T12:50:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.986+0000 7f770fe00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 0x7f76f0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.986+0000 7f770fe00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 0x7f76f0073cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f77000012f0 tx=0x7f7700009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:55.986+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f77080164d0 con 0x7f771015b660 2024-09-11T12:50:56.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:56.158+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f771006fd70 con 0x7f771015b660 2024-09-11T12:50:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.074+0000 7f7707e00640 1 -- 172.21.15.78:0/2396867703 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v42) ==== 138+0+0 (secure 0 0 0) 0x7f77080f7bc0 con 0x7f771015b660 2024-09-11T12:50:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:50:57.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 msgr2=0x7f76f0073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:57.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 0x7f76f0073cf0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f77000012f0 tx=0x7f7700009040 comp rx=0 tx=0).stop 2024-09-11T12:50:57.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 msgr2=0x7f77100cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:57.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7708009230 tx=0x7f77080a57a0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 shutdown_connections 2024-09-11T12:50:57.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f76f0071830 0x7f76f0073cf0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f771015b660 0x7f77100cb440 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7710073cb0 0x7f77100caf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 --2- 172.21.15.78:0/2396867703 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77100738d0 0x7f771013a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 >> 172.21.15.78:0/2396867703 conn(0x7f77100cd190 msgr2=0x7f771013f310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 shutdown_connections 2024-09-11T12:50:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.078+0000 7f7716e00640 1 -- 172.21.15.78:0/2396867703 wait complete. 2024-09-11T12:50:57.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:50:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- 172.21.15.78:0/1564430523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540cf790 msgr2=0x7f395415e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 --2- 172.21.15.78:0/1564430523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540cf790 0x7f395415e6f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3944009840 tx=0x7f394401c820 comp rx=0 tx=0).stop 2024-09-11T12:50:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- 172.21.15.78:0/1564430523 shutdown_connections 2024-09-11T12:50:57.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 --2- 172.21.15.78:0/1564430523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39540cf790 0x7f395415e6f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- 172.21.15.78:0/1564430523 >> 172.21.15.78:0/1564430523 conn(0x7f39540b2ec0 msgr2=0x7f39540b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- 172.21.15.78:0/1564430523 shutdown_connections 2024-09-11T12:50:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- 172.21.15.78:0/1564430523 wait complete. 2024-09-11T12:50:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 Processor -- start 2024-09-11T12:50:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- start start 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 0x7f395415d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 0x7f3954159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3954168e70 con 0x7f3954159bb0 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3954168cf0 con 0x7f39540cf790 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3954168ff0 con 0x7f39540cd290 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43124/0 (socket says 172.21.15.78:43124) 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 -- 172.21.15.78:0/153546803 learned_addr learned my addr 172.21.15.78:0/153546803 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f3959600640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 0x7f395415d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f3958c00640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 0x7f3954159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 -- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 msgr2=0x7f395415d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 0x7f395415d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 -- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 msgr2=0x7f3954159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 0x7f3954159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f395a000640 1 -- 172.21.15.78:0/153546803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3954159fd0 con 0x7f3954159bb0 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f3959600640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 0x7f395415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.258+0000 7f3958c00640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 0x7f3954159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f395a000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c780 tx=0x7f394c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:57.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c003b80 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f394c003d20 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c00aab0 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395413ee70 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395413f430 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f395415e6f0 con 0x7f3954159bb0 2024-09-11T12:50:57.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f394c00ac50 con 0x7f3954159bb0 2024-09-11T12:50:57.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.262+0000 7f3951600640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 0x7f3934073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:57.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.266+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5227+0+0 (secure 0 0 0) 0x7f394c092820 con 0x7f3954159bb0 2024-09-11T12:50:57.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.266+0000 7f3959600640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 0x7f3934073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:57.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.266+0000 7f3959600640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 0x7f3934073cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39440af000 tx=0x7f394401b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:57.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.266+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3954159fd0 con 0x7f3954159bb0 2024-09-11T12:50:57.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:57.434+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f395406f2c0 con 0x7f3954159bb0 2024-09-11T12:50:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.078+0000 7f3951600640 1 -- 172.21.15.78:0/153546803 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v43) ==== 170+0+0 (secure 0 0 0) 0x7f394c05e660 con 0x7f3954159bb0 2024-09-11T12:50:58.084 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 msgr2=0x7f3934073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 0x7f3934073cf0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f39440af000 tx=0x7f394401b060 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 msgr2=0x7f395415af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f394c00c780 tx=0x7f394c00cc40 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 shutdown_connections 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3934071830 0x7f3934073cf0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3954159bb0 0x7f395415af20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39540cf790 0x7f3954159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 --2- 172.21.15.78:0/153546803 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39540cd290 0x7f395415d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 >> 172.21.15.78:0/153546803 conn(0x7f39540b2ec0 msgr2=0x7f3954140400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:58.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 shutdown_connections 2024-09-11T12:50:58.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.082+0000 7f395c000640 1 -- 172.21.15.78:0/153546803 wait complete. 2024-09-11T12:50:58.122 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:50:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- 172.21.15.78:0/1921006742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 msgr2=0x7fa78415dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 --2- 172.21.15.78:0/1921006742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa78415dcc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa770009900 tx=0x7fa77001c7e0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- 172.21.15.78:0/1921006742 shutdown_connections 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 --2- 172.21.15.78:0/1921006742 >> v2:172.21.15.157:3300/0 conn(0x7fa78415eaf0 0x7fa784072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 --2- 172.21.15.78:0/1921006742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa78415dcc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- 172.21.15.78:0/1921006742 >> 172.21.15.78:0/1921006742 conn(0x7fa784073bb0 msgr2=0x7fa784073fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- 172.21.15.78:0/1921006742 shutdown_connections 2024-09-11T12:50:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- 172.21.15.78:0/1921006742 wait complete. 2024-09-11T12:50:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 Processor -- start 2024-09-11T12:50:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.262+0000 7fa783e00640 1 -- start start 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 0x7fa7840b0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 0x7fa7841fabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa78416ac10 con 0x7fa7840747d0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa78416aa90 con 0x7fa78415eaf0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa78416ad90 con 0x7fa78415d8e0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 0x7fa7840b0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 0x7fa7841fabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43150/0 (socket says 172.21.15.78:43150) 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 0x7fa7840b0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:59038/0 (socket says 172.21.15.78:59038) 2024-09-11T12:50:58.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 -- 172.21.15.78:0/1490588031 learned_addr learned my addr 172.21.15.78:0/1490588031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 -- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 msgr2=0x7fa7840b0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 0x7fa7840b0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 -- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 msgr2=0x7fa7841fabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 0x7fa7841fabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 -- 172.21.15.78:0/1490588031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7841fb6b0 con 0x7fa7840747d0 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783400640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 0x7fa7841fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:50:58.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa782a00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c500 tx=0x7fa77400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774017070 con 0x7fa7840747d0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7740053c0 con 0x7fa7840747d0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774012890 con 0x7fa7840747d0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7841fb990 con 0x7fa7840747d0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa784154e90 con 0x7fa7840747d0 2024-09-11T12:50:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.266+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78415dcc0 con 0x7fa7840747d0 2024-09-11T12:50:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.270+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fa77401a070 con 0x7fa7840747d0 2024-09-11T12:50:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.270+0000 7fa77aa00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 0x7fa764073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:58.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.270+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5609+0+0 (secure 0 0 0) 0x7fa774091620 con 0x7fa7840747d0 2024-09-11T12:50:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.270+0000 7fa782000640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 0x7fa764073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:58.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.270+0000 7fa782000640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 0x7fa764073cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7700af000 tx=0x7fa77001b020 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:58.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.274+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa774021080 con 0x7fa7840747d0 2024-09-11T12:50:58.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:58.470+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fa78415aad0 con 0x7fa7840747d0 2024-09-11T12:50:59.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.082+0000 7fa77aa00640 1 -- 172.21.15.78:0/1490588031 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fa77405cc40 con 0x7fa7840747d0 2024-09-11T12:50:59.087 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 6 and data pool 7 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.082+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 msgr2=0x7fa764073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.082+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 0x7fa764073cf0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa7700af000 tx=0x7fa77001b020 comp rx=0 tx=0).stop 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.082+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 msgr2=0x7fa7841553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa77400c500 tx=0x7fa77400c9c0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 shutdown_connections 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa764071830 0x7fa764073cf0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa78415eaf0 0x7fa7841fabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78415d8e0 0x7fa7840b0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 --2- 172.21.15.78:0/1490588031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7840747d0 0x7fa7841553b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 >> 172.21.15.78:0/1490588031 conn(0x7fa784073bb0 msgr2=0x7fa78415f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 shutdown_connections 2024-09-11T12:50:59.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.086+0000 7fa783e00640 1 -- 172.21.15.78:0/1490588031 wait complete. 2024-09-11T12:50:59.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 -- 172.21.15.78:0/2241883658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c0cf760 msgr2=0x7f9f4c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 --2- 172.21.15.78:0/2241883658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c0cf760 0x7f9f4c074260 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c009900 tx=0x7f9f3c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 -- 172.21.15.78:0/2241883658 shutdown_connections 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 --2- 172.21.15.78:0/2241883658 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c06fc50 0x7f9f4c0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 --2- 172.21.15.78:0/2241883658 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c0cf760 0x7f9f4c074260 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 -- 172.21.15.78:0/2241883658 >> 172.21.15.78:0/2241883658 conn(0x7f9f4c06ca80 msgr2=0x7f9f4c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 -- 172.21.15.78:0/2241883658 shutdown_connections 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.274+0000 7f9f53600640 1 -- 172.21.15.78:0/2241883658 wait complete. 2024-09-11T12:50:59.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 Processor -- start 2024-09-11T12:50:59.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- start start 2024-09-11T12:50:59.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f4c06fc50 0x7f9f4c151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c0cf760 0x7f9f4c14a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c151010 con 0x7f9f4c15aec0 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f4c15f380 con 0x7f9f4c06fc50 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4c151190 con 0x7f9f4c0cf760 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f4be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c0cf760 0x7f9f4c14a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43172/0 (socket says 172.21.15.78:43172) 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 -- 172.21.15.78:0/2562458554 learned_addr learned my addr 172.21.15.78:0/2562458554 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f50c00640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f4c06fc50 0x7f9f4c151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 -- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c0cf760 msgr2=0x7f9f4c14a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c0cf760 0x7f9f4c14a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 -- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f4c06fc50 msgr2=0x7f9f4c151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f4c06fc50 0x7f9f4c151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 -- 172.21.15.78:0/2562458554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4c14d710 con 0x7f9f4c15aec0 2024-09-11T12:50:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f51600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea50 tx=0x7f9f4400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400cdc0 con 0x7f9f4c15aec0 2024-09-11T12:50:59.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f44004c40 con 0x7f9f4c15aec0 2024-09-11T12:50:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4c1f8100 con 0x7f9f4c15aec0 2024-09-11T12:50:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44005280 con 0x7f9f4c15aec0 2024-09-11T12:50:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.278+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f4c14b220 con 0x7f9f4c15aec0 2024-09-11T12:50:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4c14d8e0 con 0x7f9f4c15aec0 2024-09-11T12:50:59.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f9f44005420 con 0x7f9f4c15aec0 2024-09-11T12:50:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f48c00640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 0x7f9f2c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9f440929c0 con 0x7f9f4c15aec0 2024-09-11T12:50:59.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f50c00640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 0x7f9f2c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.282+0000 7f9f50c00640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 0x7f9f2c073cf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0af000 tx=0x7f9f3c01b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.286+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9f4c14d8e0 con 0x7f9f4c15aec0 2024-09-11T12:50:59.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9f4c0700e0 con 0x7f9f4c15aec0 2024-09-11T12:50:59.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f48c00640 1 -- 172.21.15.78:0/2562458554 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16686 (secure 0 0 0) 0x7f9f44016030 con 0x7f9f4c15aec0 2024-09-11T12:50:59.459 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:59.459 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":44,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:58.476956+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:56.171601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:57.449946+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 msgr2=0x7f9f2c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 0x7f9f2c073cf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0af000 tx=0x7f9f3c01b120 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 msgr2=0x7f9f4c14abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9f4400ea50 tx=0x7f9f4400ef10 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 shutdown_connections 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9f2c071830 0x7f9f2c073cf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f4c15aec0 0x7f9f4c14abd0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f4c0cf760 0x7f9f4c14a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 --2- 172.21.15.78:0/2562458554 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f4c06fc50 0x7f9f4c151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.454+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 >> 172.21.15.78:0/2562458554 conn(0x7f9f4c06ca80 msgr2=0x7f9f4c159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.458+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 shutdown_connections 2024-09-11T12:50:59.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.458+0000 7f9f53600640 1 -- 172.21.15.78:0/2562458554 wait complete. 2024-09-11T12:50:59.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:50:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- 172.21.15.78:0/721842905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc070190 msgr2=0x7f51cc071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- 172.21.15.78:0/721842905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc070190 0x7f51cc071d60 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f51cc0b2f70 tx=0x7f51c401cd50 comp rx=0 tx=0).stop 2024-09-11T12:50:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- 172.21.15.78:0/721842905 shutdown_connections 2024-09-11T12:50:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- 172.21.15.78:0/721842905 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc070190 0x7f51cc071d60 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- 172.21.15.78:0/721842905 >> v2:172.21.15.157:3300/0 conn(0x7f51cc15ba80 0x7f51cc06fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- 172.21.15.78:0/721842905 >> 172.21.15.78:0/721842905 conn(0x7f51cc06ca80 msgr2=0x7f51cc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- 172.21.15.78:0/721842905 shutdown_connections 2024-09-11T12:50:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- 172.21.15.78:0/721842905 wait complete. 2024-09-11T12:50:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 Processor -- start 2024-09-11T12:50:59.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- start start 2024-09-11T12:50:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc070190 0x7f51cc154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51cc074240 0x7f51cc141760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc1678c0 con 0x7f51cc15ba80 2024-09-11T12:50:59.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51cc167740 con 0x7f51cc070190 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d4000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51cc167a40 con 0x7f51cc074240 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43182/0 (socket says 172.21.15.78:43182) 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 -- 172.21.15.78:0/894321349 learned_addr learned my addr 172.21.15.78:0/894321349 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d1600640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc070190 0x7f51cc154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d0c00640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51cc074240 0x7f51cc141760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 -- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51cc074240 msgr2=0x7f51cc141760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51cc074240 0x7f51cc141760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 -- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc070190 msgr2=0x7f51cc154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc070190 0x7f51cc154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 -- 172.21.15.78:0/894321349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51cc1f7960 con 0x7f51cc15ba80 2024-09-11T12:50:59.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.618+0000 7f51d2000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009bd0 tx=0x7f51c4009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40bb070 con 0x7f51cc15ba80 2024-09-11T12:50:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51c4004c50 con 0x7f51cc15ba80 2024-09-11T12:50:59.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51cc1f7c00 con 0x7f51cc15ba80 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c40b2040 con 0x7f51cc15ba80 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51cc141ca0 con 0x7f51cc15ba80 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51cc071d60 con 0x7f51cc15ba80 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f51c40be070 con 0x7f51cc15ba80 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 0x7f51ac073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f51c412be50 con 0x7f51cc15ba80 2024-09-11T12:50:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.622+0000 7f51d1600640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 0x7f51ac073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.626+0000 7f51d1600640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 0x7f51ac073cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51c0000f70 tx=0x7f51c0009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.626+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f51c4016970 con 0x7f51cc15ba80 2024-09-11T12:50:59.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51cc06fbf0 con 0x7f51cc15ba80 2024-09-11T12:50:59.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51c9600640 1 -- 172.21.15.78:0/894321349 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16686 (secure 0 0 0) 0x7f51c40f7a30 con 0x7f51cc15ba80 2024-09-11T12:50:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:50:59.798 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":44,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:50:58.476956+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:56.171601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:57.449946+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:50:59.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 msgr2=0x7f51ac073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 0x7f51ac073cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f51c0000f70 tx=0x7f51c0009040 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 msgr2=0x7f51cc1fab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009bd0 tx=0x7f51c4009300 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 shutdown_connections 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51ac071830 0x7f51ac073cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51cc15ba80 0x7f51cc1fab60 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51cc074240 0x7f51cc141760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 --2- 172.21.15.78:0/894321349 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51cc070190 0x7f51cc154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 >> 172.21.15.78:0/894321349 conn(0x7f51cc06ca80 msgr2=0x7f51cc15c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 shutdown_connections 2024-09-11T12:50:59.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.794+0000 7f51d4000640 1 -- 172.21.15.78:0/894321349 wait complete. 2024-09-11T12:50:59.820 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:50:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 -- 172.21.15.78:0/4179618022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 msgr2=0x7f14100b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 --2- 172.21.15.78:0/4179618022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f14100b1a10 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f14100b26f0 tx=0x7f140001c5e0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 -- 172.21.15.78:0/4179618022 shutdown_connections 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 --2- 172.21.15.78:0/4179618022 >> v2:172.21.15.157:3300/0 conn(0x7f14100ce150 0x7f141015e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 --2- 172.21.15.78:0/4179618022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f14100cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 --2- 172.21.15.78:0/4179618022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f14100b1a10 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 -- 172.21.15.78:0/4179618022 >> 172.21.15.78:0/4179618022 conn(0x7f1410071dd0 msgr2=0x7f14100721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 -- 172.21.15.78:0/4179618022 shutdown_connections 2024-09-11T12:50:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.958+0000 7f1414e00640 1 -- 172.21.15.78:0/4179618022 wait complete. 2024-09-11T12:50:59.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 Processor -- start 2024-09-11T12:50:59.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- start start 2024-09-11T12:50:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f141014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100ce150 0x7f14101fb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1410070020 con 0x7f14101518b0 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f141006fea0 con 0x7f14100ce150 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f14100701a0 con 0x7f14100b1f50 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43204/0 (socket says 172.21.15.78:43204) 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 -- 172.21.15.78:0/2406545131 learned_addr learned my addr 172.21.15.78:0/2406545131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140d400640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100ce150 0x7f14101fb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140de00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f141014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 -- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 msgr2=0x7f141014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f141014f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 -- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100ce150 msgr2=0x7f14101fb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:50:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100ce150 0x7f14101fb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:50:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 -- 172.21.15.78:0/2406545131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141014fe40 con 0x7f14101518b0 2024-09-11T12:50:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140de00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f141014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:50:59.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f140e800640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f140800e990 tx=0x7f140800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408004d30 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f140800cb00 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14101f74d0 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1408005020 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.962+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1410157350 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141015e4d0 con 0x7f14101518b0 2024-09-11T12:50:59.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f14080051c0 con 0x7f14101518b0 2024-09-11T12:50:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f1406000640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 0x7f13f0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:50:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f1408092a70 con 0x7f14101518b0 2024-09-11T12:50:59.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f140de00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 0x7f13f0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:50:59.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.966+0000 7f140de00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 0x7f13f0073cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14000af000 tx=0x7f140001ae20 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:50:59.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:50:59.970+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1408016030 con 0x7f14101518b0 2024-09-11T12:51:00.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:00.174+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f1410069440 con 0x7f14101518b0 2024-09-11T12:51:01.073 INFO:tasks.ceph.mds.a.smithi078.stderr:2024-09-11T12:51:01.070+0000 7fa78e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:51:01.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.110+0000 7f1406000640 1 -- 172.21.15.78:0/2406545131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f140805de60 con 0x7f14101518b0 2024-09-11T12:51:01.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 msgr2=0x7f13f0073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 0x7f13f0073cf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f14000af000 tx=0x7f140001ae20 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 msgr2=0x7f1410156e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f140800e990 tx=0x7f140800ee50 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 shutdown_connections 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13f0071830 0x7f13f0073cf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14101518b0 0x7f1410156e10 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14100ce150 0x7f14101fb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 --2- 172.21.15.78:0/2406545131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14100b1f50 0x7f141014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 >> 172.21.15.78:0/2406545131 conn(0x7f1410071dd0 msgr2=0x7f14100b3150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:01.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 shutdown_connections 2024-09-11T12:51:01.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.114+0000 7f1414e00640 1 -- 172.21.15.78:0/2406545131 wait complete. 2024-09-11T12:51:01.140 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:51:01.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:51:01.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- 172.21.15.78:0/314326463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 msgr2=0x7fcb04003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:01.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 --2- 172.21.15.78:0/314326463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb04003f00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fcb100016f0 tx=0x7fcb1001c750 comp rx=0 tx=0).stop 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- 172.21.15.78:0/314326463 shutdown_connections 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 --2- 172.21.15.78:0/314326463 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb04003f00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- 172.21.15.78:0/314326463 >> 172.21.15.78:0/314326463 conn(0x7fcb20071dd0 msgr2=0x7fcb200721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- 172.21.15.78:0/314326463 shutdown_connections 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- 172.21.15.78:0/314326463 wait complete. 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 Processor -- start 2024-09-11T12:51:01.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- start start 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb201fb5f0 0x7fcb20141c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb20149920 0x7fcb2014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2015f070 con 0x7fcb04003ae0 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2015eef0 con 0x7fcb201fb5f0 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb25800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb2015f1f0 con 0x7fcb20149920 2024-09-11T12:51:01.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43224/0 (socket says 172.21.15.78:43224) 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 -- 172.21.15.78:0/177648919 learned_addr learned my addr 172.21.15.78:0/177648919 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1f400640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb20149920 0x7fcb2014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1e000640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb201fb5f0 0x7fcb20141c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 -- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb20149920 msgr2=0x7fcb2014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb20149920 0x7fcb2014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 -- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb201fb5f0 msgr2=0x7fcb20141c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb201fb5f0 0x7fcb20141c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.286+0000 7fcb1ea00640 1 -- 172.21.15.78:0/177648919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb20148880 con 0x7fcb04003ae0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb1ea00640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000c10 tx=0x7fcb10003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100b8070 con 0x7fcb04003ae0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb100ae040 con 0x7fcb04003ae0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb20148b60 con 0x7fcb04003ae0 2024-09-11T12:51:01.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb100ad070 con 0x7fcb04003ae0 2024-09-11T12:51:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb20149120 con 0x7fcb04003ae0 2024-09-11T12:51:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb200b17f0 con 0x7fcb04003ae0 2024-09-11T12:51:01.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fcb10006350 con 0x7fcb04003ae0 2024-09-11T12:51:01.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 0x7fcb00073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:01.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fcb1012b140 con 0x7fcb04003ae0 2024-09-11T12:51:01.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.290+0000 7fcb1e000640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 0x7fcb00073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:01.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.294+0000 7fcb1e000640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 0x7fcb00073d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001870 tx=0x7fcb0c006040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:01.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.294+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcb100164b0 con 0x7fcb04003ae0 2024-09-11T12:51:01.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:01.494+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fcb201ed6a0 con 0x7fcb04003ae0 2024-09-11T12:51:02.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb16a00640 1 -- 172.21.15.78:0/177648919 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7fcb100f6670 con 0x7fcb04003ae0 2024-09-11T12:51:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 msgr2=0x7fcb00073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 0x7fcb00073d20 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c001870 tx=0x7fcb0c006040 comp rx=0 tx=0).stop 2024-09-11T12:51:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 msgr2=0x7fcb2014b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcb10000c10 tx=0x7fcb10003be0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 shutdown_connections 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcb00071860 0x7fcb00073d20 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb20149920 0x7fcb2014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb201fb5f0 0x7fcb20141c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 --2- 172.21.15.78:0/177648919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb04003ae0 0x7fcb2014b0c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 >> 172.21.15.78:0/177648919 conn(0x7fcb20071dd0 msgr2=0x7fcb200b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:02.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 shutdown_connections 2024-09-11T12:51:02.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.118+0000 7fcb25800640 1 -- 172.21.15.78:0/177648919 wait complete. 2024-09-11T12:51:02.146 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:51:02.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- 172.21.15.78:0/2469596485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 msgr2=0x7fe19c070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- 172.21.15.78:0/2469596485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c070a00 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe19c15dad0 tx=0x7fe18c01c830 comp rx=0 tx=0).stop 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- 172.21.15.78:0/2469596485 shutdown_connections 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- 172.21.15.78:0/2469596485 >> v2:172.21.15.157:3300/0 conn(0x7fe19c0b4db0 0x7fe19c0b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- 172.21.15.78:0/2469596485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c070a00 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- 172.21.15.78:0/2469596485 >> 172.21.15.78:0/2469596485 conn(0x7fe19c06ca80 msgr2=0x7fe19c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- 172.21.15.78:0/2469596485 shutdown_connections 2024-09-11T12:51:02.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- 172.21.15.78:0/2469596485 wait complete. 2024-09-11T12:51:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 Processor -- start 2024-09-11T12:51:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- start start 2024-09-11T12:51:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe19c0736d0 0x7fe19c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19c0b4db0 0x7fe19c1fb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c06f250 con 0x7fe19c157ae0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c06f0d0 con 0x7fe19c0b4db0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe1a1800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe19c06f3d0 con 0x7fe19c0736d0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43250/0 (socket says 172.21.15.78:43250) 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 -- 172.21.15.78:0/920170975 learned_addr learned my addr 172.21.15.78:0/920170975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19aa00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe19c0736d0 0x7fe19c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19a000640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19c0b4db0 0x7fe19c1fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 -- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe19c0736d0 msgr2=0x7fe19c155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe19c0736d0 0x7fe19c155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 -- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19c0b4db0 msgr2=0x7fe19c1fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19c0b4db0 0x7fe19c1fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 -- 172.21.15.78:0/920170975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c151590 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.286+0000 7fe19b400640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe19400ea20 tx=0x7fe19400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19400cde0 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe194004c40 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194005280 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe1a1800640 1 -- 172.21.15.78:0/920170975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c151850 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe1a1800640 1 -- 172.21.15.78:0/920170975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c151e10 con 0x7fe19c157ae0 2024-09-11T12:51:02.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c0b5220 con 0x7fe19c157ae0 2024-09-11T12:51:02.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.290+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fe194005420 con 0x7fe19c157ae0 2024-09-11T12:51:02.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.294+0000 7fe192a00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 0x7fe17c073d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.294+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe1940928d0 con 0x7fe19c157ae0 2024-09-11T12:51:02.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.294+0000 7fe19aa00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 0x7fe17c073d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.294+0000 7fe19aa00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 0x7fe17c073d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0af000 tx=0x7fe18c0b2040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:02.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.298+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe194016030 con 0x7fe19c157ae0 2024-09-11T12:51:02.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.498+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe19c163ea0 con 0x7fe19c157ae0 2024-09-11T12:51:02.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.498+0000 7fe192a00640 1 -- 172.21.15.78:0/920170975 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fe19405de10 con 0x7fe19c157ae0 2024-09-11T12:51:02.503 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:02.503 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-09-11T12:51:02:115861+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:02.503 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.498+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 msgr2=0x7fe17c073d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.498+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 0x7fe17c073d50 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0af000 tx=0x7fe18c0b2040 comp rx=0 tx=0).stop 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 msgr2=0x7fe19c152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe19400ea20 tx=0x7fe19400eee0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 shutdown_connections 2024-09-11T12:51:02.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe17c071890 0x7fe17c073d50 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe19c157ae0 0x7fe19c152610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe19c0b4db0 0x7fe19c1fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 --2- 172.21.15.78:0/920170975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe19c0736d0 0x7fe19c155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 >> 172.21.15.78:0/920170975 conn(0x7fe19c06ca80 msgr2=0x7fe19c0b2fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 shutdown_connections 2024-09-11T12:51:02.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.502+0000 7fe187e00640 1 -- 172.21.15.78:0/920170975 wait complete. 2024-09-11T12:51:02.525 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- 172.21.15.78:0/2304454393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8300b4c30 msgr2=0x7fc8300ce570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 --2- 172.21.15.78:0/2304454393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8300b4c30 0x7fc8300ce570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc830073da0 tx=0x7fc81c01c890 comp rx=0 tx=0).stop 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- 172.21.15.78:0/2304454393 shutdown_connections 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 --2- 172.21.15.78:0/2304454393 >> v2:172.21.15.157:3300/0 conn(0x7fc83006fc50 0x7fc8300ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 --2- 172.21.15.78:0/2304454393 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8300b4c30 0x7fc8300ce570 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- 172.21.15.78:0/2304454393 >> 172.21.15.78:0/2304454393 conn(0x7fc83006ca80 msgr2=0x7fc8300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- 172.21.15.78:0/2304454393 shutdown_connections 2024-09-11T12:51:02.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- 172.21.15.78:0/2304454393 wait complete. 2024-09-11T12:51:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 Processor -- start 2024-09-11T12:51:02.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.662+0000 7fc838000640 1 -- start start 2024-09-11T12:51:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8300b4c30 0x7fc8300b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83015a940 0x7fc830152a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc830167bc0 con 0x7fc83006fc50 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc830167a40 con 0x7fc83015a940 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc830167d40 con 0x7fc8300b4c30 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43274/0 (socket says 172.21.15.78:43274) 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 -- 172.21.15.78:0/1803260833 learned_addr learned my addr 172.21.15.78:0/1803260833 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 -- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8300b4c30 msgr2=0x7fc8300b0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8300b4c30 0x7fc8300b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 -- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83015a940 msgr2=0x7fc830152a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83015a940 0x7fc830152a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 -- 172.21.15.78:0/1803260833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8300b0a50 con 0x7fc83006fc50 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc835600640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c400 tx=0x7fc82400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824017070 con 0x7fc83006fc50 2024-09-11T12:51:02.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc824003600 con 0x7fc83006fc50 2024-09-11T12:51:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8300b0d30 con 0x7fc83006fc50 2024-09-11T12:51:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824003a50 con 0x7fc83006fc50 2024-09-11T12:51:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8300b12f0 con 0x7fc83006fc50 2024-09-11T12:51:02.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.666+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8300b4520 con 0x7fc83006fc50 2024-09-11T12:51:02.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.670+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fc82401a070 con 0x7fc83006fc50 2024-09-11T12:51:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.670+0000 7fc82d600640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 0x7fc810073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.670+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc824091410 con 0x7fc83006fc50 2024-09-11T12:51:02.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.670+0000 7fc834c00640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 0x7fc810073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:02.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.670+0000 7fc834c00640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 0x7fc810073cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0af000 tx=0x7fc81c019630 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:02.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.674+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc82405cff0 con 0x7fc83006fc50 2024-09-11T12:51:02.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.838+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8300cd870 con 0x7fc83006fc50 2024-09-11T12:51:02.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc82d600640 1 -- 172.21.15.78:0/1803260833 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) ==== 74+0+16673 (secure 0 0 0) 0x7fc824065230 con 0x7fc83006fc50 2024-09-11T12:51:02.845 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:02.845 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":45,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:51:00.103131+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:56.171601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:57.449946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 msgr2=0x7fc810073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 0x7fc810073cf0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc81c0af000 tx=0x7fc81c019630 comp rx=0 tx=0).stop 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 msgr2=0x7fc830145e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c400 tx=0x7fc82400c8c0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 shutdown_connections 2024-09-11T12:51:02.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc810071830 0x7fc810073cf0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc83015a940 0x7fc830152a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8300b4c30 0x7fc8300b0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 --2- 172.21.15.78:0/1803260833 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc83006fc50 0x7fc830145e80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 >> 172.21.15.78:0/1803260833 conn(0x7fc83006ca80 msgr2=0x7fc8301584e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 shutdown_connections 2024-09-11T12:51:02.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:02.842+0000 7fc838000640 1 -- 172.21.15.78:0/1803260833 wait complete. 2024-09-11T12:51:02.866 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:51:02.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 -- 172.21.15.78:0/2589722992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 msgr2=0x7f9a5c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 --2- 172.21.15.78:0/2589722992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c15c100 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c009c80 tx=0x7f9a4c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 -- 172.21.15.78:0/2589722992 shutdown_connections 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 --2- 172.21.15.78:0/2589722992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 0x7f9a5c0a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 --2- 172.21.15.78:0/2589722992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c15c100 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 -- 172.21.15.78:0/2589722992 >> 172.21.15.78:0/2589722992 conn(0x7f9a5c074b90 msgr2=0x7f9a5c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:03.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 -- 172.21.15.78:0/2589722992 shutdown_connections 2024-09-11T12:51:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.002+0000 7f9a63a00640 1 -- 172.21.15.78:0/2589722992 wait complete. 2024-09-11T12:51:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 Processor -- start 2024-09-11T12:51:03.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- start start 2024-09-11T12:51:03.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 0x7f9a5c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5c0d0bf0 0x7f9a5c0ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c169330 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1691b0 con 0x7f9a5c0d0bf0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c1694b0 con 0x7f9a5c06fdf0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43286/0 (socket says 172.21.15.78:43286) 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 -- 172.21.15.78:0/547253605 learned_addr learned my addr 172.21.15.78:0/547253605 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a61000640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 0x7f9a5c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a61a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5c0d0bf0 0x7f9a5c0ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 -- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 msgr2=0x7f9a5c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 0x7f9a5c0b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 -- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5c0d0bf0 msgr2=0x7f9a5c0ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5c0d0bf0 0x7f9a5c0ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 -- 172.21.15.78:0/547253605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c14c330 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a5be00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800d770 tx=0x7f9a4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48017020 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a48009820 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48009c50 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c14b950 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c14b670 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.006+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c0a9d00 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.010+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f9a4801a020 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.010+0000 7f9a58c00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 0x7f9a3c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.010+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f9a48092600 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.010+0000 7f9a61000640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 0x7f9a3c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.010+0000 7f9a61000640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 0x7f9a3c073cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0af000 tx=0x7f9a4c01b5e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.014+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9a48021020 con 0x7f9a5c0b14b0 2024-09-11T12:51:03.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.178+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a5c0ab3b0 con 0x7f9a3c071830 2024-09-11T12:51:03.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.182+0000 7f9a58c00640 1 -- 172.21.15.78:0/547253605 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f9a5c0ab3b0 con 0x7f9a3c071830 2024-09-11T12:51:03.189 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:03.191 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:51:03.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 msgr2=0x7f9a3c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 0x7f9a3c073cf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0af000 tx=0x7f9a4c01b5e0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 msgr2=0x7f9a5c13d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9a4800d770 tx=0x7f9a4800dc30 comp rx=0 tx=0).stop 2024-09-11T12:51:03.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 shutdown_connections 2024-09-11T12:51:03.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9a3c071830 0x7f9a3c073cf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a5c0d0bf0 0x7f9a5c0ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a5c0b14b0 0x7f9a5c13d4e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 --2- 172.21.15.78:0/547253605 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a5c06fdf0 0x7f9a5c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 >> 172.21.15.78:0/547253605 conn(0x7f9a5c074b90 msgr2=0x7f9a5c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:03.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 shutdown_connections 2024-09-11T12:51:03.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.186+0000 7f9a63a00640 1 -- 172.21.15.78:0/547253605 wait complete. 2024-09-11T12:51:03.210 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-11T12:51:03.122817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242320,"kb_used_data":5276,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732400,"statfs":{"total":773094113280,"available":772845977600,"internally_reserved":0,"allocated":5402624,"data_stored":2837968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001040"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275503+0000","last_change":"2024-09-11T12:50:59.107007+0000","last_active":"2024-09-11T12:51:01.275503+0000","last_peered":"2024-09-11T12:51:01.275503+0000","last_clean":"2024-09-11T12:51:01.275503+0000","last_became_active":"2024-09-11T12:50:59.106740+0000","last_became_peered":"2024-09-11T12:50:59.106740+0000","last_unstale":"2024-09-11T12:51:01.275503+0000","last_undegraded":"2024-09-11T12:51:01.275503+0000","last_fullsized":"2024-09-11T12:51:01.275503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:53.984965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113457+0000","last_change":"2024-09-11T12:50:59.122247+0000","last_active":"2024-09-11T12:51:00.113457+0000","last_peered":"2024-09-11T12:51:00.113457+0000","last_clean":"2024-09-11T12:51:00.113457+0000","last_became_active":"2024-09-11T12:50:59.121888+0000","last_became_peered":"2024-09-11T12:50:59.121888+0000","last_unstale":"2024-09-11T12:51:00.113457+0000","last_undegraded":"2024-09-11T12:51:00.113457+0000","last_fullsized":"2024-09-11T12:51:00.113457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:21.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095678+0000","last_change":"2024-09-11T12:50:59.101936+0000","last_active":"2024-09-11T12:51:01.095678+0000","last_peered":"2024-09-11T12:51:01.095678+0000","last_clean":"2024-09-11T12:51:01.095678+0000","last_became_active":"2024-09-11T12:50:59.101616+0000","last_became_peered":"2024-09-11T12:50:59.101616+0000","last_unstale":"2024-09-11T12:51:01.095678+0000","last_undegraded":"2024-09-11T12:51:01.095678+0000","last_fullsized":"2024-09-11T12:51:01.095678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:04.378115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945300+0000","last_change":"2024-09-11T12:50:59.121457+0000","last_active":"2024-09-11T12:51:00.945300+0000","last_peered":"2024-09-11T12:51:00.945300+0000","last_clean":"2024-09-11T12:51:00.945300+0000","last_became_active":"2024-09-11T12:50:59.121207+0000","last_became_peered":"2024-09-11T12:50:59.121207+0000","last_unstale":"2024-09-11T12:51:00.945300+0000","last_undegraded":"2024-09-11T12:51:00.945300+0000","last_fullsized":"2024-09-11T12:51:00.945300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:28.818518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112083+0000","last_change":"2024-09-11T12:50:59.121718+0000","last_active":"2024-09-11T12:51:00.112083+0000","last_peered":"2024-09-11T12:51:00.112083+0000","last_clean":"2024-09-11T12:51:00.112083+0000","last_became_active":"2024-09-11T12:50:59.121397+0000","last_became_peered":"2024-09-11T12:50:59.121397+0000","last_unstale":"2024-09-11T12:51:00.112083+0000","last_undegraded":"2024-09-11T12:51:00.112083+0000","last_fullsized":"2024-09-11T12:51:00.112083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:56.902491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053727+0000","last_change":"2024-09-11T12:50:59.120811+0000","last_active":"2024-09-11T12:51:01.053727+0000","last_peered":"2024-09-11T12:51:01.053727+0000","last_clean":"2024-09-11T12:51:01.053727+0000","last_became_active":"2024-09-11T12:50:59.120564+0000","last_became_peered":"2024-09-11T12:50:59.120564+0000","last_unstale":"2024-09-11T12:51:01.053727+0000","last_undegraded":"2024-09-11T12:51:01.053727+0000","last_fullsized":"2024-09-11T12:51:01.053727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:51.362297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275262+0000","last_change":"2024-09-11T12:50:59.113784+0000","last_active":"2024-09-11T12:51:01.275262+0000","last_peered":"2024-09-11T12:51:01.275262+0000","last_clean":"2024-09-11T12:51:01.275262+0000","last_became_active":"2024-09-11T12:50:59.113508+0000","last_became_peered":"2024-09-11T12:50:59.113508+0000","last_unstale":"2024-09-11T12:51:01.275262+0000","last_undegraded":"2024-09-11T12:51:01.275262+0000","last_fullsized":"2024-09-11T12:51:01.275262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:04.722605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944733+0000","last_change":"2024-09-11T12:50:59.127489+0000","last_active":"2024-09-11T12:51:00.944733+0000","last_peered":"2024-09-11T12:51:00.944733+0000","last_clean":"2024-09-11T12:51:00.944733+0000","last_became_active":"2024-09-11T12:50:59.127156+0000","last_became_peered":"2024-09-11T12:50:59.127156+0000","last_unstale":"2024-09-11T12:51:00.944733+0000","last_undegraded":"2024-09-11T12:51:00.944733+0000","last_fullsized":"2024-09-11T12:51:00.944733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:28:19.143693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945252+0000","last_change":"2024-09-11T12:50:59.116385+0000","last_active":"2024-09-11T12:51:00.945252+0000","last_peered":"2024-09-11T12:51:00.945252+0000","last_clean":"2024-09-11T12:51:00.945252+0000","last_became_active":"2024-09-11T12:50:59.116193+0000","last_became_peered":"2024-09-11T12:50:59.116193+0000","last_unstale":"2024-09-11T12:51:00.945252+0000","last_undegraded":"2024-09-11T12:51:00.945252+0000","last_fullsized":"2024-09-11T12:51:00.945252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:32:08.471568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945285+0000","last_change":"2024-09-11T12:50:59.126765+0000","last_active":"2024-09-11T12:51:00.945285+0000","last_peered":"2024-09-11T12:51:00.945285+0000","last_clean":"2024-09-11T12:51:00.945285+0000","last_became_active":"2024-09-11T12:50:59.126451+0000","last_became_peered":"2024-09-11T12:50:59.126451+0000","last_unstale":"2024-09-11T12:51:00.945285+0000","last_undegraded":"2024-09-11T12:51:00.945285+0000","last_fullsized":"2024-09-11T12:51:00.945285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:46:58.988322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945129+0000","last_change":"2024-09-11T12:50:59.119125+0000","last_active":"2024-09-11T12:51:00.945129+0000","last_peered":"2024-09-11T12:51:00.945129+0000","last_clean":"2024-09-11T12:51:00.945129+0000","last_became_active":"2024-09-11T12:50:59.118813+0000","last_became_peered":"2024-09-11T12:50:59.118813+0000","last_unstale":"2024-09-11T12:51:00.945129+0000","last_undegraded":"2024-09-11T12:51:00.945129+0000","last_fullsized":"2024-09-11T12:51:00.945129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:57:25.996974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113026+0000","last_change":"2024-09-11T12:50:59.102276+0000","last_active":"2024-09-11T12:51:00.113026+0000","last_peered":"2024-09-11T12:51:00.113026+0000","last_clean":"2024-09-11T12:51:00.113026+0000","last_became_active":"2024-09-11T12:50:59.100926+0000","last_became_peered":"2024-09-11T12:50:59.100926+0000","last_unstale":"2024-09-11T12:51:00.113026+0000","last_undegraded":"2024-09-11T12:51:00.113026+0000","last_fullsized":"2024-09-11T12:51:00.113026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:04.722605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111423+0000","last_change":"2024-09-11T12:50:59.104330+0000","last_active":"2024-09-11T12:51:00.111423+0000","last_peered":"2024-09-11T12:51:00.111423+0000","last_clean":"2024-09-11T12:51:00.111423+0000","last_became_active":"2024-09-11T12:50:59.104067+0000","last_became_peered":"2024-09-11T12:50:59.104067+0000","last_unstale":"2024-09-11T12:51:00.111423+0000","last_undegraded":"2024-09-11T12:51:00.111423+0000","last_fullsized":"2024-09-11T12:51:00.111423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:14:09.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944703+0000","last_change":"2024-09-11T12:50:59.116406+0000","last_active":"2024-09-11T12:51:00.944703+0000","last_peered":"2024-09-11T12:51:00.944703+0000","last_clean":"2024-09-11T12:51:00.944703+0000","last_became_active":"2024-09-11T12:50:59.116203+0000","last_became_peered":"2024-09-11T12:50:59.116203+0000","last_unstale":"2024-09-11T12:51:00.944703+0000","last_undegraded":"2024-09-11T12:51:00.944703+0000","last_fullsized":"2024-09-11T12:51:00.944703+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:47.289447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053692+0000","last_change":"2024-09-11T12:50:59.107911+0000","last_active":"2024-09-11T12:51:01.053692+0000","last_peered":"2024-09-11T12:51:01.053692+0000","last_clean":"2024-09-11T12:51:01.053692+0000","last_became_active":"2024-09-11T12:50:59.106700+0000","last_became_peered":"2024-09-11T12:50:59.106700+0000","last_unstale":"2024-09-11T12:51:01.053692+0000","last_undegraded":"2024-09-11T12:51:01.053692+0000","last_fullsized":"2024-09-11T12:51:01.053692+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:22:24.786016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112491+0000","last_change":"2024-09-11T12:50:59.127100+0000","last_active":"2024-09-11T12:51:00.112491+0000","last_peered":"2024-09-11T12:51:00.112491+0000","last_clean":"2024-09-11T12:51:00.112491+0000","last_became_active":"2024-09-11T12:50:59.126575+0000","last_became_peered":"2024-09-11T12:50:59.126575+0000","last_unstale":"2024-09-11T12:51:00.112491+0000","last_undegraded":"2024-09-11T12:51:00.112491+0000","last_fullsized":"2024-09-11T12:51:00.112491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:46.977587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095840+0000","last_change":"2024-09-11T12:50:59.096173+0000","last_active":"2024-09-11T12:51:01.095840+0000","last_peered":"2024-09-11T12:51:01.095840+0000","last_clean":"2024-09-11T12:51:01.095840+0000","last_became_active":"2024-09-11T12:50:59.095851+0000","last_became_peered":"2024-09-11T12:50:59.095851+0000","last_unstale":"2024-09-11T12:51:01.095840+0000","last_undegraded":"2024-09-11T12:51:01.095840+0000","last_fullsized":"2024-09-11T12:51:01.095840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:29:41.655196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095728+0000","last_change":"2024-09-11T12:50:59.122884+0000","last_active":"2024-09-11T12:51:01.095728+0000","last_peered":"2024-09-11T12:51:01.095728+0000","last_clean":"2024-09-11T12:51:01.095728+0000","last_became_active":"2024-09-11T12:50:59.122007+0000","last_became_peered":"2024-09-11T12:50:59.122007+0000","last_unstale":"2024-09-11T12:51:01.095728+0000","last_undegraded":"2024-09-11T12:51:01.095728+0000","last_fullsized":"2024-09-11T12:51:01.095728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:04.184506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113611+0000","last_change":"2024-09-11T12:50:59.126950+0000","last_active":"2024-09-11T12:51:00.113611+0000","last_peered":"2024-09-11T12:51:00.113611+0000","last_clean":"2024-09-11T12:51:00.113611+0000","last_became_active":"2024-09-11T12:50:59.126629+0000","last_became_peered":"2024-09-11T12:50:59.126629+0000","last_unstale":"2024-09-11T12:51:00.113611+0000","last_undegraded":"2024-09-11T12:51:00.113611+0000","last_fullsized":"2024-09-11T12:51:00.113611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:17.549541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111414+0000","last_change":"2024-09-11T12:50:59.127181+0000","last_active":"2024-09-11T12:51:00.111414+0000","last_peered":"2024-09-11T12:51:00.111414+0000","last_clean":"2024-09-11T12:51:00.111414+0000","last_became_active":"2024-09-11T12:50:59.126907+0000","last_became_peered":"2024-09-11T12:50:59.126907+0000","last_unstale":"2024-09-11T12:51:00.111414+0000","last_undegraded":"2024-09-11T12:51:00.111414+0000","last_fullsized":"2024-09-11T12:51:00.111414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:50.797002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053902+0000","last_change":"2024-09-11T12:50:59.117147+0000","last_active":"2024-09-11T12:51:01.053902+0000","last_peered":"2024-09-11T12:51:01.053902+0000","last_clean":"2024-09-11T12:51:01.053902+0000","last_became_active":"2024-09-11T12:50:59.116898+0000","last_became_peered":"2024-09-11T12:50:59.116898+0000","last_unstale":"2024-09-11T12:51:01.053902+0000","last_undegraded":"2024-09-11T12:51:01.053902+0000","last_fullsized":"2024-09-11T12:51:01.053902+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:24:46.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945037+0000","last_change":"2024-09-11T12:50:59.125165+0000","last_active":"2024-09-11T12:51:00.945037+0000","last_peered":"2024-09-11T12:51:00.945037+0000","last_clean":"2024-09-11T12:51:00.945037+0000","last_became_active":"2024-09-11T12:50:59.124861+0000","last_became_peered":"2024-09-11T12:50:59.124861+0000","last_unstale":"2024-09-11T12:51:00.945037+0000","last_undegraded":"2024-09-11T12:51:00.945037+0000","last_fullsized":"2024-09-11T12:51:00.945037+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:17.084103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112242+0000","last_change":"2024-09-11T12:50:59.116756+0000","last_active":"2024-09-11T12:51:00.112242+0000","last_peered":"2024-09-11T12:51:00.112242+0000","last_clean":"2024-09-11T12:51:00.112242+0000","last_became_active":"2024-09-11T12:50:59.116454+0000","last_became_peered":"2024-09-11T12:50:59.116454+0000","last_unstale":"2024-09-11T12:51:00.112242+0000","last_undegraded":"2024-09-11T12:51:00.112242+0000","last_fullsized":"2024-09-11T12:51:00.112242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:56.473558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112644+0000","last_change":"2024-09-11T12:50:59.116838+0000","last_active":"2024-09-11T12:51:00.112644+0000","last_peered":"2024-09-11T12:51:00.112644+0000","last_clean":"2024-09-11T12:51:00.112644+0000","last_became_active":"2024-09-11T12:50:59.116499+0000","last_became_peered":"2024-09-11T12:50:59.116499+0000","last_unstale":"2024-09-11T12:51:00.112644+0000","last_undegraded":"2024-09-11T12:51:00.112644+0000","last_fullsized":"2024-09-11T12:51:00.112644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:17.529852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113757+0000","last_change":"2024-09-11T12:50:59.112746+0000","last_active":"2024-09-11T12:51:00.113757+0000","last_peered":"2024-09-11T12:51:00.113757+0000","last_clean":"2024-09-11T12:51:00.113757+0000","last_became_active":"2024-09-11T12:50:59.111826+0000","last_became_peered":"2024-09-11T12:50:59.111826+0000","last_unstale":"2024-09-11T12:51:00.113757+0000","last_undegraded":"2024-09-11T12:51:00.113757+0000","last_fullsized":"2024-09-11T12:51:00.113757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:04.378115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111936+0000","last_change":"2024-09-11T12:50:59.122778+0000","last_active":"2024-09-11T12:51:00.111936+0000","last_peered":"2024-09-11T12:51:00.111936+0000","last_clean":"2024-09-11T12:51:00.111936+0000","last_became_active":"2024-09-11T12:50:59.122367+0000","last_became_peered":"2024-09-11T12:50:59.122367+0000","last_unstale":"2024-09-11T12:51:00.111936+0000","last_undegraded":"2024-09-11T12:51:00.111936+0000","last_fullsized":"2024-09-11T12:51:00.111936+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:21.146426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095748+0000","last_change":"2024-09-11T12:50:59.096082+0000","last_active":"2024-09-11T12:51:01.095748+0000","last_peered":"2024-09-11T12:51:01.095748+0000","last_clean":"2024-09-11T12:51:01.095748+0000","last_became_active":"2024-09-11T12:50:59.095834+0000","last_became_peered":"2024-09-11T12:50:59.095834+0000","last_unstale":"2024-09-11T12:51:01.095748+0000","last_undegraded":"2024-09-11T12:51:01.095748+0000","last_fullsized":"2024-09-11T12:51:01.095748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:57.753073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113067+0000","last_change":"2024-09-11T12:50:59.101058+0000","last_active":"2024-09-11T12:51:00.113067+0000","last_peered":"2024-09-11T12:51:00.113067+0000","last_clean":"2024-09-11T12:51:00.113067+0000","last_became_active":"2024-09-11T12:50:59.100704+0000","last_became_peered":"2024-09-11T12:50:59.100704+0000","last_unstale":"2024-09-11T12:51:00.113067+0000","last_undegraded":"2024-09-11T12:51:00.113067+0000","last_fullsized":"2024-09-11T12:51:00.113067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:12.019353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112660+0000","last_change":"2024-09-11T12:50:59.118380+0000","last_active":"2024-09-11T12:51:00.112660+0000","last_peered":"2024-09-11T12:51:00.112660+0000","last_clean":"2024-09-11T12:51:00.112660+0000","last_became_active":"2024-09-11T12:50:59.118006+0000","last_became_peered":"2024-09-11T12:50:59.118006+0000","last_unstale":"2024-09-11T12:51:00.112660+0000","last_undegraded":"2024-09-11T12:51:00.112660+0000","last_fullsized":"2024-09-11T12:51:00.112660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:53.401485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095489+0000","last_change":"2024-09-11T12:50:59.115457+0000","last_active":"2024-09-11T12:51:01.095489+0000","last_peered":"2024-09-11T12:51:01.095489+0000","last_clean":"2024-09-11T12:51:01.095489+0000","last_became_active":"2024-09-11T12:50:59.111950+0000","last_became_peered":"2024-09-11T12:50:59.111950+0000","last_unstale":"2024-09-11T12:51:01.095489+0000","last_undegraded":"2024-09-11T12:51:01.095489+0000","last_fullsized":"2024-09-11T12:51:01.095489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053694+0000","last_change":"2024-09-11T12:50:59.111483+0000","last_active":"2024-09-11T12:51:01.053694+0000","last_peered":"2024-09-11T12:51:01.053694+0000","last_clean":"2024-09-11T12:51:01.053694+0000","last_became_active":"2024-09-11T12:50:59.111216+0000","last_became_peered":"2024-09-11T12:50:59.111216+0000","last_unstale":"2024-09-11T12:51:01.053694+0000","last_undegraded":"2024-09-11T12:51:01.053694+0000","last_fullsized":"2024-09-11T12:51:01.053694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:35.614879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111298+0000","last_change":"2024-09-11T12:50:59.112810+0000","last_active":"2024-09-11T12:51:00.111298+0000","last_peered":"2024-09-11T12:51:00.111298+0000","last_clean":"2024-09-11T12:51:00.111298+0000","last_became_active":"2024-09-11T12:50:59.112518+0000","last_became_peered":"2024-09-11T12:50:59.112518+0000","last_unstale":"2024-09-11T12:51:00.111298+0000","last_undegraded":"2024-09-11T12:51:00.111298+0000","last_fullsized":"2024-09-11T12:51:00.111298+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:42:28.962546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984428+0000","last_change":"2024-09-11T12:50:59.122003+0000","last_active":"2024-09-11T12:51:00.984428+0000","last_peered":"2024-09-11T12:51:00.984428+0000","last_clean":"2024-09-11T12:51:00.984428+0000","last_became_active":"2024-09-11T12:50:59.121651+0000","last_became_peered":"2024-09-11T12:50:59.121651+0000","last_unstale":"2024-09-11T12:51:00.984428+0000","last_undegraded":"2024-09-11T12:51:00.984428+0000","last_fullsized":"2024-09-11T12:51:00.984428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":19,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054431+0000","last_change":"2024-09-11T12:50:58.107932+0000","last_active":"2024-09-11T12:51:01.054431+0000","last_peered":"2024-09-11T12:51:01.054431+0000","last_clean":"2024-09-11T12:51:01.054431+0000","last_became_active":"2024-09-11T12:50:58.107680+0000","last_became_peered":"2024-09-11T12:50:58.107680+0000","last_unstale":"2024-09-11T12:51:01.054431+0000","last_undegraded":"2024-09-11T12:51:01.054431+0000","last_fullsized":"2024-09-11T12:51:01.054431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:55.899646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113730+0000","last_change":"2024-09-11T12:50:59.104959+0000","last_active":"2024-09-11T12:51:00.113730+0000","last_peered":"2024-09-11T12:51:00.113730+0000","last_clean":"2024-09-11T12:51:00.113730+0000","last_became_active":"2024-09-11T12:50:59.103992+0000","last_became_peered":"2024-09-11T12:50:59.103992+0000","last_unstale":"2024-09-11T12:51:00.113730+0000","last_undegraded":"2024-09-11T12:51:00.113730+0000","last_fullsized":"2024-09-11T12:51:00.113730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:26.869564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095765+0000","last_change":"2024-09-11T12:50:58.107594+0000","last_active":"2024-09-11T12:51:01.095765+0000","last_peered":"2024-09-11T12:51:01.095765+0000","last_clean":"2024-09-11T12:51:01.095765+0000","last_became_active":"2024-09-11T12:50:58.107308+0000","last_became_peered":"2024-09-11T12:50:58.107308+0000","last_unstale":"2024-09-11T12:51:01.095765+0000","last_undegraded":"2024-09-11T12:51:01.095765+0000","last_fullsized":"2024-09-11T12:51:01.095765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:51.002666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113198+0000","last_change":"2024-09-11T12:50:59.107384+0000","last_active":"2024-09-11T12:51:00.113198+0000","last_peered":"2024-09-11T12:51:00.113198+0000","last_clean":"2024-09-11T12:51:00.113198+0000","last_became_active":"2024-09-11T12:50:59.107111+0000","last_became_peered":"2024-09-11T12:50:59.107111+0000","last_unstale":"2024-09-11T12:51:00.113198+0000","last_undegraded":"2024-09-11T12:51:00.113198+0000","last_fullsized":"2024-09-11T12:51:00.113198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:08.742585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984377+0000","last_change":"2024-09-11T12:50:58.102485+0000","last_active":"2024-09-11T12:51:00.984377+0000","last_peered":"2024-09-11T12:51:00.984377+0000","last_clean":"2024-09-11T12:51:00.984377+0000","last_became_active":"2024-09-11T12:50:58.102108+0000","last_became_peered":"2024-09-11T12:50:58.102108+0000","last_unstale":"2024-09-11T12:51:00.984377+0000","last_undegraded":"2024-09-11T12:51:00.984377+0000","last_fullsized":"2024-09-11T12:51:00.984377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:29.888058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984447+0000","last_change":"2024-09-11T12:50:59.110439+0000","last_active":"2024-09-11T12:51:00.984447+0000","last_peered":"2024-09-11T12:51:00.984447+0000","last_clean":"2024-09-11T12:51:00.984447+0000","last_became_active":"2024-09-11T12:50:59.110156+0000","last_became_peered":"2024-09-11T12:50:59.110156+0000","last_unstale":"2024-09-11T12:51:00.984447+0000","last_undegraded":"2024-09-11T12:51:00.984447+0000","last_fullsized":"2024-09-11T12:51:00.984447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275130+0000","last_change":"2024-09-11T12:50:58.109512+0000","last_active":"2024-09-11T12:51:01.275130+0000","last_peered":"2024-09-11T12:51:01.275130+0000","last_clean":"2024-09-11T12:51:01.275130+0000","last_became_active":"2024-09-11T12:50:58.109167+0000","last_became_peered":"2024-09-11T12:50:58.109167+0000","last_unstale":"2024-09-11T12:51:01.275130+0000","last_undegraded":"2024-09-11T12:51:01.275130+0000","last_fullsized":"2024-09-11T12:51:01.275130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:03.290154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095601+0000","last_change":"2024-09-11T12:50:59.106575+0000","last_active":"2024-09-11T12:51:01.095601+0000","last_peered":"2024-09-11T12:51:01.095601+0000","last_clean":"2024-09-11T12:51:01.095601+0000","last_became_active":"2024-09-11T12:50:59.105335+0000","last_became_peered":"2024-09-11T12:50:59.105335+0000","last_unstale":"2024-09-11T12:51:01.095601+0000","last_undegraded":"2024-09-11T12:51:01.095601+0000","last_fullsized":"2024-09-11T12:51:01.095601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:17.549541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053982+0000","last_change":"2024-09-11T12:50:58.111035+0000","last_active":"2024-09-11T12:51:01.053982+0000","last_peered":"2024-09-11T12:51:01.053982+0000","last_clean":"2024-09-11T12:51:01.053982+0000","last_became_active":"2024-09-11T12:50:58.110738+0000","last_became_peered":"2024-09-11T12:50:58.110738+0000","last_unstale":"2024-09-11T12:51:01.053982+0000","last_undegraded":"2024-09-11T12:51:01.053982+0000","last_fullsized":"2024-09-11T12:51:01.053982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:08.626408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945109+0000","last_change":"2024-09-11T12:50:59.128553+0000","last_active":"2024-09-11T12:51:00.945109+0000","last_peered":"2024-09-11T12:51:00.945109+0000","last_clean":"2024-09-11T12:51:00.945109+0000","last_became_active":"2024-09-11T12:50:59.127880+0000","last_became_peered":"2024-09-11T12:50:59.127880+0000","last_unstale":"2024-09-11T12:51:00.945109+0000","last_undegraded":"2024-09-11T12:51:00.945109+0000","last_fullsized":"2024-09-11T12:51:00.945109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:02:25.868829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095594+0000","last_change":"2024-09-11T12:50:58.098497+0000","last_active":"2024-09-11T12:51:01.095594+0000","last_peered":"2024-09-11T12:51:01.095594+0000","last_clean":"2024-09-11T12:51:01.095594+0000","last_became_active":"2024-09-11T12:50:58.098063+0000","last_became_peered":"2024-09-11T12:50:58.098063+0000","last_unstale":"2024-09-11T12:51:01.095594+0000","last_undegraded":"2024-09-11T12:51:01.095594+0000","last_fullsized":"2024-09-11T12:51:01.095594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:59.034777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944793+0000","last_change":"2024-09-11T12:50:59.127848+0000","last_active":"2024-09-11T12:51:00.944793+0000","last_peered":"2024-09-11T12:51:00.944793+0000","last_clean":"2024-09-11T12:51:00.944793+0000","last_became_active":"2024-09-11T12:50:59.126825+0000","last_became_peered":"2024-09-11T12:50:59.126825+0000","last_unstale":"2024-09-11T12:51:00.944793+0000","last_undegraded":"2024-09-11T12:51:00.944793+0000","last_fullsized":"2024-09-11T12:51:00.944793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:11:06.811241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112760+0000","last_change":"2024-09-11T12:50:58.104299+0000","last_active":"2024-09-11T12:51:00.112760+0000","last_peered":"2024-09-11T12:51:00.112760+0000","last_clean":"2024-09-11T12:51:00.112760+0000","last_became_active":"2024-09-11T12:50:58.103952+0000","last_became_peered":"2024-09-11T12:50:58.103952+0000","last_unstale":"2024-09-11T12:51:00.112760+0000","last_undegraded":"2024-09-11T12:51:00.112760+0000","last_fullsized":"2024-09-11T12:51:00.112760+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:16.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112065+0000","last_change":"2024-09-11T12:50:59.116985+0000","last_active":"2024-09-11T12:51:00.112065+0000","last_peered":"2024-09-11T12:51:00.112065+0000","last_clean":"2024-09-11T12:51:00.112065+0000","last_became_active":"2024-09-11T12:50:59.116709+0000","last_became_peered":"2024-09-11T12:50:59.116709+0000","last_unstale":"2024-09-11T12:51:00.112065+0000","last_undegraded":"2024-09-11T12:51:00.112065+0000","last_fullsized":"2024-09-11T12:51:00.112065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:30.890904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944857+0000","last_change":"2024-09-11T12:50:58.106830+0000","last_active":"2024-09-11T12:51:00.944857+0000","last_peered":"2024-09-11T12:51:00.944857+0000","last_clean":"2024-09-11T12:51:00.944857+0000","last_became_active":"2024-09-11T12:50:58.105732+0000","last_became_peered":"2024-09-11T12:50:58.105732+0000","last_unstale":"2024-09-11T12:51:00.944857+0000","last_undegraded":"2024-09-11T12:51:00.944857+0000","last_fullsized":"2024-09-11T12:51:00.944857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:14.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095669+0000","last_change":"2024-09-11T12:50:59.122159+0000","last_active":"2024-09-11T12:51:01.095669+0000","last_peered":"2024-09-11T12:51:01.095669+0000","last_clean":"2024-09-11T12:51:01.095669+0000","last_became_active":"2024-09-11T12:50:59.121884+0000","last_became_peered":"2024-09-11T12:50:59.121884+0000","last_unstale":"2024-09-11T12:51:01.095669+0000","last_undegraded":"2024-09-11T12:51:01.095669+0000","last_fullsized":"2024-09-11T12:51:01.095669+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:26.869564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095830+0000","last_change":"2024-09-11T12:50:58.097602+0000","last_active":"2024-09-11T12:51:01.095830+0000","last_peered":"2024-09-11T12:51:01.095830+0000","last_clean":"2024-09-11T12:51:01.095830+0000","last_became_active":"2024-09-11T12:50:58.097211+0000","last_became_peered":"2024-09-11T12:50:58.097211+0000","last_unstale":"2024-09-11T12:51:01.095830+0000","last_undegraded":"2024-09-11T12:51:01.095830+0000","last_fullsized":"2024-09-11T12:51:01.095830+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:36.906734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275581+0000","last_change":"2024-09-11T12:50:59.110182+0000","last_active":"2024-09-11T12:51:01.275581+0000","last_peered":"2024-09-11T12:51:01.275581+0000","last_clean":"2024-09-11T12:51:01.275581+0000","last_became_active":"2024-09-11T12:50:59.109853+0000","last_became_peered":"2024-09-11T12:50:59.109853+0000","last_unstale":"2024-09-11T12:51:01.275581+0000","last_undegraded":"2024-09-11T12:51:01.275581+0000","last_fullsized":"2024-09-11T12:51:01.275581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:48:52.073833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945425+0000","last_change":"2024-09-11T12:50:58.111691+0000","last_active":"2024-09-11T12:51:00.945425+0000","last_peered":"2024-09-11T12:51:00.945425+0000","last_clean":"2024-09-11T12:51:00.945425+0000","last_became_active":"2024-09-11T12:50:58.109298+0000","last_became_peered":"2024-09-11T12:50:58.109298+0000","last_unstale":"2024-09-11T12:51:00.945425+0000","last_undegraded":"2024-09-11T12:51:00.945425+0000","last_fullsized":"2024-09-11T12:51:00.945425+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:33.419831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054257+0000","last_change":"2024-09-11T12:50:59.120690+0000","last_active":"2024-09-11T12:51:01.054257+0000","last_peered":"2024-09-11T12:51:01.054257+0000","last_clean":"2024-09-11T12:51:01.054257+0000","last_became_active":"2024-09-11T12:50:59.120455+0000","last_became_peered":"2024-09-11T12:50:59.120455+0000","last_unstale":"2024-09-11T12:51:01.054257+0000","last_undegraded":"2024-09-11T12:51:01.054257+0000","last_fullsized":"2024-09-11T12:51:01.054257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:01.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275535+0000","last_change":"2024-09-11T12:50:58.113351+0000","last_active":"2024-09-11T12:51:01.275535+0000","last_peered":"2024-09-11T12:51:01.275535+0000","last_clean":"2024-09-11T12:51:01.275535+0000","last_became_active":"2024-09-11T12:50:58.113091+0000","last_became_peered":"2024-09-11T12:50:58.113091+0000","last_unstale":"2024-09-11T12:51:01.275535+0000","last_undegraded":"2024-09-11T12:51:01.275535+0000","last_fullsized":"2024-09-11T12:51:01.275535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:52.398639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275442+0000","last_change":"2024-09-11T12:50:59.119332+0000","last_active":"2024-09-11T12:51:01.275442+0000","last_peered":"2024-09-11T12:51:01.275442+0000","last_clean":"2024-09-11T12:51:01.275442+0000","last_became_active":"2024-09-11T12:50:59.119036+0000","last_became_peered":"2024-09-11T12:50:59.119036+0000","last_unstale":"2024-09-11T12:51:01.275442+0000","last_undegraded":"2024-09-11T12:51:01.275442+0000","last_fullsized":"2024-09-11T12:51:01.275442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:08.742585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":21,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111452+0000","last_change":"2024-09-11T12:50:58.100258+0000","last_active":"2024-09-11T12:51:00.111452+0000","last_peered":"2024-09-11T12:51:00.111452+0000","last_clean":"2024-09-11T12:51:00.111452+0000","last_became_active":"2024-09-11T12:50:58.099931+0000","last_became_peered":"2024-09-11T12:50:58.099931+0000","last_unstale":"2024-09-11T12:51:00.111452+0000","last_undegraded":"2024-09-11T12:51:00.111452+0000","last_fullsized":"2024-09-11T12:51:00.111452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:46:18.132465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112775+0000","last_change":"2024-09-11T12:50:59.126925+0000","last_active":"2024-09-11T12:51:00.112775+0000","last_peered":"2024-09-11T12:51:00.112775+0000","last_clean":"2024-09-11T12:51:00.112775+0000","last_became_active":"2024-09-11T12:50:59.126506+0000","last_became_peered":"2024-09-11T12:50:59.126506+0000","last_unstale":"2024-09-11T12:51:00.112775+0000","last_undegraded":"2024-09-11T12:51:00.112775+0000","last_fullsized":"2024-09-11T12:51:00.112775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054012+0000","last_change":"2024-09-11T12:50:58.112901+0000","last_active":"2024-09-11T12:51:01.054012+0000","last_peered":"2024-09-11T12:51:01.054012+0000","last_clean":"2024-09-11T12:51:01.054012+0000","last_became_active":"2024-09-11T12:50:58.112637+0000","last_became_peered":"2024-09-11T12:50:58.112637+0000","last_unstale":"2024-09-11T12:51:01.054012+0000","last_undegraded":"2024-09-11T12:51:01.054012+0000","last_fullsized":"2024-09-11T12:51:01.054012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:11.016508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945015+0000","last_change":"2024-09-11T12:50:59.128324+0000","last_active":"2024-09-11T12:51:00.945015+0000","last_peered":"2024-09-11T12:51:00.945015+0000","last_clean":"2024-09-11T12:51:00.945015+0000","last_became_active":"2024-09-11T12:50:59.127494+0000","last_became_peered":"2024-09-11T12:50:59.127494+0000","last_unstale":"2024-09-11T12:51:00.945015+0000","last_undegraded":"2024-09-11T12:51:00.945015+0000","last_fullsized":"2024-09-11T12:51:00.945015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:40:24.792102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275120+0000","last_change":"2024-09-11T12:50:58.113306+0000","last_active":"2024-09-11T12:51:01.275120+0000","last_peered":"2024-09-11T12:51:01.275120+0000","last_clean":"2024-09-11T12:51:01.275120+0000","last_became_active":"2024-09-11T12:50:58.112967+0000","last_became_peered":"2024-09-11T12:50:58.112967+0000","last_unstale":"2024-09-11T12:51:01.275120+0000","last_undegraded":"2024-09-11T12:51:01.275120+0000","last_fullsized":"2024-09-11T12:51:01.275120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:16.546696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053690+0000","last_change":"2024-09-11T12:50:59.124336+0000","last_active":"2024-09-11T12:51:01.053690+0000","last_peered":"2024-09-11T12:51:01.053690+0000","last_clean":"2024-09-11T12:51:01.053690+0000","last_became_active":"2024-09-11T12:50:59.123995+0000","last_became_peered":"2024-09-11T12:50:59.123995+0000","last_unstale":"2024-09-11T12:51:01.053690+0000","last_undegraded":"2024-09-11T12:51:01.053690+0000","last_fullsized":"2024-09-11T12:51:01.053690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:52.884639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053827+0000","last_change":"2024-09-11T12:50:58.113044+0000","last_active":"2024-09-11T12:51:01.053827+0000","last_peered":"2024-09-11T12:51:01.053827+0000","last_clean":"2024-09-11T12:51:01.053827+0000","last_became_active":"2024-09-11T12:50:58.112656+0000","last_became_peered":"2024-09-11T12:50:58.112656+0000","last_unstale":"2024-09-11T12:51:01.053827+0000","last_undegraded":"2024-09-11T12:51:01.053827+0000","last_fullsized":"2024-09-11T12:51:01.053827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:56.750228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275264+0000","last_change":"2024-09-11T12:50:59.119373+0000","last_active":"2024-09-11T12:51:01.275264+0000","last_peered":"2024-09-11T12:51:01.275264+0000","last_clean":"2024-09-11T12:51:01.275264+0000","last_became_active":"2024-09-11T12:50:59.119045+0000","last_became_peered":"2024-09-11T12:50:59.119045+0000","last_unstale":"2024-09-11T12:51:01.275264+0000","last_undegraded":"2024-09-11T12:51:01.275264+0000","last_fullsized":"2024-09-11T12:51:01.275264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:12.019353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":23,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944959+0000","last_change":"2024-09-11T12:50:58.101697+0000","last_active":"2024-09-11T12:51:00.944959+0000","last_peered":"2024-09-11T12:51:00.944959+0000","last_clean":"2024-09-11T12:51:00.944959+0000","last_became_active":"2024-09-11T12:50:58.101246+0000","last_became_peered":"2024-09-11T12:50:58.101246+0000","last_unstale":"2024-09-11T12:51:00.944959+0000","last_undegraded":"2024-09-11T12:51:00.944959+0000","last_fullsized":"2024-09-11T12:51:00.944959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:56.440536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111977+0000","last_change":"2024-09-11T12:50:59.105311+0000","last_active":"2024-09-11T12:51:00.111977+0000","last_peered":"2024-09-11T12:51:00.111977+0000","last_clean":"2024-09-11T12:51:00.111977+0000","last_became_active":"2024-09-11T12:50:59.104895+0000","last_became_peered":"2024-09-11T12:50:59.104895+0000","last_unstale":"2024-09-11T12:51:00.111977+0000","last_undegraded":"2024-09-11T12:51:00.111977+0000","last_fullsized":"2024-09-11T12:51:00.111977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:13.195879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111910+0000","last_change":"2024-09-11T12:50:58.110928+0000","last_active":"2024-09-11T12:51:00.111910+0000","last_peered":"2024-09-11T12:51:00.111910+0000","last_clean":"2024-09-11T12:51:00.111910+0000","last_became_active":"2024-09-11T12:50:58.110278+0000","last_became_peered":"2024-09-11T12:50:58.110278+0000","last_unstale":"2024-09-11T12:51:00.111910+0000","last_undegraded":"2024-09-11T12:51:00.111910+0000","last_fullsized":"2024-09-11T12:51:00.111910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:25.822358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945295+0000","last_change":"2024-09-11T12:50:59.130694+0000","last_active":"2024-09-11T12:51:00.945295+0000","last_peered":"2024-09-11T12:51:00.945295+0000","last_clean":"2024-09-11T12:51:00.945295+0000","last_became_active":"2024-09-11T12:50:59.130361+0000","last_became_peered":"2024-09-11T12:50:59.130361+0000","last_unstale":"2024-09-11T12:51:00.945295+0000","last_undegraded":"2024-09-11T12:51:00.945295+0000","last_fullsized":"2024-09-11T12:51:00.945295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:52.913127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095796+0000","last_change":"2024-09-11T12:50:58.102158+0000","last_active":"2024-09-11T12:51:01.095796+0000","last_peered":"2024-09-11T12:51:01.095796+0000","last_clean":"2024-09-11T12:51:01.095796+0000","last_became_active":"2024-09-11T12:50:58.101875+0000","last_became_peered":"2024-09-11T12:50:58.101875+0000","last_unstale":"2024-09-11T12:51:01.095796+0000","last_undegraded":"2024-09-11T12:51:01.095796+0000","last_fullsized":"2024-09-11T12:51:01.095796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:12:44.915885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112405+0000","last_change":"2024-09-11T12:50:59.124661+0000","last_active":"2024-09-11T12:51:00.112405+0000","last_peered":"2024-09-11T12:51:00.112405+0000","last_clean":"2024-09-11T12:51:00.112405+0000","last_became_active":"2024-09-11T12:50:59.123996+0000","last_became_peered":"2024-09-11T12:50:59.123996+0000","last_unstale":"2024-09-11T12:51:00.112405+0000","last_undegraded":"2024-09-11T12:51:00.112405+0000","last_fullsized":"2024-09-11T12:51:00.112405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:24.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113027+0000","last_change":"2024-09-11T12:50:58.104431+0000","last_active":"2024-09-11T12:51:00.113027+0000","last_peered":"2024-09-11T12:51:00.113027+0000","last_clean":"2024-09-11T12:51:00.113027+0000","last_became_active":"2024-09-11T12:50:58.104126+0000","last_became_peered":"2024-09-11T12:50:58.104126+0000","last_unstale":"2024-09-11T12:51:00.113027+0000","last_undegraded":"2024-09-11T12:51:00.113027+0000","last_fullsized":"2024-09-11T12:51:00.113027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:03.290154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113360+0000","last_change":"2024-09-11T12:50:59.116037+0000","last_active":"2024-09-11T12:51:00.113360+0000","last_peered":"2024-09-11T12:51:00.113360+0000","last_clean":"2024-09-11T12:51:00.113360+0000","last_became_active":"2024-09-11T12:50:59.112693+0000","last_became_peered":"2024-09-11T12:50:59.112693+0000","last_unstale":"2024-09-11T12:51:00.113360+0000","last_undegraded":"2024-09-11T12:51:00.113360+0000","last_fullsized":"2024-09-11T12:51:00.113360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054248+0000","last_change":"2024-09-11T12:50:58.113638+0000","last_active":"2024-09-11T12:51:01.054248+0000","last_peered":"2024-09-11T12:51:01.054248+0000","last_clean":"2024-09-11T12:51:01.054248+0000","last_became_active":"2024-09-11T12:50:58.113438+0000","last_became_peered":"2024-09-11T12:50:58.113438+0000","last_unstale":"2024-09-11T12:51:01.054248+0000","last_undegraded":"2024-09-11T12:51:01.054248+0000","last_fullsized":"2024-09-11T12:51:01.054248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:03.719760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053792+0000","last_change":"2024-09-11T12:50:59.123907+0000","last_active":"2024-09-11T12:51:01.053792+0000","last_peered":"2024-09-11T12:51:01.053792+0000","last_clean":"2024-09-11T12:51:01.053792+0000","last_became_active":"2024-09-11T12:50:59.120116+0000","last_became_peered":"2024-09-11T12:50:59.120116+0000","last_unstale":"2024-09-11T12:51:01.053792+0000","last_undegraded":"2024-09-11T12:51:01.053792+0000","last_fullsized":"2024-09-11T12:51:01.053792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:41.783189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054146+0000","last_change":"2024-09-11T12:50:58.110448+0000","last_active":"2024-09-11T12:51:01.054146+0000","last_peered":"2024-09-11T12:51:01.054146+0000","last_clean":"2024-09-11T12:51:01.054146+0000","last_became_active":"2024-09-11T12:50:58.110166+0000","last_became_peered":"2024-09-11T12:50:58.110166+0000","last_unstale":"2024-09-11T12:51:01.054146+0000","last_undegraded":"2024-09-11T12:51:01.054146+0000","last_fullsized":"2024-09-11T12:51:01.054146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:20.183851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984210+0000","last_change":"2024-09-11T12:50:59.114929+0000","last_active":"2024-09-11T12:51:00.984210+0000","last_peered":"2024-09-11T12:51:00.984210+0000","last_clean":"2024-09-11T12:51:00.984210+0000","last_became_active":"2024-09-11T12:50:59.114735+0000","last_became_peered":"2024-09-11T12:50:59.114735+0000","last_unstale":"2024-09-11T12:51:00.984210+0000","last_undegraded":"2024-09-11T12:51:00.984210+0000","last_fullsized":"2024-09-11T12:51:00.984210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:53.401485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112125+0000","last_change":"2024-09-11T12:50:58.103023+0000","last_active":"2024-09-11T12:51:00.112125+0000","last_peered":"2024-09-11T12:51:00.112125+0000","last_clean":"2024-09-11T12:51:00.112125+0000","last_became_active":"2024-09-11T12:50:58.100839+0000","last_became_peered":"2024-09-11T12:50:58.100839+0000","last_unstale":"2024-09-11T12:51:00.112125+0000","last_undegraded":"2024-09-11T12:51:00.112125+0000","last_fullsized":"2024-09-11T12:51:00.112125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:52.398639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111338+0000","last_change":"2024-09-11T12:50:59.108219+0000","last_active":"2024-09-11T12:51:00.111338+0000","last_peered":"2024-09-11T12:51:00.111338+0000","last_clean":"2024-09-11T12:51:00.111338+0000","last_became_active":"2024-09-11T12:50:59.107942+0000","last_became_peered":"2024-09-11T12:50:59.107942+0000","last_unstale":"2024-09-11T12:51:00.111338+0000","last_undegraded":"2024-09-11T12:51:00.111338+0000","last_fullsized":"2024-09-11T12:51:00.111338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:35.356098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944692+0000","last_change":"2024-09-11T12:50:58.108999+0000","last_active":"2024-09-11T12:51:00.944692+0000","last_peered":"2024-09-11T12:51:00.944692+0000","last_clean":"2024-09-11T12:51:00.944692+0000","last_became_active":"2024-09-11T12:50:58.108338+0000","last_became_peered":"2024-09-11T12:50:58.108338+0000","last_unstale":"2024-09-11T12:51:00.944692+0000","last_undegraded":"2024-09-11T12:51:00.944692+0000","last_fullsized":"2024-09-11T12:51:00.944692+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:17.423142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944675+0000","last_change":"2024-09-11T12:50:59.119018+0000","last_active":"2024-09-11T12:51:00.944675+0000","last_peered":"2024-09-11T12:51:00.944675+0000","last_clean":"2024-09-11T12:51:00.944675+0000","last_became_active":"2024-09-11T12:50:59.118731+0000","last_became_peered":"2024-09-11T12:50:59.118731+0000","last_unstale":"2024-09-11T12:51:00.944675+0000","last_undegraded":"2024-09-11T12:51:00.944675+0000","last_fullsized":"2024-09-11T12:51:00.944675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:39.896831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.274996+0000","last_change":"2024-09-11T12:50:58.112202+0000","last_active":"2024-09-11T12:51:01.274996+0000","last_peered":"2024-09-11T12:51:01.274996+0000","last_clean":"2024-09-11T12:51:01.274996+0000","last_became_active":"2024-09-11T12:50:58.111922+0000","last_became_peered":"2024-09-11T12:50:58.111922+0000","last_unstale":"2024-09-11T12:51:01.274996+0000","last_undegraded":"2024-09-11T12:51:01.274996+0000","last_fullsized":"2024-09-11T12:51:01.274996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:45.974742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113816+0000","last_change":"2024-09-11T12:50:59.127186+0000","last_active":"2024-09-11T12:51:00.113816+0000","last_peered":"2024-09-11T12:51:00.113816+0000","last_clean":"2024-09-11T12:51:00.113816+0000","last_became_active":"2024-09-11T12:50:59.126674+0000","last_became_peered":"2024-09-11T12:50:59.126674+0000","last_unstale":"2024-09-11T12:51:00.113816+0000","last_undegraded":"2024-09-11T12:51:00.113816+0000","last_fullsized":"2024-09-11T12:51:00.113816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:04.184506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054333+0000","last_change":"2024-09-11T12:50:58.108624+0000","last_active":"2024-09-11T12:51:01.054333+0000","last_peered":"2024-09-11T12:51:01.054333+0000","last_clean":"2024-09-11T12:51:01.054333+0000","last_became_active":"2024-09-11T12:50:58.108292+0000","last_became_peered":"2024-09-11T12:50:58.108292+0000","last_unstale":"2024-09-11T12:51:01.054333+0000","last_undegraded":"2024-09-11T12:51:01.054333+0000","last_fullsized":"2024-09-11T12:51:01.054333+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:07.739739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945430+0000","last_change":"2024-09-11T12:50:59.121606+0000","last_active":"2024-09-11T12:51:00.945430+0000","last_peered":"2024-09-11T12:51:00.945430+0000","last_clean":"2024-09-11T12:51:00.945430+0000","last_became_active":"2024-09-11T12:50:59.121350+0000","last_became_peered":"2024-09-11T12:50:59.121350+0000","last_unstale":"2024-09-11T12:51:00.945430+0000","last_undegraded":"2024-09-11T12:51:00.945430+0000","last_fullsized":"2024-09-11T12:51:00.945430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:15.983967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275535+0000","last_change":"2024-09-11T12:50:58.114147+0000","last_active":"2024-09-11T12:51:01.275535+0000","last_peered":"2024-09-11T12:51:01.275535+0000","last_clean":"2024-09-11T12:51:01.275535+0000","last_became_active":"2024-09-11T12:50:58.113725+0000","last_became_peered":"2024-09-11T12:50:58.113725+0000","last_unstale":"2024-09-11T12:51:01.275535+0000","last_undegraded":"2024-09-11T12:51:01.275535+0000","last_fullsized":"2024-09-11T12:51:01.275535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:27.063998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275443+0000","last_change":"2024-09-11T12:50:59.113289+0000","last_active":"2024-09-11T12:51:01.275443+0000","last_peered":"2024-09-11T12:51:01.275443+0000","last_clean":"2024-09-11T12:51:01.275443+0000","last_became_active":"2024-09-11T12:50:59.112316+0000","last_became_peered":"2024-09-11T12:50:59.112316+0000","last_unstale":"2024-09-11T12:51:01.275443+0000","last_undegraded":"2024-09-11T12:51:01.275443+0000","last_fullsized":"2024-09-11T12:51:01.275443+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:21.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.983991+0000","last_change":"2024-09-11T12:50:58.095325+0000","last_active":"2024-09-11T12:51:00.983991+0000","last_peered":"2024-09-11T12:51:00.983991+0000","last_clean":"2024-09-11T12:51:00.983991+0000","last_became_active":"2024-09-11T12:50:58.095063+0000","last_became_peered":"2024-09-11T12:50:58.095063+0000","last_unstale":"2024-09-11T12:51:00.983991+0000","last_undegraded":"2024-09-11T12:51:00.983991+0000","last_fullsized":"2024-09-11T12:51:00.983991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:56.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053911+0000","last_change":"2024-09-11T12:50:59.120274+0000","last_active":"2024-09-11T12:51:01.053911+0000","last_peered":"2024-09-11T12:51:01.053911+0000","last_clean":"2024-09-11T12:51:01.053911+0000","last_became_active":"2024-09-11T12:50:59.120001+0000","last_became_peered":"2024-09-11T12:50:59.120001+0000","last_unstale":"2024-09-11T12:51:01.053911+0000","last_undegraded":"2024-09-11T12:51:01.053911+0000","last_fullsized":"2024-09-11T12:51:01.053911+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:53:11.771016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275235+0000","last_change":"2024-09-11T12:50:58.112226+0000","last_active":"2024-09-11T12:51:01.275235+0000","last_peered":"2024-09-11T12:51:01.275235+0000","last_clean":"2024-09-11T12:51:01.275235+0000","last_became_active":"2024-09-11T12:50:58.110227+0000","last_became_peered":"2024-09-11T12:50:58.110227+0000","last_unstale":"2024-09-11T12:51:01.275235+0000","last_undegraded":"2024-09-11T12:51:01.275235+0000","last_fullsized":"2024-09-11T12:51:01.275235+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:16.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053892+0000","last_change":"2024-09-11T12:50:59.119682+0000","last_active":"2024-09-11T12:51:01.053892+0000","last_peered":"2024-09-11T12:51:01.053892+0000","last_clean":"2024-09-11T12:51:01.053892+0000","last_became_active":"2024-09-11T12:50:59.119485+0000","last_became_peered":"2024-09-11T12:50:59.119485+0000","last_unstale":"2024-09-11T12:51:01.053892+0000","last_undegraded":"2024-09-11T12:51:01.053892+0000","last_fullsized":"2024-09-11T12:51:01.053892+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:50:55.065714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095669+0000","last_change":"2024-09-11T12:50:58.110929+0000","last_active":"2024-09-11T12:51:01.095669+0000","last_peered":"2024-09-11T12:51:01.095669+0000","last_clean":"2024-09-11T12:51:01.095669+0000","last_became_active":"2024-09-11T12:50:58.109988+0000","last_became_peered":"2024-09-11T12:50:58.109988+0000","last_unstale":"2024-09-11T12:51:01.095669+0000","last_undegraded":"2024-09-11T12:51:01.095669+0000","last_fullsized":"2024-09-11T12:51:01.095669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:33.156439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112809+0000","last_change":"2024-09-11T12:50:59.106259+0000","last_active":"2024-09-11T12:51:00.112809+0000","last_peered":"2024-09-11T12:51:00.112809+0000","last_clean":"2024-09-11T12:51:00.112809+0000","last_became_active":"2024-09-11T12:50:59.105964+0000","last_became_peered":"2024-09-11T12:50:59.105964+0000","last_unstale":"2024-09-11T12:51:00.112809+0000","last_undegraded":"2024-09-11T12:51:00.112809+0000","last_fullsized":"2024-09-11T12:51:00.112809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:09.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984127+0000","last_change":"2024-09-11T12:50:58.107206+0000","last_active":"2024-09-11T12:51:00.984127+0000","last_peered":"2024-09-11T12:51:00.984127+0000","last_clean":"2024-09-11T12:51:00.984127+0000","last_became_active":"2024-09-11T12:50:58.106241+0000","last_became_peered":"2024-09-11T12:50:58.106241+0000","last_unstale":"2024-09-11T12:51:00.984127+0000","last_undegraded":"2024-09-11T12:51:00.984127+0000","last_fullsized":"2024-09-11T12:51:00.984127+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:35.146304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095543+0000","last_change":"2024-09-11T12:50:59.107286+0000","last_active":"2024-09-11T12:51:01.095543+0000","last_peered":"2024-09-11T12:51:01.095543+0000","last_clean":"2024-09-11T12:51:01.095543+0000","last_became_active":"2024-09-11T12:50:59.106556+0000","last_became_peered":"2024-09-11T12:50:59.106556+0000","last_unstale":"2024-09-11T12:51:01.095543+0000","last_undegraded":"2024-09-11T12:51:01.095543+0000","last_fullsized":"2024-09-11T12:51:01.095543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945199+0000","last_change":"2024-09-11T12:50:58.113126+0000","last_active":"2024-09-11T12:51:00.945199+0000","last_peered":"2024-09-11T12:51:00.945199+0000","last_clean":"2024-09-11T12:51:00.945199+0000","last_became_active":"2024-09-11T12:50:58.112864+0000","last_became_peered":"2024-09-11T12:50:58.112864+0000","last_unstale":"2024-09-11T12:51:00.945199+0000","last_undegraded":"2024-09-11T12:51:00.945199+0000","last_fullsized":"2024-09-11T12:51:00.945199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:55.613106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945473+0000","last_change":"2024-09-11T12:50:59.119354+0000","last_active":"2024-09-11T12:51:00.945473+0000","last_peered":"2024-09-11T12:51:00.945473+0000","last_clean":"2024-09-11T12:51:00.945473+0000","last_became_active":"2024-09-11T12:50:59.119126+0000","last_became_peered":"2024-09-11T12:50:59.119126+0000","last_unstale":"2024-09-11T12:51:00.945473+0000","last_undegraded":"2024-09-11T12:51:00.945473+0000","last_fullsized":"2024-09-11T12:51:00.945473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:29:52.277471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113662+0000","last_change":"2024-09-11T12:50:58.110730+0000","last_active":"2024-09-11T12:51:00.113662+0000","last_peered":"2024-09-11T12:51:00.113662+0000","last_clean":"2024-09-11T12:51:00.113662+0000","last_became_active":"2024-09-11T12:50:58.110432+0000","last_became_peered":"2024-09-11T12:50:58.110432+0000","last_unstale":"2024-09-11T12:51:00.113662+0000","last_undegraded":"2024-09-11T12:51:00.113662+0000","last_fullsized":"2024-09-11T12:51:00.113662+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:27.063998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275012+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:51:01.275012+0000","last_peered":"2024-09-11T12:51:01.275012+0000","last_clean":"2024-09-11T12:51:01.275012+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:51:01.275012+0000","last_undegraded":"2024-09-11T12:51:01.275012+0000","last_fullsized":"2024-09-11T12:51:01.275012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":797797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":208011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":208951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":207977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":208973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":791441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:51:03.215 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 -- 172.21.15.78:0/42764117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4780 msgr2=0x7fe510073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 --2- 172.21.15.78:0/42764117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4780 0x7fe510073f70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe500009900 tx=0x7fe50001c8a0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 -- 172.21.15.78:0/42764117 shutdown_connections 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 --2- 172.21.15.78:0/42764117 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100b4b60 0x7fe51015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 --2- 172.21.15.78:0/42764117 >> v2:172.21.15.157:3300/0 conn(0x7fe5100ccb20 0x7fe51015b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 --2- 172.21.15.78:0/42764117 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4780 0x7fe510073f70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 -- 172.21.15.78:0/42764117 >> 172.21.15.78:0/42764117 conn(0x7fe51006ca80 msgr2=0x7fe5100b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 -- 172.21.15.78:0/42764117 shutdown_connections 2024-09-11T12:51:03.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.386+0000 7fe516200640 1 -- 172.21.15.78:0/42764117 wait complete. 2024-09-11T12:51:03.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 Processor -- start 2024-09-11T12:51:03.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- start start 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5100b4780 0x7fe510073870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 0x7fe51014b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe51006f080 con 0x7fe5100b4b60 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe51006ef00 con 0x7fe5100b4780 2024-09-11T12:51:03.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe51006f200 con 0x7fe5100ccb20 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43294/0 (socket says 172.21.15.78:43294) 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 -- 172.21.15.78:0/1740803067 learned_addr learned my addr 172.21.15.78:0/1740803067 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50f400640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5100b4780 0x7fe510073870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50fe00640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 0x7fe51014b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 -- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 msgr2=0x7fe51014b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 0x7fe51014b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 -- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5100b4780 msgr2=0x7fe510073870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5100b4780 0x7fe510073870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 -- 172.21.15.78:0/1740803067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe510071fb0 con 0x7fe5100b4b60 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50fe00640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 0x7fe51014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:03.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe50ea00640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00d750 tx=0x7fe4fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc017070 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4fc009d20 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe510072290 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc00de60 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.390+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe510072850 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe510070050 con 0x7fe5100b4b60 2024-09-11T12:51:03.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fe4fc004280 con 0x7fe5100b4b60 2024-09-11T12:51:03.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe507400640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 0x7fe4f0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fe4fc092520 con 0x7fe5100b4b60 2024-09-11T12:51:03.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe50f400640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 0x7fe4f0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.394+0000 7fe50f400640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 0x7fe4f0073d20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5000af000 tx=0x7fe50001b0e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.398+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe4fc020020 con 0x7fe5100b4b60 2024-09-11T12:51:03.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.562+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe51006fa90 con 0x7fe4f0071860 2024-09-11T12:51:03.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.566+0000 7fe507400640 1 -- 172.21.15.78:0/1740803067 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fe5000f9840 con 0x7fe4f0071860 2024-09-11T12:51:03.574 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:03.575 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:51:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 msgr2=0x7fe4f0073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 0x7fe4f0073d20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe5000af000 tx=0x7fe50001b0e0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 msgr2=0x7fe510164e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc00d750 tx=0x7fe4fc00dc10 comp rx=0 tx=0).stop 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 shutdown_connections 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe4f0071860 0x7fe4f0073d20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5100ccb20 0x7fe51014b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5100b4b60 0x7fe510164e10 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 --2- 172.21.15.78:0/1740803067 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5100b4780 0x7fe510073870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 >> 172.21.15.78:0/1740803067 conn(0x7fe51006ca80 msgr2=0x7fe51014c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 shutdown_connections 2024-09-11T12:51:03.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.570+0000 7fe516200640 1 -- 172.21.15.78:0/1740803067 wait complete. 2024-09-11T12:51:03.594 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-09-11T12:51:03.122817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":185,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242320,"kb_used_data":5276,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732400,"statfs":{"total":773094113280,"available":772845977600,"internally_reserved":0,"allocated":5402624,"data_stored":2837968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001040"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275503+0000","last_change":"2024-09-11T12:50:59.107007+0000","last_active":"2024-09-11T12:51:01.275503+0000","last_peered":"2024-09-11T12:51:01.275503+0000","last_clean":"2024-09-11T12:51:01.275503+0000","last_became_active":"2024-09-11T12:50:59.106740+0000","last_became_peered":"2024-09-11T12:50:59.106740+0000","last_unstale":"2024-09-11T12:51:01.275503+0000","last_undegraded":"2024-09-11T12:51:01.275503+0000","last_fullsized":"2024-09-11T12:51:01.275503+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:53.984965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113457+0000","last_change":"2024-09-11T12:50:59.122247+0000","last_active":"2024-09-11T12:51:00.113457+0000","last_peered":"2024-09-11T12:51:00.113457+0000","last_clean":"2024-09-11T12:51:00.113457+0000","last_became_active":"2024-09-11T12:50:59.121888+0000","last_became_peered":"2024-09-11T12:50:59.121888+0000","last_unstale":"2024-09-11T12:51:00.113457+0000","last_undegraded":"2024-09-11T12:51:00.113457+0000","last_fullsized":"2024-09-11T12:51:00.113457+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:21.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095678+0000","last_change":"2024-09-11T12:50:59.101936+0000","last_active":"2024-09-11T12:51:01.095678+0000","last_peered":"2024-09-11T12:51:01.095678+0000","last_clean":"2024-09-11T12:51:01.095678+0000","last_became_active":"2024-09-11T12:50:59.101616+0000","last_became_peered":"2024-09-11T12:50:59.101616+0000","last_unstale":"2024-09-11T12:51:01.095678+0000","last_undegraded":"2024-09-11T12:51:01.095678+0000","last_fullsized":"2024-09-11T12:51:01.095678+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:04.378115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945300+0000","last_change":"2024-09-11T12:50:59.121457+0000","last_active":"2024-09-11T12:51:00.945300+0000","last_peered":"2024-09-11T12:51:00.945300+0000","last_clean":"2024-09-11T12:51:00.945300+0000","last_became_active":"2024-09-11T12:50:59.121207+0000","last_became_peered":"2024-09-11T12:50:59.121207+0000","last_unstale":"2024-09-11T12:51:00.945300+0000","last_undegraded":"2024-09-11T12:51:00.945300+0000","last_fullsized":"2024-09-11T12:51:00.945300+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:28.818518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112083+0000","last_change":"2024-09-11T12:50:59.121718+0000","last_active":"2024-09-11T12:51:00.112083+0000","last_peered":"2024-09-11T12:51:00.112083+0000","last_clean":"2024-09-11T12:51:00.112083+0000","last_became_active":"2024-09-11T12:50:59.121397+0000","last_became_peered":"2024-09-11T12:50:59.121397+0000","last_unstale":"2024-09-11T12:51:00.112083+0000","last_undegraded":"2024-09-11T12:51:00.112083+0000","last_fullsized":"2024-09-11T12:51:00.112083+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:56.902491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053727+0000","last_change":"2024-09-11T12:50:59.120811+0000","last_active":"2024-09-11T12:51:01.053727+0000","last_peered":"2024-09-11T12:51:01.053727+0000","last_clean":"2024-09-11T12:51:01.053727+0000","last_became_active":"2024-09-11T12:50:59.120564+0000","last_became_peered":"2024-09-11T12:50:59.120564+0000","last_unstale":"2024-09-11T12:51:01.053727+0000","last_undegraded":"2024-09-11T12:51:01.053727+0000","last_fullsized":"2024-09-11T12:51:01.053727+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:51.362297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275262+0000","last_change":"2024-09-11T12:50:59.113784+0000","last_active":"2024-09-11T12:51:01.275262+0000","last_peered":"2024-09-11T12:51:01.275262+0000","last_clean":"2024-09-11T12:51:01.275262+0000","last_became_active":"2024-09-11T12:50:59.113508+0000","last_became_peered":"2024-09-11T12:50:59.113508+0000","last_unstale":"2024-09-11T12:51:01.275262+0000","last_undegraded":"2024-09-11T12:51:01.275262+0000","last_fullsized":"2024-09-11T12:51:01.275262+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:04.722605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944733+0000","last_change":"2024-09-11T12:50:59.127489+0000","last_active":"2024-09-11T12:51:00.944733+0000","last_peered":"2024-09-11T12:51:00.944733+0000","last_clean":"2024-09-11T12:51:00.944733+0000","last_became_active":"2024-09-11T12:50:59.127156+0000","last_became_peered":"2024-09-11T12:50:59.127156+0000","last_unstale":"2024-09-11T12:51:00.944733+0000","last_undegraded":"2024-09-11T12:51:00.944733+0000","last_fullsized":"2024-09-11T12:51:00.944733+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:28:19.143693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945252+0000","last_change":"2024-09-11T12:50:59.116385+0000","last_active":"2024-09-11T12:51:00.945252+0000","last_peered":"2024-09-11T12:51:00.945252+0000","last_clean":"2024-09-11T12:51:00.945252+0000","last_became_active":"2024-09-11T12:50:59.116193+0000","last_became_peered":"2024-09-11T12:50:59.116193+0000","last_unstale":"2024-09-11T12:51:00.945252+0000","last_undegraded":"2024-09-11T12:51:00.945252+0000","last_fullsized":"2024-09-11T12:51:00.945252+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:32:08.471568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945285+0000","last_change":"2024-09-11T12:50:59.126765+0000","last_active":"2024-09-11T12:51:00.945285+0000","last_peered":"2024-09-11T12:51:00.945285+0000","last_clean":"2024-09-11T12:51:00.945285+0000","last_became_active":"2024-09-11T12:50:59.126451+0000","last_became_peered":"2024-09-11T12:50:59.126451+0000","last_unstale":"2024-09-11T12:51:00.945285+0000","last_undegraded":"2024-09-11T12:51:00.945285+0000","last_fullsized":"2024-09-11T12:51:00.945285+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:46:58.988322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945129+0000","last_change":"2024-09-11T12:50:59.119125+0000","last_active":"2024-09-11T12:51:00.945129+0000","last_peered":"2024-09-11T12:51:00.945129+0000","last_clean":"2024-09-11T12:51:00.945129+0000","last_became_active":"2024-09-11T12:50:59.118813+0000","last_became_peered":"2024-09-11T12:50:59.118813+0000","last_unstale":"2024-09-11T12:51:00.945129+0000","last_undegraded":"2024-09-11T12:51:00.945129+0000","last_fullsized":"2024-09-11T12:51:00.945129+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:57:25.996974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113026+0000","last_change":"2024-09-11T12:50:59.102276+0000","last_active":"2024-09-11T12:51:00.113026+0000","last_peered":"2024-09-11T12:51:00.113026+0000","last_clean":"2024-09-11T12:51:00.113026+0000","last_became_active":"2024-09-11T12:50:59.100926+0000","last_became_peered":"2024-09-11T12:50:59.100926+0000","last_unstale":"2024-09-11T12:51:00.113026+0000","last_undegraded":"2024-09-11T12:51:00.113026+0000","last_fullsized":"2024-09-11T12:51:00.113026+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:04.722605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111423+0000","last_change":"2024-09-11T12:50:59.104330+0000","last_active":"2024-09-11T12:51:00.111423+0000","last_peered":"2024-09-11T12:51:00.111423+0000","last_clean":"2024-09-11T12:51:00.111423+0000","last_became_active":"2024-09-11T12:50:59.104067+0000","last_became_peered":"2024-09-11T12:50:59.104067+0000","last_unstale":"2024-09-11T12:51:00.111423+0000","last_undegraded":"2024-09-11T12:51:00.111423+0000","last_fullsized":"2024-09-11T12:51:00.111423+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:14:09.955217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944703+0000","last_change":"2024-09-11T12:50:59.116406+0000","last_active":"2024-09-11T12:51:00.944703+0000","last_peered":"2024-09-11T12:51:00.944703+0000","last_clean":"2024-09-11T12:51:00.944703+0000","last_became_active":"2024-09-11T12:50:59.116203+0000","last_became_peered":"2024-09-11T12:50:59.116203+0000","last_unstale":"2024-09-11T12:51:00.944703+0000","last_undegraded":"2024-09-11T12:51:00.944703+0000","last_fullsized":"2024-09-11T12:51:00.944703+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:47.289447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053692+0000","last_change":"2024-09-11T12:50:59.107911+0000","last_active":"2024-09-11T12:51:01.053692+0000","last_peered":"2024-09-11T12:51:01.053692+0000","last_clean":"2024-09-11T12:51:01.053692+0000","last_became_active":"2024-09-11T12:50:59.106700+0000","last_became_peered":"2024-09-11T12:50:59.106700+0000","last_unstale":"2024-09-11T12:51:01.053692+0000","last_undegraded":"2024-09-11T12:51:01.053692+0000","last_fullsized":"2024-09-11T12:51:01.053692+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:22:24.786016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112491+0000","last_change":"2024-09-11T12:50:59.127100+0000","last_active":"2024-09-11T12:51:00.112491+0000","last_peered":"2024-09-11T12:51:00.112491+0000","last_clean":"2024-09-11T12:51:00.112491+0000","last_became_active":"2024-09-11T12:50:59.126575+0000","last_became_peered":"2024-09-11T12:50:59.126575+0000","last_unstale":"2024-09-11T12:51:00.112491+0000","last_undegraded":"2024-09-11T12:51:00.112491+0000","last_fullsized":"2024-09-11T12:51:00.112491+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:46.977587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095840+0000","last_change":"2024-09-11T12:50:59.096173+0000","last_active":"2024-09-11T12:51:01.095840+0000","last_peered":"2024-09-11T12:51:01.095840+0000","last_clean":"2024-09-11T12:51:01.095840+0000","last_became_active":"2024-09-11T12:50:59.095851+0000","last_became_peered":"2024-09-11T12:50:59.095851+0000","last_unstale":"2024-09-11T12:51:01.095840+0000","last_undegraded":"2024-09-11T12:51:01.095840+0000","last_fullsized":"2024-09-11T12:51:01.095840+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:29:41.655196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095728+0000","last_change":"2024-09-11T12:50:59.122884+0000","last_active":"2024-09-11T12:51:01.095728+0000","last_peered":"2024-09-11T12:51:01.095728+0000","last_clean":"2024-09-11T12:51:01.095728+0000","last_became_active":"2024-09-11T12:50:59.122007+0000","last_became_peered":"2024-09-11T12:50:59.122007+0000","last_unstale":"2024-09-11T12:51:01.095728+0000","last_undegraded":"2024-09-11T12:51:01.095728+0000","last_fullsized":"2024-09-11T12:51:01.095728+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:04.184506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113611+0000","last_change":"2024-09-11T12:50:59.126950+0000","last_active":"2024-09-11T12:51:00.113611+0000","last_peered":"2024-09-11T12:51:00.113611+0000","last_clean":"2024-09-11T12:51:00.113611+0000","last_became_active":"2024-09-11T12:50:59.126629+0000","last_became_peered":"2024-09-11T12:50:59.126629+0000","last_unstale":"2024-09-11T12:51:00.113611+0000","last_undegraded":"2024-09-11T12:51:00.113611+0000","last_fullsized":"2024-09-11T12:51:00.113611+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:17.549541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111414+0000","last_change":"2024-09-11T12:50:59.127181+0000","last_active":"2024-09-11T12:51:00.111414+0000","last_peered":"2024-09-11T12:51:00.111414+0000","last_clean":"2024-09-11T12:51:00.111414+0000","last_became_active":"2024-09-11T12:50:59.126907+0000","last_became_peered":"2024-09-11T12:50:59.126907+0000","last_unstale":"2024-09-11T12:51:00.111414+0000","last_undegraded":"2024-09-11T12:51:00.111414+0000","last_fullsized":"2024-09-11T12:51:00.111414+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:50.797002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053902+0000","last_change":"2024-09-11T12:50:59.117147+0000","last_active":"2024-09-11T12:51:01.053902+0000","last_peered":"2024-09-11T12:51:01.053902+0000","last_clean":"2024-09-11T12:51:01.053902+0000","last_became_active":"2024-09-11T12:50:59.116898+0000","last_became_peered":"2024-09-11T12:50:59.116898+0000","last_unstale":"2024-09-11T12:51:01.053902+0000","last_undegraded":"2024-09-11T12:51:01.053902+0000","last_fullsized":"2024-09-11T12:51:01.053902+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:24:46.684745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945037+0000","last_change":"2024-09-11T12:50:59.125165+0000","last_active":"2024-09-11T12:51:00.945037+0000","last_peered":"2024-09-11T12:51:00.945037+0000","last_clean":"2024-09-11T12:51:00.945037+0000","last_became_active":"2024-09-11T12:50:59.124861+0000","last_became_peered":"2024-09-11T12:50:59.124861+0000","last_unstale":"2024-09-11T12:51:00.945037+0000","last_undegraded":"2024-09-11T12:51:00.945037+0000","last_fullsized":"2024-09-11T12:51:00.945037+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:17.084103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112242+0000","last_change":"2024-09-11T12:50:59.116756+0000","last_active":"2024-09-11T12:51:00.112242+0000","last_peered":"2024-09-11T12:51:00.112242+0000","last_clean":"2024-09-11T12:51:00.112242+0000","last_became_active":"2024-09-11T12:50:59.116454+0000","last_became_peered":"2024-09-11T12:50:59.116454+0000","last_unstale":"2024-09-11T12:51:00.112242+0000","last_undegraded":"2024-09-11T12:51:00.112242+0000","last_fullsized":"2024-09-11T12:51:00.112242+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:56.473558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112644+0000","last_change":"2024-09-11T12:50:59.116838+0000","last_active":"2024-09-11T12:51:00.112644+0000","last_peered":"2024-09-11T12:51:00.112644+0000","last_clean":"2024-09-11T12:51:00.112644+0000","last_became_active":"2024-09-11T12:50:59.116499+0000","last_became_peered":"2024-09-11T12:50:59.116499+0000","last_unstale":"2024-09-11T12:51:00.112644+0000","last_undegraded":"2024-09-11T12:51:00.112644+0000","last_fullsized":"2024-09-11T12:51:00.112644+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:17.529852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113757+0000","last_change":"2024-09-11T12:50:59.112746+0000","last_active":"2024-09-11T12:51:00.113757+0000","last_peered":"2024-09-11T12:51:00.113757+0000","last_clean":"2024-09-11T12:51:00.113757+0000","last_became_active":"2024-09-11T12:50:59.111826+0000","last_became_peered":"2024-09-11T12:50:59.111826+0000","last_unstale":"2024-09-11T12:51:00.113757+0000","last_undegraded":"2024-09-11T12:51:00.113757+0000","last_fullsized":"2024-09-11T12:51:00.113757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:04.378115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111936+0000","last_change":"2024-09-11T12:50:59.122778+0000","last_active":"2024-09-11T12:51:00.111936+0000","last_peered":"2024-09-11T12:51:00.111936+0000","last_clean":"2024-09-11T12:51:00.111936+0000","last_became_active":"2024-09-11T12:50:59.122367+0000","last_became_peered":"2024-09-11T12:50:59.122367+0000","last_unstale":"2024-09-11T12:51:00.111936+0000","last_undegraded":"2024-09-11T12:51:00.111936+0000","last_fullsized":"2024-09-11T12:51:00.111936+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:21.146426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095748+0000","last_change":"2024-09-11T12:50:59.096082+0000","last_active":"2024-09-11T12:51:01.095748+0000","last_peered":"2024-09-11T12:51:01.095748+0000","last_clean":"2024-09-11T12:51:01.095748+0000","last_became_active":"2024-09-11T12:50:59.095834+0000","last_became_peered":"2024-09-11T12:50:59.095834+0000","last_unstale":"2024-09-11T12:51:01.095748+0000","last_undegraded":"2024-09-11T12:51:01.095748+0000","last_fullsized":"2024-09-11T12:51:01.095748+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:57.753073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113067+0000","last_change":"2024-09-11T12:50:59.101058+0000","last_active":"2024-09-11T12:51:00.113067+0000","last_peered":"2024-09-11T12:51:00.113067+0000","last_clean":"2024-09-11T12:51:00.113067+0000","last_became_active":"2024-09-11T12:50:59.100704+0000","last_became_peered":"2024-09-11T12:50:59.100704+0000","last_unstale":"2024-09-11T12:51:00.113067+0000","last_undegraded":"2024-09-11T12:51:00.113067+0000","last_fullsized":"2024-09-11T12:51:00.113067+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:12.019353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112660+0000","last_change":"2024-09-11T12:50:59.118380+0000","last_active":"2024-09-11T12:51:00.112660+0000","last_peered":"2024-09-11T12:51:00.112660+0000","last_clean":"2024-09-11T12:51:00.112660+0000","last_became_active":"2024-09-11T12:50:59.118006+0000","last_became_peered":"2024-09-11T12:50:59.118006+0000","last_unstale":"2024-09-11T12:51:00.112660+0000","last_undegraded":"2024-09-11T12:51:00.112660+0000","last_fullsized":"2024-09-11T12:51:00.112660+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:53.401485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095489+0000","last_change":"2024-09-11T12:50:59.115457+0000","last_active":"2024-09-11T12:51:01.095489+0000","last_peered":"2024-09-11T12:51:01.095489+0000","last_clean":"2024-09-11T12:51:01.095489+0000","last_became_active":"2024-09-11T12:50:59.111950+0000","last_became_peered":"2024-09-11T12:50:59.111950+0000","last_unstale":"2024-09-11T12:51:01.095489+0000","last_undegraded":"2024-09-11T12:51:01.095489+0000","last_fullsized":"2024-09-11T12:51:01.095489+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053694+0000","last_change":"2024-09-11T12:50:59.111483+0000","last_active":"2024-09-11T12:51:01.053694+0000","last_peered":"2024-09-11T12:51:01.053694+0000","last_clean":"2024-09-11T12:51:01.053694+0000","last_became_active":"2024-09-11T12:50:59.111216+0000","last_became_peered":"2024-09-11T12:50:59.111216+0000","last_unstale":"2024-09-11T12:51:01.053694+0000","last_undegraded":"2024-09-11T12:51:01.053694+0000","last_fullsized":"2024-09-11T12:51:01.053694+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:35.614879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111298+0000","last_change":"2024-09-11T12:50:59.112810+0000","last_active":"2024-09-11T12:51:00.111298+0000","last_peered":"2024-09-11T12:51:00.111298+0000","last_clean":"2024-09-11T12:51:00.111298+0000","last_became_active":"2024-09-11T12:50:59.112518+0000","last_became_peered":"2024-09-11T12:50:59.112518+0000","last_unstale":"2024-09-11T12:51:00.111298+0000","last_undegraded":"2024-09-11T12:51:00.111298+0000","last_fullsized":"2024-09-11T12:51:00.111298+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:42:28.962546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984428+0000","last_change":"2024-09-11T12:50:59.122003+0000","last_active":"2024-09-11T12:51:00.984428+0000","last_peered":"2024-09-11T12:51:00.984428+0000","last_clean":"2024-09-11T12:51:00.984428+0000","last_became_active":"2024-09-11T12:50:59.121651+0000","last_became_peered":"2024-09-11T12:50:59.121651+0000","last_unstale":"2024-09-11T12:51:00.984428+0000","last_undegraded":"2024-09-11T12:51:00.984428+0000","last_fullsized":"2024-09-11T12:51:00.984428+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"44'4","reported_seq":19,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054431+0000","last_change":"2024-09-11T12:50:58.107932+0000","last_active":"2024-09-11T12:51:01.054431+0000","last_peered":"2024-09-11T12:51:01.054431+0000","last_clean":"2024-09-11T12:51:01.054431+0000","last_became_active":"2024-09-11T12:50:58.107680+0000","last_became_peered":"2024-09-11T12:50:58.107680+0000","last_unstale":"2024-09-11T12:51:01.054431+0000","last_undegraded":"2024-09-11T12:51:01.054431+0000","last_fullsized":"2024-09-11T12:51:01.054431+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:55.899646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113730+0000","last_change":"2024-09-11T12:50:59.104959+0000","last_active":"2024-09-11T12:51:00.113730+0000","last_peered":"2024-09-11T12:51:00.113730+0000","last_clean":"2024-09-11T12:51:00.113730+0000","last_became_active":"2024-09-11T12:50:59.103992+0000","last_became_peered":"2024-09-11T12:50:59.103992+0000","last_unstale":"2024-09-11T12:51:00.113730+0000","last_undegraded":"2024-09-11T12:51:00.113730+0000","last_fullsized":"2024-09-11T12:51:00.113730+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:26.869564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095765+0000","last_change":"2024-09-11T12:50:58.107594+0000","last_active":"2024-09-11T12:51:01.095765+0000","last_peered":"2024-09-11T12:51:01.095765+0000","last_clean":"2024-09-11T12:51:01.095765+0000","last_became_active":"2024-09-11T12:50:58.107308+0000","last_became_peered":"2024-09-11T12:50:58.107308+0000","last_unstale":"2024-09-11T12:51:01.095765+0000","last_undegraded":"2024-09-11T12:51:01.095765+0000","last_fullsized":"2024-09-11T12:51:01.095765+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:51.002666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113198+0000","last_change":"2024-09-11T12:50:59.107384+0000","last_active":"2024-09-11T12:51:00.113198+0000","last_peered":"2024-09-11T12:51:00.113198+0000","last_clean":"2024-09-11T12:51:00.113198+0000","last_became_active":"2024-09-11T12:50:59.107111+0000","last_became_peered":"2024-09-11T12:50:59.107111+0000","last_unstale":"2024-09-11T12:51:00.113198+0000","last_undegraded":"2024-09-11T12:51:00.113198+0000","last_fullsized":"2024-09-11T12:51:00.113198+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:08.742585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984377+0000","last_change":"2024-09-11T12:50:58.102485+0000","last_active":"2024-09-11T12:51:00.984377+0000","last_peered":"2024-09-11T12:51:00.984377+0000","last_clean":"2024-09-11T12:51:00.984377+0000","last_became_active":"2024-09-11T12:50:58.102108+0000","last_became_peered":"2024-09-11T12:50:58.102108+0000","last_unstale":"2024-09-11T12:51:00.984377+0000","last_undegraded":"2024-09-11T12:51:00.984377+0000","last_fullsized":"2024-09-11T12:51:00.984377+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:29.888058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984447+0000","last_change":"2024-09-11T12:50:59.110439+0000","last_active":"2024-09-11T12:51:00.984447+0000","last_peered":"2024-09-11T12:51:00.984447+0000","last_clean":"2024-09-11T12:51:00.984447+0000","last_became_active":"2024-09-11T12:50:59.110156+0000","last_became_peered":"2024-09-11T12:50:59.110156+0000","last_unstale":"2024-09-11T12:51:00.984447+0000","last_undegraded":"2024-09-11T12:51:00.984447+0000","last_fullsized":"2024-09-11T12:51:00.984447+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275130+0000","last_change":"2024-09-11T12:50:58.109512+0000","last_active":"2024-09-11T12:51:01.275130+0000","last_peered":"2024-09-11T12:51:01.275130+0000","last_clean":"2024-09-11T12:51:01.275130+0000","last_became_active":"2024-09-11T12:50:58.109167+0000","last_became_peered":"2024-09-11T12:50:58.109167+0000","last_unstale":"2024-09-11T12:51:01.275130+0000","last_undegraded":"2024-09-11T12:51:01.275130+0000","last_fullsized":"2024-09-11T12:51:01.275130+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:03.290154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095601+0000","last_change":"2024-09-11T12:50:59.106575+0000","last_active":"2024-09-11T12:51:01.095601+0000","last_peered":"2024-09-11T12:51:01.095601+0000","last_clean":"2024-09-11T12:51:01.095601+0000","last_became_active":"2024-09-11T12:50:59.105335+0000","last_became_peered":"2024-09-11T12:50:59.105335+0000","last_unstale":"2024-09-11T12:51:01.095601+0000","last_undegraded":"2024-09-11T12:51:01.095601+0000","last_fullsized":"2024-09-11T12:51:01.095601+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:17.549541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053982+0000","last_change":"2024-09-11T12:50:58.111035+0000","last_active":"2024-09-11T12:51:01.053982+0000","last_peered":"2024-09-11T12:51:01.053982+0000","last_clean":"2024-09-11T12:51:01.053982+0000","last_became_active":"2024-09-11T12:50:58.110738+0000","last_became_peered":"2024-09-11T12:50:58.110738+0000","last_unstale":"2024-09-11T12:51:01.053982+0000","last_undegraded":"2024-09-11T12:51:01.053982+0000","last_fullsized":"2024-09-11T12:51:01.053982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:08.626408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945109+0000","last_change":"2024-09-11T12:50:59.128553+0000","last_active":"2024-09-11T12:51:00.945109+0000","last_peered":"2024-09-11T12:51:00.945109+0000","last_clean":"2024-09-11T12:51:00.945109+0000","last_became_active":"2024-09-11T12:50:59.127880+0000","last_became_peered":"2024-09-11T12:50:59.127880+0000","last_unstale":"2024-09-11T12:51:00.945109+0000","last_undegraded":"2024-09-11T12:51:00.945109+0000","last_fullsized":"2024-09-11T12:51:00.945109+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:02:25.868829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095594+0000","last_change":"2024-09-11T12:50:58.098497+0000","last_active":"2024-09-11T12:51:01.095594+0000","last_peered":"2024-09-11T12:51:01.095594+0000","last_clean":"2024-09-11T12:51:01.095594+0000","last_became_active":"2024-09-11T12:50:58.098063+0000","last_became_peered":"2024-09-11T12:50:58.098063+0000","last_unstale":"2024-09-11T12:51:01.095594+0000","last_undegraded":"2024-09-11T12:51:01.095594+0000","last_fullsized":"2024-09-11T12:51:01.095594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:59.034777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944793+0000","last_change":"2024-09-11T12:50:59.127848+0000","last_active":"2024-09-11T12:51:00.944793+0000","last_peered":"2024-09-11T12:51:00.944793+0000","last_clean":"2024-09-11T12:51:00.944793+0000","last_became_active":"2024-09-11T12:50:59.126825+0000","last_became_peered":"2024-09-11T12:50:59.126825+0000","last_unstale":"2024-09-11T12:51:00.944793+0000","last_undegraded":"2024-09-11T12:51:00.944793+0000","last_fullsized":"2024-09-11T12:51:00.944793+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:11:06.811241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112760+0000","last_change":"2024-09-11T12:50:58.104299+0000","last_active":"2024-09-11T12:51:00.112760+0000","last_peered":"2024-09-11T12:51:00.112760+0000","last_clean":"2024-09-11T12:51:00.112760+0000","last_became_active":"2024-09-11T12:50:58.103952+0000","last_became_peered":"2024-09-11T12:50:58.103952+0000","last_unstale":"2024-09-11T12:51:00.112760+0000","last_undegraded":"2024-09-11T12:51:00.112760+0000","last_fullsized":"2024-09-11T12:51:00.112760+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:16.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112065+0000","last_change":"2024-09-11T12:50:59.116985+0000","last_active":"2024-09-11T12:51:00.112065+0000","last_peered":"2024-09-11T12:51:00.112065+0000","last_clean":"2024-09-11T12:51:00.112065+0000","last_became_active":"2024-09-11T12:50:59.116709+0000","last_became_peered":"2024-09-11T12:50:59.116709+0000","last_unstale":"2024-09-11T12:51:00.112065+0000","last_undegraded":"2024-09-11T12:51:00.112065+0000","last_fullsized":"2024-09-11T12:51:00.112065+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:30.890904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944857+0000","last_change":"2024-09-11T12:50:58.106830+0000","last_active":"2024-09-11T12:51:00.944857+0000","last_peered":"2024-09-11T12:51:00.944857+0000","last_clean":"2024-09-11T12:51:00.944857+0000","last_became_active":"2024-09-11T12:50:58.105732+0000","last_became_peered":"2024-09-11T12:50:58.105732+0000","last_unstale":"2024-09-11T12:51:00.944857+0000","last_undegraded":"2024-09-11T12:51:00.944857+0000","last_fullsized":"2024-09-11T12:51:00.944857+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:39:14.071837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095669+0000","last_change":"2024-09-11T12:50:59.122159+0000","last_active":"2024-09-11T12:51:01.095669+0000","last_peered":"2024-09-11T12:51:01.095669+0000","last_clean":"2024-09-11T12:51:01.095669+0000","last_became_active":"2024-09-11T12:50:59.121884+0000","last_became_peered":"2024-09-11T12:50:59.121884+0000","last_unstale":"2024-09-11T12:51:01.095669+0000","last_undegraded":"2024-09-11T12:51:01.095669+0000","last_fullsized":"2024-09-11T12:51:01.095669+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:26.869564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095830+0000","last_change":"2024-09-11T12:50:58.097602+0000","last_active":"2024-09-11T12:51:01.095830+0000","last_peered":"2024-09-11T12:51:01.095830+0000","last_clean":"2024-09-11T12:51:01.095830+0000","last_became_active":"2024-09-11T12:50:58.097211+0000","last_became_peered":"2024-09-11T12:50:58.097211+0000","last_unstale":"2024-09-11T12:51:01.095830+0000","last_undegraded":"2024-09-11T12:51:01.095830+0000","last_fullsized":"2024-09-11T12:51:01.095830+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:36.906734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275581+0000","last_change":"2024-09-11T12:50:59.110182+0000","last_active":"2024-09-11T12:51:01.275581+0000","last_peered":"2024-09-11T12:51:01.275581+0000","last_clean":"2024-09-11T12:51:01.275581+0000","last_became_active":"2024-09-11T12:50:59.109853+0000","last_became_peered":"2024-09-11T12:50:59.109853+0000","last_unstale":"2024-09-11T12:51:01.275581+0000","last_undegraded":"2024-09-11T12:51:01.275581+0000","last_fullsized":"2024-09-11T12:51:01.275581+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:48:52.073833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945425+0000","last_change":"2024-09-11T12:50:58.111691+0000","last_active":"2024-09-11T12:51:00.945425+0000","last_peered":"2024-09-11T12:51:00.945425+0000","last_clean":"2024-09-11T12:51:00.945425+0000","last_became_active":"2024-09-11T12:50:58.109298+0000","last_became_peered":"2024-09-11T12:50:58.109298+0000","last_unstale":"2024-09-11T12:51:00.945425+0000","last_undegraded":"2024-09-11T12:51:00.945425+0000","last_fullsized":"2024-09-11T12:51:00.945425+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:33.419831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054257+0000","last_change":"2024-09-11T12:50:59.120690+0000","last_active":"2024-09-11T12:51:01.054257+0000","last_peered":"2024-09-11T12:51:01.054257+0000","last_clean":"2024-09-11T12:51:01.054257+0000","last_became_active":"2024-09-11T12:50:59.120455+0000","last_became_peered":"2024-09-11T12:50:59.120455+0000","last_unstale":"2024-09-11T12:51:01.054257+0000","last_undegraded":"2024-09-11T12:51:01.054257+0000","last_fullsized":"2024-09-11T12:51:01.054257+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:01.279697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"44'3","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275535+0000","last_change":"2024-09-11T12:50:58.113351+0000","last_active":"2024-09-11T12:51:01.275535+0000","last_peered":"2024-09-11T12:51:01.275535+0000","last_clean":"2024-09-11T12:51:01.275535+0000","last_became_active":"2024-09-11T12:50:58.113091+0000","last_became_peered":"2024-09-11T12:50:58.113091+0000","last_unstale":"2024-09-11T12:51:01.275535+0000","last_undegraded":"2024-09-11T12:51:01.275535+0000","last_fullsized":"2024-09-11T12:51:01.275535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:52.398639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275442+0000","last_change":"2024-09-11T12:50:59.119332+0000","last_active":"2024-09-11T12:51:01.275442+0000","last_peered":"2024-09-11T12:51:01.275442+0000","last_clean":"2024-09-11T12:51:01.275442+0000","last_became_active":"2024-09-11T12:50:59.119036+0000","last_became_peered":"2024-09-11T12:50:59.119036+0000","last_unstale":"2024-09-11T12:51:01.275442+0000","last_undegraded":"2024-09-11T12:51:01.275442+0000","last_fullsized":"2024-09-11T12:51:01.275442+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:08.742585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"44'6","reported_seq":21,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111452+0000","last_change":"2024-09-11T12:50:58.100258+0000","last_active":"2024-09-11T12:51:00.111452+0000","last_peered":"2024-09-11T12:51:00.111452+0000","last_clean":"2024-09-11T12:51:00.111452+0000","last_became_active":"2024-09-11T12:50:58.099931+0000","last_became_peered":"2024-09-11T12:50:58.099931+0000","last_unstale":"2024-09-11T12:51:00.111452+0000","last_undegraded":"2024-09-11T12:51:00.111452+0000","last_fullsized":"2024-09-11T12:51:00.111452+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:46:18.132465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112775+0000","last_change":"2024-09-11T12:50:59.126925+0000","last_active":"2024-09-11T12:51:00.112775+0000","last_peered":"2024-09-11T12:51:00.112775+0000","last_clean":"2024-09-11T12:51:00.112775+0000","last_became_active":"2024-09-11T12:50:59.126506+0000","last_became_peered":"2024-09-11T12:50:59.126506+0000","last_unstale":"2024-09-11T12:51:00.112775+0000","last_undegraded":"2024-09-11T12:51:00.112775+0000","last_fullsized":"2024-09-11T12:51:00.112775+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:28.066844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054012+0000","last_change":"2024-09-11T12:50:58.112901+0000","last_active":"2024-09-11T12:51:01.054012+0000","last_peered":"2024-09-11T12:51:01.054012+0000","last_clean":"2024-09-11T12:51:01.054012+0000","last_became_active":"2024-09-11T12:50:58.112637+0000","last_became_peered":"2024-09-11T12:50:58.112637+0000","last_unstale":"2024-09-11T12:51:01.054012+0000","last_undegraded":"2024-09-11T12:51:01.054012+0000","last_fullsized":"2024-09-11T12:51:01.054012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:11.016508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945015+0000","last_change":"2024-09-11T12:50:59.128324+0000","last_active":"2024-09-11T12:51:00.945015+0000","last_peered":"2024-09-11T12:51:00.945015+0000","last_clean":"2024-09-11T12:51:00.945015+0000","last_became_active":"2024-09-11T12:50:59.127494+0000","last_became_peered":"2024-09-11T12:50:59.127494+0000","last_unstale":"2024-09-11T12:51:00.945015+0000","last_undegraded":"2024-09-11T12:51:00.945015+0000","last_fullsized":"2024-09-11T12:51:00.945015+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:40:24.792102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275120+0000","last_change":"2024-09-11T12:50:58.113306+0000","last_active":"2024-09-11T12:51:01.275120+0000","last_peered":"2024-09-11T12:51:01.275120+0000","last_clean":"2024-09-11T12:51:01.275120+0000","last_became_active":"2024-09-11T12:50:58.112967+0000","last_became_peered":"2024-09-11T12:50:58.112967+0000","last_unstale":"2024-09-11T12:51:01.275120+0000","last_undegraded":"2024-09-11T12:51:01.275120+0000","last_fullsized":"2024-09-11T12:51:01.275120+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:16.546696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053690+0000","last_change":"2024-09-11T12:50:59.124336+0000","last_active":"2024-09-11T12:51:01.053690+0000","last_peered":"2024-09-11T12:51:01.053690+0000","last_clean":"2024-09-11T12:51:01.053690+0000","last_became_active":"2024-09-11T12:50:59.123995+0000","last_became_peered":"2024-09-11T12:50:59.123995+0000","last_unstale":"2024-09-11T12:51:01.053690+0000","last_undegraded":"2024-09-11T12:51:01.053690+0000","last_fullsized":"2024-09-11T12:51:01.053690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:52.884639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053827+0000","last_change":"2024-09-11T12:50:58.113044+0000","last_active":"2024-09-11T12:51:01.053827+0000","last_peered":"2024-09-11T12:51:01.053827+0000","last_clean":"2024-09-11T12:51:01.053827+0000","last_became_active":"2024-09-11T12:50:58.112656+0000","last_became_peered":"2024-09-11T12:50:58.112656+0000","last_unstale":"2024-09-11T12:51:01.053827+0000","last_undegraded":"2024-09-11T12:51:01.053827+0000","last_fullsized":"2024-09-11T12:51:01.053827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:24:56.750228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275264+0000","last_change":"2024-09-11T12:50:59.119373+0000","last_active":"2024-09-11T12:51:01.275264+0000","last_peered":"2024-09-11T12:51:01.275264+0000","last_clean":"2024-09-11T12:51:01.275264+0000","last_became_active":"2024-09-11T12:50:59.119045+0000","last_became_peered":"2024-09-11T12:50:59.119045+0000","last_unstale":"2024-09-11T12:51:01.275264+0000","last_undegraded":"2024-09-11T12:51:01.275264+0000","last_fullsized":"2024-09-11T12:51:01.275264+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:12.019353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"44'8","reported_seq":23,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944959+0000","last_change":"2024-09-11T12:50:58.101697+0000","last_active":"2024-09-11T12:51:00.944959+0000","last_peered":"2024-09-11T12:51:00.944959+0000","last_clean":"2024-09-11T12:51:00.944959+0000","last_became_active":"2024-09-11T12:50:58.101246+0000","last_became_peered":"2024-09-11T12:50:58.101246+0000","last_unstale":"2024-09-11T12:51:00.944959+0000","last_undegraded":"2024-09-11T12:51:00.944959+0000","last_fullsized":"2024-09-11T12:51:00.944959+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:56.440536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111977+0000","last_change":"2024-09-11T12:50:59.105311+0000","last_active":"2024-09-11T12:51:00.111977+0000","last_peered":"2024-09-11T12:51:00.111977+0000","last_clean":"2024-09-11T12:51:00.111977+0000","last_became_active":"2024-09-11T12:50:59.104895+0000","last_became_peered":"2024-09-11T12:50:59.104895+0000","last_unstale":"2024-09-11T12:51:00.111977+0000","last_undegraded":"2024-09-11T12:51:00.111977+0000","last_fullsized":"2024-09-11T12:51:00.111977+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:03:13.195879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111910+0000","last_change":"2024-09-11T12:50:58.110928+0000","last_active":"2024-09-11T12:51:00.111910+0000","last_peered":"2024-09-11T12:51:00.111910+0000","last_clean":"2024-09-11T12:51:00.111910+0000","last_became_active":"2024-09-11T12:50:58.110278+0000","last_became_peered":"2024-09-11T12:50:58.110278+0000","last_unstale":"2024-09-11T12:51:00.111910+0000","last_undegraded":"2024-09-11T12:51:00.111910+0000","last_fullsized":"2024-09-11T12:51:00.111910+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:25.822358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945295+0000","last_change":"2024-09-11T12:50:59.130694+0000","last_active":"2024-09-11T12:51:00.945295+0000","last_peered":"2024-09-11T12:51:00.945295+0000","last_clean":"2024-09-11T12:51:00.945295+0000","last_became_active":"2024-09-11T12:50:59.130361+0000","last_became_peered":"2024-09-11T12:50:59.130361+0000","last_unstale":"2024-09-11T12:51:00.945295+0000","last_undegraded":"2024-09-11T12:51:00.945295+0000","last_fullsized":"2024-09-11T12:51:00.945295+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:52.913127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095796+0000","last_change":"2024-09-11T12:50:58.102158+0000","last_active":"2024-09-11T12:51:01.095796+0000","last_peered":"2024-09-11T12:51:01.095796+0000","last_clean":"2024-09-11T12:51:01.095796+0000","last_became_active":"2024-09-11T12:50:58.101875+0000","last_became_peered":"2024-09-11T12:50:58.101875+0000","last_unstale":"2024-09-11T12:51:01.095796+0000","last_undegraded":"2024-09-11T12:51:01.095796+0000","last_fullsized":"2024-09-11T12:51:01.095796+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:12:44.915885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112405+0000","last_change":"2024-09-11T12:50:59.124661+0000","last_active":"2024-09-11T12:51:00.112405+0000","last_peered":"2024-09-11T12:51:00.112405+0000","last_clean":"2024-09-11T12:51:00.112405+0000","last_became_active":"2024-09-11T12:50:59.123996+0000","last_became_peered":"2024-09-11T12:50:59.123996+0000","last_unstale":"2024-09-11T12:51:00.112405+0000","last_undegraded":"2024-09-11T12:51:00.112405+0000","last_fullsized":"2024-09-11T12:51:00.112405+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:24.655318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"44'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113027+0000","last_change":"2024-09-11T12:50:58.104431+0000","last_active":"2024-09-11T12:51:00.113027+0000","last_peered":"2024-09-11T12:51:00.113027+0000","last_clean":"2024-09-11T12:51:00.113027+0000","last_became_active":"2024-09-11T12:50:58.104126+0000","last_became_peered":"2024-09-11T12:50:58.104126+0000","last_unstale":"2024-09-11T12:51:00.113027+0000","last_undegraded":"2024-09-11T12:51:00.113027+0000","last_fullsized":"2024-09-11T12:51:00.113027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:03.290154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113360+0000","last_change":"2024-09-11T12:50:59.116037+0000","last_active":"2024-09-11T12:51:00.113360+0000","last_peered":"2024-09-11T12:51:00.113360+0000","last_clean":"2024-09-11T12:51:00.113360+0000","last_became_active":"2024-09-11T12:50:59.112693+0000","last_became_peered":"2024-09-11T12:50:59.112693+0000","last_unstale":"2024-09-11T12:51:00.113360+0000","last_undegraded":"2024-09-11T12:51:00.113360+0000","last_fullsized":"2024-09-11T12:51:00.113360+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054248+0000","last_change":"2024-09-11T12:50:58.113638+0000","last_active":"2024-09-11T12:51:01.054248+0000","last_peered":"2024-09-11T12:51:01.054248+0000","last_clean":"2024-09-11T12:51:01.054248+0000","last_became_active":"2024-09-11T12:50:58.113438+0000","last_became_peered":"2024-09-11T12:50:58.113438+0000","last_unstale":"2024-09-11T12:51:01.054248+0000","last_undegraded":"2024-09-11T12:51:01.054248+0000","last_fullsized":"2024-09-11T12:51:01.054248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:03.719760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053792+0000","last_change":"2024-09-11T12:50:59.123907+0000","last_active":"2024-09-11T12:51:01.053792+0000","last_peered":"2024-09-11T12:51:01.053792+0000","last_clean":"2024-09-11T12:51:01.053792+0000","last_became_active":"2024-09-11T12:50:59.120116+0000","last_became_peered":"2024-09-11T12:50:59.120116+0000","last_unstale":"2024-09-11T12:51:01.053792+0000","last_undegraded":"2024-09-11T12:51:01.053792+0000","last_fullsized":"2024-09-11T12:51:01.053792+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:41.783189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054146+0000","last_change":"2024-09-11T12:50:58.110448+0000","last_active":"2024-09-11T12:51:01.054146+0000","last_peered":"2024-09-11T12:51:01.054146+0000","last_clean":"2024-09-11T12:51:01.054146+0000","last_became_active":"2024-09-11T12:50:58.110166+0000","last_became_peered":"2024-09-11T12:50:58.110166+0000","last_unstale":"2024-09-11T12:51:01.054146+0000","last_undegraded":"2024-09-11T12:51:01.054146+0000","last_fullsized":"2024-09-11T12:51:01.054146+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:20.183851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984210+0000","last_change":"2024-09-11T12:50:59.114929+0000","last_active":"2024-09-11T12:51:00.984210+0000","last_peered":"2024-09-11T12:51:00.984210+0000","last_clean":"2024-09-11T12:51:00.984210+0000","last_became_active":"2024-09-11T12:50:59.114735+0000","last_became_peered":"2024-09-11T12:50:59.114735+0000","last_unstale":"2024-09-11T12:51:00.984210+0000","last_undegraded":"2024-09-11T12:51:00.984210+0000","last_fullsized":"2024-09-11T12:51:00.984210+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:53.401485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112125+0000","last_change":"2024-09-11T12:50:58.103023+0000","last_active":"2024-09-11T12:51:00.112125+0000","last_peered":"2024-09-11T12:51:00.112125+0000","last_clean":"2024-09-11T12:51:00.112125+0000","last_became_active":"2024-09-11T12:50:58.100839+0000","last_became_peered":"2024-09-11T12:50:58.100839+0000","last_unstale":"2024-09-11T12:51:00.112125+0000","last_undegraded":"2024-09-11T12:51:00.112125+0000","last_fullsized":"2024-09-11T12:51:00.112125+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:52.398639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.111338+0000","last_change":"2024-09-11T12:50:59.108219+0000","last_active":"2024-09-11T12:51:00.111338+0000","last_peered":"2024-09-11T12:51:00.111338+0000","last_clean":"2024-09-11T12:51:00.111338+0000","last_became_active":"2024-09-11T12:50:59.107942+0000","last_became_peered":"2024-09-11T12:50:59.107942+0000","last_unstale":"2024-09-11T12:51:00.111338+0000","last_undegraded":"2024-09-11T12:51:00.111338+0000","last_fullsized":"2024-09-11T12:51:00.111338+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:31:35.356098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944692+0000","last_change":"2024-09-11T12:50:58.108999+0000","last_active":"2024-09-11T12:51:00.944692+0000","last_peered":"2024-09-11T12:51:00.944692+0000","last_clean":"2024-09-11T12:51:00.944692+0000","last_became_active":"2024-09-11T12:50:58.108338+0000","last_became_peered":"2024-09-11T12:50:58.108338+0000","last_unstale":"2024-09-11T12:51:00.944692+0000","last_undegraded":"2024-09-11T12:51:00.944692+0000","last_fullsized":"2024-09-11T12:51:00.944692+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:28:17.423142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.944675+0000","last_change":"2024-09-11T12:50:59.119018+0000","last_active":"2024-09-11T12:51:00.944675+0000","last_peered":"2024-09-11T12:51:00.944675+0000","last_clean":"2024-09-11T12:51:00.944675+0000","last_became_active":"2024-09-11T12:50:59.118731+0000","last_became_peered":"2024-09-11T12:50:59.118731+0000","last_unstale":"2024-09-11T12:51:00.944675+0000","last_undegraded":"2024-09-11T12:51:00.944675+0000","last_fullsized":"2024-09-11T12:51:00.944675+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:39.896831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.274996+0000","last_change":"2024-09-11T12:50:58.112202+0000","last_active":"2024-09-11T12:51:01.274996+0000","last_peered":"2024-09-11T12:51:01.274996+0000","last_clean":"2024-09-11T12:51:01.274996+0000","last_became_active":"2024-09-11T12:50:58.111922+0000","last_became_peered":"2024-09-11T12:50:58.111922+0000","last_unstale":"2024-09-11T12:51:01.274996+0000","last_undegraded":"2024-09-11T12:51:01.274996+0000","last_fullsized":"2024-09-11T12:51:01.274996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:45.974742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113816+0000","last_change":"2024-09-11T12:50:59.127186+0000","last_active":"2024-09-11T12:51:00.113816+0000","last_peered":"2024-09-11T12:51:00.113816+0000","last_clean":"2024-09-11T12:51:00.113816+0000","last_became_active":"2024-09-11T12:50:59.126674+0000","last_became_peered":"2024-09-11T12:50:59.126674+0000","last_unstale":"2024-09-11T12:51:00.113816+0000","last_undegraded":"2024-09-11T12:51:00.113816+0000","last_fullsized":"2024-09-11T12:51:00.113816+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:13:04.184506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"45'1","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.054333+0000","last_change":"2024-09-11T12:50:58.108624+0000","last_active":"2024-09-11T12:51:01.054333+0000","last_peered":"2024-09-11T12:51:01.054333+0000","last_clean":"2024-09-11T12:51:01.054333+0000","last_became_active":"2024-09-11T12:50:58.108292+0000","last_became_peered":"2024-09-11T12:50:58.108292+0000","last_unstale":"2024-09-11T12:51:01.054333+0000","last_undegraded":"2024-09-11T12:51:01.054333+0000","last_fullsized":"2024-09-11T12:51:01.054333+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:07.739739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945430+0000","last_change":"2024-09-11T12:50:59.121606+0000","last_active":"2024-09-11T12:51:00.945430+0000","last_peered":"2024-09-11T12:51:00.945430+0000","last_clean":"2024-09-11T12:51:00.945430+0000","last_became_active":"2024-09-11T12:50:59.121350+0000","last_became_peered":"2024-09-11T12:50:59.121350+0000","last_unstale":"2024-09-11T12:51:00.945430+0000","last_undegraded":"2024-09-11T12:51:00.945430+0000","last_fullsized":"2024-09-11T12:51:00.945430+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:15.983967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275535+0000","last_change":"2024-09-11T12:50:58.114147+0000","last_active":"2024-09-11T12:51:01.275535+0000","last_peered":"2024-09-11T12:51:01.275535+0000","last_clean":"2024-09-11T12:51:01.275535+0000","last_became_active":"2024-09-11T12:50:58.113725+0000","last_became_peered":"2024-09-11T12:50:58.113725+0000","last_unstale":"2024-09-11T12:51:01.275535+0000","last_undegraded":"2024-09-11T12:51:01.275535+0000","last_fullsized":"2024-09-11T12:51:01.275535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:27.063998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275443+0000","last_change":"2024-09-11T12:50:59.113289+0000","last_active":"2024-09-11T12:51:01.275443+0000","last_peered":"2024-09-11T12:51:01.275443+0000","last_clean":"2024-09-11T12:51:01.275443+0000","last_became_active":"2024-09-11T12:50:59.112316+0000","last_became_peered":"2024-09-11T12:50:59.112316+0000","last_unstale":"2024-09-11T12:51:01.275443+0000","last_undegraded":"2024-09-11T12:51:01.275443+0000","last_fullsized":"2024-09-11T12:51:01.275443+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:21.186697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.983991+0000","last_change":"2024-09-11T12:50:58.095325+0000","last_active":"2024-09-11T12:51:00.983991+0000","last_peered":"2024-09-11T12:51:00.983991+0000","last_clean":"2024-09-11T12:51:00.983991+0000","last_became_active":"2024-09-11T12:50:58.095063+0000","last_became_peered":"2024-09-11T12:50:58.095063+0000","last_unstale":"2024-09-11T12:51:00.983991+0000","last_undegraded":"2024-09-11T12:51:00.983991+0000","last_fullsized":"2024-09-11T12:51:00.983991+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:58:56.031297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053911+0000","last_change":"2024-09-11T12:50:59.120274+0000","last_active":"2024-09-11T12:51:01.053911+0000","last_peered":"2024-09-11T12:51:01.053911+0000","last_clean":"2024-09-11T12:51:01.053911+0000","last_became_active":"2024-09-11T12:50:59.120001+0000","last_became_peered":"2024-09-11T12:50:59.120001+0000","last_unstale":"2024-09-11T12:51:01.053911+0000","last_undegraded":"2024-09-11T12:51:01.053911+0000","last_fullsized":"2024-09-11T12:51:01.053911+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:53:11.771016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275235+0000","last_change":"2024-09-11T12:50:58.112226+0000","last_active":"2024-09-11T12:51:01.275235+0000","last_peered":"2024-09-11T12:51:01.275235+0000","last_clean":"2024-09-11T12:51:01.275235+0000","last_became_active":"2024-09-11T12:50:58.110227+0000","last_became_peered":"2024-09-11T12:50:58.110227+0000","last_unstale":"2024-09-11T12:51:01.275235+0000","last_undegraded":"2024-09-11T12:51:01.275235+0000","last_fullsized":"2024-09-11T12:51:01.275235+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:16.527007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.053892+0000","last_change":"2024-09-11T12:50:59.119682+0000","last_active":"2024-09-11T12:51:01.053892+0000","last_peered":"2024-09-11T12:51:01.053892+0000","last_clean":"2024-09-11T12:51:01.053892+0000","last_became_active":"2024-09-11T12:50:59.119485+0000","last_became_peered":"2024-09-11T12:50:59.119485+0000","last_unstale":"2024-09-11T12:51:01.053892+0000","last_undegraded":"2024-09-11T12:51:01.053892+0000","last_fullsized":"2024-09-11T12:51:01.053892+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:50:55.065714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095669+0000","last_change":"2024-09-11T12:50:58.110929+0000","last_active":"2024-09-11T12:51:01.095669+0000","last_peered":"2024-09-11T12:51:01.095669+0000","last_clean":"2024-09-11T12:51:01.095669+0000","last_became_active":"2024-09-11T12:50:58.109988+0000","last_became_peered":"2024-09-11T12:50:58.109988+0000","last_unstale":"2024-09-11T12:51:01.095669+0000","last_undegraded":"2024-09-11T12:51:01.095669+0000","last_fullsized":"2024-09-11T12:51:01.095669+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:42:33.156439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.112809+0000","last_change":"2024-09-11T12:50:59.106259+0000","last_active":"2024-09-11T12:51:00.112809+0000","last_peered":"2024-09-11T12:51:00.112809+0000","last_clean":"2024-09-11T12:51:00.112809+0000","last_became_active":"2024-09-11T12:50:59.105964+0000","last_became_peered":"2024-09-11T12:50:59.105964+0000","last_unstale":"2024-09-11T12:51:00.112809+0000","last_undegraded":"2024-09-11T12:51:00.112809+0000","last_fullsized":"2024-09-11T12:51:00.112809+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:09.629253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.984127+0000","last_change":"2024-09-11T12:50:58.107206+0000","last_active":"2024-09-11T12:51:00.984127+0000","last_peered":"2024-09-11T12:51:00.984127+0000","last_clean":"2024-09-11T12:51:00.984127+0000","last_became_active":"2024-09-11T12:50:58.106241+0000","last_became_peered":"2024-09-11T12:50:58.106241+0000","last_unstale":"2024-09-11T12:51:00.984127+0000","last_undegraded":"2024-09-11T12:51:00.984127+0000","last_fullsized":"2024-09-11T12:51:00.984127+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:35.146304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.095543+0000","last_change":"2024-09-11T12:50:59.107286+0000","last_active":"2024-09-11T12:51:01.095543+0000","last_peered":"2024-09-11T12:51:01.095543+0000","last_clean":"2024-09-11T12:51:01.095543+0000","last_became_active":"2024-09-11T12:50:59.106556+0000","last_became_peered":"2024-09-11T12:50:59.106556+0000","last_unstale":"2024-09-11T12:51:01.095543+0000","last_undegraded":"2024-09-11T12:51:01.095543+0000","last_fullsized":"2024-09-11T12:51:01.095543+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:06:04.293000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945199+0000","last_change":"2024-09-11T12:50:58.113126+0000","last_active":"2024-09-11T12:51:00.945199+0000","last_peered":"2024-09-11T12:51:00.945199+0000","last_clean":"2024-09-11T12:51:00.945199+0000","last_became_active":"2024-09-11T12:50:58.112864+0000","last_became_peered":"2024-09-11T12:50:58.112864+0000","last_unstale":"2024-09-11T12:51:00.945199+0000","last_undegraded":"2024-09-11T12:51:00.945199+0000","last_fullsized":"2024-09-11T12:51:00.945199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:55.613106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.945473+0000","last_change":"2024-09-11T12:50:59.119354+0000","last_active":"2024-09-11T12:51:00.945473+0000","last_peered":"2024-09-11T12:51:00.945473+0000","last_clean":"2024-09-11T12:51:00.945473+0000","last_became_active":"2024-09-11T12:50:59.119126+0000","last_became_peered":"2024-09-11T12:50:59.119126+0000","last_unstale":"2024-09-11T12:51:00.945473+0000","last_undegraded":"2024-09-11T12:51:00.945473+0000","last_fullsized":"2024-09-11T12:51:00.945473+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:58.079017+0000","last_clean_scrub_stamp":"2024-09-11T12:50:58.079017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:29:52.277471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"44'2","reported_seq":17,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:00.113662+0000","last_change":"2024-09-11T12:50:58.110730+0000","last_active":"2024-09-11T12:51:00.113662+0000","last_peered":"2024-09-11T12:51:00.113662+0000","last_clean":"2024-09-11T12:51:00.113662+0000","last_became_active":"2024-09-11T12:50:58.110432+0000","last_became_peered":"2024-09-11T12:50:58.110432+0000","last_unstale":"2024-09-11T12:51:00.113662+0000","last_undegraded":"2024-09-11T12:51:00.113662+0000","last_fullsized":"2024-09-11T12:51:00.113662+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:50:57.076171+0000","last_clean_scrub_stamp":"2024-09-11T12:50:57.076171+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:30:27.063998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":151,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T12:51:01.275012+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:51:01.275012+0000","last_peered":"2024-09-11T12:51:01.275012+0000","last_clean":"2024-09-11T12:51:01.275012+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:51:01.275012+0000","last_undegraded":"2024-09-11T12:51:01.275012+0000","last_fullsized":"2024-09-11T12:51:01.275012+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32280,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339560,"statfs":{"total":96636764160,"available":96603709440,"internally_reserved":0,"allocated":1118208,"data_stored":797797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509461,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31704,"kb_used_data":516,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340136,"statfs":{"total":96636764160,"available":96604299264,"internally_reserved":0,"allocated":528384,"data_stored":208011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509461,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509461,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":532480,"data_stored":208951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509461,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27548,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344292,"statfs":{"total":96636764160,"available":96608555008,"internally_reserved":0,"allocated":532480,"data_stored":207977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509461,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":536576,"data_stored":208973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":524288,"data_stored":207409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542165,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":791441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:51:03.598 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:51:03.598 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:51:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 -- 172.21.15.78:0/3488840532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 msgr2=0x7f055c15f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 --2- 172.21.15.78:0/3488840532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c15f410 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f0554009cb0 tx=0x7f055401cce0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 -- 172.21.15.78:0/3488840532 shutdown_connections 2024-09-11T12:51:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 --2- 172.21.15.78:0/3488840532 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c15f410 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 -- 172.21.15.78:0/3488840532 >> 172.21.15.78:0/3488840532 conn(0x7f055c06f3a0 msgr2=0x7f055c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 -- 172.21.15.78:0/3488840532 shutdown_connections 2024-09-11T12:51:03.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.778+0000 7f0563000640 1 -- 172.21.15.78:0/3488840532 wait complete. 2024-09-11T12:51:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 Processor -- start 2024-09-11T12:51:03.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- start start 2024-09-11T12:51:03.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 0x7f055c0cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 0x7f055c14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f055c16a030 con 0x7f055c15cbb0 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f055c169eb0 con 0x7f055c155ca0 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f055c16a1b0 con 0x7f055c156080 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43316/0 (socket says 172.21.15.78:43316) 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 -- 172.21.15.78:0/4155104489 learned_addr learned my addr 172.21.15.78:0/4155104489 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f055be00640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 0x7f055c0cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f055b400640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 0x7f055c14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 -- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 msgr2=0x7f055c14b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 0x7f055c14b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 -- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 msgr2=0x7f055c0cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 0x7f055c0cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 -- 172.21.15.78:0/4155104489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c14f3e0 con 0x7f055c15cbb0 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f055b400640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 0x7f055c14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f055be00640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 0x7f055c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0561000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0554009bd0 tx=0x7f05540093d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540b2040 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05540ad7f0 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05540adc20 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c14f6c0 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c14fc80 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.782+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c15f410 con 0x7f055c15cbb0 2024-09-11T12:51:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.786+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f0554004000 con 0x7f055c15cbb0 2024-09-11T12:51:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.786+0000 7f0553e00640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 0x7f053c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.786+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f055412cb00 con 0x7f055c15cbb0 2024-09-11T12:51:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.786+0000 7f055be00640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 0x7f053c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:03.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.786+0000 7f055be00640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 0x7f053c073cf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b3c50 tx=0x7f054c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:03.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:03.790+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0554016990 con 0x7f055c15cbb0 2024-09-11T12:51:04.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.014+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f055c0695e0 con 0x7f055c15cbb0 2024-09-11T12:51:04.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0553e00640 1 -- 172.21.15.78:0/4155104489 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7f055412ce20 con 0x7f055c15cbb0 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:51:04.022 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:51:04.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 msgr2=0x7f053c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 0x7f053c073cf0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f055c0b3c50 tx=0x7f054c009040 comp rx=0 tx=0).stop 2024-09-11T12:51:04.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 msgr2=0x7f055c14b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0554009bd0 tx=0x7f05540093d0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 shutdown_connections 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f053c071830 0x7f053c073cf0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f055c15cbb0 0x7f055c14b8c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f055c156080 0x7f055c14b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 --2- 172.21.15.78:0/4155104489 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f055c155ca0 0x7f055c0cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 >> 172.21.15.78:0/4155104489 conn(0x7f055c06f3a0 msgr2=0x7f055c0cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 shutdown_connections 2024-09-11T12:51:04.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.018+0000 7f0563000640 1 -- 172.21.15.78:0/4155104489 wait complete. 2024-09-11T12:51:04.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- 172.21.15.78:0/645643180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c8070360 msgr2=0x7fb8c815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 --2- 172.21.15.78:0/645643180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c8070360 0x7fb8c815b350 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009900 tx=0x7fb8c001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- 172.21.15.78:0/645643180 shutdown_connections 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 --2- 172.21.15.78:0/645643180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c8070360 0x7fb8c815b350 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- 172.21.15.78:0/645643180 >> 172.21.15.78:0/645643180 conn(0x7fb8c806ca80 msgr2=0x7fb8c80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- 172.21.15.78:0/645643180 shutdown_connections 2024-09-11T12:51:04.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- 172.21.15.78:0/645643180 wait complete. 2024-09-11T12:51:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 Processor -- start 2024-09-11T12:51:04.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- start start 2024-09-11T12:51:04.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 0x7fb8c81fb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c8070910 0x7fb8c80b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c8168ec0 con 0x7fb8c80741e0 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb8c815beb0 con 0x7fb8c8070360 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8cec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb8c8169040 con 0x7fb8c8070910 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8ccc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8ccc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40644/0 (socket says 172.21.15.78:40644) 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8ccc00640 1 -- 172.21.15.78:0/1578974482 learned_addr learned my addr 172.21.15.78:0/1578974482 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8c7400640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c8070910 0x7fb8c80b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.182+0000 7fb8c7e00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 0x7fb8c81fb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 -- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c8070910 msgr2=0x7fb8c80b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c8070910 0x7fb8c80b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 -- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 msgr2=0x7fb8c81fb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 0x7fb8c81fb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 -- 172.21.15.78:0/1578974482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8c80b02c0 con 0x7fb8c80741e0 2024-09-11T12:51:04.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8c7e00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 0x7fb8c81fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8ccc00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009230 tx=0x7fb8c00a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c00b2040 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8c0004230 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003600 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8c80b0580 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8c814c840 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8c815b350 con 0x7fb8c80741e0 2024-09-11T12:51:04.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fb8c00bf020 con 0x7fb8c80741e0 2024-09-11T12:51:04.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 0x7fb8a8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb8c012c350 con 0x7fb8c80741e0 2024-09-11T12:51:04.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.186+0000 7fb8c7e00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 0x7fb8a8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.190+0000 7fb8c7e00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 0x7fb8a8073cf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80012f0 tx=0x7fb8b8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.190+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb8c00c6020 con 0x7fb8c80741e0 2024-09-11T12:51:04.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb8c814ed00 con 0x7fb8c80741e0 2024-09-11T12:51:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8bfe00640 1 -- 172.21.15.78:0/1578974482 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5229 (secure 0 0 0) 0x7fb8c00164d0 con 0x7fb8c80741e0 2024-09-11T12:51:04.394 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:04.394 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":26,"btime":"2024-09-11T12:51:02:115861+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:04.394 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 26 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 msgr2=0x7fb8a8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 0x7fb8a8073cf0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80012f0 tx=0x7fb8b8009040 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 msgr2=0x7fb8c813b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009230 tx=0x7fb8c00a56b0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 shutdown_connections 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8a8071830 0x7fb8a8073cf0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb8c80741e0 0x7fb8c813b290 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb8c8070910 0x7fb8c80b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 --2- 172.21.15.78:0/1578974482 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb8c8070360 0x7fb8c81fb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 >> 172.21.15.78:0/1578974482 conn(0x7fb8c806ca80 msgr2=0x7fb8c8072a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 shutdown_connections 2024-09-11T12:51:04.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.390+0000 7fb8cec00640 1 -- 172.21.15.78:0/1578974482 wait complete. 2024-09-11T12:51:04.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:50:58.476924+0000', 'modified': '2024-09-11T12:51:01.500477+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5271}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5271': {'gid': 5271, 'name': 'a', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/2225802748', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6833', 'nonce': 2225802748}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 2225802748}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5271, 'qdb_cluster': [5271]} 2024-09-11T12:51:04.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:51:04.818 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-11T12:51:04.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- 172.21.15.78:0/2786897685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 msgr2=0x7fc61c0b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 --2- 172.21.15.78:0/2786897685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c0b4520 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009870 tx=0x7fc60c01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- 172.21.15.78:0/2786897685 shutdown_connections 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 --2- 172.21.15.78:0/2786897685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c0b4520 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- 172.21.15.78:0/2786897685 >> 172.21.15.78:0/2786897685 conn(0x7fc61c06ca80 msgr2=0x7fc61c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- 172.21.15.78:0/2786897685 shutdown_connections 2024-09-11T12:51:04.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- 172.21.15.78:0/2786897685 wait complete. 2024-09-11T12:51:04.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 Processor -- start 2024-09-11T12:51:04.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.958+0000 7fc622e00640 1 -- start start 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc61c0cf580 0x7fc61c1424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c1690b0 con 0x7fc61c15a940 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c168f30 con 0x7fc61c0cf580 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c169230 con 0x7fc61c06fc50 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc620e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:40458/0 (socket says 172.21.15.78:40458) 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 -- 172.21.15.78:0/3576387671 learned_addr learned my addr 172.21.15.78:0/3576387671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61b400640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc61c0cf580 0x7fc61c1424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 -- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc61c0cf580 msgr2=0x7fc61c1424b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc61c0cf580 0x7fc61c1424b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 -- 172.21.15.78:0/3576387671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 msgr2=0x7fc61c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 -- 172.21.15.78:0/3576387671 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61c149d90 con 0x7fc61c06fc50 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc620e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:51:04.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc61be00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009790 tx=0x7fc60c005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc613e00640 1 -- 172.21.15.78:0/3576387671 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0b2040 con 0x7fc61c06fc50 2024-09-11T12:51:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc613e00640 1 -- 172.21.15.78:0/3576387671 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc60c0044b0 con 0x7fc61c06fc50 2024-09-11T12:51:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc613e00640 1 -- 172.21.15.78:0/3576387671 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c003800 con 0x7fc61c06fc50 2024-09-11T12:51:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61c14a070 con 0x7fc61c06fc50 2024-09-11T12:51:04.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.962+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc61c14a630 con 0x7fc61c06fc50 2024-09-11T12:51:04.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc613e00640 1 -- 172.21.15.78:0/3576387671 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fc60c0bd060 con 0x7fc61c06fc50 2024-09-11T12:51:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc613e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 0x7fc5fc073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc613e00640 1 -- 172.21.15.78:0/3576387671 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc60c12ba60 con 0x7fc61c06fc50 2024-09-11T12:51:04.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc61b400640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 0x7fc5fc073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc6262d1000 1 Processor -- start 2024-09-11T12:51:04.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc6262d1000 1 -- start start 2024-09-11T12:51:04.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.966+0000 7fc61b400640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 0x7fc5fc073cf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc61c166340 tx=0x7fc608009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a43c0 0x56421d956e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d957720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56421d7b4270 con 0x56421d9a3aa0 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56421d7a3770 con 0x56421d957340 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56421d7a77b0 con 0x56421d9a43c0 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40668/0 (socket says 172.21.15.78:40668) 2024-09-11T12:51:04.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 -- 172.21.15.78:0/3519521576 learned_addr learned my addr 172.21.15.78:0/3519521576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 -- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a43c0 msgr2=0x56421d956e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a43c0 0x56421d956e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 -- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 msgr2=0x56421d957720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d957720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 -- 172.21.15.78:0/3519521576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421d742f90 con 0x56421d9a3aa0 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc61be00640 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0f6be0 tx=0x7fc60c0f6830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2380924f0e095f8d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc610c00640 1 -- 172.21.15.78:0/3519521576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0fb050 con 0x56421d9a3aa0 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc610c00640 1 -- 172.21.15.78:0/3519521576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc60c0f6ee0 con 0x56421d9a3aa0 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc610c00640 1 -- 172.21.15.78:0/3519521576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc60c0fb430 con 0x56421d9a3aa0 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- 172.21.15.78:0/3519521576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 msgr2=0x56421d9a3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0f6be0 tx=0x7fc60c0f6830 comp rx=0 tx=0).stop 2024-09-11T12:51:04.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- 172.21.15.78:0/3519521576 shutdown_connections 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d957720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a43c0 0x56421d956e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3519521576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9a3e80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- 172.21.15.78:0/3519521576 >> 172.21.15.78:0/3519521576 conn(0x56421d984b60 msgr2=0x56421d986f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- 172.21.15.78:0/3519521576 shutdown_connections 2024-09-11T12:51:04.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- 172.21.15.78:0/3519521576 wait complete. 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 -- start start 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.970+0000 7fc6262d1000 1 Processor -- start 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d957340 0x56421d9a3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d9a43c0 0x56421d965290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56421d873290 con 0x56421d9a3aa0 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56421d74fab0 con 0x56421d9a43c0 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56421d74d170 con 0x56421d957340 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 10 client.0 _ll_register_callbacks cb 0x56421d944830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:51:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40670/0 (socket says 172.21.15.78:40670) 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 -- 172.21.15.78:0/775259950 learned_addr learned my addr 172.21.15.78:0/775259950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61be00640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d957340 0x56421d9a3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc620e00640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d9a43c0 0x56421d965290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 -- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d957340 msgr2=0x56421d9a3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d957340 0x56421d9a3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 -- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d9a43c0 msgr2=0x56421d965290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d9a43c0 0x56421d965290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 -- 172.21.15.78:0/775259950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421d8286f0 con 0x56421d9a3aa0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61b400640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc608000f70 tx=0x7fc608007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d7404aba1ef26a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608007020 con 0x56421d9a3aa0 2024-09-11T12:51:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56421d73cf10 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc6080071c0 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc60802a050 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 10 client.5415 fetch_fsmap learned FSMap version 26 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x56421d7ffc40 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56421d73cf10 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fc60802d020 con 0x56421d9a3aa0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 10 client.5415 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 10 client.5415 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 20 client.5415 populate_metadata read hostname 'smithi078' 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d965fd0 0x56421d9b5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 4 client.5415 mds_command: new command op to 5271 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 --> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56421d7b61c0 con 0x56421d965fd0 2024-09-11T12:51:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61be00640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d965fd0 0x56421d9b5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc61be00640 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d965fd0 0x56421d9b5b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb075354afcf528 server_cookie=47fdde4838226941 in_seq=0 out_seq=0 2024-09-11T12:51:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 10 client.5415 ms_handle_connect on v2:172.21.15.78:6833/2225802748 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 1 -- 172.21.15.78:0/775259950 <== mds.0 v2:172.21.15.78:6833/2225802748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56421d7b61c0 con 0x56421d965fd0 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc605600640 10 client.5415 handle_command_reply: tid=0 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.974+0000 7fc6262d1000 1 client.5415 shutdown 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 msgr2=0x56421d964d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc608000f70 tx=0x7fc608007cf0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 shutdown_connections 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d965fd0 0x56421d9b5b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d9a43c0 0x56421d965290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d964d50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc605600640 0 client.5415 ms_handle_reset on v2:172.21.15.78:6833/2225802748 2024-09-11T12:51:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 --2- 172.21.15.78:0/775259950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d957340 0x56421d9a3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 >> 172.21.15.78:0/775259950 conn(0x56421d984b60 msgr2=0x56421d956500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 shutdown_connections 2024-09-11T12:51:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 1 -- 172.21.15.78:0/775259950 wait complete. 2024-09-11T12:51:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.978+0000 7fc6262d1000 20 client.5415 trim_cache size 0 max 16384 2024-09-11T12:51:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 Processor -- start 2024-09-11T12:51:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 -- start start 2024-09-11T12:51:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d984b60 0x56421d982e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a3aa0 0x56421d964d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c201050 con 0x56421d957340 2024-09-11T12:51:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc61c147460 con 0x56421d984b60 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc61c1f96d0 con 0x56421d9a3aa0 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40684/0 (socket says 172.21.15.78:40684) 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 -- 172.21.15.78:0/3851393184 learned_addr learned my addr 172.21.15.78:0/3851393184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 -- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a3aa0 msgr2=0x56421d964d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a3aa0 0x56421d964d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 -- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d984b60 msgr2=0x56421d982e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d984b60 0x56421d982e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.994+0000 7fc61b400640 1 -- 172.21.15.78:0/3851393184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56421d758e70 con 0x56421d957340 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc61b400640 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc608007ae0 tx=0x7fc608013890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2f5a3160dad927 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc607e00640 1 -- 172.21.15.78:0/3851393184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608032030 con 0x56421d957340 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc607e00640 1 -- 172.21.15.78:0/3851393184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc608013980 con 0x56421d957340 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc607e00640 1 -- 172.21.15.78:0/3851393184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc608013dd0 con 0x56421d957340 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/3851393184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 msgr2=0x56421d9833a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fc608007ae0 tx=0x7fc608013890 comp rx=0 tx=0).stop 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/3851393184 shutdown_connections 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d9a3aa0 0x56421d964d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d957340 0x56421d9833a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- 172.21.15.78:0/3851393184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d984b60 0x56421d982e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/3851393184 >> 172.21.15.78:0/3851393184 conn(0x56421d970770 msgr2=0x56421d95b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/3851393184 shutdown_connections 2024-09-11T12:51:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/3851393184 wait complete. 2024-09-11T12:51:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- start start 2024-09-11T12:51:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 Processor -- start 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d964d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d984b60 0x56421d965290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x56421d7abde0 con 0x56421d9a3aa0 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x56421d7a6ac0 con 0x56421d957340 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x56421d7b6ea0 con 0x56421d984b60 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 10 client.0 _ll_register_callbacks cb 0x7fc608001df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40690/0 (socket says 172.21.15.78:40690) 2024-09-11T12:51:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 -- 172.21.15.78:0/873677172 learned_addr learned my addr 172.21.15.78:0/873677172 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc61b400640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d984b60 0x56421d965290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc61be00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d964d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 -- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d984b60 msgr2=0x56421d965290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d984b60 0x56421d965290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 -- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 msgr2=0x56421d964d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d964d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 -- 172.21.15.78:0/873677172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56421d72dbf0 con 0x56421d9a3aa0 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc61be00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d964d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc620e00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc614003d90 tx=0x7fc614012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257907a543e64e6e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc605600640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:51:05.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61401a070 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc614010b00 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:04.998+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc608013980 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fc614025050 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 10 client.5421 fetch_fsmap learned FSMap version 26 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x56421d749c60 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc608013980 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7fc614028020 con 0x56421d9a3aa0 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 10 client.5421 fetch_fsmap finished waiting for FSMap version 26 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 10 client.5421 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 20 client.5421 populate_metadata read hostname 'smithi078' 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d9a43c0 0x56421da1c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 4 client.5421 mds_command: new command op to 5271 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 --> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] -- command(tid 0: {"prefix": "status"}) -- 0x56421d7b1c60 con 0x56421d9a43c0 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc61be00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d9a43c0 0x56421da1c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 10 client.5421 ms_handle_connect on v2:172.21.15.78:6833/2225802748 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc61be00640 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d9a43c0 0x56421da1c100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f90d06bfa854e50 server_cookie=34d6d99b377632f3 in_seq=0 out_seq=0 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 1 -- 172.21.15.78:0/873677172 <== mds.0 v2:172.21.15.78:6833/2225802748 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56421d7b1c60 con 0x56421d9a43c0 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 10 client.5421 handle_command_reply: tid=0 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 client.5421 shutdown 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 msgr2=0x56421d9d7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc614003d90 tx=0x7fc614012ef0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 shutdown_connections 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:6833/2225802748,v1:172.21.15.78:6835/2225802748] conn(0x56421d9a43c0 0x56421da1c100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x56421d9a3aa0 0x56421d9d7e60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x56421d984b60 0x56421d965290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc605600640 0 client.5421 ms_handle_reset on v2:172.21.15.78:6833/2225802748 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 --2- 172.21.15.78:0/873677172 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x56421d957340 0x56421d964d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 >> 172.21.15.78:0/873677172 conn(0x56421d970770 msgr2=0x56421d9b0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 shutdown_connections 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 1 -- 172.21.15.78:0/873677172 wait complete. 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc6262d1000 20 client.5421 trim_cache size 0 max 16384 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5271, 2024-09-11T12:51:05.011 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9188194210000002, 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 26, 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 45, 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 45, 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.421153579 2024-09-11T12:51:05.012 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 msgr2=0x7fc5fc073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 0x7fc5fc073cf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc61c166340 tx=0x7fc608009040 comp rx=0 tx=0).stop 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 msgr2=0x7fc61c149770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc60c009790 tx=0x7fc60c005770 comp rx=0 tx=0).stop 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 shutdown_connections 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc5fc071830 0x7fc5fc073cf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61c15a940 0x7fc61c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc61c0cf580 0x7fc61c1424b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 --2- 172.21.15.78:0/3576387671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61c06fc50 0x7fc61c149770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 >> 172.21.15.78:0/3576387671 conn(0x7fc61c06ca80 msgr2=0x7fc61c158150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 shutdown_connections 2024-09-11T12:51:05.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.002+0000 7fc622e00640 1 -- 172.21.15.78:0/3576387671 wait complete. 2024-09-11T12:51:05.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- 172.21.15.78:0/3496598708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c0b2dc0 msgr2=0x7fa88c0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 --2- 172.21.15.78:0/3496598708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c0b31f0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa878009900 tx=0x7fa87801c820 comp rx=0 tx=0).stop 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- 172.21.15.78:0/3496598708 shutdown_connections 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 --2- 172.21.15.78:0/3496598708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15dcc0 0x7fa88c15e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 --2- 172.21.15.78:0/3496598708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c0b31f0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- 172.21.15.78:0/3496598708 >> 172.21.15.78:0/3496598708 conn(0x7fa88c0b19a0 msgr2=0x7fa88c070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- 172.21.15.78:0/3496598708 shutdown_connections 2024-09-11T12:51:05.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- 172.21.15.78:0/3496598708 wait complete. 2024-09-11T12:51:05.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 Processor -- start 2024-09-11T12:51:05.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.166+0000 7fa892200640 1 -- start start 2024-09-11T12:51:05.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c06f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15c390 0x7fa88c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c168de0 con 0x7fa88c15dcc0 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa88c168c60 con 0x7fa88c0b2dc0 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa88c168f60 con 0x7fa88c15c390 2024-09-11T12:51:05.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40710/0 (socket says 172.21.15.78:40710) 2024-09-11T12:51:05.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 -- 172.21.15.78:0/2219249036 learned_addr learned my addr 172.21.15.78:0/2219249036 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:05.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa890e00640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c06f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa88be00640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15c390 0x7fa88c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 -- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15c390 msgr2=0x7fa88c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15c390 0x7fa88c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 -- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa88c0b2dc0 msgr2=0x7fa88c06f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c06f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 -- 172.21.15.78:0/2219249036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88c150a90 con 0x7fa88c15dcc0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa891800640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ea80 tx=0x7fa88400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884004b30 con 0x7fa88c15dcc0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa884004cf0 con 0x7fa88c15dcc0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa884005280 con 0x7fa88c15dcc0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88c146270 con 0x7fa88c15dcc0 2024-09-11T12:51:05.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88c146830 con 0x7fa88c15dcc0 2024-09-11T12:51:05.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.170+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa88c0b31f0 con 0x7fa88c15dcc0 2024-09-11T12:51:05.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.174+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fa884005420 con 0x7fa88c15dcc0 2024-09-11T12:51:05.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.174+0000 7fa888c00640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 0x7fa86c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:05.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.174+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fa884092c90 con 0x7fa88c15dcc0 2024-09-11T12:51:05.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.174+0000 7fa890e00640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 0x7fa86c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:05.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.174+0000 7fa890e00640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 0x7fa86c073cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa880007fd0 tx=0x7fa880009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:05.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.178+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa88c150a90 con 0x7fa88c15dcc0 2024-09-11T12:51:05.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.374+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa88c074e00 con 0x7fa88c15dcc0 2024-09-11T12:51:05.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa888c00640 1 -- 172.21.15.78:0/2219249036 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fa88405e110 con 0x7fa88c15dcc0 2024-09-11T12:51:05.381 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:51:05.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 msgr2=0x7fa86c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 0x7fa86c073cf0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa880007fd0 tx=0x7fa880009040 comp rx=0 tx=0).stop 2024-09-11T12:51:05.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 msgr2=0x7fa88c14a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fa88400ea80 tx=0x7fa88400ef40 comp rx=0 tx=0).stop 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 shutdown_connections 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa86c071830 0x7fa86c073cf0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa88c15dcc0 0x7fa88c14a390 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa88c15c390 0x7fa88c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 --2- 172.21.15.78:0/2219249036 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa88c0b2dc0 0x7fa88c06f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 >> 172.21.15.78:0/2219249036 conn(0x7fa88c0b19a0 msgr2=0x7fa88c148470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 shutdown_connections 2024-09-11T12:51:05.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05.378+0000 7fa892200640 1 -- 172.21.15.78:0/2219249036 wait complete. 2024-09-11T12:51:05.401 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:51:05.402 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:51:05.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.403 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.412 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42369sec preferred_lft 42369sec 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.413 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:51:05.414 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:05.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.415 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:51:05.469 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:51:05.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.472 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:51:05.527 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:51:05.527 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:51:05.527 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:51:05.527 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:51:05.527 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:05.528 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:51:05.528 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:51:05.528 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:51:05.528 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:51:05.528 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:51:05.528 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:51:05.529 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:51:05.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:51:05.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:51:05.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.661 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:51:05.673 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:51:05.673 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:51:05.673 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:51:05.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.674 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:51:05.735 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:51:05.735 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:51:05.784 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:51:05.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:51:05.785 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:51:05.835 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:51:05.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:05.923 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:05.924 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:51:05.939 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:05.939 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:51:05.939 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:51:05.940 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:51:05.940 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:51:05.944 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:51:05.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:05.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:51:05.963 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:51:06.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:51:06.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:51:06.067 INFO:teuthology.orchestra.run.smithi078.stdout:client5018 2024-09-11T12:51:06.067 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:51:06.067 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:06.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- 172.21.15.78:0/437789813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 msgr2=0x7f8c7c005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 --2- 172.21.15.78:0/437789813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c7c005da0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8c88009b40 tx=0x7f8c8801d220 comp rx=0 tx=0).stop 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- 172.21.15.78:0/437789813 shutdown_connections 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 --2- 172.21.15.78:0/437789813 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c7c005da0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- 172.21.15.78:0/437789813 >> 172.21.15.78:0/437789813 conn(0x7f8c9806ca80 msgr2=0x7f8c980b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- 172.21.15.78:0/437789813 shutdown_connections 2024-09-11T12:51:06.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- 172.21.15.78:0/437789813 wait complete. 2024-09-11T12:51:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 Processor -- start 2024-09-11T12:51:06.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.254+0000 7f8c9d400640 1 -- start start 2024-09-11T12:51:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c981f7f00 0x7f8c98158ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 0x7f8c981e8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c980703b0 con 0x7f8c7c005920 2024-09-11T12:51:06.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98070230 con 0x7f8c981f7f00 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c98070530 con 0x7f8c981e6830 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40728/0 (socket says 172.21.15.78:40728) 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 -- 172.21.15.78:0/4139364358 learned_addr learned my addr 172.21.15.78:0/4139364358 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c97200640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 0x7f8c981e8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c95e00640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c981f7f00 0x7f8c98158ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 -- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 msgr2=0x7f8c981e8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 0x7f8c981e8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 -- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c981f7f00 msgr2=0x7f8c98158ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c981f7f00 0x7f8c98158ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 -- 172.21.15.78:0/4139364358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c981e8510 con 0x7f8c7c005920 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c97200640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 0x7f8c981e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:06.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c96800640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000e390 tx=0x7f8c8000e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80015020 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c80010980 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c981f8750 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80010db0 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981f8d10 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9813eb10 con 0x7f8c7c005920 2024-09-11T12:51:06.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.258+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f8c80018020 con 0x7f8c7c005920 2024-09-11T12:51:06.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.262+0000 7f8c8ea00640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 0x7f8c74073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:06.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.262+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f8c80092820 con 0x7f8c7c005920 2024-09-11T12:51:06.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.262+0000 7f8c95e00640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 0x7f8c74073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:06.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.262+0000 7f8c95e00640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 0x7f8c74073d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c880ad000 tx=0x7f8c88019fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:06.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.266+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8c8001f020 con 0x7f8c7c005920 2024-09-11T12:51:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8c9813e830 con 0x7f8c7c005920 2024-09-11T12:51:06.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c8ea00640 1 -- 172.21.15.78:0/4139364358 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f8c8005e260 con 0x7f8c7c005920 2024-09-11T12:51:06.469 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:51:06.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 msgr2=0x7f8c74073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:06.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 0x7f8c74073d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8c880ad000 tx=0x7f8c88019fc0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 msgr2=0x7f8c981fbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:06.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000e390 tx=0x7f8c8000e850 comp rx=0 tx=0).stop 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 shutdown_connections 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8c74071860 0x7f8c74073d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c981e6830 0x7f8c981e8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c981f7f00 0x7f8c98158ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 --2- 172.21.15.78:0/4139364358 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c7c005920 0x7f8c981fbfb0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 >> 172.21.15.78:0/4139364358 conn(0x7f8c9806ca80 msgr2=0x7f8c9813f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 shutdown_connections 2024-09-11T12:51:06.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:06.466+0000 7f8c9d400640 1 -- 172.21.15.78:0/4139364358 wait complete. 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:51:06.493 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:51:06.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.494 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:51:06.504 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:06.505 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.506 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:51:06.507 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:51:06.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.508 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:51:06.572 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:51:06.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.574 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:51:06.632 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:51:06.633 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:51:06.634 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:51:06.728 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:51:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:51:06.772 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:51:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:51:06.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.779 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:51:06.788 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:51:06.788 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:51:06.789 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:51:06.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.789 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:51:06.852 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:51:06.852 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:51:06.902 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:51:06.902 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:51:06.902 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:51:06.951 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:51:06.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:06.952 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:07.040 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:07.040 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:51:07.053 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:51:07.054 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:51:07.054 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:51:07.054 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:51:07.054 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:51:07.057 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:51:07.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:07.058 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:51:07.077 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:51:07.170 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:51:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:51:07.174 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:51:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:51:07.180 INFO:teuthology.orchestra.run.smithi157.stdout:client5436 2024-09-11T12:51:07.180 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:51:07.180 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:07.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:51:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/1882669327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c074060 msgr2=0x7ff01c071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 --2- 172.21.15.78:0/1882669327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c074060 0x7ff01c071e10 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff00800aa10 tx=0x7ff00801dc00 comp rx=0 tx=0).stop 2024-09-11T12:51:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/1882669327 shutdown_connections 2024-09-11T12:51:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 --2- 172.21.15.78:0/1882669327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c074060 0x7ff01c071e10 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:07.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/1882669327 >> 172.21.15.78:0/1882669327 conn(0x7ff01c0b3050 msgr2=0x7ff01c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/1882669327 shutdown_connections 2024-09-11T12:51:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/1882669327 wait complete. 2024-09-11T12:51:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 Processor -- start 2024-09-11T12:51:07.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- start start 2024-09-11T12:51:07.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:07.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff01c074060 0x7ff01c155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 0x7ff01c15c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c070640 con 0x7ff01c072350 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff01c0704c0 con 0x7ff01c151050 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff01c0707c0 con 0x7ff01c074060 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40738/0 (socket says 172.21.15.78:40738) 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 -- 172.21.15.78:0/3991072910 learned_addr learned my addr 172.21.15.78:0/3991072910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01b400640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 0x7ff01c15c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 -- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff01c074060 msgr2=0x7ff01c155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff01c074060 0x7ff01c155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 -- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 msgr2=0x7ff01c15c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:07.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 0x7ff01c15c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 -- 172.21.15.78:0/3991072910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01c1fae70 con 0x7ff01c072350 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01b400640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 0x7ff01c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff01aa00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c790 tx=0x7ff00c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c003b80 con 0x7ff01c072350 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff00c003d20 con 0x7ff01c072350 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c00aa90 con 0x7ff01c072350 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.338+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01c1fb150 con 0x7ff01c072350 2024-09-11T12:51:07.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff01c1fb710 con 0x7ff01c072350 2024-09-11T12:51:07.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01c071e10 con 0x7ff01c072350 2024-09-11T12:51:07.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7ff00c00ac30 con 0x7ff01c072350 2024-09-11T12:51:07.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff013400640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 0x7feffc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:07.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7ff00c091f50 con 0x7ff01c072350 2024-09-11T12:51:07.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.342+0000 7ff01a000640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 0x7feffc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:07.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.346+0000 7ff01a000640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 0x7feffc073d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b0000 tx=0x7ff00801a9a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:07.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.346+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff00c05db30 con 0x7ff01c072350 2024-09-11T12:51:07.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:07.546+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7ff01c0b3960 con 0x7ff01c072350 2024-09-11T12:51:08.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff013400640 1 -- 172.21.15.78:0/3991072910 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7ff00c05d430 con 0x7ff01c072350 2024-09-11T12:51:08.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 msgr2=0x7feffc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 0x7feffc073d20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b0000 tx=0x7ff00801a9a0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 msgr2=0x7ff01c141430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff00c00c790 tx=0x7ff00c00cc50 comp rx=0 tx=0).stop 2024-09-11T12:51:08.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 shutdown_connections 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feffc071860 0x7feffc073d20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff01c151050 0x7ff01c15c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff01c074060 0x7ff01c155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 --2- 172.21.15.78:0/3991072910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01c072350 0x7ff01c141430 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 >> 172.21.15.78:0/3991072910 conn(0x7ff01c0b3050 msgr2=0x7ff01c14ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 shutdown_connections 2024-09-11T12:51:08.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.150+0000 7ff021e00640 1 -- 172.21.15.78:0/3991072910 wait complete. 2024-09-11T12:51:08.174 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:51:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- 172.21.15.78:0/3494720662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8003c40 msgr2=0x7fd9e8004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 --2- 172.21.15.78:0/3494720662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8003c40 0x7fd9e8004040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4009b40 tx=0x7fd9f401d220 comp rx=0 tx=0).stop 2024-09-11T12:51:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- 172.21.15.78:0/3494720662 shutdown_connections 2024-09-11T12:51:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 --2- 172.21.15.78:0/3494720662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9e8003c40 0x7fd9e8004040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- 172.21.15.78:0/3494720662 >> 172.21.15.78:0/3494720662 conn(0x7fda0406ca80 msgr2=0x7fda040b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- 172.21.15.78:0/3494720662 shutdown_connections 2024-09-11T12:51:08.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- 172.21.15.78:0/3494720662 wait complete. 2024-09-11T12:51:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 Processor -- start 2024-09-11T12:51:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.314+0000 7fda0b000640 1 -- start start 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e8003c40 0x7fda041f7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda04160b00 0x7fda0414e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda04152620 con 0x7fda040738c0 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda041524a0 con 0x7fda04160b00 2024-09-11T12:51:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda041527a0 con 0x7fd9e8003c40 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40758/0 (socket says 172.21.15.78:40758) 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 -- 172.21.15.78:0/1192782669 learned_addr learned my addr 172.21.15.78:0/1192782669 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03e00640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e8003c40 0x7fda041f7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda09000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda04160b00 0x7fda0414e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 -- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e8003c40 msgr2=0x7fda041f7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e8003c40 0x7fda041f7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 -- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda04160b00 msgr2=0x7fda0414e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda04160b00 0x7fda0414e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 -- 172.21.15.78:0/1192782669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda041f83c0 con 0x7fda040738c0 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda03400640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0003e50 tx=0x7fd9f0001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0015070 con 0x7fda040738c0 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f0011900 con 0x7fda040738c0 2024-09-11T12:51:08.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f0011d50 con 0x7fda040738c0 2024-09-11T12:51:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda041f86a0 con 0x7fda040738c0 2024-09-11T12:51:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda04149c30 con 0x7fda040738c0 2024-09-11T12:51:08.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.318+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda04149e30 con 0x7fda040738c0 2024-09-11T12:51:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.322+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fd9f0018070 con 0x7fda040738c0 2024-09-11T12:51:08.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.322+0000 7fd9fbe00640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 0x7fd9e4073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.322+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fd9f0092dc0 con 0x7fda040738c0 2024-09-11T12:51:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.322+0000 7fda03e00640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 0x7fd9e4073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.322+0000 7fda03e00640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 0x7fd9e4073cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ad000 tx=0x7fd9f4019fc0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:08.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.326+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fda04149e30 con 0x7fda040738c0 2024-09-11T12:51:08.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.514+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fda04050ba0 con 0x7fd9e4071830 2024-09-11T12:51:08.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.514+0000 7fd9fbe00640 1 -- 172.21.15.78:0/1192782669 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fda04050ba0 con 0x7fd9e4071830 2024-09-11T12:51:08.519 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:51:08.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:51:08.520 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:51:08.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:51:08.520 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 msgr2=0x7fd9e4073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 0x7fd9e4073cf0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40ad000 tx=0x7fd9f4019fc0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 msgr2=0x7fda04071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd9f0003e50 tx=0x7fd9f0001810 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 shutdown_connections 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd9e4071830 0x7fd9e4073cf0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda04160b00 0x7fda0414e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda040738c0 0x7fda04071d60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 --2- 172.21.15.78:0/1192782669 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd9e8003c40 0x7fda041f7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 >> 172.21.15.78:0/1192782669 conn(0x7fda0406ca80 msgr2=0x7fda040b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:08.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 shutdown_connections 2024-09-11T12:51:08.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.518+0000 7fda0b000640 1 -- 172.21.15.78:0/1192782669 wait complete. 2024-09-11T12:51:08.541 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:51:08.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- 172.21.15.78:0/1144715192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34340ce150 msgr2=0x7f343406fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- 172.21.15.78:0/1144715192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34340ce150 0x7f343406fb80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f342c009840 tx=0x7f342c01c7e0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- 172.21.15.78:0/1144715192 shutdown_connections 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- 172.21.15.78:0/1144715192 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34340ce150 0x7f343406fb80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- 172.21.15.78:0/1144715192 >> v2:172.21.15.157:3300/0 conn(0x7f34340b2120 0x7f34340cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- 172.21.15.78:0/1144715192 >> 172.21.15.78:0/1144715192 conn(0x7f3434071dd0 msgr2=0x7f34340721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- 172.21.15.78:0/1144715192 shutdown_connections 2024-09-11T12:51:08.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- 172.21.15.78:0/1144715192 wait complete. 2024-09-11T12:51:08.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 Processor -- start 2024-09-11T12:51:08.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- start start 2024-09-11T12:51:08.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 0x7f3434142b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 0x7f34341f8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3434167960 con 0x7f34341518b0 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f34341677e0 con 0x7f34340ce150 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3439400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3434167ae0 con 0x7f34340b2120 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40764/0 (socket says 172.21.15.78:40764) 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 -- 172.21.15.78:0/577624672 learned_addr learned my addr 172.21.15.78:0/577624672 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3431e00640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 0x7f34341f8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3432800640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 0x7f3434142b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 -- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 msgr2=0x7f3434142b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 0x7f3434142b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 -- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 msgr2=0x7f34341f8ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 0x7f34341f8ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 -- 172.21.15.78:0/577624672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34341484c0 con 0x7f34341518b0 2024-09-11T12:51:08.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3431e00640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 0x7f34341f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3432800640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 0x7f3434142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:08.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.682+0000 7f3433200640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f342c0193f0 tx=0x7f342c0a55b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0ba020 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f342c0043b0 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34341f91e0 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c0b2040 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34340b0b10 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34340cdc10 con 0x7f34341518b0 2024-09-11T12:51:08.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f342c005260 con 0x7f34341518b0 2024-09-11T12:51:08.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 0x7f3414073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:08.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f342c12c220 con 0x7f34341518b0 2024-09-11T12:51:08.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.686+0000 7f3432800640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 0x7f3414073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:08.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.690+0000 7f3432800640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 0x7f3414073cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34240015b0 tx=0x7f3424009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:08.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.690+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f342c0c4020 con 0x7f34341518b0 2024-09-11T12:51:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.854+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f343414e9e0 con 0x7f34341518b0 2024-09-11T12:51:08.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f342aa00640 1 -- 172.21.15.78:0/577624672 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f342c0163d0 con 0x7f34341518b0 2024-09-11T12:51:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 msgr2=0x7f3414073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 0x7f3414073cf0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f34240015b0 tx=0x7f3424009040 comp rx=0 tx=0).stop 2024-09-11T12:51:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 msgr2=0x7f34340b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f342c0193f0 tx=0x7f342c0a55b0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 shutdown_connections 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3414071830 0x7f3414073cf0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f34341518b0 0x7f34340b1310 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f34340ce150 0x7f34341f8ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 --2- 172.21.15.78:0/577624672 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f34340b2120 0x7f3434142b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 >> 172.21.15.78:0/577624672 conn(0x7f3434071dd0 msgr2=0x7f3434158210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 shutdown_connections 2024-09-11T12:51:08.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:08.858+0000 7f3439400640 1 -- 172.21.15.78:0/577624672 wait complete. 2024-09-11T12:51:08.883 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:51:08.883 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:08.891 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:51:08.892 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:08.940 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- 172.21.15.78:0/3369940449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01300052f0 msgr2=0x7f013000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 --2- 172.21.15.78:0/3369940449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01300052f0 0x7f013000b9d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0144006fd0 tx=0x7f014401caf0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- 172.21.15.78:0/3369940449 shutdown_connections 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 --2- 172.21.15.78:0/3369940449 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01300052f0 0x7f013000b9d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- 172.21.15.78:0/3369940449 >> 172.21.15.78:0/3369940449 conn(0x7f014c06ca80 msgr2=0x7f014c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- 172.21.15.78:0/3369940449 shutdown_connections 2024-09-11T12:51:09.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- 172.21.15.78:0/3369940449 wait complete. 2024-09-11T12:51:09.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 Processor -- start 2024-09-11T12:51:09.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.118+0000 7f0153a00640 1 -- start start 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01300052f0 0x7f014c1596a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f014c159be0 0x7f014c153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f014c0745f0 con 0x7f014c1f2d90 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f014c074470 con 0x7f014c159be0 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f014c0cfd00 con 0x7f01300052f0 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40770/0 (socket says 172.21.15.78:40770) 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 -- 172.21.15.78:0/570686437 learned_addr learned my addr 172.21.15.78:0/570686437 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0151a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f014c159be0 0x7f014c153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0151000640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01300052f0 0x7f014c1596a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 -- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01300052f0 msgr2=0x7f014c1596a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01300052f0 0x7f014c1596a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 -- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f014c159be0 msgr2=0x7f014c153d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f014c159be0 0x7f014c153d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 -- 172.21.15.78:0/570686437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014c0ab240 con 0x7f014c1f2d90 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f014be00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0138000fc0 tx=0x7f0138001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138015070 con 0x7f014c1f2d90 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0138010980 con 0x7f014c1f2d90 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014c0a95e0 con 0x7f014c1f2d90 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138010db0 con 0x7f014c1f2d90 2024-09-11T12:51:09.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014c0a9ba0 con 0x7f014c1f2d90 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014c0a9da0 con 0x7f014c1f2d90 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.122+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f0138018070 con 0x7f014c1f2d90 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.126+0000 7f0148c00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 0x7f012c073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.126+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f01380924e0 con 0x7f014c1f2d90 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.126+0000 7f0151000640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 0x7f012c073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.126+0000 7f0151000640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 0x7f012c073d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f013c005fd0 tx=0x7f013c00b260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:09.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.130+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f014c0a9da0 con 0x7f014c1f2d90 2024-09-11T12:51:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.306+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f014c070340 con 0x7f014c1f2d90 2024-09-11T12:51:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.306+0000 7f0148c00640 1 -- 172.21.15.78:0/570686437 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7f013805dfb0 con 0x7f014c1f2d90 2024-09-11T12:51:09.312 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 msgr2=0x7f012c073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 0x7f012c073d20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f013c005fd0 tx=0x7f013c00b260 comp rx=0 tx=0).stop 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 msgr2=0x7f014c1537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0138000fc0 tx=0x7f0138001830 comp rx=0 tx=0).stop 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 shutdown_connections 2024-09-11T12:51:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f012c071860 0x7f012c073d20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f014c159be0 0x7f014c153d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014c1f2d90 0x7f014c1537f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 --2- 172.21.15.78:0/570686437 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01300052f0 0x7f014c1596a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 >> 172.21.15.78:0/570686437 conn(0x7f014c06ca80 msgr2=0x7f014c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 shutdown_connections 2024-09-11T12:51:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.310+0000 7f0153a00640 1 -- 172.21.15.78:0/570686437 wait complete. 2024-09-11T12:51:09.333 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:51:09.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:09.333 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:51:09.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:51:09.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:09.360 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:51:09.412 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:51:09.412 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:09.422 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:51:09.422 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:51:09.473 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:51:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- 172.21.15.78:0/1357948285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f80736d0 msgr2=0x7fc7f8157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- 172.21.15.78:0/1357948285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f80736d0 0x7fc7f8157ae0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc7e4009900 tx=0x7fc7e401c850 comp rx=0 tx=0).stop 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- 172.21.15.78:0/1357948285 shutdown_connections 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- 172.21.15.78:0/1357948285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f806eb40 0x7fc7f806efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- 172.21.15.78:0/1357948285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f80736d0 0x7fc7f8157ae0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- 172.21.15.78:0/1357948285 >> 172.21.15.78:0/1357948285 conn(0x7fc7f806ca80 msgr2=0x7fc7f80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- 172.21.15.78:0/1357948285 shutdown_connections 2024-09-11T12:51:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- 172.21.15.78:0/1357948285 wait complete. 2024-09-11T12:51:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 Processor -- start 2024-09-11T12:51:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- start start 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806eb40 0x7fc7f81581a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80736d0 0x7fc7f8151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80b52b0 con 0x7fc7f8070a00 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc7f80b5130 con 0x7fc7f806eb40 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.610+0000 7fc7fee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc7f8167e10 con 0x7fc7f80736d0 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40778/0 (socket says 172.21.15.78:40778) 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 -- 172.21.15.78:0/3459961034 learned_addr learned my addr 172.21.15.78:0/3459961034 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7e00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806eb40 0x7fc7f81581a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7fce00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80736d0 0x7fc7f8151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 -- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80736d0 msgr2=0x7fc7f8151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80736d0 0x7fc7f8151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 -- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806eb40 msgr2=0x7fc7f81581a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806eb40 0x7fc7f81581a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 -- 172.21.15.78:0/3459961034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7f81fb7c0 con 0x7fc7f8070a00 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7f7400640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc7e401d0f0 tx=0x7fc7e40a6bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e40ac070 con 0x7fc7f8070a00 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7e4004120 con 0x7fc7f8070a00 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4005040 con 0x7fc7f8070a00 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7f81fbaa0 con 0x7fc7f8070a00 2024-09-11T12:51:09.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7f81586e0 con 0x7fc7f8070a00 2024-09-11T12:51:09.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7f806efb0 con 0x7fc7f8070a00 2024-09-11T12:51:09.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.614+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fc7e40af070 con 0x7fc7f8070a00 2024-09-11T12:51:09.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.618+0000 7fc7efe00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 0x7fc7d8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:09.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.618+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fc7e412bfb0 con 0x7fc7f8070a00 2024-09-11T12:51:09.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.618+0000 7fc7f7e00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 0x7fc7d8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:09.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.618+0000 7fc7f7e00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 0x7fc7d8073cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8007fd0 tx=0x7fc7e8007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:09.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.622+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc7e40164d0 con 0x7fc7f8070a00 2024-09-11T12:51:09.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.798+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc7f80b3920 con 0x7fc7f8070a00 2024-09-11T12:51:09.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.798+0000 7fc7efe00640 1 -- 172.21.15.78:0/3459961034 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v45) ==== 80+0+0 (secure 0 0 0) 0x7fc7e40f7430 con 0x7fc7f8070a00 2024-09-11T12:51:09.803 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 msgr2=0x7fc7d8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 0x7fc7d8073cf0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8007fd0 tx=0x7fc7e8007e40 comp rx=0 tx=0).stop 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 msgr2=0x7fc7f81fae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc7e401d0f0 tx=0x7fc7e40a6bd0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 shutdown_connections 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc7d8071830 0x7fc7d8073cf0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc7f80736d0 0x7fc7f8151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc7f8070a00 0x7fc7f81fae90 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 --2- 172.21.15.78:0/3459961034 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc7f806eb40 0x7fc7f81581a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:09.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 >> 172.21.15.78:0/3459961034 conn(0x7fc7f806ca80 msgr2=0x7fc7f81538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:09.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 shutdown_connections 2024-09-11T12:51:09.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:09.802+0000 7fc7fee00640 1 -- 172.21.15.78:0/3459961034 wait complete. 2024-09-11T12:51:09.826 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:51:09.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:09.826 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:51:09.854 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:51:09.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:51:09.854 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:51:09.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 -- 172.21.15.78:0/3685711581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7328070c30 msgr2=0x7f73280cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 --2- 172.21.15.78:0/3685711581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7328070c30 0x7f73280cd910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7314009840 tx=0x7f731401c820 comp rx=0 tx=0).stop 2024-09-11T12:51:10.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 -- 172.21.15.78:0/3685711581 shutdown_connections 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 --2- 172.21.15.78:0/3685711581 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7328070c30 0x7f73280cd910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 --2- 172.21.15.78:0/3685711581 >> v2:172.21.15.157:3300/0 conn(0x7f7328159fc0 0x7f73280706f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 -- 172.21.15.78:0/3685711581 >> 172.21.15.78:0/3685711581 conn(0x7f732806ca80 msgr2=0x7f73280b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 -- 172.21.15.78:0/3685711581 shutdown_connections 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 -- 172.21.15.78:0/3685711581 wait complete. 2024-09-11T12:51:10.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.042+0000 7f732f600640 1 Processor -- start 2024-09-11T12:51:10.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- start start 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73280cde50 0x7f732813e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f732806f6a0 con 0x7f73280c4ff0 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f732806f520 con 0x7f7328159fc0 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f732806f820 con 0x7f73280cde50 2024-09-11T12:51:10.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7327e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73280cde50 0x7f732813e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7327e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:57516/0 (socket says 172.21.15.78:57516) 2024-09-11T12:51:10.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40780/0 (socket says 172.21.15.78:40780) 2024-09-11T12:51:10.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7327e00640 1 -- 172.21.15.78:0/1667176642 learned_addr learned my addr 172.21.15.78:0/1667176642 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:10.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 -- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73280cde50 msgr2=0x7f732813e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73280cde50 0x7f732813e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 -- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 msgr2=0x7f73280b14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 -- 172.21.15.78:0/1667176642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f732815d140 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7327e00640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732d600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f732000c3c0 tx=0x7f7320007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320011790 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7320017070 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7320011ce0 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f732815d420 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73280b1a00 con 0x7f73280c4ff0 2024-09-11T12:51:10.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.046+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73280cd910 con 0x7f73280c4ff0 2024-09-11T12:51:10.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.050+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f732001c460 con 0x7f73280c4ff0 2024-09-11T12:51:10.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.050+0000 7f7324c00640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 0x7f7308073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.050+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f7320091db0 con 0x7f73280c4ff0 2024-09-11T12:51:10.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.050+0000 7f732cc00640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 0x7f7308073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.050+0000 7f732cc00640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 0x7f7308073cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7318009980 tx=0x7f7318009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:10.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.054+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7320096070 con 0x7f73280c4ff0 2024-09-11T12:51:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7328070900 con 0x7f73280c4ff0 2024-09-11T12:51:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f7324c00640 1 -- 172.21.15.78:0/1667176642 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f732005d340 con 0x7f73280c4ff0 2024-09-11T12:51:10.262 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:10.262 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 msgr2=0x7f7308073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 0x7f7308073cf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7318009980 tx=0x7f7318009500 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 msgr2=0x7f73280ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f732000c3c0 tx=0x7f7320007f90 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 shutdown_connections 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7308071830 0x7f7308073cf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f73280c4ff0 0x7f73280ae1f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7328159fc0 0x7f73280b14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 --2- 172.21.15.78:0/1667176642 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f73280cde50 0x7f732813e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 >> 172.21.15.78:0/1667176642 conn(0x7f732806ca80 msgr2=0x7f7328154970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:10.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 shutdown_connections 2024-09-11T12:51:10.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.258+0000 7f732f600640 1 -- 172.21.15.78:0/1667176642 wait complete. 2024-09-11T12:51:10.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:10.443 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T12:51:10.443 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-11T12:51:10.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol63180 2024-09-11T12:51:10.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- 172.21.15.78:0/616287992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01640052f0 msgr2=0x7f016400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 --2- 172.21.15.78:0/616287992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01640052f0 0x7f016400b9d0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0178001dc0 tx=0x7f017801cb30 comp rx=0 tx=0).stop 2024-09-11T12:51:10.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f017e800640 1 -- 172.21.15.78:0/616287992 reap_dead start 2024-09-11T12:51:10.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- 172.21.15.78:0/616287992 shutdown_connections 2024-09-11T12:51:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- 172.21.15.78:0/616287992 >> 172.21.15.78:0/616287992 conn(0x7f0180071dd0 msgr2=0x7f01800721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- 172.21.15.78:0/616287992 shutdown_connections 2024-09-11T12:51:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- 172.21.15.78:0/616287992 wait complete. 2024-09-11T12:51:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 Processor -- start 2024-09-11T12:51:10.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- start start 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01640052f0 0x7f0180145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018015d4e0 0x7f01801f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01800cdfa0 con 0x7f01800aab50 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01800cde20 con 0x7f018015d4e0 2024-09-11T12:51:10.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0185200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0180070290 con 0x7f01640052f0 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40798/0 (socket says 172.21.15.78:40798) 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 -- 172.21.15.78:0/2537500072 learned_addr learned my addr 172.21.15.78:0/2537500072 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f017de00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01640052f0 0x7f0180145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f017e800640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018015d4e0 0x7f01801f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 -- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01640052f0 msgr2=0x7f0180145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01640052f0 0x7f0180145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 -- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018015d4e0 msgr2=0x7f01801f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018015d4e0 0x7f01801f41b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.586+0000 7f0177e00640 1 -- 172.21.15.78:0/2537500072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01801f4ee0 con 0x7f01800aab50 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0177e00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f016800f9b0 tx=0x7f016800fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016800bc50 con 0x7f01800aab50 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0168011440 con 0x7f01800aab50 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168011850 con 0x7f01800aab50 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01800a9d90 con 0x7f01800aab50 2024-09-11T12:51:10.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01800aa350 con 0x7f01800aab50 2024-09-11T12:51:10.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01800707f0 con 0x7f01800aab50 2024-09-11T12:51:10.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7f0168019020 con 0x7f01800aab50 2024-09-11T12:51:10.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 0x7f015c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f0168092a50 con 0x7f01800aab50 2024-09-11T12:51:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.590+0000 7f017de00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 0x7f015c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:10.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.594+0000 7f017de00640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 0x7f015c073cf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0170001ec0 tx=0x7f0170006730 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:10.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.594+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f016805def0 con 0x7f01800aab50 2024-09-11T12:51:10.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.782+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol63180", "target": ["mon-mgr", ""]}) -- 0x7f0180147fd0 con 0x7f015c071830 2024-09-11T12:51:10.796 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:51:10.792+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:51:10.796 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:51:10.792+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:51:10.796 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:51:10.792+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:51:10.796 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:51:10.792+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:51:10.797 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:51:10.792+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:51:10.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.826+0000 7f0176000640 1 -- 172.21.15.78:0/2537500072 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0180147fd0 con 0x7f015c071830 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 msgr2=0x7f015c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 0x7f015c073cf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0170001ec0 tx=0x7f0170006730 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 msgr2=0x7f018015cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f016800f9b0 tx=0x7f016800fe70 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 shutdown_connections 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f015c071830 0x7f015c073cf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f018015d4e0 0x7f01801f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01800aab50 0x7f018015cfa0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 --2- 172.21.15.78:0/2537500072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01640052f0 0x7f0180145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 >> 172.21.15.78:0/2537500072 conn(0x7f0180071dd0 msgr2=0x7f0180150440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 shutdown_connections 2024-09-11T12:51:10.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.830+0000 7f0185200640 1 -- 172.21.15.78:0/2537500072 wait complete. 2024-09-11T12:51:10.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 -- 172.21.15.78:0/953860225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6806fd00 msgr2=0x7fdf6815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 --2- 172.21.15.78:0/953860225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6806fd00 0x7fdf6815ed00 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdf58009c80 tx=0x7fdf5801cda0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 -- 172.21.15.78:0/953860225 shutdown_connections 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 --2- 172.21.15.78:0/953860225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf68154e20 0x7fdf68071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 --2- 172.21.15.78:0/953860225 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf6806fd00 0x7fdf6815ed00 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 -- 172.21.15.78:0/953860225 >> 172.21.15.78:0/953860225 conn(0x7fdf6806c650 msgr2=0x7fdf680d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 -- 172.21.15.78:0/953860225 shutdown_connections 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.990+0000 7fdf6de00640 1 -- 172.21.15.78:0/953860225 wait complete. 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 Processor -- start 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- start start 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 0x7fdf6814dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 0x7fdf680d3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6816b300 con 0x7fdf680b4520 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6816b180 con 0x7fdf68154e20 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6816b480 con 0x7fdf6806fd00 2024-09-11T12:51:11.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40814/0 (socket says 172.21.15.78:40814) 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 -- 172.21.15.78:0/2341101944 learned_addr learned my addr 172.21.15.78:0/2341101944 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf67400640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 0x7fdf680d3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 -- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 msgr2=0x7fdf6814dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66a00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 0x7fdf6814dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 0x7fdf6814dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 -- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 msgr2=0x7fdf680d3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 0x7fdf680d3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 -- 172.21.15.78:0/2341101944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf68157900 con 0x7fdf680b4520 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf67400640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 0x7fdf680d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66a00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 0x7fdf6814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf66000640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c400 tx=0x7fdf5400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54017070 con 0x7fdf680b4520 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf540053c0 con 0x7fdf680b4520 2024-09-11T12:51:11.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf54003880 con 0x7fdf680b4520 2024-09-11T12:51:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf681fae90 con 0x7fdf680b4520 2024-09-11T12:51:11.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.994+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf681fb450 con 0x7fdf680b4520 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf681fb650 con 0x7fdf680b4520 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 6) ==== 95085+0+0 (secure 0 0 0) 0x7fdf5401a070 con 0x7fdf680b4520 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf5f400640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 0x7fdf48073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fdf54091540 con 0x7fdf680b4520 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf66a00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 0x7fdf48073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:10.998+0000 7fdf66a00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 0x7fdf48073d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf580af000 tx=0x7fdf5801b5e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:11.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.002+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdf681fb650 con 0x7fdf680b4520 2024-09-11T12:51:11.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.170+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fdf54058c20 con 0x7fdf680b4520 2024-09-11T12:51:11.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.206+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf6815ed00 con 0x7fdf680b4520 2024-09-11T12:51:11.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf5f400640 1 -- 172.21.15.78:0/2341101944 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fdf54001130 con 0x7fdf680b4520 2024-09-11T12:51:11.213 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:11.213 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:11.213 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 msgr2=0x7fdf48073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 0x7fdf48073d20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fdf580af000 tx=0x7fdf5801b5e0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 msgr2=0x7fdf680d2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdf5400c400 tx=0x7fdf5400c8c0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 shutdown_connections 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf48071860 0x7fdf48073d20 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf68154e20 0x7fdf680d3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf680b4520 0x7fdf680d2be0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 --2- 172.21.15.78:0/2341101944 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf6806fd00 0x7fdf6814dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 >> 172.21.15.78:0/2341101944 conn(0x7fdf6806c650 msgr2=0x7fdf680b4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 shutdown_connections 2024-09-11T12:51:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.210+0000 7fdf6de00640 1 -- 172.21.15.78:0/2341101944 wait complete. 2024-09-11T12:51:11.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:11.393 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.57999432299999998,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.971248859485495,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:11.393 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.971248859485495 }, "state": "open", "uptime": 0.579994323 } ] 2024-09-11T12:51:11.393 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 -- 172.21.15.78:0/1055927352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 msgr2=0x7f6c3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 --2- 172.21.15.78:0/1055927352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c3800b9d0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c730 tx=0x7f6c4c01c700 comp rx=0 tx=0).stop 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 -- 172.21.15.78:0/1055927352 shutdown_connections 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 --2- 172.21.15.78:0/1055927352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c3800b9d0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 -- 172.21.15.78:0/1055927352 >> 172.21.15.78:0/1055927352 conn(0x7f6c5406d870 msgr2=0x7f6c5406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 -- 172.21.15.78:0/1055927352 shutdown_connections 2024-09-11T12:51:11.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.530+0000 7f6c59000640 1 -- 172.21.15.78:0/1055927352 wait complete. 2024-09-11T12:51:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 Processor -- start 2024-09-11T12:51:11.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 -- start start 2024-09-11T12:51:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 0x7f6c54159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c54159ab0 0x7f6c5413a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c540736f0 con 0x7f6c38005320 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c540cfde0 con 0x7f6c54159ab0 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c54073870 con 0x7f6c541538b0 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40830/0 (socket says 172.21.15.78:40830) 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 -- 172.21.15.78:0/1574687513 learned_addr learned my addr 172.21.15.78:0/1574687513 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c52a00640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 0x7f6c54159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53e00640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c54159ab0 0x7f6c5413a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 -- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 msgr2=0x7f6c54159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 0x7f6c54159570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 -- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c54159ab0 msgr2=0x7f6c5413a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c54159ab0 0x7f6c5413a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 -- 172.21.15.78:0/1574687513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c54137e00 con 0x7f6c38005320 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c52a00640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 0x7f6c54159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c53400640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c44001bf0 tx=0x7f6c44012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44010b40 con 0x7f6c38005320 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c44017030 con 0x7f6c38005320 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c5413ae90 con 0x7f6c38005320 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.534+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44012e60 con 0x7f6c38005320 2024-09-11T12:51:11.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c541530b0 con 0x7f6c38005320 2024-09-11T12:51:11.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c5415ab00 con 0x7f6c38005320 2024-09-11T12:51:11.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f6c44013450 con 0x7f6c38005320 2024-09-11T12:51:11.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c4b400640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 0x7f6c34073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:11.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7f6c44095e20 con 0x7f6c38005320 2024-09-11T12:51:11.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.538+0000 7f6c52a00640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 0x7f6c34073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:11.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.542+0000 7f6c52a00640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 0x7f6c34073db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c54154680 tx=0x7f6c4000d570 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:11.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.542+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6c44061890 con 0x7f6c38005320 2024-09-11T12:51:11.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:51:11.565 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:51:11.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.742+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c54159e90 con 0x7f6c38005320 2024-09-11T12:51:11.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c4b400640 1 -- 172.21.15.78:0/1574687513 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f6c44061190 con 0x7f6c38005320 2024-09-11T12:51:11.749 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:11.749 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 msgr2=0x7f6c34073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 0x7f6c34073db0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6c54154680 tx=0x7f6c4000d570 comp rx=0 tx=0).stop 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 msgr2=0x7f6c541378c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6c44001bf0 tx=0x7f6c44012b20 comp rx=0 tx=0).stop 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 shutdown_connections 2024-09-11T12:51:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6c340718f0 0x7f6c34073db0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c54159ab0 0x7f6c5413a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c541538b0 0x7f6c54159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 --2- 172.21.15.78:0/1574687513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c38005320 0x7f6c541378c0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 >> 172.21.15.78:0/1574687513 conn(0x7f6c5406d870 msgr2=0x7f6c54071dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 shutdown_connections 2024-09-11T12:51:11.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:11.746+0000 7f6c59000640 1 -- 172.21.15.78:0/1574687513 wait complete. 2024-09-11T12:51:11.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:11.929 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":1.115989076,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.944056268016883,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:11.929 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.944056268016883 }, "state": "open", "uptime": 1.115989076 } ] 2024-09-11T12:51:11.929 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:16.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:17.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- 172.21.15.78:0/3814328125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 msgr2=0x7fca34003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 --2- 172.21.15.78:0/3814328125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca34003e40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fca40009b40 tx=0x7fca4001d250 comp rx=0 tx=0).stop 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- 172.21.15.78:0/3814328125 shutdown_connections 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 --2- 172.21.15.78:0/3814328125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca34003e40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- 172.21.15.78:0/3814328125 >> 172.21.15.78:0/3814328125 conn(0x7fca5006ca80 msgr2=0x7fca500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- 172.21.15.78:0/3814328125 shutdown_connections 2024-09-11T12:51:17.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- 172.21.15.78:0/3814328125 wait complete. 2024-09-11T12:51:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 Processor -- start 2024-09-11T12:51:17.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- start start 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 0x7fca50159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca501e9360 0x7fca501e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca5006f2b0 con 0x7fca34003a60 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca5006f130 con 0x7fca500b1bd0 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca56a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca5006f430 con 0x7fca501e9360 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:17.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55342/0 (socket says 172.21.15.78:55342) 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 -- 172.21.15.78:0/1152218248 learned_addr learned my addr 172.21.15.78:0/1152218248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca54a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca501e9360 0x7fca501e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4f400640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 0x7fca50159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 -- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca501e9360 msgr2=0x7fca501e6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca501e9360 0x7fca501e6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 -- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 msgr2=0x7fca50159600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 0x7fca50159600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4fe00640 1 -- 172.21.15.78:0/1152218248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca501fbb90 con 0x7fca34003a60 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.077+0000 7fca4f400640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 0x7fca50159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca4fe00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fca4001d080 tx=0x7fca400050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca400bb030 con 0x7fca34003a60 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca40004900 con 0x7fca34003a60 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40004d50 con 0x7fca34003a60 2024-09-11T12:51:17.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca501fbe70 con 0x7fca34003a60 2024-09-11T12:51:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca501e8b60 con 0x7fca34003a60 2024-09-11T12:51:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5015e540 con 0x7fca34003a60 2024-09-11T12:51:17.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fca400be030 con 0x7fca34003a60 2024-09-11T12:51:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 0x7fca30073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fca4012d440 con 0x7fca34003a60 2024-09-11T12:51:17.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.081+0000 7fca4f400640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 0x7fca30073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:17.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.085+0000 7fca4f400640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 0x7fca30073d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca500750b0 tx=0x7fca3c006b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:17.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.085+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fca40016f60 con 0x7fca34003a60 2024-09-11T12:51:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca5015cf40 con 0x7fca34003a60 2024-09-11T12:51:17.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca47e00640 1 -- 172.21.15.78:0/1152218248 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fca40101160 con 0x7fca34003a60 2024-09-11T12:51:17.294 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:17.294 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:17.294 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 msgr2=0x7fca30073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 0x7fca30073d80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fca500750b0 tx=0x7fca3c006b30 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 msgr2=0x7fca501fb290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fca4001d080 tx=0x7fca400050a0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 shutdown_connections 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca300718c0 0x7fca30073d80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca501e9360 0x7fca501e6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca500b1bd0 0x7fca50159600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 --2- 172.21.15.78:0/1152218248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca34003a60 0x7fca501fb290 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 >> 172.21.15.78:0/1152218248 conn(0x7fca5006ca80 msgr2=0x7fca5015ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 shutdown_connections 2024-09-11T12:51:17.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:17.289+0000 7fca56a00640 1 -- 172.21.15.78:0/1152218248 wait complete. 2024-09-11T12:51:17.315 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:17.473 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":6.6599323349999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.66476241873211,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:17.474 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.66476241873211 }, "state": "open", "uptime": 6.659932335 } ] 2024-09-11T12:51:17.474 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:22.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3805646181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5cc003a90 msgr2=0x7fb5cc003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3805646181 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5cc003a90 0x7fb5cc003eb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009ec0 tx=0x7fb5d801d5a0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e9e00640 1 -- 172.21.15.78:0/3805646181 reap_dead start 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3805646181 shutdown_connections 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3805646181 >> 172.21.15.78:0/3805646181 conn(0x7fb5e40ceab0 msgr2=0x7fb5e40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3805646181 shutdown_connections 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3805646181 wait complete. 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 Processor -- start 2024-09-11T12:51:22.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- start start 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5cc003a90 0x7fb5e414df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e415e170 0x7fb5e415e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4167fb0 con 0x7fb5e4164050 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4167e30 con 0x7fb5e415e170 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5ebe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e4168130 con 0x7fb5cc003a90 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55372/0 (socket says 172.21.15.78:55372) 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 -- 172.21.15.78:0/3099128563 learned_addr learned my addr 172.21.15.78:0/3099128563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5cc003a90 0x7fb5e414df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:22.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e9e00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e415e170 0x7fb5e415e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 -- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5cc003a90 msgr2=0x7fb5e414df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5cc003a90 0x7fb5e414df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 -- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e415e170 msgr2=0x7fb5e415e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e415e170 0x7fb5e415e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 -- 172.21.15.78:0/3099128563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e41644d0 con 0x7fb5e4164050 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.625+0000 7fb5e8a00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00066b0 tx=0x7fb5d00039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0015070 con 0x7fb5e4164050 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5d0011920 con 0x7fb5e4164050 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0011d70 con 0x7fb5e4164050 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e414e6d0 con 0x7fb5e4164050 2024-09-11T12:51:22.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e414ec90 con 0x7fb5e4164050 2024-09-11T12:51:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e415eb30 con 0x7fb5e4164050 2024-09-11T12:51:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fb5d0018070 con 0x7fb5e4164050 2024-09-11T12:51:22.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 0x7fb5c4073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5709+0+0 (secure 0 0 0) 0x7fb5d0092e60 con 0x7fb5e4164050 2024-09-11T12:51:22.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.629+0000 7fb5e9400640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 0x7fb5c4073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:22.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.633+0000 7fb5e9400640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 0x7fb5c4073d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80af000 tx=0x7fb5d80b3040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:22.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.633+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb5d005e200 con 0x7fb5e4164050 2024-09-11T12:51:22.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.833+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e4070480 con 0x7fb5e4164050 2024-09-11T12:51:22.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5e1600640 1 -- 172.21.15.78:0/3099128563 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fb5d0062750 con 0x7fb5e4164050 2024-09-11T12:51:22.841 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:22.841 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:22.841 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 msgr2=0x7fb5c4073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 0x7fb5c4073d80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80af000 tx=0x7fb5d80b3040 comp rx=0 tx=0).stop 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 msgr2=0x7fb5e414fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb5d00066b0 tx=0x7fb5d00039d0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 shutdown_connections 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb5c40718c0 0x7fb5c4073d80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb5e415e170 0x7fb5e415e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb5e4164050 0x7fb5e414fae0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 --2- 172.21.15.78:0/3099128563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb5cc003a90 0x7fb5e414df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:22.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 >> 172.21.15.78:0/3099128563 conn(0x7fb5e40ceab0 msgr2=0x7fb5e40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:22.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 shutdown_connections 2024-09-11T12:51:22.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:22.837+0000 7fb5ebe00640 1 -- 172.21.15.78:0/3099128563 wait complete. 2024-09-11T12:51:22.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:23.025 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.211872589,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.388628092084119,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:23.026 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.38862809208412 }, "state": "open", "uptime": 12.211872589 } ] 2024-09-11T12:51:23.026 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:28.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:28.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- 172.21.15.78:0/3631957123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 msgr2=0x7f912015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:28.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 --2- 172.21.15.78:0/3631957123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f912015ba80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f9108009900 tx=0x7f910801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- 172.21.15.78:0/3631957123 shutdown_connections 2024-09-11T12:51:28.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 --2- 172.21.15.78:0/3631957123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912015e5b0 0x7f912015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 --2- 172.21.15.78:0/3631957123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f912015ba80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- 172.21.15.78:0/3631957123 >> 172.21.15.78:0/3631957123 conn(0x7f912006ca80 msgr2=0x7f91200b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:28.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- 172.21.15.78:0/3631957123 shutdown_connections 2024-09-11T12:51:28.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- 172.21.15.78:0/3631957123 wait complete. 2024-09-11T12:51:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 Processor -- start 2024-09-11T12:51:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.173+0000 7f9125800640 1 -- start start 2024-09-11T12:51:28.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912006fc50 0x7f91201520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 0x7f91201f8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f91201680d0 con 0x7f9120074240 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9120167f50 con 0x7f912015e5b0 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9120168250 con 0x7f912006fc50 2024-09-11T12:51:28.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39324/0 (socket says 172.21.15.78:39324) 2024-09-11T12:51:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 -- 172.21.15.78:0/3986040955 learned_addr learned my addr 172.21.15.78:0/3986040955 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911ea00640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912006fc50 0x7f91201520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911f400640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 0x7f91201f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:28.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 -- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912006fc50 msgr2=0x7f91201520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912006fc50 0x7f91201520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 -- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 msgr2=0x7f91201f8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 0x7f91201f8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 -- 172.21.15.78:0/3986040955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9120144070 con 0x7f9120074240 2024-09-11T12:51:28.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911f400640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 0x7f91201f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:28.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f911e000640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f911000eb00 tx=0x7f911000c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:28.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110004b30 con 0x7f9120074240 2024-09-11T12:51:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9110004de0 con 0x7f9120074240 2024-09-11T12:51:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9110005280 con 0x7f9120074240 2024-09-11T12:51:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9120144350 con 0x7f9120074240 2024-09-11T12:51:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9120144e30 con 0x7f9120074240 2024-09-11T12:51:28.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912014c6a0 con 0x7f9120074240 2024-09-11T12:51:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.177+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f9110005420 con 0x7f9120074240 2024-09-11T12:51:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.181+0000 7f9116a00640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 0x7f90fc073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.181+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f9110092c70 con 0x7f9120074240 2024-09-11T12:51:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.181+0000 7f911ea00640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 0x7f90fc073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:28.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.181+0000 7f911ea00640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 0x7f90fc073d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91080af000 tx=0x7f910801b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:28.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.185+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9110016030 con 0x7f9120074240 2024-09-11T12:51:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.385+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f912015ba60 con 0x7f9120074240 2024-09-11T12:51:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.385+0000 7f9116a00640 1 -- 172.21.15.78:0/3986040955 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f911005df50 con 0x7f9120074240 2024-09-11T12:51:28.391 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:28.391 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:28.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 msgr2=0x7f90fc073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:28.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 0x7f90fc073d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91080af000 tx=0x7f910801b0f0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 msgr2=0x7f9120143b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f911000eb00 tx=0x7f911000c270 comp rx=0 tx=0).stop 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 shutdown_connections 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90fc0718c0 0x7f90fc073d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f912015e5b0 0x7f91201f8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9120074240 0x7f9120143b30 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 --2- 172.21.15.78:0/3986040955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f912006fc50 0x7f91201520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 >> 172.21.15.78:0/3986040955 conn(0x7f912006ca80 msgr2=0x7f91201516f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 shutdown_connections 2024-09-11T12:51:28.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:28.389+0000 7f9125800640 1 -- 172.21.15.78:0/3986040955 wait complete. 2024-09-11T12:51:28.412 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:28.572 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":17.755813117999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.116403610203157,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:28.572 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.116403610203157 }, "state": "open", "uptime": 17.755813118 } ] 2024-09-11T12:51:28.572 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:33.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- 172.21.15.78:0/2084706852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c0b14d0 msgr2=0x7fb40c166c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 --2- 172.21.15.78:0/2084706852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c0b14d0 0x7fb40c166c10 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb408009c80 tx=0x7fb40801cce0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- 172.21.15.78:0/2084706852 shutdown_connections 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 --2- 172.21.15.78:0/2084706852 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c0b14d0 0x7fb40c166c10 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- 172.21.15.78:0/2084706852 >> 172.21.15.78:0/2084706852 conn(0x7fb40c0cf830 msgr2=0x7fb40c0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- 172.21.15.78:0/2084706852 shutdown_connections 2024-09-11T12:51:33.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- 172.21.15.78:0/2084706852 wait complete. 2024-09-11T12:51:33.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 Processor -- start 2024-09-11T12:51:33.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.721+0000 7fb414800640 1 -- start start 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40c0a75d0 0x7fb40c15b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 0x7fb40c151db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c16a2c0 con 0x7fb40c15f080 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c16a140 con 0x7fb40c0b14d0 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c16a440 con 0x7fb40c0a75d0 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:33.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39348/0 (socket says 172.21.15.78:39348) 2024-09-11T12:51:33.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 -- 172.21.15.78:0/2692028844 learned_addr learned my addr 172.21.15.78:0/2692028844 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:33.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb411400640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 0x7fb40c151db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:33.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb411e00640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40c0a75d0 0x7fb40c15b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 -- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40c0a75d0 msgr2=0x7fb40c15b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40c0a75d0 0x7fb40c15b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 -- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 msgr2=0x7fb40c151db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 0x7fb40c151db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 -- 172.21.15.78:0/2692028844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c1f9240 con 0x7fb40c15f080 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb411400640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 0x7fb40c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb412800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb40801b390 tx=0x7fb4080a5bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080bb070 con 0x7fb40c15f080 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb408004d60 con 0x7fb40c15f080 2024-09-11T12:51:33.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c165420 con 0x7fb40c15f080 2024-09-11T12:51:33.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4080b2040 con 0x7fb40c15f080 2024-09-11T12:51:33.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c1659e0 con 0x7fb40c15f080 2024-09-11T12:51:33.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c0b0dc0 con 0x7fb40c15f080 2024-09-11T12:51:33.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.725+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fb4080be070 con 0x7fb40c15f080 2024-09-11T12:51:33.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.729+0000 7fb406000640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 0x7fb3f0073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:33.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.729+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fb40812c370 con 0x7fb40c15f080 2024-09-11T12:51:33.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.729+0000 7fb411e00640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 0x7fb3f0073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:33.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.729+0000 7fb411e00640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 0x7fb3f0073d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb400001650 tx=0x7fb400009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:33.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.733+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb408016990 con 0x7fb40c15f080 2024-09-11T12:51:33.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.929+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb40c0d1360 con 0x7fb40c15f080 2024-09-11T12:51:33.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.929+0000 7fb406000640 1 -- 172.21.15.78:0/2692028844 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fb4080f7e30 con 0x7fb40c15f080 2024-09-11T12:51:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:33.936 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:33.936 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 msgr2=0x7fb3f0073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 0x7fb3f0073d80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb400001650 tx=0x7fb400009040 comp rx=0 tx=0).stop 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 msgr2=0x7fb40c1522f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb40801b390 tx=0x7fb4080a5bf0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 shutdown_connections 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb3f00718c0 0x7fb3f0073d80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb40c15f080 0x7fb40c1522f0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb40c0b14d0 0x7fb40c151db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 --2- 172.21.15.78:0/2692028844 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb40c0a75d0 0x7fb40c15b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:33.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 >> 172.21.15.78:0/2692028844 conn(0x7fb40c0cf830 msgr2=0x7fb40c155340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:33.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 shutdown_connections 2024-09-11T12:51:33.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:33.933+0000 7fb414800640 1 -- 172.21.15.78:0/2692028844 wait complete. 2024-09-11T12:51:33.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:34.116 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":23.303753801999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.847451194835003,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:34.117 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.847451194835003 }, "state": "open", "uptime": 23.303753802 } ] 2024-09-11T12:51:34.117 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:39.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:39.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 -- 172.21.15.78:0/26166371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb080b14d0 msgr2=0x7ffb08166c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/26166371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb080b14d0 0x7ffb08166c10 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009c80 tx=0x7ffb0001cce0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 -- 172.21.15.78:0/26166371 shutdown_connections 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/26166371 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb080b14d0 0x7ffb08166c10 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 -- 172.21.15.78:0/26166371 >> 172.21.15.78:0/26166371 conn(0x7ffb080cf830 msgr2=0x7ffb080cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 -- 172.21.15.78:0/26166371 shutdown_connections 2024-09-11T12:51:39.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.269+0000 7ffb0fe00640 1 -- 172.21.15.78:0/26166371 wait complete. 2024-09-11T12:51:39.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 Processor -- start 2024-09-11T12:51:39.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- start start 2024-09-11T12:51:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb080a75d0 0x7ffb081f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b14d0 0x7ffb08152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:39.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0816a270 con 0x7ffb0815f080 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb0816a0f0 con 0x7ffb080b14d0 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb0816a3f0 con 0x7ffb080a75d0 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55460/0 (socket says 172.21.15.78:55460) 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 -- 172.21.15.78:0/1409818391 learned_addr learned my addr 172.21.15.78:0/1409818391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0d400640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb080a75d0 0x7ffb081f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0ca00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b14d0 0x7ffb08152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 -- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb080a75d0 msgr2=0x7ffb081f9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb080a75d0 0x7ffb081f9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 -- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b14d0 msgr2=0x7ffb08152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b14d0 0x7ffb08152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 -- 172.21.15.78:0/1409818391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb081f9c40 con 0x7ffb0815f080 2024-09-11T12:51:39.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0de00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009bd0 tx=0x7ffb000a5bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb000bb070 con 0x7ffb0815f080 2024-09-11T12:51:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb00004d40 con 0x7ffb0815f080 2024-09-11T12:51:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb000b2040 con 0x7ffb0815f080 2024-09-11T12:51:39.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb08156560 con 0x7ffb0815f080 2024-09-11T12:51:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb081f8f00 con 0x7ffb0815f080 2024-09-11T12:51:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.273+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb080b0dc0 con 0x7ffb0815f080 2024-09-11T12:51:39.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.277+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7ffb000be070 con 0x7ffb0815f080 2024-09-11T12:51:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.277+0000 7ffb05600640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 0x7ffae8073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.277+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7ffb0012c1c0 con 0x7ffb0815f080 2024-09-11T12:51:39.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.277+0000 7ffb0d400640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 0x7ffae8073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:39.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.277+0000 7ffb0d400640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 0x7ffae8073db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffafc007fd0 tx=0x7ffafc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:39.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.281+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ffb00016990 con 0x7ffb0815f080 2024-09-11T12:51:39.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.477+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb080d1080 con 0x7ffb0815f080 2024-09-11T12:51:39.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb05600640 1 -- 172.21.15.78:0/1409818391 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7ffb000f7670 con 0x7ffb0815f080 2024-09-11T12:51:39.481 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:39.481 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:39.481 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 msgr2=0x7ffae8073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 0x7ffae8073db0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ffafc007fd0 tx=0x7ffafc009040 comp rx=0 tx=0).stop 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 msgr2=0x7ffb08155b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009bd0 tx=0x7ffb000a5bf0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 shutdown_connections 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7ffae80718f0 0x7ffae8073db0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb0815f080 0x7ffb08155b80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb080b14d0 0x7ffb08152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 --2- 172.21.15.78:0/1409818391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb080a75d0 0x7ffb081f9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 >> 172.21.15.78:0/1409818391 conn(0x7ffb080cf830 msgr2=0x7ffb0815c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 shutdown_connections 2024-09-11T12:51:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:39.481+0000 7ffb0fe00640 1 -- 172.21.15.78:0/1409818391 wait complete. 2024-09-11T12:51:39.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:39.661 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":11,"uptime":28.851694687999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.581924324849673,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:39.662 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.581924324849673 }, "state": "open", "uptime": 28.851694688 } ] 2024-09-11T12:51:39.662 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:41.675 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:51:41.680 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:51:44.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:44.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- 172.21.15.78:0/2931399496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c4004810 msgr2=0x7f07c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:44.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2931399496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07c4004810 0x7f07c4004c90 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f07c800ac50 tx=0x7f07c801e070 comp rx=0 tx=0).stop 2024-09-11T12:51:44.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2400640 1 -- 172.21.15.78:0/2931399496 reap_dead start 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- 172.21.15.78:0/2931399496 shutdown_connections 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- 172.21.15.78:0/2931399496 >> 172.21.15.78:0/2931399496 conn(0x7f07dc0cfeb0 msgr2=0x7f07dc0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- 172.21.15.78:0/2931399496 shutdown_connections 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- 172.21.15.78:0/2931399496 wait complete. 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 Processor -- start 2024-09-11T12:51:44.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.813+0000 7f07e2e00640 1 -- start start 2024-09-11T12:51:44.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:44.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07dc139a30 0x7f07dc15a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 0x7f07dc13e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cdd40 con 0x7f07dc15ae40 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc0cdbc0 con 0x7f07c4004810 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07dc0b3250 con 0x7f07dc139a30 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 0x7f07dc13e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59442/0 (socket says 172.21.15.78:59442) 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 -- 172.21.15.78:0/2938188774 learned_addr learned my addr 172.21.15.78:0/2938188774 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1000640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07dc139a30 0x7f07dc15a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 -- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07dc139a30 msgr2=0x7f07dc15a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07dc139a30 0x7f07dc15a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:44.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 -- 172.21.15.78:0/2938188774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 msgr2=0x7f07dc13e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:44.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 0x7f07dc13e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:44.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 -- 172.21.15.78:0/2938188774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc1f8d40 con 0x7f07c4004810 2024-09-11T12:51:44.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2400640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 0x7f07dc13e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:51:44.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e1a00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07d0003ef0 tx=0x7f07d0001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:44.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0011640 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07d0016020 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc1f7e90 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0011a80 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1f8410 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.817+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc0b3720 con 0x7f07c4004810 2024-09-11T12:51:44.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.821+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f07d0011c20 con 0x7f07c4004810 2024-09-11T12:51:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.821+0000 7f07d9600640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 0x7f07bc073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:44.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.821+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f07d0093420 con 0x7f07c4004810 2024-09-11T12:51:44.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.821+0000 7f07e1000640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 0x7f07bc073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:44.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.821+0000 7f07e1000640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 0x7f07bc073db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07c80ae000 tx=0x7f07c801ae10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:44.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:44.825+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f07d0020020 con 0x7f07c4004810 2024-09-11T12:51:45.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.025+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07dc15a6e0 con 0x7f07c4004810 2024-09-11T12:51:45.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.025+0000 7f07d9600640 1 -- 172.21.15.78:0/2938188774 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f07d005e800 con 0x7f07c4004810 2024-09-11T12:51:45.028 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:45.029 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:45.029 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 msgr2=0x7f07bc073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:45.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 0x7f07bc073db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f07c80ae000 tx=0x7f07c801ae10 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 msgr2=0x7f07dc15b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f07d0003ef0 tx=0x7f07d0001640 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 shutdown_connections 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f07bc0718f0 0x7f07bc073db0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07dc15ae40 0x7f07dc13e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07dc139a30 0x7f07dc15a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 --2- 172.21.15.78:0/2938188774 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07c4004810 0x7f07dc15b480 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 >> 172.21.15.78:0/2938188774 conn(0x7f07dc0cfeb0 msgr2=0x7f07dc0b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 shutdown_connections 2024-09-11T12:51:45.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:45.029+0000 7f07e2e00640 1 -- 172.21.15.78:0/2938188774 wait complete. 2024-09-11T12:51:45.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:45.210 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":34.399635773999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.319779370585032,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:45.210 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.31977937058503 }, "state": "open", "uptime": 34.399635774 } ] 2024-09-11T12:51:45.210 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:50.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:50.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 -- 172.21.15.78:0/1940237075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4074260 msgr2=0x7f51a415aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:50.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 --2- 172.21.15.78:0/1940237075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4074260 0x7f51a415aec0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f518c009900 tx=0x7f518c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 -- 172.21.15.78:0/1940237075 shutdown_connections 2024-09-11T12:51:50.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 --2- 172.21.15.78:0/1940237075 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a4074260 0x7f51a415aec0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 -- 172.21.15.78:0/1940237075 >> 172.21.15.78:0/1940237075 conn(0x7f51a406ca80 msgr2=0x7f51a40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:50.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 -- 172.21.15.78:0/1940237075 shutdown_connections 2024-09-11T12:51:50.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.353+0000 7f51a9c00640 1 -- 172.21.15.78:0/1940237075 wait complete. 2024-09-11T12:51:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 Processor -- start 2024-09-11T12:51:50.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 -- start start 2024-09-11T12:51:50.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a406fc50 0x7f51a41f9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:50.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 0x7f51a41490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:50.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51a41519a0 con 0x7f51a40cf760 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51a4151820 con 0x7f51a4074260 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51a4151b20 con 0x7f51a406fc50 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.357+0000 7f51a3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52626/0 (socket says 172.21.15.78:52626) 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 -- 172.21.15.78:0/3985824174 learned_addr learned my addr 172.21.15.78:0/3985824174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a2a00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a406fc50 0x7f51a41f9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519be00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 0x7f51a41490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 -- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a406fc50 msgr2=0x7f51a41f9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a406fc50 0x7f51a41f9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 -- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 msgr2=0x7f51a41490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:50.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 0x7f51a41490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 -- 172.21.15.78:0/3985824174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a414cc70 con 0x7f51a40cf760 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519be00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 0x7f51a41490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a3400640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c780 tx=0x7f519c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c004b70 con 0x7f51a40cf760 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f519c004e00 con 0x7f51a40cf760 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f519c00aa90 con 0x7f51a40cf760 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a41f5d90 con 0x7f51a40cf760 2024-09-11T12:51:50.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a41f6350 con 0x7f51a40cf760 2024-09-11T12:51:50.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a415aec0 con 0x7f51a40cf760 2024-09-11T12:51:50.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.361+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f519c00ac30 con 0x7f51a40cf760 2024-09-11T12:51:50.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.365+0000 7f519aa00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 0x7f5180073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:50.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.365+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f519c092540 con 0x7f51a40cf760 2024-09-11T12:51:50.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.365+0000 7f51a2a00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 0x7f5180073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:50.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.365+0000 7f51a2a00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 0x7f5180073d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5194007fd0 tx=0x7f5194009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:50.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.369+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f519c020070 con 0x7f51a40cf760 2024-09-11T12:51:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51a4073420 con 0x7f51a40cf760 2024-09-11T12:51:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f519aa00640 1 -- 172.21.15.78:0/3985824174 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f519c05df70 con 0x7f51a40cf760 2024-09-11T12:51:50.570 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:50.570 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:50.570 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 msgr2=0x7f5180073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 0x7f5180073d80 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f5194007fd0 tx=0x7f5194009040 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 msgr2=0x7f51a406eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f519c00c780 tx=0x7f519c00cc40 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 shutdown_connections 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f51800718c0 0x7f5180073d80 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51a40cf760 0x7f51a406eb40 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51a4074260 0x7f51a41490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 --2- 172.21.15.78:0/3985824174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51a406fc50 0x7f51a41f9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:50.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 >> 172.21.15.78:0/3985824174 conn(0x7f51a406ca80 msgr2=0x7f51a40b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 shutdown_connections 2024-09-11T12:51:50.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:50.569+0000 7f51a9c00640 1 -- 172.21.15.78:0/3985824174 wait complete. 2024-09-11T12:51:50.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:50.751 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":39.939577114000002,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.061344064419515,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:50.752 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.061344064419515 }, "state": "open", "uptime": 39.939577114 } ] 2024-09-11T12:51:50.752 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:51:55.753 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:51:55.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- 172.21.15.78:0/1216449629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda9c005920 msgr2=0x7fda9c005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:55.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 --2- 172.21.15.78:0/1216449629 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda9c005920 0x7fda9c005da0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4001860 tx=0x7fdaa401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:51:55.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdab7400640 1 -- 172.21.15.78:0/1216449629 reap_dead start 2024-09-11T12:51:55.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- 172.21.15.78:0/1216449629 shutdown_connections 2024-09-11T12:51:55.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- 172.21.15.78:0/1216449629 >> 172.21.15.78:0/1216449629 conn(0x7fdab806ca80 msgr2=0x7fdab80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:55.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- 172.21.15.78:0/1216449629 shutdown_connections 2024-09-11T12:51:55.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- 172.21.15.78:0/1216449629 wait complete. 2024-09-11T12:51:55.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 Processor -- start 2024-09-11T12:51:55.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- start start 2024-09-11T12:51:55.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda9c005920 0x7fdab814d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab814c7b0 0x7fdab8163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdab806f2e0 con 0x7fdab80b14c0 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdab806f160 con 0x7fdab814c7b0 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdabd600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdab806f460 con 0x7fda9c005920 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdab6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdab6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56768/0 (socket says 172.21.15.78:56768) 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdab6000640 1 -- 172.21.15.78:0/641067706 learned_addr learned my addr 172.21.15.78:0/641067706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.909+0000 7fdab6a00640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda9c005920 0x7fdab814d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab7400640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab814c7b0 0x7fdab8163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 -- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda9c005920 msgr2=0x7fdab814d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda9c005920 0x7fdab814d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 -- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab814c7b0 msgr2=0x7fdab8163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab814c7b0 0x7fdab8163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 -- 172.21.15.78:0/641067706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdab81fb0a0 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdab6000640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401cdc0 tx=0x7fdaa4003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4001480 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaa4006090 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdab81fb380 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa40068d0 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdab81fb940 con 0x7fdab80b14c0 2024-09-11T12:51:55.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdab80b4db0 con 0x7fdab80b14c0 2024-09-11T12:51:55.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fdaa4006240 con 0x7fdab80b14c0 2024-09-11T12:51:55.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.913+0000 7fdaaea00640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 0x7fda98073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:51:55.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.917+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fdaa412bff0 con 0x7fdab80b14c0 2024-09-11T12:51:55.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.917+0000 7fdab6a00640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 0x7fda98073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:51:55.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.917+0000 7fdab6a00640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 0x7fda98073db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8000c10 tx=0x7fdaa8001030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:51:55.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:55.921+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdaa40aa070 con 0x7fdab80b14c0 2024-09-11T12:51:56.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.121+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdab814ae00 con 0x7fdab80b14c0 2024-09-11T12:51:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.121+0000 7fdaaea00640 1 -- 172.21.15.78:0/641067706 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fdaa4016690 con 0x7fdab80b14c0 2024-09-11T12:51:56.123 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:51:56.124 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:51:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:51:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 msgr2=0x7fda98073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 0x7fda98073db0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8000c10 tx=0x7fdaa8001030 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 msgr2=0x7fdab815d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fdaa401cdc0 tx=0x7fdaa4003bc0 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 shutdown_connections 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fda980718f0 0x7fda98073db0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdab814c7b0 0x7fdab8163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdab80b14c0 0x7fdab815d350 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 --2- 172.21.15.78:0/641067706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda9c005920 0x7fdab814d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 >> 172.21.15.78:0/641067706 conn(0x7fdab806ca80 msgr2=0x7fdab8155d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 shutdown_connections 2024-09-11T12:51:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:51:56.125+0000 7fdabd600640 1 -- 172.21.15.78:0/641067706 wait complete. 2024-09-11T12:51:56.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:51:56.304 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":45.491518513999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.80564650522393,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:51:56.304 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.80564650522393 }, "state": "open", "uptime": 45.491518514 } ] 2024-09-11T12:51:56.305 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:52:01.305 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:01.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- 172.21.15.78:0/1472932006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca84004810 msgr2=0x7fca84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:01.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/1472932006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca84004810 0x7fca84004c90 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fca88001860 tx=0x7fca8801efb0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- 172.21.15.78:0/1472932006 shutdown_connections 2024-09-11T12:52:01.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/1472932006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca84004810 0x7fca84004c90 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- 172.21.15.78:0/1472932006 >> 172.21.15.78:0/1472932006 conn(0x7fcaa006ca80 msgr2=0x7fcaa00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:01.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- 172.21.15.78:0/1472932006 shutdown_connections 2024-09-11T12:52:01.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- 172.21.15.78:0/1472932006 wait complete. 2024-09-11T12:52:01.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 Processor -- start 2024-09-11T12:52:01.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.457+0000 7fcaa6a00640 1 -- start start 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca84004810 0x7fcaa00c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcaa0147950 0x7fcaa015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa00cf280 con 0x7fcaa00ad280 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa015d330 con 0x7fca84004810 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcaa00cf400 con 0x7fcaa0147950 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56796/0 (socket says 172.21.15.78:56796) 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 -- 172.21.15.78:0/2500870798 learned_addr learned my addr 172.21.15.78:0/2500870798 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:01.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca9fe00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca84004810 0x7fcaa00c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa4a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcaa0147950 0x7fcaa015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 -- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcaa0147950 msgr2=0x7fcaa015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcaa0147950 0x7fcaa015a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 -- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca84004810 msgr2=0x7fcaa00c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca84004810 0x7fcaa00c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 -- 172.21.15.78:0/2500870798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa00c54f0 con 0x7fcaa00ad280 2024-09-11T12:52:01.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca97400640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca88005040 tx=0x7fca88006950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003800 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca880b7030 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa00ac4c0 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca88003cf0 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa00aca80 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.461+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaa00b4520 con 0x7fcaa00ad280 2024-09-11T12:52:01.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.465+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fca880b3420 con 0x7fcaa00ad280 2024-09-11T12:52:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.465+0000 7fca9cc00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 0x7fca7c073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:01.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.465+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fca8812f880 con 0x7fcaa00ad280 2024-09-11T12:52:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.465+0000 7fca9fe00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 0x7fca7c073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:01.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.465+0000 7fca9fe00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 0x7fca7c073d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90001f50 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:01.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.469+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fca88018d60 con 0x7fcaa00ad280 2024-09-11T12:52:01.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.669+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaa00cdbf0 con 0x7fcaa00ad280 2024-09-11T12:52:01.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.669+0000 7fca9cc00640 1 -- 172.21.15.78:0/2500870798 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fca880fadc0 con 0x7fcaa00ad280 2024-09-11T12:52:01.671 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:01.671 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:01.671 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 msgr2=0x7fca7c073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 0x7fca7c073d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fca90006fd0 tx=0x7fca90001f50 comp rx=0 tx=0).stop 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 msgr2=0x7fcaa01f4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fca88005040 tx=0x7fca88006950 comp rx=0 tx=0).stop 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 shutdown_connections 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca7c0718c0 0x7fca7c073d80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcaa0147950 0x7fcaa015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcaa00ad280 0x7fcaa01f4b80 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 --2- 172.21.15.78:0/2500870798 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca84004810 0x7fcaa00c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:01.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 >> 172.21.15.78:0/2500870798 conn(0x7fcaa006ca80 msgr2=0x7fcaa00cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:01.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 shutdown_connections 2024-09-11T12:52:01.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:01.673+0000 7fcaa6a00640 1 -- 172.21.15.78:0/2500870798 wait complete. 2024-09-11T12:52:01.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:52:01.852 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":51.043460099999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.553208003496916,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:52:01.852 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.553208003496916 }, "state": "open", "uptime": 51.0434601 } ] 2024-09-11T12:52:01.852 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:52:06.854 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:07.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- 172.21.15.78:0/4005192868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 msgr2=0x7fac5006ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:07.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 --2- 172.21.15.78:0/4005192868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac5006ecf0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fac48009bc0 tx=0x7fac4801cba0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- 172.21.15.78:0/4005192868 shutdown_connections 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 --2- 172.21.15.78:0/4005192868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac5006ecf0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 --2- 172.21.15.78:0/4005192868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac5015bc60 0x7fac50073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- 172.21.15.78:0/4005192868 >> 172.21.15.78:0/4005192868 conn(0x7fac5006ca80 msgr2=0x7fac500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- 172.21.15.78:0/4005192868 shutdown_connections 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- 172.21.15.78:0/4005192868 wait complete. 2024-09-11T12:52:07.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 Processor -- start 2024-09-11T12:52:07.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.005+0000 7fac56800640 1 -- start start 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 0x7fac50154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 0x7fac5014e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac50168c50 con 0x7fac5015bc60 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac50168ad0 con 0x7fac500ccb20 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac50168dd0 con 0x7fac500b4780 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 0x7fac5014e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:07.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 0x7fac50154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:07.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:33340/0 (socket says 172.21.15.78:33340) 2024-09-11T12:52:07.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 -- 172.21.15.78:0/2601833680 learned_addr learned my addr 172.21.15.78:0/2601833680 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:07.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4f400640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 0x7fac50154d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:55616/0 (socket says 172.21.15.78:55616) 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 -- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 msgr2=0x7fac50154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 0x7fac50154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 -- 172.21.15.78:0/2601833680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 msgr2=0x7fac5014e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 0x7fac5014e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 -- 172.21.15.78:0/2601833680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5014e8c0 con 0x7fac500ccb20 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4fe00640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 0x7fac5014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac4ea00640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ee30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:07.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c00cd80 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac3c004020 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5014eba0 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004470 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac50154590 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.009+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac5006ecf0 con 0x7fac500ccb20 2024-09-11T12:52:07.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.013+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fac3c005030 con 0x7fac500ccb20 2024-09-11T12:52:07.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.013+0000 7fac47400640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 0x7fac30073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:07.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.013+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fac3c092a80 con 0x7fac500ccb20 2024-09-11T12:52:07.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.013+0000 7fac4f400640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 0x7fac30073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:07.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.013+0000 7fac4f400640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 0x7fac30073d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f70 tx=0x7fac40009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:07.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.017+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fac3c05de60 con 0x7fac500ccb20 2024-09-11T12:52:07.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac5014bc40 con 0x7fac500ccb20 2024-09-11T12:52:07.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac47400640 1 -- 172.21.15.78:0/2601833680 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fac3c090e20 con 0x7fac500ccb20 2024-09-11T12:52:07.217 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:07.217 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:07.218 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 msgr2=0x7fac30073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 0x7fac30073d80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac40000f70 tx=0x7fac40009040 comp rx=0 tx=0).stop 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 msgr2=0x7fac50152cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00e970 tx=0x7fac3c00ee30 comp rx=0 tx=0).stop 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 shutdown_connections 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac300718c0 0x7fac30073d80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac5015bc60 0x7fac5014e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac500ccb20 0x7fac50152cc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 --2- 172.21.15.78:0/2601833680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac500b4780 0x7fac50154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 >> 172.21.15.78:0/2601833680 conn(0x7fac5006ca80 msgr2=0x7fac501560b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 shutdown_connections 2024-09-11T12:52:07.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:07.217+0000 7fac56800640 1 -- 172.21.15.78:0/2601833680 wait complete. 2024-09-11T12:52:07.239 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:52:07.401 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":56.591401896999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.30416542664171,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:52:07.401 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.30416542664171 }, "state": "open", "uptime": 56.591401897 } ] 2024-09-11T12:52:07.401 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:52:11.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:52:11.945 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:52:12.402 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 -- 172.21.15.78:0/3219350741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500cf7c0 msgr2=0x7fef5015cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 --2- 172.21.15.78:0/3219350741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500cf7c0 0x7fef5015cad0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fef3c009900 tx=0x7fef3c01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 -- 172.21.15.78:0/3219350741 shutdown_connections 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 --2- 172.21.15.78:0/3219350741 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500cf7c0 0x7fef5015cad0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 --2- 172.21.15.78:0/3219350741 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef500aed20 0x7fef500cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 -- 172.21.15.78:0/3219350741 >> 172.21.15.78:0/3219350741 conn(0x7fef5006ca80 msgr2=0x7fef500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 -- 172.21.15.78:0/3219350741 shutdown_connections 2024-09-11T12:52:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.553+0000 7fef56e00640 1 -- 172.21.15.78:0/3219350741 wait complete. 2024-09-11T12:52:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 Processor -- start 2024-09-11T12:52:12.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- start start 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef500cf7c0 0x7fef50153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 0x7fef501540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fef500cd0d0 con 0x7fef500aed20 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fef50152d80 con 0x7fef500cf7c0 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fef500cd250 con 0x7fef5015d010 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52560/0 (socket says 172.21.15.78:52560) 2024-09-11T12:52:12.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 -- 172.21.15.78:0/3032936162 learned_addr learned my addr 172.21.15.78:0/3032936162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4f400640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef500cf7c0 0x7fef50153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef54e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 0x7fef501540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 -- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 msgr2=0x7fef501540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 0x7fef501540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 -- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef500cf7c0 msgr2=0x7fef50153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef500cf7c0 0x7fef50153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 -- 172.21.15.78:0/3032936162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef50143f70 con 0x7fef500aed20 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef54e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 0x7fef501540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef4fe00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d980 tx=0x7fef4000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40017070 con 0x7fef500aed20 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef40009d60 con 0x7fef500aed20 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef40005280 con 0x7fef500aed20 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef50144250 con 0x7fef500aed20 2024-09-11T12:52:12.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef501506d0 con 0x7fef500aed20 2024-09-11T12:52:12.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.557+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef500cf280 con 0x7fef500aed20 2024-09-11T12:52:12.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.561+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fef40005420 con 0x7fef500aed20 2024-09-11T12:52:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.561+0000 7fef47e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 0x7fef30073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.561+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fef400923d0 con 0x7fef500aed20 2024-09-11T12:52:12.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.561+0000 7fef4f400640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 0x7fef30073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:12.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.561+0000 7fef4f400640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 0x7fef30073d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0af000 tx=0x7fef3c01b0f0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:12.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.565+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fef4005d730 con 0x7fef500aed20 2024-09-11T12:52:12.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef5006f6c0 con 0x7fef500aed20 2024-09-11T12:52:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef47e00640 1 -- 172.21.15.78:0/3032936162 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fef40061c80 con 0x7fef500aed20 2024-09-11T12:52:12.765 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:12.765 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:12.765 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 msgr2=0x7fef30073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 0x7fef30073d80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef3c0af000 tx=0x7fef3c01b0f0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 msgr2=0x7fef50142ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fef4000d980 tx=0x7fef4000de40 comp rx=0 tx=0).stop 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 shutdown_connections 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fef300718c0 0x7fef30073d80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fef5015d010 0x7fef501540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fef500cf7c0 0x7fef50153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 --2- 172.21.15.78:0/3032936162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fef500aed20 0x7fef50142ce0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 >> 172.21.15.78:0/3032936162 conn(0x7fef5006ca80 msgr2=0x7fef5006fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 shutdown_connections 2024-09-11T12:52:12.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:12.765+0000 7fef56e00640 1 -- 172.21.15.78:0/3032936162 wait complete. 2024-09-11T12:52:12.787 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:52:12.946 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":62.135343910000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.058470947946223,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:52:12.946 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.058470947946223 }, "state": "open", "uptime": 62.13534391 } ] 2024-09-11T12:52:12.946 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:52:17.948 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:18.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 -- 172.21.15.78:0/3590927781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 msgr2=0x7f6ad40d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 --2- 172.21.15.78:0/3590927781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad40d1080 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6ad415d270 tx=0x7f6acc01cb80 comp rx=0 tx=0).stop 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 -- 172.21.15.78:0/3590927781 shutdown_connections 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 --2- 172.21.15.78:0/3590927781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad40d1080 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 --2- 172.21.15.78:0/3590927781 >> v2:172.21.15.157:3300/0 conn(0x7f6ad406fdf0 0x7f6ad40a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 -- 172.21.15.78:0/3590927781 >> 172.21.15.78:0/3590927781 conn(0x7f6ad4074b90 msgr2=0x7f6ad40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 -- 172.21.15.78:0/3590927781 shutdown_connections 2024-09-11T12:52:18.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.101+0000 7f6ad9000640 1 -- 172.21.15.78:0/3590927781 wait complete. 2024-09-11T12:52:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 Processor -- start 2024-09-11T12:52:18.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- start start 2024-09-11T12:52:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad406fdf0 0x7f6ad414ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ad40b14b0 0x7f6ad4146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:18.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4168e60 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ad4168ce0 con 0x7f6ad406fdf0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ad4168fe0 con 0x7f6ad40b14b0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45864/0 (socket says 172.21.15.78:45864) 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 -- 172.21.15.78:0/876106302 learned_addr learned my addr 172.21.15.78:0/876106302 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad1400640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ad40b14b0 0x7f6ad4146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad1e00640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad406fdf0 0x7f6ad414ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 -- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ad40b14b0 msgr2=0x7f6ad4146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ad40b14b0 0x7f6ad4146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 -- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad406fdf0 msgr2=0x7f6ad414ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad406fdf0 0x7f6ad414ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 -- 172.21.15.78:0/876106302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ad414f490 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad2800640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6acc01a7b0 tx=0x7f6acc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:18.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0bc070 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6acc0042f0 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6acc0b2040 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ad41fadd0 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.105+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ad41fb390 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ad40a9d00 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f6acc0bf070 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6aca000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 0x7f6ab4073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:18.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f6acc12bf10 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6ad1e00640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 0x7f6ab4073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:18.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.109+0000 7f6ad1e00640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 0x7f6ab4073d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4001690 tx=0x7f6ac4009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:18.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.113+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6acc016770 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.317+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ad415e030 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.317+0000 7f6aca000640 1 -- 172.21.15.78:0/876106302 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f6acc0f7a70 con 0x7f6ad40d0bf0 2024-09-11T12:52:18.320 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:18.320 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:18.320 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 msgr2=0x7f6ab4073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 0x7f6ab4073d80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6ac4001690 tx=0x7f6ac4009040 comp rx=0 tx=0).stop 2024-09-11T12:52:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 msgr2=0x7f6ad415ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:18.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6acc01a7b0 tx=0x7f6acc009310 comp rx=0 tx=0).stop 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 shutdown_connections 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ab40718c0 0x7f6ab4073d80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ad40d0bf0 0x7f6ad415ab90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ad40b14b0 0x7f6ad4146650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 --2- 172.21.15.78:0/876106302 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ad406fdf0 0x7f6ad414ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 >> 172.21.15.78:0/876106302 conn(0x7f6ad4074b90 msgr2=0x7f6ad414cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 shutdown_connections 2024-09-11T12:52:18.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:18.321+0000 7f6ad9000640 1 -- 172.21.15.78:0/876106302 wait complete. 2024-09-11T12:52:18.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:52:18.502 INFO:teuthology.orchestra.run.smithi078.stdout:[{"id":5048,"entity":{"name":{"type":"client","num":5048},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":4256568404}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":67.691285968000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.815381872461799,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5048 172.21.15.157:0/4256568404","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:52:18.503 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 4256568404, "type": "any" }, "name": { "num": 5048, "type": "client" } }, "id": 5048, "inst": "client.5048 172.21.15.157:0/4256568404", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.8153818724618 }, "state": "open", "uptime": 67.691285968 } ] 2024-09-11T12:52:18.503 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-11T12:52:23.504 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- 172.21.15.78:0/267398287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea2415d010 msgr2=0x7fea240b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- 172.21.15.78:0/267398287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea2415d010 0x7fea240b4520 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fea240b2980 tx=0x7fea1c01cd50 comp rx=0 tx=0).stop 2024-09-11T12:52:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- 172.21.15.78:0/267398287 shutdown_connections 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- 172.21.15.78:0/267398287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea2415d010 0x7fea240b4520 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- 172.21.15.78:0/267398287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea240cf7c0 0x7fea2415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- 172.21.15.78:0/267398287 >> v2:172.21.15.157:3300/0 conn(0x7fea240aed20 0x7fea240cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- 172.21.15.78:0/267398287 >> 172.21.15.78:0/267398287 conn(0x7fea2406ca80 msgr2=0x7fea240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- 172.21.15.78:0/267398287 shutdown_connections 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- 172.21.15.78:0/267398287 wait complete. 2024-09-11T12:52:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 Processor -- start 2024-09-11T12:52:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- start start 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 0x7fea24145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2415d010 0x7fea2414d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea240cd460 con 0x7fea240aed20 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea240cd2e0 con 0x7fea240cf7c0 2024-09-11T12:52:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea2ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea240cd5e0 con 0x7fea2415d010 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:45886/0 (socket says 172.21.15.78:45886) 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 -- 172.21.15.78:0/2153039333 learned_addr learned my addr 172.21.15.78:0/2153039333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29a00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2415d010 0x7fea2414d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea23e00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 0x7fea24145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 -- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2415d010 msgr2=0x7fea2414d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2415d010 0x7fea2414d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 -- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 msgr2=0x7fea24145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 0x7fea24145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea29000640 1 -- 172.21.15.78:0/2153039333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea240725a0 con 0x7fea240aed20 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.653+0000 7fea23e00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 0x7fea24145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea29000640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d770 tx=0x7fea1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14017070 con 0x7fea240aed20 2024-09-11T12:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea14009d20 con 0x7fea240aed20 2024-09-11T12:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea140106c0 con 0x7fea240aed20 2024-09-11T12:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea24072880 con 0x7fea240aed20 2024-09-11T12:52:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea24072e40 con 0x7fea240aed20 2024-09-11T12:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea240cdb00 con 0x7fea240aed20 2024-09-11T12:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fea1401a070 con 0x7fea240aed20 2024-09-11T12:52:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 0x7fea04073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.657+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fea140923b0 con 0x7fea240aed20 2024-09-11T12:52:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.661+0000 7fea23e00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 0x7fea04073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.661+0000 7fea23e00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 0x7fea04073d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea100058d0 tx=0x7fea10005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:23.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.665+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fea1405dd80 con 0x7fea240aed20 2024-09-11T12:52:23.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.861+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea24145f90 con 0x7fea240aed20 2024-09-11T12:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.861+0000 7fea20c00640 1 -- 172.21.15.78:0/2153039333 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fea1405d680 con 0x7fea240aed20 2024-09-11T12:52:23.863 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:23.863 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:23.863 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 msgr2=0x7fea04073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 0x7fea04073d80 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fea100058d0 tx=0x7fea10005860 comp rx=0 tx=0).stop 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 msgr2=0x7fea241476a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d770 tx=0x7fea1400dc30 comp rx=0 tx=0).stop 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 shutdown_connections 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea040718c0 0x7fea04073d80 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea2415d010 0x7fea2414d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea240cf7c0 0x7fea24145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 --2- 172.21.15.78:0/2153039333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea240aed20 0x7fea241476a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 >> 172.21.15.78:0/2153039333 conn(0x7fea2406ca80 msgr2=0x7fea2406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 shutdown_connections 2024-09-11T12:52:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:23.865+0000 7fea2ba00640 1 -- 172.21.15.78:0/2153039333 wait complete. 2024-09-11T12:52:23.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json session ls 2024-09-11T12:52:24.043 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T12:52:24.044 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-11T12:52:24.044 DEBUG:tasks.ceph_test_case:wait_until_true: success in 65s and 0 retries 2024-09-11T12:52:24.044 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:52:24.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:52:24.151 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:52:24.243 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:52:24.243 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:52:24.244 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:52:24.244 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:24.253 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:52:24.253 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:52:24.253 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:52:24.253 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:24.262 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:52:24.263 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:52:24.263 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:52:24.263 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:24.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:52:24.305 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:52:24.305 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:52:24.305 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:24.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:52:24.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- 172.21.15.78:0/1939563137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2738074240 msgr2=0x7f273815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- 172.21.15.78:0/1939563137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2738074240 0x7f273815ba80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f272c009c80 tx=0x7f272c01cd70 comp rx=0 tx=0).stop 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- 172.21.15.78:0/1939563137 shutdown_connections 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- 172.21.15.78:0/1939563137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 0x7f273815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- 172.21.15.78:0/1939563137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2738074240 0x7f273815ba80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- 172.21.15.78:0/1939563137 >> 172.21.15.78:0/1939563137 conn(0x7f273806ca80 msgr2=0x7f27380b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- 172.21.15.78:0/1939563137 shutdown_connections 2024-09-11T12:52:24.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- 172.21.15.78:0/1939563137 wait complete. 2024-09-11T12:52:24.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 Processor -- start 2024-09-11T12:52:24.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- start start 2024-09-11T12:52:24.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 0x7f273806eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 0x7f2738161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2738167740 con 0x7f273815e5b0 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2738072910 con 0x7f2738074240 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.489+0000 7f2740600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27381678c0 con 0x7f2738070360 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39986/0 (socket says 172.21.15.78:39986) 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 -- 172.21.15.78:0/3305586635 learned_addr learned my addr 172.21.15.78:0/3305586635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273dc00640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 0x7f273806eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273d200640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 0x7f2738161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 -- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 msgr2=0x7f273806eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 0x7f273806eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 -- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 msgr2=0x7f2738161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 0x7f2738161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 -- 172.21.15.78:0/3305586635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27381fb3f0 con 0x7f273815e5b0 2024-09-11T12:52:24.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273d200640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 0x7f2738161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f273e600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f273000b410 tx=0x7f273000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730015020 con 0x7f273815e5b0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2730011940 con 0x7f273815e5b0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730011d90 con 0x7f273815e5b0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27381fa630 con 0x7f273815e5b0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27381faed0 con 0x7f273815e5b0 2024-09-11T12:52:24.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27381fb0d0 con 0x7f273815e5b0 2024-09-11T12:52:24.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.493+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f2730018020 con 0x7f273815e5b0 2024-09-11T12:52:24.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.497+0000 7f2735600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 0x7f2718073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.497+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f27300921c0 con 0x7f273815e5b0 2024-09-11T12:52:24.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.497+0000 7f273dc00640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 0x7f2718073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.497+0000 7f273dc00640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 0x7f2718073d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f272c0af000 tx=0x7f272c01b5b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:24.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.497+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f27381fb0d0 con 0x7f273815e5b0 2024-09-11T12:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.697+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27381fa800 con 0x7f273815e5b0 2024-09-11T12:52:24.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2735600640 1 -- 172.21.15.78:0/3305586635 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f273001f020 con 0x7f273815e5b0 2024-09-11T12:52:24.701 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:24.701 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:24.701 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 msgr2=0x7f2718073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 0x7f2718073d80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f272c0af000 tx=0x7f272c01b5b0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 msgr2=0x7f2738161880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f273000b410 tx=0x7f273000b8d0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 shutdown_connections 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f27180718c0 0x7f2718073d80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f273815e5b0 0x7f2738161880 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2738074240 0x7f2738161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 --2- 172.21.15.78:0/3305586635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2738070360 0x7f273806eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 >> 172.21.15.78:0/3305586635 conn(0x7f273806ca80 msgr2=0x7f2738159a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:24.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 shutdown_connections 2024-09-11T12:52:24.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.701+0000 7f2740600640 1 -- 172.21.15.78:0/3305586635 wait complete. 2024-09-11T12:52:24.722 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 -- 172.21.15.78:0/1985985494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c15e5b0 msgr2=0x7f3d0c15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1985985494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c15e5b0 0x7f3d0c15e9f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009900 tx=0x7f3cf801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 -- 172.21.15.78:0/1985985494 shutdown_connections 2024-09-11T12:52:24.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1985985494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c15e5b0 0x7f3d0c15e9f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1985985494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c074240 0x7f3d0c15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 -- 172.21.15.78:0/1985985494 >> 172.21.15.78:0/1985985494 conn(0x7f3d0c06ca80 msgr2=0x7f3d0c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 -- 172.21.15.78:0/1985985494 shutdown_connections 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.861+0000 7f3d13a00640 1 -- 172.21.15.78:0/1985985494 wait complete. 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 Processor -- start 2024-09-11T12:52:24.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- start start 2024-09-11T12:52:24.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d0c06fc50 0x7f3d0c1fb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c15e5b0 0x7f3d0c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c1680d0 con 0x7f3d0c074240 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d0c167f50 con 0x7f3d0c06fc50 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d0c168250 con 0x7f3d0c15e5b0 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:39992/0 (socket says 172.21.15.78:39992) 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 -- 172.21.15.78:0/1577224517 learned_addr learned my addr 172.21.15.78:0/1577224517 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d11000640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d0c06fc50 0x7f3d0c1fb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d11a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c15e5b0 0x7f3d0c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 -- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c15e5b0 msgr2=0x7f3d0c06eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c15e5b0 0x7f3d0c06eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 -- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d0c06fc50 msgr2=0x7f3d0c1fb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d0c06fc50 0x7f3d0c1fb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 -- 172.21.15.78:0/1577224517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d0c160f70 con 0x7f3d0c074240 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d0be00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009850 tx=0x7f3cf8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf80b2040 con 0x7f3d0c074240 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cf8005070 con 0x7f3d0c074240 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8005480 con 0x7f3d0c074240 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d0c150e90 con 0x7f3d0c074240 2024-09-11T12:52:24.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d0c1614f0 con 0x7f3d0c074240 2024-09-11T12:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.865+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d0c15ba80 con 0x7f3d0c074240 2024-09-11T12:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.869+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7f3cf80046d0 con 0x7f3d0c074240 2024-09-11T12:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.869+0000 7f3d08c00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 0x7f3cec073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:24.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.869+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f3cf812c3f0 con 0x7f3d0c074240 2024-09-11T12:52:24.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.869+0000 7f3d11000640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 0x7f3cec073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:24.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.869+0000 7f3d11000640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 0x7f3cec073d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005c50 tx=0x7f3cfc005be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:24.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:24.873+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3cf80164d0 con 0x7f3d0c074240 2024-09-11T12:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d0c0695e0 con 0x7f3d0c074240 2024-09-11T12:52:25.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d08c00640 1 -- 172.21.15.78:0/1577224517 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f3cf80f7f30 con 0x7f3d0c074240 2024-09-11T12:52:25.073 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:25.074 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:25.074 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:25.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 msgr2=0x7f3cec073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 0x7f3cec073d80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3cfc005c50 tx=0x7f3cfc005be0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 msgr2=0x7f3d0c144790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8009850 tx=0x7f3cf8004040 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 shutdown_connections 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3cec0718c0 0x7f3cec073d80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d0c15e5b0 0x7f3d0c06eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d0c074240 0x7f3d0c144790 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 --2- 172.21.15.78:0/1577224517 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d0c06fc50 0x7f3d0c1fb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 >> 172.21.15.78:0/1577224517 conn(0x7f3d0c06ca80 msgr2=0x7f3d0c15da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 shutdown_connections 2024-09-11T12:52:25.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.073+0000 7f3d13a00640 1 -- 172.21.15.78:0/1577224517 wait complete. 2024-09-11T12:52:25.094 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 -- 172.21.15.78:0/1441038739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 msgr2=0x7fc28415b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 --2- 172.21.15.78:0/1441038739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28415b350 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009c80 tx=0x7fc27c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 -- 172.21.15.78:0/1441038739 shutdown_connections 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 --2- 172.21.15.78:0/1441038739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28415b350 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 -- 172.21.15.78:0/1441038739 >> 172.21.15.78:0/1441038739 conn(0x7fc28406ca80 msgr2=0x7fc2840b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 -- 172.21.15.78:0/1441038739 shutdown_connections 2024-09-11T12:52:25.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.233+0000 7fc28c400640 1 -- 172.21.15.78:0/1441038739 wait complete. 2024-09-11T12:52:25.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 Processor -- start 2024-09-11T12:52:25.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- start start 2024-09-11T12:52:25.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 0x7fc2840ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 0x7fc2840aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc284168ec0 con 0x7fc284070360 2024-09-11T12:52:25.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc28415beb0 con 0x7fc2840741e0 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc284169040 con 0x7fc284070910 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40004/0 (socket says 172.21.15.78:40004) 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 -- 172.21.15.78:0/1965831202 learned_addr learned my addr 172.21.15.78:0/1965831202 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289000640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 0x7fc2840ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28a400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 0x7fc2840aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 -- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 msgr2=0x7fc2840ae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 0x7fc2840ae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 -- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 msgr2=0x7fc2840aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 0x7fc2840aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 -- 172.21.15.78:0/1965831202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2841f88a0 con 0x7fc284070360 2024-09-11T12:52:25.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289000640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 0x7fc2840ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28a400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 0x7fc2840aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc289a00640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c3f0 tx=0x7fc27800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278017070 con 0x7fc284070360 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2780053c0 con 0x7fc284070360 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc278003860 con 0x7fc284070360 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2841f8b80 con 0x7fc284070360 2024-09-11T12:52:25.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2841f9160 con 0x7fc284070360 2024-09-11T12:52:25.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.237+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28415b350 con 0x7fc284070360 2024-09-11T12:52:25.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.241+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 7) ==== 95136+0+0 (secure 0 0 0) 0x7fc27801a070 con 0x7fc284070360 2024-09-11T12:52:25.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.241+0000 7fc281600640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 0x7fc264073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.241+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc278091480 con 0x7fc284070360 2024-09-11T12:52:25.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.241+0000 7fc289000640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 0x7fc264073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.241+0000 7fc289000640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 0x7fc264073d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc2841fc370 tx=0x7fc270009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.245+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc27805d010 con 0x7fc284070360 2024-09-11T12:52:25.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.373+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fc278058ba0 con 0x7fc284070360 2024-09-11T12:52:25.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.413+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc284152030 con 0x7fc284070360 2024-09-11T12:52:25.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.413+0000 7fc281600640 1 -- 172.21.15.78:0/1965831202 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16764 (secure 0 0 0) 0x7fc278083460 con 0x7fc284070360 2024-09-11T12:52:25.416 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:25.417 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:51:24.200425+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:56.171601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:57.449946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.4","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 msgr2=0x7fc264073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 0x7fc264073d80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc2841fc370 tx=0x7fc270009040 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 msgr2=0x7fc28406f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc27800c3f0 tx=0x7fc27800c8b0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 shutdown_connections 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc2640718c0 0x7fc264073d80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc2840741e0 0x7fc2840aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc284070910 0x7fc2840ae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 --2- 172.21.15.78:0/1965831202 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc284070360 0x7fc28406f250 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 >> 172.21.15.78:0/1965831202 conn(0x7fc28406ca80 msgr2=0x7fc2840b1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 shutdown_connections 2024-09-11T12:52:25.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.417+0000 7fc28c400640 1 -- 172.21.15.78:0/1965831202 wait complete. 2024-09-11T12:52:25.438 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:52:25.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:25.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- 172.21.15.78:0/761523523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 msgr2=0x7fe5e000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 --2- 172.21.15.78:0/761523523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5e000b9d0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe5f40016f0 tx=0x7fe5f401c7b0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- 172.21.15.78:0/761523523 shutdown_connections 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 --2- 172.21.15.78:0/761523523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5e000b9d0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- 172.21.15.78:0/761523523 >> 172.21.15.78:0/761523523 conn(0x7fe5fc06ca80 msgr2=0x7fe5fc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- 172.21.15.78:0/761523523 shutdown_connections 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- 172.21.15.78:0/761523523 wait complete. 2024-09-11T12:52:25.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 Processor -- start 2024-09-11T12:52:25.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- start start 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 0x7fe5fc1f75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5fc1f7b00 0x7fe5fc0b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc0ccca0 con 0x7fe5e00051d0 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5fc0ccb20 con 0x7fe5fc0ac600 2024-09-11T12:52:25.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe600c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe5fc15e510 con 0x7fe5fc1f7b00 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe5f9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe5f9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40032/0 (socket says 172.21.15.78:40032) 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.581+0000 7fe5f9e00640 1 -- 172.21.15.78:0/2393050960 learned_addr learned my addr 172.21.15.78:0/2393050960 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5fa800640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5fc1f7b00 0x7fe5fc0b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9400640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 0x7fe5fc1f75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 -- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5fc1f7b00 msgr2=0x7fe5fc0b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5fc1f7b00 0x7fe5fc0b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 -- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 msgr2=0x7fe5fc1f75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 0x7fe5fc1f75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 -- 172.21.15.78:0/2393050960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5fc1f8060 con 0x7fe5e00051d0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9400640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 0x7fe5fc1f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f9e00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec006fd0 tx=0x7fe5ec0067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec00fab0 con 0x7fe5e00051d0 2024-09-11T12:52:25.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5ec016020 con 0x7fe5e00051d0 2024-09-11T12:52:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec011430 con 0x7fe5e00051d0 2024-09-11T12:52:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5fc1f8340 con 0x7fe5e00051d0 2024-09-11T12:52:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5fc0ae7e0 con 0x7fe5e00051d0 2024-09-11T12:52:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5fc0ae9e0 con 0x7fe5e00051d0 2024-09-11T12:52:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.585+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fe5ec0115d0 con 0x7fe5e00051d0 2024-09-11T12:52:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.589+0000 7fe5f2000640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 0x7fe5dc073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.589+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fe5ec092e20 con 0x7fe5e00051d0 2024-09-11T12:52:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.589+0000 7fe5f9400640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 0x7fe5dc073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.589+0000 7fe5f9400640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 0x7fe5dc073d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc15a620 tx=0x7fe5e80091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.593+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe5fc0ae9e0 con 0x7fe5e00051d0 2024-09-11T12:52:25.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.789+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe5fc06fdc0 con 0x7fe5e00051d0 2024-09-11T12:52:25.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.789+0000 7fe5f2000640 1 -- 172.21.15.78:0/2393050960 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7fe5ec020020 con 0x7fe5e00051d0 2024-09-11T12:52:25.792 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:25.792 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:25.792 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 msgr2=0x7fe5dc073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 0x7fe5dc073d20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc15a620 tx=0x7fe5e80091e0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 msgr2=0x7fe5fc0aed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec006fd0 tx=0x7fe5ec0067a0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 shutdown_connections 2024-09-11T12:52:25.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe5dc071860 0x7fe5dc073d20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5fc1f7b00 0x7fe5fc0b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5fc0ac600 0x7fe5fc1f75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 --2- 172.21.15.78:0/2393050960 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5e00051d0 0x7fe5fc0aed00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 >> 172.21.15.78:0/2393050960 conn(0x7fe5fc06ca80 msgr2=0x7fe5fc072b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 shutdown_connections 2024-09-11T12:52:25.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.793+0000 7fe600c00640 1 -- 172.21.15.78:0/2393050960 wait complete. 2024-09-11T12:52:25.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 -- 172.21.15.78:0/929313712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 msgr2=0x7f0c9c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 --2- 172.21.15.78:0/929313712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c0700c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009c80 tx=0x7f0c9401cda0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 -- 172.21.15.78:0/929313712 shutdown_connections 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 --2- 172.21.15.78:0/929313712 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c0700c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 --2- 172.21.15.78:0/929313712 >> v2:172.21.15.157:3300/0 conn(0x7f0c9c13d840 0x7f0c9c140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 -- 172.21.15.78:0/929313712 >> 172.21.15.78:0/929313712 conn(0x7f0c9c06ca80 msgr2=0x7f0c9c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 -- 172.21.15.78:0/929313712 shutdown_connections 2024-09-11T12:52:25.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.953+0000 7f0ca3800640 1 -- 172.21.15.78:0/929313712 wait complete. 2024-09-11T12:52:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 Processor -- start 2024-09-11T12:52:25.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- start start 2024-09-11T12:52:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c06fc50 0x7f0c9c164f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c1679a0 con 0x7f0c9c13d840 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9c167820 con 0x7f0c9c06fc50 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0c9c167b20 con 0x7f0c9c0cf460 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40044/0 (socket says 172.21.15.78:40044) 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 -- 172.21.15.78:0/997213617 learned_addr learned my addr 172.21.15.78:0/997213617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca0e00640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c06fc50 0x7f0c9c164f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0c9be00640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 -- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 msgr2=0x7f0c9c157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 -- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c06fc50 msgr2=0x7f0c9c164f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c06fc50 0x7f0c9c164f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 -- 172.21.15.78:0/997213617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9c153ea0 con 0x7f0c9c13d840 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca1800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009bd0 tx=0x7f0c940092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940bb070 con 0x7f0c9c13d840 2024-09-11T12:52:25.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9c154180 con 0x7f0c9c13d840 2024-09-11T12:52:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c94004d40 con 0x7f0c9c13d840 2024-09-11T12:52:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c940b2040 con 0x7f0c9c13d840 2024-09-11T12:52:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.957+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9c155550 con 0x7f0c9c13d840 2024-09-11T12:52:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c9c0700c0 con 0x7f0c9c13d840 2024-09-11T12:52:25.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f0c940a5bc0 con 0x7f0c9c13d840 2024-09-11T12:52:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0c98c00640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 0x7f0c7c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:25.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f0c9412be10 con 0x7f0c9c13d840 2024-09-11T12:52:25.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0ca0e00640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 0x7f0c7c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:25.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.961+0000 7f0ca0e00640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 0x7f0c7c073cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c13cda0 tx=0x7f0c900091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:25.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:25.965+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0c94016990 con 0x7f0c9c13d840 2024-09-11T12:52:26.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c9c072860 con 0x7f0c9c13d840 2024-09-11T12:52:26.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0c98c00640 1 -- 172.21.15.78:0/997213617 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5229 (secure 0 0 0) 0x7f0c940f7390 con 0x7f0c9c13d840 2024-09-11T12:52:26.169 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:26.170 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":27,"btime":"2024-09-11T12:51:08:150143+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:50:58.476924+0000","modified":"2024-09-11T12:51:01.500477+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5271},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5271":{"gid":5271,"name":"a","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/2225802748","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2225802748},{"type":"v1","addr":"172.21.15.78:6835","nonce":2225802748}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5271,"qdb_cluster":[5271]},"id":3}]} 2024-09-11T12:52:26.170 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 27 2024-09-11T12:52:26.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 msgr2=0x7f0c7c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 0x7f0c7c073cf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c13cda0 tx=0x7f0c900091e0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 msgr2=0x7f0c9c158070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0c94009bd0 tx=0x7f0c940092e0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 shutdown_connections 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0c7c071830 0x7f0c7c073cf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0c9c13d840 0x7f0c9c158070 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0c9c0cf460 0x7f0c9c157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 --2- 172.21.15.78:0/997213617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0c9c06fc50 0x7f0c9c164f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 >> 172.21.15.78:0/997213617 conn(0x7f0c9c06ca80 msgr2=0x7f0c9c15e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 shutdown_connections 2024-09-11T12:52:26.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.169+0000 7f0ca3800640 1 -- 172.21.15.78:0/997213617 wait complete. 2024-09-11T12:52:26.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- 172.21.15.78:0/2629161249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015eda0 msgr2=0x7f938015cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 --2- 172.21.15.78:0/2629161249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015eda0 0x7f938015cf00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9370009900 tx=0x7f937001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- 172.21.15.78:0/2629161249 shutdown_connections 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 --2- 172.21.15.78:0/2629161249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015eda0 0x7f938015cf00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- 172.21.15.78:0/2629161249 >> 172.21.15.78:0/2629161249 conn(0x7f938006ca80 msgr2=0x7f93800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- 172.21.15.78:0/2629161249 shutdown_connections 2024-09-11T12:52:26.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- 172.21.15.78:0/2629161249 wait complete. 2024-09-11T12:52:26.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 Processor -- start 2024-09-11T12:52:26.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 -- start start 2024-09-11T12:52:26.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93800b4520 0x7f938013bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.333+0000 7f9387800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015eda0 0x7f93801fb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9380169fc0 con 0x7f938015c5f0 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9380169e40 con 0x7f938015eda0 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f938016a140 con 0x7f93800b4520 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40056/0 (socket says 172.21.15.78:40056) 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 -- 172.21.15.78:0/733757455 learned_addr learned my addr 172.21.15.78:0/733757455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9384e00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93800b4520 0x7f938013bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9385800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015eda0 0x7f93801fb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 -- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93800b4520 msgr2=0x7f938013bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93800b4520 0x7f938013bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 -- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015eda0 msgr2=0x7f93801fb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015eda0 0x7f93801fb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 -- 172.21.15.78:0/733757455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f938013c170 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937fe00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c4b0 tx=0x7f936c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0052d0 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f936c005470 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f938013c450 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c012870 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f938013ca10 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938015cf00 con 0x7f938015c5f0 2024-09-11T12:52:26.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.337+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f936c003600 con 0x7f938015c5f0 2024-09-11T12:52:26.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.341+0000 7f937cc00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 0x7f9360073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.341+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7f936c091b70 con 0x7f938015c5f0 2024-09-11T12:52:26.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.341+0000 7f9384e00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 0x7f9360073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.341+0000 7f9384e00640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 0x7f9360073d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93700af000 tx=0x7f937001b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:26.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.345+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f936c05d0c0 con 0x7f938015c5f0 2024-09-11T12:52:26.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.513+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9380071010 con 0x7f938015c5f0 2024-09-11T12:52:26.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f937cc00640 1 -- 172.21.15.78:0/733757455 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16764 (secure 0 0 0) 0x7f936c061610 con 0x7f938015c5f0 2024-09-11T12:52:26.517 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:26.517 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:51:24.200425+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:50:56.171601+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-11T12:50:57.449946+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.4","mappings":[{"from":0,"to":5}]},{"pgid":"7.35","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 msgr2=0x7f9360073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 0x7f9360073d20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f93700af000 tx=0x7f937001b120 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 msgr2=0x7f9380154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f936c00c4b0 tx=0x7f936c00c970 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 shutdown_connections 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9360071860 0x7f9360073d20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f938015eda0 0x7f93801fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f938015c5f0 0x7f9380154e30 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 --2- 172.21.15.78:0/733757455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93800b4520 0x7f938013bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 >> 172.21.15.78:0/733757455 conn(0x7f938006ca80 msgr2=0x7f9380150c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 shutdown_connections 2024-09-11T12:52:26.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.517+0000 7f9387800640 1 -- 172.21.15.78:0/733757455 wait complete. 2024-09-11T12:52:26.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 -- 172.21.15.78:0/1845914518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1380052f0 msgr2=0x7fc13800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 --2- 172.21.15.78:0/1845914518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1380052f0 0x7fc13800b9d0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00bbb0 tx=0x7fc14c01ce60 comp rx=0 tx=0).stop 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 -- 172.21.15.78:0/1845914518 shutdown_connections 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 --2- 172.21.15.78:0/1845914518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1380052f0 0x7fc13800b9d0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 -- 172.21.15.78:0/1845914518 >> 172.21.15.78:0/1845914518 conn(0x7fc15406ca80 msgr2=0x7fc1540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 -- 172.21.15.78:0/1845914518 shutdown_connections 2024-09-11T12:52:26.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.681+0000 7fc15ac00640 1 -- 172.21.15.78:0/1845914518 wait complete. 2024-09-11T12:52:26.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 Processor -- start 2024-09-11T12:52:26.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- start start 2024-09-11T12:52:26.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1380052f0 0x7fc154150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540aeaa0 0x7fc15414cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc15415e730 con 0x7fc15414d9b0 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc15415e5b0 con 0x7fc1540aeaa0 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc15415e8b0 con 0x7fc1380052f0 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40062/0 (socket says 172.21.15.78:40062) 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 -- 172.21.15.78:0/404460170 learned_addr learned my addr 172.21.15.78:0/404460170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153e00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1380052f0 0x7fc154150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc158c00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540aeaa0 0x7fc15414cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 -- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1380052f0 msgr2=0x7fc154150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1380052f0 0x7fc154150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 -- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540aeaa0 msgr2=0x7fc15414cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540aeaa0 0x7fc15414cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 -- 172.21.15.78:0/404460170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15414de30 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc153400640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc140003f60 tx=0x7fc140003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140011640 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc140016020 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140011a60 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15414e0b0 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15414e670 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.685+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15415ee10 con 0x7fc15414d9b0 2024-09-11T12:52:26.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.689+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fc140011c00 con 0x7fc15414d9b0 2024-09-11T12:52:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.689+0000 7fc14be00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 0x7fc134073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.689+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5767+0+0 (secure 0 0 0) 0x7fc140092db0 con 0x7fc15414d9b0 2024-09-11T12:52:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.689+0000 7fc153e00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 0x7fc134073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.689+0000 7fc153e00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 0x7fc134073cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc144001360 tx=0x7fc144006b30 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:26.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.693+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc140097070 con 0x7fc15414d9b0 2024-09-11T12:52:26.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.893+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc1540cebf0 con 0x7fc15414d9b0 2024-09-11T12:52:26.894 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:52:26.893+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:52:26.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.893+0000 7fc14be00640 1 -- 172.21.15.78:0/404460170 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7fc140020020 con 0x7fc15414d9b0 2024-09-11T12:52:26.896 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:52:26.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 msgr2=0x7fc134073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 0x7fc134073cf0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc144001360 tx=0x7fc144006b30 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 msgr2=0x7fc15414f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc140003f60 tx=0x7fc140003f90 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 shutdown_connections 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc134071830 0x7fc134073cf0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1540aeaa0 0x7fc15414cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc15414d9b0 0x7fc15414f150 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 --2- 172.21.15.78:0/404460170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1380052f0 0x7fc154150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 >> 172.21.15.78:0/404460170 conn(0x7fc15406ca80 msgr2=0x7fc1540b4c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 shutdown_connections 2024-09-11T12:52:26.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:26.897+0000 7fc15ac00640 1 -- 172.21.15.78:0/404460170 wait complete. 2024-09-11T12:52:26.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:52:26.962 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 -- 172.21.15.78:0/3163994363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 msgr2=0x7fc89c0b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 --2- 172.21.15.78:0/3163994363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c0b3400 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc894009bc0 tx=0x7fc89401cba0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 -- 172.21.15.78:0/3163994363 shutdown_connections 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 --2- 172.21.15.78:0/3163994363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c0b3400 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 --2- 172.21.15.78:0/3163994363 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c150a70 0x7fc89c0b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 -- 172.21.15.78:0/3163994363 >> 172.21.15.78:0/3163994363 conn(0x7fc89c075a10 msgr2=0x7fc89c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 -- 172.21.15.78:0/3163994363 shutdown_connections 2024-09-11T12:52:27.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 -- 172.21.15.78:0/3163994363 wait complete. 2024-09-11T12:52:27.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.065+0000 7fc8a2800640 1 Processor -- start 2024-09-11T12:52:27.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- start start 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c150a70 0x7fc89c14b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c14db30 0x7fc89c0ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c166930 con 0x7fc89c15ac40 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c0b5680 con 0x7fc89c150a70 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc89c166ab0 con 0x7fc89c14db30 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:27.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40092/0 (socket says 172.21.15.78:40092) 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 -- 172.21.15.78:0/1072473544 learned_addr learned my addr 172.21.15.78:0/1072473544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc89be00640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c14db30 0x7fc89c0ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc89b400640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c150a70 0x7fc89c14b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 -- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c14db30 msgr2=0x7fc89c0ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c14db30 0x7fc89c0ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 -- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c150a70 msgr2=0x7fc89c14b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c150a70 0x7fc89c14b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 -- 172.21.15.78:0/1072473544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c14b980 con 0x7fc89c15ac40 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893e00640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c430 tx=0x7fc88400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884011790 con 0x7fc89c15ac40 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc884017070 con 0x7fc89c15ac40 2024-09-11T12:52:27.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc884011d20 con 0x7fc89c15ac40 2024-09-11T12:52:27.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c1f74e0 con 0x7fc89c15ac40 2024-09-11T12:52:27.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1f7780 con 0x7fc89c15ac40 2024-09-11T12:52:27.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.069+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc89c15a530 con 0x7fc89c15ac40 2024-09-11T12:52:27.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.073+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fc88401c460 con 0x7fc89c15ac40 2024-09-11T12:52:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.073+0000 7fc893400640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 0x7fc878073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:27.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.073+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7fc884091ff0 con 0x7fc89c15ac40 2024-09-11T12:52:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.073+0000 7fc89b400640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 0x7fc878073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:27.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.073+0000 7fc89b400640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 0x7fc878073cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00afd0 tx=0x7fc88c00c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:27.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.077+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc88405d460 con 0x7fc89c15ac40 2024-09-11T12:52:27.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.285+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc89c0b5ac0 con 0x7fc89c15ac40 2024-09-11T12:52:27.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.901+0000 7fc893400640 1 -- 172.21.15.78:0/1072473544 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fc8840619b0 con 0x7fc89c15ac40 2024-09-11T12:52:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 msgr2=0x7fc878073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 0x7fc878073cf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc88c00afd0 tx=0x7fc88c00c040 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 msgr2=0x7fc89c156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fc88400c430 tx=0x7fc88400c8f0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 shutdown_connections 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc878071830 0x7fc878073cf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc89c14db30 0x7fc89c0ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc89c15ac40 0x7fc89c156940 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 --2- 172.21.15.78:0/1072473544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc89c150a70 0x7fc89c14b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 >> 172.21.15.78:0/1072473544 conn(0x7fc89c075a10 msgr2=0x7fc89c0b0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 shutdown_connections 2024-09-11T12:52:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:27.905+0000 7fc8a2800640 1 -- 172.21.15.78:0/1072473544 wait complete. 2024-09-11T12:52:27.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 -- 172.21.15.78:0/2967767687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0888004b70 msgr2=0x7f0888004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2967767687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0888004b70 0x7f0888004f90 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0894006fd0 tx=0x7f089401ca60 comp rx=0 tx=0).stop 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 -- 172.21.15.78:0/2967767687 shutdown_connections 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2967767687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0888004b70 0x7f0888004f90 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 -- 172.21.15.78:0/2967767687 >> 172.21.15.78:0/2967767687 conn(0x7f08a00b3050 msgr2=0x7f08a00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 -- 172.21.15.78:0/2967767687 shutdown_connections 2024-09-11T12:52:28.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.113+0000 7f08a7e00640 1 -- 172.21.15.78:0/2967767687 wait complete. 2024-09-11T12:52:28.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 Processor -- start 2024-09-11T12:52:28.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- start start 2024-09-11T12:52:28.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0888004b70 0x7f08a014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:28.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a014da60 0x7f08a01fb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0072500 con 0x7f08a01fbb20 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08a0072380 con 0x7f08a014da60 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08a0072680 con 0x7f0888004b70 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40124/0 (socket says 172.21.15.78:40124) 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 -- 172.21.15.78:0/2479748013 learned_addr learned my addr 172.21.15.78:0/2479748013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5400640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0888004b70 0x7f08a014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:28.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a4a00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a014da60 0x7f08a01fb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:28.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 -- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0888004b70 msgr2=0x7f08a014d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:28.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0888004b70 0x7f08a014d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 -- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a014da60 msgr2=0x7f08a01fb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a014da60 0x7f08a01fb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 -- 172.21.15.78:0/2479748013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08a014dfe0 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a5e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4250 tx=0x7f0898011b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898004250 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0898016070 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0898005280 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08a01f7a50 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.117+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08a01f8010 con 0x7f08a01fbb20 2024-09-11T12:52:28.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08a006f790 con 0x7f08a01fbb20 2024-09-11T12:52:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f0898018030 con 0x7f08a01fbb20 2024-09-11T12:52:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f089d600640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 0x7f0880073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:28.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5853+0+0 (secure 0 0 0) 0x7f0898092110 con 0x7f08a01fbb20 2024-09-11T12:52:28.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f08a5400640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 0x7f0880073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:28.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.121+0000 7f08a5400640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 0x7f0880073d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08940aa000 tx=0x7f08940b1040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:28.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.125+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f089805d630 con 0x7f08a01fbb20 2024-09-11T12:52:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.293+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08a0070ce0 con 0x7f08a01fbb20 2024-09-11T12:52:28.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f089d600640 1 -- 172.21.15.78:0/2479748013 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v49) ==== 181+0+0 (secure 0 0 0) 0x7f0898061b80 con 0x7f08a01fbb20 2024-09-11T12:52:28.905 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:52:28.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 msgr2=0x7f0880073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:28.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 0x7f0880073d20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f08940aa000 tx=0x7f08940b1040 comp rx=0 tx=0).stop 2024-09-11T12:52:28.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 msgr2=0x7f08a014c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:28.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f08a00b4250 tx=0x7f0898011b10 comp rx=0 tx=0).stop 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 shutdown_connections 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0880071860 0x7f0880073d20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08a01fbb20 0x7f08a014c250 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08a014da60 0x7f08a01fb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 --2- 172.21.15.78:0/2479748013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0888004b70 0x7f08a014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 >> 172.21.15.78:0/2479748013 conn(0x7f08a00b3050 msgr2=0x7f08a015e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 shutdown_connections 2024-09-11T12:52:28.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:28.905+0000 7f08a7e00640 1 -- 172.21.15.78:0/2479748013 wait complete. 2024-09-11T12:52:28.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- 172.21.15.78:0/3425404103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c004810 msgr2=0x7f150c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 --2- 172.21.15.78:0/3425404103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f150c004810 0x7f150c004c90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1514009b30 tx=0x7f151401d210 comp rx=0 tx=0).stop 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152d600640 1 -- 172.21.15.78:0/3425404103 reap_dead start 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- 172.21.15.78:0/3425404103 shutdown_connections 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- 172.21.15.78:0/3425404103 >> 172.21.15.78:0/3425404103 conn(0x7f15280b1c20 msgr2=0x7f1528070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- 172.21.15.78:0/3425404103 shutdown_connections 2024-09-11T12:52:29.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- 172.21.15.78:0/3425404103 wait complete. 2024-09-11T12:52:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 Processor -- start 2024-09-11T12:52:29.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- start start 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 0x7f152815e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1528161460 0x7f15281618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1528166700 con 0x7f1528150fa0 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1528166580 con 0x7f150c004810 2024-09-11T12:52:29.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f152e000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1528166880 con 0x7f1528161460 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.077+0000 7f1527e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40142/0 (socket says 172.21.15.78:40142) 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 0x7f152815e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 -- 172.21.15.78:0/1350096815 learned_addr learned my addr 172.21.15.78:0/1350096815 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152d600640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1528161460 0x7f15281618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 -- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1528161460 msgr2=0x7f15281618e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1528161460 0x7f15281618e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 -- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 msgr2=0x7f152815e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 0x7f152815e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 -- 172.21.15.78:0/1350096815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15281f76e0 con 0x7f1528150fa0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152cc00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 0x7f152815e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1527e00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f15140af040 tx=0x7f1514005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140b1040 con 0x7f1528150fa0 2024-09-11T12:52:29.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15140ac7b0 con 0x7f1528150fa0 2024-09-11T12:52:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1528162300 con 0x7f1528150fa0 2024-09-11T12:52:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15140acc00 con 0x7f1528150fa0 2024-09-11T12:52:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f152806cb80 con 0x7f1528150fa0 2024-09-11T12:52:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f152815c890 con 0x7f1528150fa0 2024-09-11T12:52:29.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.081+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f15140046d0 con 0x7f1528150fa0 2024-09-11T12:52:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.085+0000 7f1524c00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 0x7f1508073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.085+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5415+0+0 (secure 0 0 0) 0x7f151412cc00 con 0x7f1528150fa0 2024-09-11T12:52:29.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.085+0000 7f152cc00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 0x7f1508073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:29.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.085+0000 7f152cc00640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 0x7f1508073d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15180128a0 tx=0x7f1518009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:29.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.089+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1514016f20 con 0x7f1528150fa0 2024-09-11T12:52:29.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.261+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f152815f690 con 0x7f1528150fa0 2024-09-11T12:52:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.909+0000 7f1524c00640 1 -- 172.21.15.78:0/1350096815 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v50) ==== 169+0+0 (secure 0 0 0) 0x7f15140f8250 con 0x7f1528150fa0 2024-09-11T12:52:29.910 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:52:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 msgr2=0x7f1508073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 0x7f1508073d20 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15180128a0 tx=0x7f1518009500 comp rx=0 tx=0).stop 2024-09-11T12:52:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 msgr2=0x7f152814d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f15140af040 tx=0x7f1514005620 comp rx=0 tx=0).stop 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 shutdown_connections 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1508071860 0x7f1508073d20 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f15180128a0 tx=0x7f1518009500 comp rx=0 tx=0).stop 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1528161460 0x7f15281618e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1528150fa0 0x7f152814d3c0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 --2- 172.21.15.78:0/1350096815 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f150c004810 0x7f152815e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 >> 172.21.15.78:0/1350096815 conn(0x7f15280b1c20 msgr2=0x7f15281652b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 shutdown_connections 2024-09-11T12:52:29.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:29.913+0000 7f152e000640 1 -- 172.21.15.78:0/1350096815 wait complete. 2024-09-11T12:52:29.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:52:30.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- 172.21.15.78:0/3764808829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 msgr2=0x7f13d8070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 --2- 172.21.15.78:0/3764808829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d8070a00 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f13d80b58b0 tx=0x7f13cc01ccd0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- 172.21.15.78:0/3764808829 shutdown_connections 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 --2- 172.21.15.78:0/3764808829 >> v2:172.21.15.157:3300/0 conn(0x7f13d80b4db0 0x7f13d80b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 --2- 172.21.15.78:0/3764808829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80736d0 0x7f13d815af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 --2- 172.21.15.78:0/3764808829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d8070a00 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- 172.21.15.78:0/3764808829 >> 172.21.15.78:0/3764808829 conn(0x7f13d806ca80 msgr2=0x7f13d80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- 172.21.15.78:0/3764808829 shutdown_connections 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- 172.21.15.78:0/3764808829 wait complete. 2024-09-11T12:52:30.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 Processor -- start 2024-09-11T12:52:30.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.085+0000 7f13dfe00640 1 -- start start 2024-09-11T12:52:30.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d80736d0 0x7f13d814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 0x7f13d814d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13d806f0c0 con 0x7f13d8157ae0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13d806ef40 con 0x7f13d80736d0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13d806f240 con 0x7f13d80b4db0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40146/0 (socket says 172.21.15.78:40146) 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 -- 172.21.15.78:0/2516822681 learned_addr learned my addr 172.21.15.78:0/2516822681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dca00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 0x7f13d814d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dd400640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d80736d0 0x7f13d814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 -- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 msgr2=0x7f13d814d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 0x7f13d814d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 -- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d80736d0 msgr2=0x7f13d814f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d80736d0 0x7f13d814f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 -- 172.21.15.78:0/2516822681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d80afb90 con 0x7f13d8157ae0 2024-09-11T12:52:30.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dca00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 0x7f13d814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:30.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dde00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f13d000d750 tx=0x7f13d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:30.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d0017070 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d0009d20 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d000de60 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d8154680 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d80afe30 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13d80b0030 con 0x7f13d8157ae0 2024-09-11T12:52:30.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.089+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f13d0005280 con 0x7f13d8157ae0 2024-09-11T12:52:30.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.093+0000 7f13d5600640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 0x7f13b8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.093+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f13d0091fa0 con 0x7f13d8157ae0 2024-09-11T12:52:30.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.093+0000 7f13dd400640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 0x7f13b8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.093+0000 7f13dd400640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 0x7f13b8073cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc01b510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:30.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.097+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f13d80b0030 con 0x7f13d8157ae0 2024-09-11T12:52:30.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.261+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f13d80ce0e0 con 0x7f13d8157ae0 2024-09-11T12:52:30.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.265+0000 7f13d5600640 1 -- 172.21.15.78:0/2516822681 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7f13d005d690 con 0x7f13d8157ae0 2024-09-11T12:52:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 msgr2=0x7f13b8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 0x7f13b8073cf0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0af000 tx=0x7f13cc01b510 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 msgr2=0x7f13d814df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f13d000d750 tx=0x7f13d000dc10 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 shutdown_connections 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f13b8071830 0x7f13b8073cf0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13d8157ae0 0x7f13d814df20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13d80b4db0 0x7f13d814d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 --2- 172.21.15.78:0/2516822681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13d80736d0 0x7f13d814f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 >> 172.21.15.78:0/2516822681 conn(0x7f13d806ca80 msgr2=0x7f13d8071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 shutdown_connections 2024-09-11T12:52:30.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.269+0000 7f13dfe00640 1 -- 172.21.15.78:0/2516822681 wait complete. 2024-09-11T12:52:30.290 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-11T12:52:30.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- 172.21.15.78:0/1192141355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 msgr2=0x7fab2415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 --2- 172.21.15.78:0/1192141355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab2415a650 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fab18009870 tx=0x7fab1801c820 comp rx=0 tx=0).stop 2024-09-11T12:52:30.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- 172.21.15.78:0/1192141355 shutdown_connections 2024-09-11T12:52:30.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 --2- 172.21.15.78:0/1192141355 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab24070020 0x7fab240704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 --2- 172.21.15.78:0/1192141355 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab2415a650 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- 172.21.15.78:0/1192141355 >> 172.21.15.78:0/1192141355 conn(0x7fab240cfeb0 msgr2=0x7fab240d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- 172.21.15.78:0/1192141355 shutdown_connections 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- 172.21.15.78:0/1192141355 wait complete. 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 Processor -- start 2024-09-11T12:52:30.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.437+0000 7fab2b000640 1 -- start start 2024-09-11T12:52:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab24070020 0x7fab24138de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab2415e650 0x7fab2413cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406f420 con 0x7fab24143940 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab2406f2a0 con 0x7fab24070020 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab2406f5a0 con 0x7fab2415e650 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40160/0 (socket says 172.21.15.78:40160) 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab24070020 0x7fab24138de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab2415e650 0x7fab2413cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 -- 172.21.15.78:0/2963489897 learned_addr learned my addr 172.21.15.78:0/2963489897 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 -- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab2415e650 msgr2=0x7fab2413cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab2415e650 0x7fab2413cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 -- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab24070020 msgr2=0x7fab24138de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab24070020 0x7fab24138de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 -- 172.21.15.78:0/2963489897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2415dfa0 con 0x7fab24143940 2024-09-11T12:52:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab29200640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c460 tx=0x7fab1000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10016020 con 0x7fab24143940 2024-09-11T12:52:30.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab10012940 con 0x7fab24143940 2024-09-11T12:52:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab2413a470 con 0x7fab24143940 2024-09-11T12:52:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab10012d90 con 0x7fab24143940 2024-09-11T12:52:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab240afcb0 con 0x7fab24143940 2024-09-11T12:52:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab240704b0 con 0x7fab24143940 2024-09-11T12:52:30.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.441+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fab10019020 con 0x7fab24143940 2024-09-11T12:52:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.445+0000 7fab21600640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 0x7fab04073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.445+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fab10091cf0 con 0x7fab24143940 2024-09-11T12:52:30.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.445+0000 7fab29c00640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 0x7fab04073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:30.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.445+0000 7fab29c00640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 0x7fab04073cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fab180ae000 tx=0x7fab1801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:30.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.449+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fab10020020 con 0x7fab24143940 2024-09-11T12:52:30.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.673+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7fab2413f630 con 0x7fab24143940 2024-09-11T12:52:30.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab21600640 1 -- 172.21.15.78:0/2963489897 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v156) ==== 140+0+0 (secure 0 0 0) 0x7fab1005d640 con 0x7fab24143940 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 msgr2=0x7fab04073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 0x7fab04073cf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fab180ae000 tx=0x7fab1801b060 comp rx=0 tx=0).stop 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 msgr2=0x7fab24139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fab1000c460 tx=0x7fab1000c920 comp rx=0 tx=0).stop 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 shutdown_connections 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fab04071830 0x7fab04073cf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab2415e650 0x7fab2413cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab24143940 0x7fab24139dd0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 --2- 172.21.15.78:0/2963489897 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab24070020 0x7fab24138de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 >> 172.21.15.78:0/2963489897 conn(0x7fab240cfeb0 msgr2=0x7fab2413e940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 shutdown_connections 2024-09-11T12:52:30.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:30.913+0000 7fab2b000640 1 -- 172.21.15.78:0/2963489897 wait complete. 2024-09-11T12:52:30.934 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-11T12:52:30.935 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-11T12:52:30.935 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:52:30.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:31.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.073+0000 7fac2e400640 1 -- 172.21.15.78:0/3350394662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac0c004810 msgr2=0x7fac0c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.073+0000 7fac2e400640 1 --2- 172.21.15.78:0/3350394662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac0c004810 0x7fac0c004c90 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fac14009b60 tx=0x7fac1401d270 comp rx=0 tx=0).stop 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.073+0000 7fac27e00640 1 -- 172.21.15.78:0/3350394662 reap_dead start 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- 172.21.15.78:0/3350394662 shutdown_connections 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- 172.21.15.78:0/3350394662 >> 172.21.15.78:0/3350394662 conn(0x7fac2806ca80 msgr2=0x7fac280b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- 172.21.15.78:0/3350394662 shutdown_connections 2024-09-11T12:52:31.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- 172.21.15.78:0/3350394662 wait complete. 2024-09-11T12:52:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 Processor -- start 2024-09-11T12:52:31.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- start start 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c004810 0x7fac2815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 0x7fac281f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac280743b0 con 0x7fac2815fc70 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac28074230 con 0x7fac0c004810 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac2e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac2806f400 con 0x7fac281f58e0 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40184/0 (socket says 172.21.15.78:40184) 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 -- 172.21.15.78:0/82910394 learned_addr learned my addr 172.21.15.78:0/82910394 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac27e00640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 0x7fac281f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac27400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c004810 0x7fac2815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 -- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 msgr2=0x7fac281f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 0x7fac281f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 -- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c004810 msgr2=0x7fac2815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c004810 0x7fac2815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac26a00640 1 -- 172.21.15.78:0/82910394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac2815c610 con 0x7fac2815fc70 2024-09-11T12:52:31.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.077+0000 7fac27e00640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 0x7fac281f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac26a00640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac14009a80 tx=0x7fac14005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140ba020 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac140ac7b0 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140b1040 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac28149f60 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac2814a520 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac2815b550 con 0x7fac2815fc70 2024-09-11T12:52:31.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fac140bd020 con 0x7fac2815fc70 2024-09-11T12:52:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 0x7fac08073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.081+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fac1412cb60 con 0x7fac2815fc70 2024-09-11T12:52:31.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.085+0000 7fac27400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 0x7fac08073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.085+0000 7fac27400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 0x7fac08073cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac180015d0 tx=0x7fac18001300 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.085+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fac140c4030 con 0x7fac2815fc70 2024-09-11T12:52:31.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.253+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac2814f740 con 0x7fac2815fc70 2024-09-11T12:52:31.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.253+0000 7fac1f400640 1 -- 172.21.15.78:0/82910394 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13032 (secure 0 0 0) 0x7fac14016f80 con 0x7fac2815fc70 2024-09-11T12:52:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:31.255 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":50,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:29.906334+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/2225802748":"2024-09-12T12:52:26.895177+0000","172.21.15.78:6833/2225802748":"2024-09-12T12:52:26.895177+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 msgr2=0x7fac08073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 0x7fac08073cf0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fac180015d0 tx=0x7fac18001300 comp rx=0 tx=0).stop 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 msgr2=0x7fac281600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fac14009a80 tx=0x7fac14005640 comp rx=0 tx=0).stop 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 shutdown_connections 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fac08071830 0x7fac08073cf0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac281f58e0 0x7fac281f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac2815fc70 0x7fac281600d0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 --2- 172.21.15.78:0/82910394 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac0c004810 0x7fac2815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 >> 172.21.15.78:0/82910394 conn(0x7fac2806ca80 msgr2=0x7fac280b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 shutdown_connections 2024-09-11T12:52:31.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.257+0000 7fac2e400640 1 -- 172.21.15.78:0/82910394 wait complete. 2024-09-11T12:52:31.277 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:52:31.277 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 -- 172.21.15.78:0/1929988249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 msgr2=0x7feb780b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 --2- 172.21.15.78:0/1929988249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb780b1a10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7feb68009840 tx=0x7feb6801c820 comp rx=0 tx=0).stop 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 -- 172.21.15.78:0/1929988249 shutdown_connections 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 --2- 172.21.15.78:0/1929988249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780ce320 0x7feb7806fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 --2- 172.21.15.78:0/1929988249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb780b1a10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 -- 172.21.15.78:0/1929988249 >> 172.21.15.78:0/1929988249 conn(0x7feb78071dd0 msgr2=0x7feb780721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 -- 172.21.15.78:0/1929988249 shutdown_connections 2024-09-11T12:52:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.421+0000 7feb7da00640 1 -- 172.21.15.78:0/1929988249 wait complete. 2024-09-11T12:52:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 Processor -- start 2024-09-11T12:52:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- start start 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1f50 0x7feb78145ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 0x7feb78158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb78167960 con 0x7feb781518b0 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb781677e0 con 0x7feb780ce320 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feb78167ae0 con 0x7feb780b1f50 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40206/0 (socket says 172.21.15.78:40206) 2024-09-11T12:52:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 -- 172.21.15.78:0/396513709 learned_addr learned my addr 172.21.15.78:0/396513709 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb76a00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1f50 0x7feb78145ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb6fe00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 0x7feb78158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 -- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1f50 msgr2=0x7feb78145ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1f50 0x7feb78145ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 -- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 msgr2=0x7feb78158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 0x7feb78158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 -- 172.21.15.78:0/396513709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb781faa50 con 0x7feb781518b0 2024-09-11T12:52:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb6fe00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 0x7feb78158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb77400640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb7000cad0 tx=0x7feb7000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000ae20 con 0x7feb781518b0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb7000f450 con 0x7feb781518b0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb781fad30 con 0x7feb781518b0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7000f8a0 con 0x7feb781518b0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb781fb2f0 con 0x7feb781518b0 2024-09-11T12:52:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.425+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb781fb940 con 0x7feb781518b0 2024-09-11T12:52:31.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.429+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7feb7000fa40 con 0x7feb781518b0 2024-09-11T12:52:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.429+0000 7feb6ea00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 0x7feb54073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.429+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7feb70092950 con 0x7feb781518b0 2024-09-11T12:52:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.429+0000 7feb76a00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 0x7feb54073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.429+0000 7feb76a00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 0x7feb54073d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb680af000 tx=0x7feb6801b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.433+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7feb70014030 con 0x7feb781518b0 2024-09-11T12:52:31.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feb781461d0 con 0x7feb781518b0 2024-09-11T12:52:31.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb6ea00640 1 -- 172.21.15.78:0/396513709 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7feb7005e690 con 0x7feb781518b0 2024-09-11T12:52:31.602 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 msgr2=0x7feb54073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 0x7feb54073d20 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7feb680af000 tx=0x7feb6801b060 comp rx=0 tx=0).stop 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 msgr2=0x7feb781548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7feb7000cad0 tx=0x7feb7000cf90 comp rx=0 tx=0).stop 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 shutdown_connections 2024-09-11T12:52:31.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feb54071860 0x7feb54073d20 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb781518b0 0x7feb781548a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb780ce320 0x7feb78158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 --2- 172.21.15.78:0/396513709 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1f50 0x7feb78145ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 >> 172.21.15.78:0/396513709 conn(0x7feb78071dd0 msgr2=0x7feb7815af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 shutdown_connections 2024-09-11T12:52:31.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.601+0000 7feb7da00640 1 -- 172.21.15.78:0/396513709 wait complete. 2024-09-11T12:52:31.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- 172.21.15.78:0/144668069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 msgr2=0x7fa7b0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 --2- 172.21.15.78:0/144668069 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7b0003e30 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009b60 tx=0x7fa7bc01d220 comp rx=0 tx=0).stop 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7ce600640 1 -- 172.21.15.78:0/144668069 reap_dead start 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- 172.21.15.78:0/144668069 shutdown_connections 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- 172.21.15.78:0/144668069 >> 172.21.15.78:0/144668069 conn(0x7fa7c806ca80 msgr2=0x7fa7c80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- 172.21.15.78:0/144668069 shutdown_connections 2024-09-11T12:52:31.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- 172.21.15.78:0/144668069 wait complete. 2024-09-11T12:52:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 Processor -- start 2024-09-11T12:52:31.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.765+0000 7fa7d0600640 1 -- start start 2024-09-11T12:52:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c814d2a0 0x7fa7c813ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c814e800 0x7fa7c81faa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81529b0 con 0x7fa7b0003a10 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8152830 con 0x7fa7c814d2a0 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa7c8152b30 con 0x7fa7c814e800 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40230/0 (socket says 172.21.15.78:40230) 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 -- 172.21.15.78:0/188184850 learned_addr learned my addr 172.21.15.78:0/188184850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7ce600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c814e800 0x7fa7c81faa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cd200640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c814d2a0 0x7fa7c813ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 -- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c814e800 msgr2=0x7fa7c81faa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c814e800 0x7fa7c81faa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 -- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c814d2a0 msgr2=0x7fa7c813ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c814d2a0 0x7fa7c813ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 -- 172.21.15.78:0/188184850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c80736c0 con 0x7fa7b0003a10 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7cdc00640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc01d050 tx=0x7fa7bc005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc0b1040 con 0x7fa7b0003a10 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7bc0049c0 con 0x7fa7b0003a10 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc009730 con 0x7fa7b0003a10 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c814d760 con 0x7fa7b0003a10 2024-09-11T12:52:31.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c814dd20 con 0x7fa7b0003a10 2024-09-11T12:52:31.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c80cf280 con 0x7fa7b0003a10 2024-09-11T12:52:31.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.769+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fa7bc0bf060 con 0x7fa7b0003a10 2024-09-11T12:52:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.773+0000 7fa7c5600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 0x7fa7a8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.773+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fa7bc12ba40 con 0x7fa7b0003a10 2024-09-11T12:52:31.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.773+0000 7fa7cd200640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 0x7fa7a8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:31.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.773+0000 7fa7cd200640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 0x7fa7a8073cf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81fa9e0 tx=0x7fa7b40091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:31.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:31.777+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa7c80736c0 con 0x7fa7b0003a10 2024-09-11T12:52:32.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.001+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fa7c80af770 con 0x7fa7b0003a10 2024-09-11T12:52:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.917+0000 7fa7c5600640 1 -- 172.21.15.78:0/188184850 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v158) ==== 164+0+0 (secure 0 0 0) 0x7fa7bc016fc0 con 0x7fa7b0003a10 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 msgr2=0x7fa7a8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 0x7fa7a8073cf0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7c81fa9e0 tx=0x7fa7b40091e0 comp rx=0 tx=0).stop 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 msgr2=0x7fa7c8073180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc01d050 tx=0x7fa7bc005640 comp rx=0 tx=0).stop 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 shutdown_connections 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa7a8071830 0x7fa7a8073cf0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7c814e800 0x7fa7c81faa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7c814d2a0 0x7fa7c813ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 --2- 172.21.15.78:0/188184850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7b0003a10 0x7fa7c8073180 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 >> 172.21.15.78:0/188184850 conn(0x7fa7c806ca80 msgr2=0x7fa7c80ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 shutdown_connections 2024-09-11T12:52:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:32.921+0000 7fa7d0600640 1 -- 172.21.15.78:0/188184850 wait complete. 2024-09-11T12:52:32.946 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- 172.21.15.78:0/3167267613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fceb4003a10 msgr2=0x7fceb4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 --2- 172.21.15.78:0/3167267613 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fceb4003a10 0x7fceb4003e30 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009ec0 tx=0x7fcec001d5a0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced4e00640 1 -- 172.21.15.78:0/3167267613 reap_dead start 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- 172.21.15.78:0/3167267613 shutdown_connections 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- 172.21.15.78:0/3167267613 >> 172.21.15.78:0/3167267613 conn(0x7fced006ca80 msgr2=0x7fced00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- 172.21.15.78:0/3167267613 shutdown_connections 2024-09-11T12:52:33.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- 172.21.15.78:0/3167267613 wait complete. 2024-09-11T12:52:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 Processor -- start 2024-09-11T12:52:33.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- start start 2024-09-11T12:52:33.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fceb4003a10 0x7fced01f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 0x7fced01661b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fced0070d00 con 0x7fced01512e0 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fced0070b80 con 0x7fced0165d30 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fced013e740 con 0x7fceb4003a10 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40252/0 (socket says 172.21.15.78:40252) 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 -- 172.21.15.78:0/1428349106 learned_addr learned my addr 172.21.15.78:0/1428349106 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecfe00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fceb4003a10 0x7fced01f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced4e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 0x7fced01661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 -- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fceb4003a10 msgr2=0x7fced01f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fceb4003a10 0x7fced01f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 -- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 msgr2=0x7fced01661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 0x7fced01661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 -- 172.21.15.78:0/1428349106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fced0153940 con 0x7fced01512e0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fced4e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 0x7fced01661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.093+0000 7fcecf400640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011ab0 tx=0x7fcebc011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc01e070 con 0x7fced01512e0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcebc00fc50 con 0x7fced01512e0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc019410 con 0x7fced01512e0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fced0151ed0 con 0x7fced01512e0 2024-09-11T12:52:33.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fced0153be0 con 0x7fced01512e0 2024-09-11T12:52:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fced013ecd0 con 0x7fced01512e0 2024-09-11T12:52:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fcebc021070 con 0x7fced01512e0 2024-09-11T12:52:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 0x7fceb0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fcebc0925b0 con 0x7fced01512e0 2024-09-11T12:52:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.097+0000 7fcecfe00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 0x7fceb0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.101+0000 7fcecfe00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 0x7fceb0073d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad000 tx=0x7fcec001a340 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:33.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.101+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcebc05dcc0 con 0x7fced01512e0 2024-09-11T12:52:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.269+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fced00cf9a0 con 0x7fced01512e0 2024-09-11T12:52:33.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fcec7e00640 1 -- 172.21.15.78:0/1428349106 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7fcebc062210 con 0x7fced01512e0 2024-09-11T12:52:33.272 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:52:33.272 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:52:33.273 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:52:33.274 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:52:33.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:52:33.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 msgr2=0x7fceb0073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 0x7fceb0073d20 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcec00ad000 tx=0x7fcec001a340 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 msgr2=0x7fced013c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fcebc011ab0 tx=0x7fcebc011f70 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 shutdown_connections 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fceb0071860 0x7fceb0073d20 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fced0165d30 0x7fced01661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fced01512e0 0x7fced013c930 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 --2- 172.21.15.78:0/1428349106 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fceb4003a10 0x7fced01f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 >> 172.21.15.78:0/1428349106 conn(0x7fced006ca80 msgr2=0x7fced0072f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 shutdown_connections 2024-09-11T12:52:33.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.273+0000 7fced6e00640 1 -- 172.21.15.78:0/1428349106 wait complete. 2024-09-11T12:52:33.294 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:52:33.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- 172.21.15.78:0/2793004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c15c3f0 msgr2=0x7fd12c0b2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- 172.21.15.78:0/2793004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c15c3f0 0x7fd12c0b2a00 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fd12c0af5b0 tx=0x7fd11801c600 comp rx=0 tx=0).stop 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- 172.21.15.78:0/2793004501 shutdown_connections 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- 172.21.15.78:0/2793004501 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c15c3f0 0x7fd12c0b2a00 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- 172.21.15.78:0/2793004501 >> v2:172.21.15.157:3300/0 conn(0x7fd12c0b2f40 0x7fd12c0731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- 172.21.15.78:0/2793004501 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c145190 0x7fd12c15beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- 172.21.15.78:0/2793004501 >> 172.21.15.78:0/2793004501 conn(0x7fd12c075c10 msgr2=0x7fd12c0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- 172.21.15.78:0/2793004501 shutdown_connections 2024-09-11T12:52:33.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- 172.21.15.78:0/2793004501 wait complete. 2024-09-11T12:52:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 Processor -- start 2024-09-11T12:52:33.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- start start 2024-09-11T12:52:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2f40 0x7fd12c142980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 0x7fd12c13afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c15e0e0 con 0x7fd12c145190 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd12c15df60 con 0x7fd12c15c3f0 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd12b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd12c15e260 con 0x7fd12c0b2f40 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd129400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd129400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:40268/0 (socket says 172.21.15.78:40268) 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.437+0000 7fd129400640 1 -- 172.21.15.78:0/3347120545 learned_addr learned my addr 172.21.15.78:0/3347120545 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129e00640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2f40 0x7fd12c142980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd12a800640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 0x7fd12c13afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 -- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2f40 msgr2=0x7fd12c142980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2f40 0x7fd12c142980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 -- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 msgr2=0x7fd12c13afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 0x7fd12c13afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 -- 172.21.15.78:0/3347120545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd12c06eb50 con 0x7fd12c145190 2024-09-11T12:52:33.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd12a800640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 0x7fd12c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:33.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd129400640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ac040 tx=0x7fd1180aa4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118005220 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1180054b0 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd12c06edd0 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd118004930 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd12c06f390 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12c15beb0 con 0x7fd12c145190 2024-09-11T12:52:33.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fd1180bf060 con 0x7fd12c145190 2024-09-11T12:52:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.441+0000 7fd122000640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 0x7fd10c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.445+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7fd11812b600 con 0x7fd12c145190 2024-09-11T12:52:33.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.445+0000 7fd129e00640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 0x7fd10c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:33.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.445+0000 7fd129e00640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 0x7fd10c073cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009f30 tx=0x7fd11c009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:33.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.445+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd1180164b0 con 0x7fd12c145190 2024-09-11T12:52:33.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.613+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fd12c0afc30 con 0x7fd12c145190 2024-09-11T12:52:33.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd122000640 1 -- 172.21.15.78:0/3347120545 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7fd1180f73e0 con 0x7fd12c145190 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 msgr2=0x7fd10c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 0x7fd10c073cf0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd11c009f30 tx=0x7fd11c009500 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 msgr2=0x7fd12c1f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd1180ac040 tx=0x7fd1180aa4b0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 shutdown_connections 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd10c071830 0x7fd10c073cf0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd12c15c3f0 0x7fd12c13afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd12c145190 0x7fd12c1f7c70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 --2- 172.21.15.78:0/3347120545 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd12c0b2f40 0x7fd12c142980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:33.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 >> 172.21.15.78:0/3347120545 conn(0x7fd12c075c10 msgr2=0x7fd12c1434c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 shutdown_connections 2024-09-11T12:52:33.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:33.617+0000 7fd12b200640 1 -- 172.21.15.78:0/3347120545 wait complete. 2024-09-11T12:52:33.639 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:52:33.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:52:33.731 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:52:33.820 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:52:33.820 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:52:33.820 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:33.829 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:52:33.830 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:52:33.830 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:33.839 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:52:33.839 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:52:33.839 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:33.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:52:33.885 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:52:33.885 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:33.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:52:33.895 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:52:33.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 -- 172.21.15.78:0/3279758016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd815df00 msgr2=0x7f8bd815c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 --2- 172.21.15.78:0/3279758016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd815df00 0x7f8bd815c1d0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8009840 tx=0x7f8bc801c790 comp rx=0 tx=0).stop 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 -- 172.21.15.78:0/3279758016 shutdown_connections 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 --2- 172.21.15.78:0/3279758016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd80c31d0 0x7f8bd80c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 --2- 172.21.15.78:0/3279758016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd815df00 0x7f8bd815c1d0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 -- 172.21.15.78:0/3279758016 >> 172.21.15.78:0/3279758016 conn(0x7f8bd806c650 msgr2=0x7f8bd806ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 -- 172.21.15.78:0/3279758016 shutdown_connections 2024-09-11T12:52:34.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 -- 172.21.15.78:0/3279758016 wait complete. 2024-09-11T12:52:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.073+0000 7f8bdde00640 1 Processor -- start 2024-09-11T12:52:34.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- start start 2024-09-11T12:52:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd80c65a0 0x7f8bd8153fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815df00 0x7f8bd8157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd81679e0 con 0x7f8bd813ed60 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd8167860 con 0x7f8bd80c65a0 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd8167b60 con 0x7f8bd815df00 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42708/0 (socket says 172.21.15.78:42708) 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 -- 172.21.15.78:0/2292101875 learned_addr learned my addr 172.21.15.78:0/2292101875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd6000640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815df00 0x7f8bd8157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd6a00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd80c65a0 0x7f8bd8153fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 -- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815df00 msgr2=0x7f8bd8157e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815df00 0x7f8bd8157e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 -- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd80c65a0 msgr2=0x7f8bd8153fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd80c65a0 0x7f8bd8153fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 -- 172.21.15.78:0/2292101875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd813f390 con 0x7f8bd813ed60 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bd7400640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000cbb0 tx=0x7f8bd000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0016020 con 0x7f8bd813ed60 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bd000aae0 con 0x7f8bd813ed60 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd813f670 con 0x7f8bd813ed60 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0012420 con 0x7f8bd813ed60 2024-09-11T12:52:34.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd813fc30 con 0x7f8bd813ed60 2024-09-11T12:52:34.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.077+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd80c6060 con 0x7f8bd813ed60 2024-09-11T12:52:34.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.081+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f8bd0019020 con 0x7f8bd813ed60 2024-09-11T12:52:34.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.081+0000 7f8bcf400640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 0x7f8bb8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.081+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7f8bd0092050 con 0x7f8bd813ed60 2024-09-11T12:52:34.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.081+0000 7f8bd6a00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 0x7f8bb8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.081+0000 7f8bd6a00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 0x7f8bb8073d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80af000 tx=0x7f8bc801afd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:34.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.085+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8bd0020020 con 0x7f8bd813ed60 2024-09-11T12:52:34.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.281+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bd813e7d0 con 0x7f8bd813ed60 2024-09-11T12:52:34.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bcf400640 1 -- 172.21.15.78:0/2292101875 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3917 (secure 0 0 0) 0x7f8bd005d850 con 0x7f8bd813ed60 2024-09-11T12:52:34.285 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:34.285 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":29,"btime":"2024-09-11T12:52:27:897741+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4946,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/1357206180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":1357206180},{"type":"v1","addr":"172.21.15.157:6837","nonce":1357206180}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5224,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2138637535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2138637535},{"type":"v1","addr":"172.21.15.157:6836","nonce":2138637535}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5268,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3359530796","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3359530796},{"type":"v1","addr":"172.21.15.78:6834","nonce":3359530796}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3702585791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3702585791},{"type":"v1","addr":"172.21.15.78:6835","nonce":3702585791}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-11T12:52:34.285 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 29 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 msgr2=0x7f8bb8073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 0x7f8bb8073d20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8bc80af000 tx=0x7f8bc801afd0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 msgr2=0x7f8bd8138030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000cbb0 tx=0x7f8bd000cca0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 shutdown_connections 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8bb8071860 0x7f8bb8073d20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8bd813ed60 0x7f8bd8138030 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8bd815df00 0x7f8bd8157e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 --2- 172.21.15.78:0/2292101875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8bd80c65a0 0x7f8bd8153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 >> 172.21.15.78:0/2292101875 conn(0x7f8bd806c650 msgr2=0x7f8bd813da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:34.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 shutdown_connections 2024-09-11T12:52:34.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.285+0000 7f8bdde00640 1 -- 172.21.15.78:0/2292101875 wait complete. 2024-09-11T12:52:34.305 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:52:34.305 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:52:34.306 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:52:34.306 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:52:34.306 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:52:34.306 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:52:34.306 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:52:34.306 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:52:34.306 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:52:34.307 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:52:34.307 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:52:34.307 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:52:34.307 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:52:34.307 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:52:34.307 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:52:34.307 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:52:34.321 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:52:34.321 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:52:34.322 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:52:34.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:52:34.323 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:52:34.323 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:52:34.324 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:52:34.324 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:52:34.367 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:52:34.368 DEBUG:teuthology.parallel:result is None 2024-09-11T12:52:34.369 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:52:34.369 DEBUG:teuthology.parallel:result is None 2024-09-11T12:52:34.370 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:52:34.371 DEBUG:teuthology.parallel:result is None 2024-09-11T12:52:34.372 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:52:34.372 DEBUG:teuthology.parallel:result is None 2024-09-11T12:52:34.373 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:52:34.457 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:52:34.458 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:52:34.459 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:52:34.459 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:52:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 -- 172.21.15.78:0/1544681319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0181518f0 msgr2=0x7ff018073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 --2- 172.21.15.78:0/1544681319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0181518f0 0x7ff018073c70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009900 tx=0x7ff00c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 -- 172.21.15.78:0/1544681319 shutdown_connections 2024-09-11T12:52:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 --2- 172.21.15.78:0/1544681319 >> v2:172.21.15.157:3300/0 conn(0x7ff0180cfeb0 0x7ff01815c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 --2- 172.21.15.78:0/1544681319 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0181518f0 0x7ff018073c70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 -- 172.21.15.78:0/1544681319 >> 172.21.15.78:0/1544681319 conn(0x7ff01806f3a0 msgr2=0x7ff0180cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 -- 172.21.15.78:0/1544681319 shutdown_connections 2024-09-11T12:52:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.521+0000 7ff01fc00640 1 -- 172.21.15.78:0/1544681319 wait complete. 2024-09-11T12:52:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 Processor -- start 2024-09-11T12:52:34.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- start start 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0180cfeb0 0x7ff018142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0181518f0 0x7ff01815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff018167740 con 0x7ff01815ae40 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff0181675c0 con 0x7ff0181518f0 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff0181678c0 con 0x7ff0180cfeb0 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42740/0 (socket says 172.21.15.78:42740) 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 -- 172.21.15.78:0/2572711120 learned_addr learned my addr 172.21.15.78:0/2572711120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff017e00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0181518f0 0x7ff01815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01d200640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0180cfeb0 0x7ff018142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 -- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0180cfeb0 msgr2=0x7ff018142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0180cfeb0 0x7ff018142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 -- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0181518f0 msgr2=0x7ff01815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0181518f0 0x7ff01815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 -- 172.21.15.78:0/2572711120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0181426b0 con 0x7ff01815ae40 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01dc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff01000ec80 tx=0x7ff01000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010003b80 con 0x7ff01815ae40 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff010003d20 con 0x7ff01815ae40 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0181477e0 con 0x7ff01815ae40 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff010004280 con 0x7ff01815ae40 2024-09-11T12:52:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0181f8dd0 con 0x7ff01815ae40 2024-09-11T12:52:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.525+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff01815c610 con 0x7ff01815ae40 2024-09-11T12:52:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.529+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7ff010004420 con 0x7ff01815ae40 2024-09-11T12:52:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.529+0000 7ff014c00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 0x7feff8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.529+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 4943+0+0 (secure 0 0 0) 0x7ff010092c10 con 0x7ff01815ae40 2024-09-11T12:52:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.529+0000 7ff01d200640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 0x7feff8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.529+0000 7ff01d200640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 0x7feff8073cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0af000 tx=0x7ff00c01b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:34.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.533+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff01005e9e0 con 0x7ff01815ae40 2024-09-11T12:52:34.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.709+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff01814fb00 con 0x7ff01815ae40 2024-09-11T12:52:34.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff014c00640 1 -- 172.21.15.78:0/2572711120 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v52) ==== 97+0+0 (secure 0 0 0) 0x7ff01005e2e0 con 0x7ff01815ae40 2024-09-11T12:52:34.937 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 msgr2=0x7feff8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 0x7feff8073cf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff00c0af000 tx=0x7ff00c01b120 comp rx=0 tx=0).stop 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 msgr2=0x7ff01815ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff01000ec80 tx=0x7ff01000c6a0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 shutdown_connections 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7feff8071830 0x7feff8073cf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff01815ae40 0x7ff01815ee70 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff0181518f0 0x7ff01815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 --2- 172.21.15.78:0/2572711120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0180cfeb0 0x7ff018142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 >> 172.21.15.78:0/2572711120 conn(0x7ff01806f3a0 msgr2=0x7ff018152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 shutdown_connections 2024-09-11T12:52:34.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:34.937+0000 7ff01fc00640 1 -- 172.21.15.78:0/2572711120 wait complete. 2024-09-11T12:52:34.961 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 -- 172.21.15.78:0/3049612502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40736d0 msgr2=0x7f07a415af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 --2- 172.21.15.78:0/3049612502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40736d0 0x7f07a415af90 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f0790009840 tx=0x7f079001c790 comp rx=0 tx=0).stop 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 -- 172.21.15.78:0/3049612502 shutdown_connections 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 --2- 172.21.15.78:0/3049612502 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40736d0 0x7f07a415af90 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 --2- 172.21.15.78:0/3049612502 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a4157ae0 0x7f07a4070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 -- 172.21.15.78:0/3049612502 >> 172.21.15.78:0/3049612502 conn(0x7f07a406ca80 msgr2=0x7f07a40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 -- 172.21.15.78:0/3049612502 shutdown_connections 2024-09-11T12:52:35.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.101+0000 7f07ab600640 1 -- 172.21.15.78:0/3049612502 wait complete. 2024-09-11T12:52:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 Processor -- start 2024-09-11T12:52:35.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- start start 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a40736d0 0x7f07a41f8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 0x7f07a415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f07a4166f10 con 0x7f07a40b4db0 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f07a40b5730 con 0x7f07a4157ae0 2024-09-11T12:52:35.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f07a4167090 con 0x7f07a40736d0 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42766/0 (socket says 172.21.15.78:42766) 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 -- 172.21.15.78:0/1213837012 learned_addr learned my addr 172.21.15.78:0/1213837012 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a8c00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a40736d0 0x7f07a41f8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a9600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 0x7f07a415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 -- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a40736d0 msgr2=0x7f07a41f8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a40736d0 0x7f07a41f8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 -- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 msgr2=0x7f07a415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 0x7f07a415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 -- 172.21.15.78:0/1213837012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a415a3b0 con 0x7f07a40b4db0 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a9600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 0x7f07a415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:35.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a3e00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0790009790 tx=0x7f07900a6900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07900ac070 con 0x7f07a40b4db0 2024-09-11T12:52:35.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07900a6de0 con 0x7f07a40b4db0 2024-09-11T12:52:35.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0790003840 con 0x7f07a40b4db0 2024-09-11T12:52:35.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a41fbaf0 con 0x7f07a40b4db0 2024-09-11T12:52:35.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.105+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a41fc0b0 con 0x7f07a40b4db0 2024-09-11T12:52:35.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a415af90 con 0x7f07a40b4db0 2024-09-11T12:52:35.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f07900af070 con 0x7f07a40b4db0 2024-09-11T12:52:35.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07a0c00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 0x7f0784073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f079012bea0 con 0x7f07a40b4db0 2024-09-11T12:52:35.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07a8c00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 0x7f0784073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.109+0000 7f07a8c00640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 0x7f0784073d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f07a415e800 tx=0x7f079400b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.113+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f07900b6070 con 0x7f07a40b4db0 2024-09-11T12:52:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f07a4072950 con 0x7f07a40b4db0 2024-09-11T12:52:35.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07a0c00640 1 -- 172.21.15.78:0/1213837012 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f0790016410 con 0x7f07a40b4db0 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:52:35.338 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:52:35.340 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:52:35.342 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:52:35.348 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:52:35.349 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.350 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:52:35.351 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:52:35.352 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.352 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.352 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:52:35.352 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 msgr2=0x7f0784073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 0x7f0784073d20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f07a415e800 tx=0x7f079400b3e0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 msgr2=0x7f07a41426d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0790009790 tx=0x7f07900a6900 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 shutdown_connections 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0784071860 0x7f0784073d20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07a4157ae0 0x7f07a415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07a40b4db0 0x7f07a41426d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 --2- 172.21.15.78:0/1213837012 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07a40736d0 0x7f07a41f8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 >> 172.21.15.78:0/1213837012 conn(0x7f07a406ca80 msgr2=0x7f07a40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 shutdown_connections 2024-09-11T12:52:35.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.337+0000 7f07ab600640 1 -- 172.21.15.78:0/1213837012 wait complete. 2024-09-11T12:52:35.359 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- 172.21.15.78:0/179262641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 msgr2=0x7fcfb00b51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/179262641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb00b51f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8009900 tx=0x7fcfa801c850 comp rx=0 tx=0).stop 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- 172.21.15.78:0/179262641 shutdown_connections 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/179262641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb00b51f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/179262641 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 0x7fcfb0070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- 172.21.15.78:0/179262641 >> 172.21.15.78:0/179262641 conn(0x7fcfb006ca80 msgr2=0x7fcfb00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- 172.21.15.78:0/179262641 shutdown_connections 2024-09-11T12:52:35.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- 172.21.15.78:0/179262641 wait complete. 2024-09-11T12:52:35.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 Processor -- start 2024-09-11T12:52:35.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- start start 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 0x7fcfb015d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 0x7fcfb015b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb006f2e0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcfb006f160 con 0x7fcfb00736d0 2024-09-11T12:52:35.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfb4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcfb006f460 con 0x7fcfb0157ae0 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfad400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfad400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42798/0 (socket says 172.21.15.78:42798) 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfad400640 1 -- 172.21.15.78:0/1835523546 learned_addr learned my addr 172.21.15.78:0/1835523546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfade00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 0x7fcfb015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.501+0000 7fcfae800640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 0x7fcfb015b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 -- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 msgr2=0x7fcfb015b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 0x7fcfb015b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 -- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 msgr2=0x7fcfb015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 0x7fcfb015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 -- 172.21.15.78:0/1835523546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfb01f72a0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfade00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 0x7fcfb015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:35.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfad400640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c3c0 tx=0x7fcf9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c0052d0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf9c007020 con 0x7fcfb00b4db0 2024-09-11T12:52:35.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfb015d6c0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf9c007470 con 0x7fcfb00b4db0 2024-09-11T12:52:35.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfb01fb050 con 0x7fcfb00b4db0 2024-09-11T12:52:35.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfb00b51f0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.505+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fcf9c0067e0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.509+0000 7fcfa6000640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 0x7fcf90073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.509+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fcf9c0913c0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.509+0000 7fcfade00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 0x7fcf90073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.509+0000 7fcfade00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 0x7fcf90073cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0005c50 tx=0x7fcfa0005be0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.509+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcfb01f72a0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.677+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fcfb0159fd0 con 0x7fcfb00b4db0 2024-09-11T12:52:35.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.677+0000 7fcfa6000640 1 -- 172.21.15.78:0/1835523546 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7fcf9c05d260 con 0x7fcfb00b4db0 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:35.680 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 msgr2=0x7fcf90073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 0x7fcf90073cf0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcfa0005c50 tx=0x7fcfa0005be0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 msgr2=0x7fcfb01479c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcf9c00c3c0 tx=0x7fcf9c00c880 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 shutdown_connections 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fcf90071830 0x7fcf90073cf0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcfb0157ae0 0x7fcfb015b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcfb00b4db0 0x7fcfb01479c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 --2- 172.21.15.78:0/1835523546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcfb00736d0 0x7fcfb015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 >> 172.21.15.78:0/1835523546 conn(0x7fcfb006ca80 msgr2=0x7fcfb0156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 shutdown_connections 2024-09-11T12:52:35.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.681+0000 7fcfb4a00640 1 -- 172.21.15.78:0/1835523546 wait complete. 2024-09-11T12:52:35.701 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:52:35.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:52:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 -- 172.21.15.78:0/3876488481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f815ecd0 msgr2=0x7f10f8073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 --2- 172.21.15.78:0/3876488481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f815ecd0 0x7f10f8073670 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f10f815be40 tx=0x7f10e401c810 comp rx=0 tx=0).stop 2024-09-11T12:52:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 -- 172.21.15.78:0/3876488481 shutdown_connections 2024-09-11T12:52:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 --2- 172.21.15.78:0/3876488481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f815ecd0 0x7f10f8073670 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 --2- 172.21.15.78:0/3876488481 >> v2:172.21.15.157:3300/0 conn(0x7f10f8073bb0 0x7f10f80caef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 --2- 172.21.15.78:0/3876488481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f81581e0 0x7f10f80c39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 -- 172.21.15.78:0/3876488481 >> 172.21.15.78:0/3876488481 conn(0x7f10f80cd190 msgr2=0x7f10f80cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 -- 172.21.15.78:0/3876488481 shutdown_connections 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.845+0000 7f10ffe00640 1 -- 172.21.15.78:0/3876488481 wait complete. 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 Processor -- start 2024-09-11T12:52:35.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- start start 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f81581e0 0x7f10f814ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f815ecd0 0x7f10f8151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10f81619f0 con 0x7f10f8073bb0 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10f8161870 con 0x7f10f81581e0 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10f8161b70 con 0x7f10f815ecd0 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42814/0 (socket says 172.21.15.78:42814) 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 -- 172.21.15.78:0/2919893571 learned_addr learned my addr 172.21.15.78:0/2919893571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fca00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f81581e0 0x7f10f814ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fde00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f815ecd0 0x7f10f8151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 -- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f815ecd0 msgr2=0x7f10f8151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f815ecd0 0x7f10f8151f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 -- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f81581e0 msgr2=0x7f10f814ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f81581e0 0x7f10f814ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:35.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 -- 172.21.15.78:0/2919893571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10f81511b0 con 0x7f10f8073bb0 2024-09-11T12:52:35.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10fd400640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00eb00 tx=0x7f10ec00c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec004b30 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10ec004de0 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10ec005280 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10f8151490 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10f8151a50 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.849+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10f80c39e0 con 0x7f10f8073bb0 2024-09-11T12:52:35.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.853+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f10ec005420 con 0x7f10f8073bb0 2024-09-11T12:52:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.853+0000 7f10f5600640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 0x7f10d8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.853+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f10ec0928b0 con 0x7f10f8073bb0 2024-09-11T12:52:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.853+0000 7f10fca00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 0x7f10d8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:35.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.853+0000 7f10fca00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 0x7f10d8073cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10e40af000 tx=0x7f10e401b050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:35.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:35.857+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f10ec016030 con 0x7f10f8073bb0 2024-09-11T12:52:36.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.029+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f10f8157700 con 0x7f10f8073bb0 2024-09-11T12:52:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.949+0000 7f10f5600640 1 -- 172.21.15.78:0/2919893571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f10ec05e6d0 con 0x7f10f8073bb0 2024-09-11T12:52:36.948 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:52:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.949+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 msgr2=0x7f10d8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.949+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 0x7f10d8073cf0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f10e40af000 tx=0x7f10e401b050 comp rx=0 tx=0).stop 2024-09-11T12:52:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.949+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 msgr2=0x7f10f814e440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:36.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.949+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f10ec00eb00 tx=0x7f10ec00c270 comp rx=0 tx=0).stop 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 shutdown_connections 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f10d8071830 0x7f10d8073cf0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10f815ecd0 0x7f10f8151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10f81581e0 0x7f10f814ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 --2- 172.21.15.78:0/2919893571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10f8073bb0 0x7f10f814e440 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 >> 172.21.15.78:0/2919893571 conn(0x7f10f80cd190 msgr2=0x7f10f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 shutdown_connections 2024-09-11T12:52:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:36.953+0000 7f10ffe00640 1 -- 172.21.15.78:0/2919893571 wait complete. 2024-09-11T12:52:36.978 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:52:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 -- 172.21.15.78:0/4222438649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 msgr2=0x7f20a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 --2- 172.21.15.78:0/4222438649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20a4004c90 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f20ac00bae0 tx=0x7f20ac01cd60 comp rx=0 tx=0).stop 2024-09-11T12:52:37.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 -- 172.21.15.78:0/4222438649 shutdown_connections 2024-09-11T12:52:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 --2- 172.21.15.78:0/4222438649 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20a4004c90 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 -- 172.21.15.78:0/4222438649 >> 172.21.15.78:0/4222438649 conn(0x7f20c00b2ec0 msgr2=0x7f20c00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 -- 172.21.15.78:0/4222438649 shutdown_connections 2024-09-11T12:52:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.129+0000 7f20c6400640 1 -- 172.21.15.78:0/4222438649 wait complete. 2024-09-11T12:52:37.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 Processor -- start 2024-09-11T12:52:37.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- start start 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c00ae2d0 0x7f20c01f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 0x7f20c00afaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0070630 con 0x7f20a4004810 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20c00704b0 con 0x7f20c00ae2d0 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00707b0 con 0x7f20c00af690 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42822/0 (socket says 172.21.15.78:42822) 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 -- 172.21.15.78:0/3777728578 learned_addr learned my addr 172.21.15.78:0/3777728578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:37.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bea00640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c00ae2d0 0x7f20c01f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bfe00640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 0x7f20c00afaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 -- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 msgr2=0x7f20c00afaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 0x7f20c00afaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 -- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c00ae2d0 msgr2=0x7f20c01f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c00ae2d0 0x7f20c01f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 -- 172.21.15.78:0/3777728578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c01f5060 con 0x7f20a4004810 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bfe00640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 0x7f20c00afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20bf400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f20b0012670 tx=0x7f20b0012b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0010b40 con 0x7f20a4004810 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b0017030 con 0x7f20a4004810 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c0156f20 con 0x7f20a4004810 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b0013450 con 0x7f20a4004810 2024-09-11T12:52:37.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c0143170 con 0x7f20a4004810 2024-09-11T12:52:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.133+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c00b0030 con 0x7f20a4004810 2024-09-11T12:52:37.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.137+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f20b00135f0 con 0x7f20a4004810 2024-09-11T12:52:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.137+0000 7f20b7400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 0x7f20a0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.137+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7f20b0096070 con 0x7f20a4004810 2024-09-11T12:52:37.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.137+0000 7f20bea00640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 0x7f20a0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:37.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.137+0000 7f20bea00640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 0x7f20a0073cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b0000 tx=0x7f20ac0a7040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:37.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.141+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f20b0061eb0 con 0x7f20a4004810 2024-09-11T12:52:37.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.309+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20c0152660 con 0x7f20a4004810 2024-09-11T12:52:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.953+0000 7f20b7400640 1 -- 172.21.15.78:0/3777728578 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7f20b00617b0 con 0x7f20a4004810 2024-09-11T12:52:37.954 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:52:37.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 msgr2=0x7f20a0073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:37.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 0x7f20a0073cf0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f20ac0b0000 tx=0x7f20ac0a7040 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 msgr2=0x7f20c01f4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f20b0012670 tx=0x7f20b0012b30 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 shutdown_connections 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f20a0071830 0x7f20a0073cf0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20c00af690 0x7f20c00afaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c00ae2d0 0x7f20c01f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 --2- 172.21.15.78:0/3777728578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004810 0x7f20c01f4b20 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 >> 172.21.15.78:0/3777728578 conn(0x7f20c00b2ec0 msgr2=0x7f20c015b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 shutdown_connections 2024-09-11T12:52:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:37.957+0000 7f20c6400640 1 -- 172.21.15.78:0/3777728578 wait complete. 2024-09-11T12:52:37.994 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:52:38.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- 172.21.15.78:0/653013207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc5d0 msgr2=0x7f19c415ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 --2- 172.21.15.78:0/653013207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc5d0 0x7f19c415ca30 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f19b0009900 tx=0x7f19b001c850 comp rx=0 tx=0).stop 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- 172.21.15.78:0/653013207 shutdown_connections 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 --2- 172.21.15.78:0/653013207 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc5d0 0x7f19c415ca30 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 --2- 172.21.15.78:0/653013207 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc1f0 0x7f19c40b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- 172.21.15.78:0/653013207 >> 172.21.15.78:0/653013207 conn(0x7f19c4071dd0 msgr2=0x7f19c40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- 172.21.15.78:0/653013207 shutdown_connections 2024-09-11T12:52:38.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- 172.21.15.78:0/653013207 wait complete. 2024-09-11T12:52:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 Processor -- start 2024-09-11T12:52:38.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.137+0000 7f19cbe00640 1 -- start start 2024-09-11T12:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc5d0 0x7f19c415d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 0x7f19c4151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4168ea0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19c4168d20 con 0x7f19c413f000 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f19c4169020 con 0x7f19c40cc5d0 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42836/0 (socket says 172.21.15.78:42836) 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 -- 172.21.15.78:0/2446445577 learned_addr learned my addr 172.21.15.78:0/2446445577 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c8a00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc5d0 0x7f19c415d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9e00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 0x7f19c4151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 -- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc5d0 msgr2=0x7f19c415d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc5d0 0x7f19c415d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 -- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 msgr2=0x7f19c4151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 0x7f19c4151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 -- 172.21.15.78:0/2446445577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19c4138a00 con 0x7f19c40cc1f0 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9e00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 0x7f19c4151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:38.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c9400640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f19b800eb30 tx=0x7f19b800c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b800ce50 con 0x7f19c40cc1f0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19b8011470 con 0x7f19c40cc1f0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19b80118a0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19c413a000 con 0x7f19c40cc1f0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19c4138fb0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.141+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19c40b2de0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.145+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f19b801a400 con 0x7f19c40cc1f0 2024-09-11T12:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.145+0000 7f19c1600640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 0x7f19a4073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.145+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7f19b80933c0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.145+0000 7f19c8a00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 0x7f19a4073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:38.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.145+0000 7f19c8a00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 0x7f19a4073cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19b00af000 tx=0x7f19b001b090 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:38.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.149+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f19b8028020 con 0x7f19c40cc1f0 2024-09-11T12:52:38.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.349+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f19c40ccdb0 con 0x7f19c40cc1f0 2024-09-11T12:52:38.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.957+0000 7f19c1600640 1 -- 172.21.15.78:0/2446445577 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f19b8016030 con 0x7f19c40cc1f0 2024-09-11T12:52:38.956 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 8 and data pool 9 2024-09-11T12:52:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.957+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 msgr2=0x7f19a4073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.957+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 0x7f19a4073cf0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f19b00af000 tx=0x7f19b001b090 comp rx=0 tx=0).stop 2024-09-11T12:52:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.957+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 msgr2=0x7f19c415b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:38.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.957+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f19b800eb30 tx=0x7f19b800c270 comp rx=0 tx=0).stop 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 shutdown_connections 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f19a4071830 0x7f19a4073cf0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19c413f000 0x7f19c4151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f19c40cc5d0 0x7f19c415d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 --2- 172.21.15.78:0/2446445577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f19c40cc1f0 0x7f19c415b250 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 >> 172.21.15.78:0/2446445577 conn(0x7f19c4071dd0 msgr2=0x7f19c413c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 shutdown_connections 2024-09-11T12:52:38.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:38.961+0000 7f19cbe00640 1 -- 172.21.15.78:0/2446445577 wait complete. 2024-09-11T12:52:39.002 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 -- 172.21.15.78:0/236614322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40b14b0 msgr2=0x7f1ac415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/236614322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40b14b0 0x7f1ac415c100 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1ab4009900 tx=0x7f1ab401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 -- 172.21.15.78:0/236614322 shutdown_connections 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/236614322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac40d0bf0 0x7f1ac40d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/236614322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40b14b0 0x7f1ac415c100 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 -- 172.21.15.78:0/236614322 >> 172.21.15.78:0/236614322 conn(0x7f1ac4074b90 msgr2=0x7f1ac40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 -- 172.21.15.78:0/236614322 shutdown_connections 2024-09-11T12:52:39.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 -- 172.21.15.78:0/236614322 wait complete. 2024-09-11T12:52:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.141+0000 7f1ac8e00640 1 Processor -- start 2024-09-11T12:52:39.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- start start 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac406fdf0 0x7f1ac4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b14b0 0x7f1ac4150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4168e50 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac4168cd0 con 0x7f1ac40b14b0 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac4168fd0 con 0x7f1ac406fdf0 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42854/0 (socket says 172.21.15.78:42854) 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 -- 172.21.15.78:0/2148036131 learned_addr learned my addr 172.21.15.78:0/2148036131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:39.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac1400640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b14b0 0x7f1ac4150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac1e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac406fdf0 0x7f1ac4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 -- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac406fdf0 msgr2=0x7f1ac4150a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac406fdf0 0x7f1ac4150a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 -- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b14b0 msgr2=0x7f1ac4150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b14b0 0x7f1ac4150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 -- 172.21.15.78:0/2148036131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac41f80a0 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac2800640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e970 tx=0x7f1abc00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc004d30 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1abc00cb20 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc005020 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac41f8380 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac41f7db0 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.145+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ac40ce250 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.149+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f1abc0051c0 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.149+0000 7f1aba000640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 0x7f1aa4073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.149+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f1abc092930 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.149+0000 7f1ac1e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 0x7f1aa4073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.149+0000 7f1ac1e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 0x7f1aa4073cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40af000 tx=0x7f1ab401b120 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.153+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1abc016030 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.317+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1ac414eca0 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1aba000640 1 -- 172.21.15.78:0/2148036131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16738 (secure 0 0 0) 0x7f1abc05de60 con 0x7f1ac40d0bf0 2024-09-11T12:52:39.320 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:39.321 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":57,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:38.352072+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:52:36.041400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-11T12:52:37.321031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:39.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 msgr2=0x7f1aa4073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 0x7f1aa4073cf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40af000 tx=0x7f1ab401b120 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 msgr2=0x7f1ac4149030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00e970 tx=0x7f1abc00ee30 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 shutdown_connections 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1aa4071830 0x7f1aa4073cf0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ac40d0bf0 0x7f1ac4149030 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ac40b14b0 0x7f1ac4150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 --2- 172.21.15.78:0/2148036131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ac406fdf0 0x7f1ac4150a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 >> 172.21.15.78:0/2148036131 conn(0x7f1ac4074b90 msgr2=0x7f1ac41527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 shutdown_connections 2024-09-11T12:52:39.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.321+0000 7f1ac8e00640 1 -- 172.21.15.78:0/2148036131 wait complete. 2024-09-11T12:52:39.342 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- 172.21.15.78:0/3454801409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8073c70 msgr2=0x7f1cb815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3454801409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8073c70 0x7f1cb815ae40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009bc0 tx=0x7f1cb001cba0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- 172.21.15.78:0/3454801409 shutdown_connections 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3454801409 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb8073c70 0x7f1cb815ae40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- 172.21.15.78:0/3454801409 >> 172.21.15.78:0/3454801409 conn(0x7f1cb806f3a0 msgr2=0x7f1cb80cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- 172.21.15.78:0/3454801409 shutdown_connections 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- 172.21.15.78:0/3454801409 wait complete. 2024-09-11T12:52:39.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 Processor -- start 2024-09-11T12:52:39.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- start start 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cb8073c70 0x7f1cb8153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb81518f0 0x7f1cb815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb8168960 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb81687e0 con 0x7f1cb81518f0 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cbe200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb8168ae0 con 0x7f1cb8073c70 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.485+0000 7f1cb6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42870/0 (socket says 172.21.15.78:42870) 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 -- 172.21.15.78:0/3698948660 learned_addr learned my addr 172.21.15.78:0/3698948660 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb7e00640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb81518f0 0x7f1cb815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb7400640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cb8073c70 0x7f1cb8153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 -- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cb8073c70 msgr2=0x7f1cb8153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cb8073c70 0x7f1cb8153fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 -- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb81518f0 msgr2=0x7f1cb815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb81518f0 0x7f1cb815e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 -- 172.21.15.78:0/3698948660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb8149d90 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cb6a00640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c4b0 tx=0x7f1ca400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca40052d0 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ca4005470 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca4012870 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb814a070 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb814a630 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb815ae40 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.489+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f1ca4003600 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.493+0000 7f1caf400640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 0x7f1c98073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.493+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f1ca4091bc0 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.493+0000 7f1cb7400640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 0x7f1c98073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.493+0000 7f1cb7400640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 0x7f1c98073d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80046c0 tx=0x7f1ca8007cd0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.493+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1ca405d0c0 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.661+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1cb8165880 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1caf400640 1 -- 172.21.15.78:0/3698948660 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16738 (secure 0 0 0) 0x7f1ca4061610 con 0x7f1cb80cfeb0 2024-09-11T12:52:39.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:39.665 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":57,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:38.352072+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:52:36.041400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-11T12:52:37.321031+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 msgr2=0x7f1c98073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 0x7f1c98073d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80046c0 tx=0x7f1ca8007cd0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 msgr2=0x7f1cb8154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400c4b0 tx=0x7f1ca400c970 comp rx=0 tx=0).stop 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 shutdown_connections 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1c98071860 0x7f1c98073d20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb81518f0 0x7f1cb815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80cfeb0 0x7f1cb8154b60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 --2- 172.21.15.78:0/3698948660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1cb8073c70 0x7f1cb8153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 >> 172.21.15.78:0/3698948660 conn(0x7f1cb806f3a0 msgr2=0x7f1cb8150830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:39.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 shutdown_connections 2024-09-11T12:52:39.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.665+0000 7f1cbe200640 1 -- 172.21.15.78:0/3698948660 wait complete. 2024-09-11T12:52:39.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:52:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- 172.21.15.78:0/3719390717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40b3040 msgr2=0x7f7ba40cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 --2- 172.21.15.78:0/3719390717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40b3040 0x7f7ba40cf6e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4158240 tx=0x7f7b9401cd50 comp rx=0 tx=0).stop 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- 172.21.15.78:0/3719390717 shutdown_connections 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 --2- 172.21.15.78:0/3719390717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40cd770 0x7f7ba40cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 --2- 172.21.15.78:0/3719390717 >> v2:172.21.15.157:3300/0 conn(0x7f7ba40b3420 0x7f7ba415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 --2- 172.21.15.78:0/3719390717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40b3040 0x7f7ba40cf6e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- 172.21.15.78:0/3719390717 >> 172.21.15.78:0/3719390717 conn(0x7f7ba4071dd0 msgr2=0x7f7ba40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- 172.21.15.78:0/3719390717 shutdown_connections 2024-09-11T12:52:39.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- 172.21.15.78:0/3719390717 wait complete. 2024-09-11T12:52:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 Processor -- start 2024-09-11T12:52:39.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.829+0000 7f7baa800640 1 -- start start 2024-09-11T12:52:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba40b3040 0x7f7ba413cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40b3420 0x7f7ba40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40aedc0 con 0x7f7ba40cd770 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ba40aec40 con 0x7f7ba40b3040 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ba40aef40 con 0x7f7ba40b3420 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42890/0 (socket says 172.21.15.78:42890) 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 -- 172.21.15.78:0/3814665625 learned_addr learned my addr 172.21.15.78:0/3814665625 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3400640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba40b3040 0x7f7ba413cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba2a00640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40b3420 0x7f7ba40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 -- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40b3420 msgr2=0x7f7ba40ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40b3420 0x7f7ba40ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 -- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba40b3040 msgr2=0x7f7ba413cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba40b3040 0x7f7ba413cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 -- 172.21.15.78:0/3814665625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ba4151810 con 0x7f7ba40cd770 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7ba3e00640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00d5e0 tx=0x7f7b9c00daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c017070 con 0x7f7ba40cd770 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b9c005310 con 0x7f7ba40cd770 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0047b0 con 0x7f7ba40cd770 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ba4152120 con 0x7f7ba40cd770 2024-09-11T12:52:39.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ba4151db0 con 0x7f7ba40cd770 2024-09-11T12:52:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ba415a700 con 0x7f7ba40cd770 2024-09-11T12:52:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.833+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f7b9c004950 con 0x7f7ba40cd770 2024-09-11T12:52:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.837+0000 7f7b9b400640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 0x7f7b80073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:39.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.837+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f7b9c091f50 con 0x7f7ba40cd770 2024-09-11T12:52:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.837+0000 7f7ba3400640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 0x7f7b80073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:39.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.837+0000 7f7ba3400640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 0x7f7b80073d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b9401b590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:39.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:39.841+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7b9c05d480 con 0x7f7ba40cd770 2024-09-11T12:52:40.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.045+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7ba4157c90 con 0x7f7ba40cd770 2024-09-11T12:52:40.945 INFO:tasks.ceph.mds.c.smithi078.stderr:2024-09-11T12:52:40.945+0000 7fb5b1600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:52:40.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.981+0000 7f7b9b400640 1 -- 172.21.15.78:0/3814665625 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7f7b9c0619d0 con 0x7f7ba40cd770 2024-09-11T12:52:40.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.981+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 msgr2=0x7f7b80073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:40.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.981+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 0x7f7b80073d20 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b940af000 tx=0x7f7b9401b590 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 msgr2=0x7f7ba40addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00d5e0 tx=0x7f7b9c00daa0 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 shutdown_connections 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7b80071860 0x7f7b80073d20 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ba40cd770 0x7f7ba40addb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ba40b3420 0x7f7ba40ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 --2- 172.21.15.78:0/3814665625 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ba40b3040 0x7f7ba413cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 >> 172.21.15.78:0/3814665625 conn(0x7f7ba4071dd0 msgr2=0x7f7ba40721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 shutdown_connections 2024-09-11T12:52:40.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:40.985+0000 7f7baa800640 1 -- 172.21.15.78:0/3814665625 wait complete. 2024-09-11T12:52:41.005 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:52:41.005 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 -- 172.21.15.78:0/1969673520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 msgr2=0x7f00540cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 --2- 172.21.15.78:0/1969673520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f00540cdc00 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f005406d1a0 tx=0x7f004c01dbb0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 -- 172.21.15.78:0/1969673520 shutdown_connections 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 --2- 172.21.15.78:0/1969673520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f00540cdc00 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 --2- 172.21.15.78:0/1969673520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 0x7f005415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 --2- 172.21.15.78:0/1969673520 >> v2:172.21.15.157:3300/0 conn(0x7f00540b3040 0x7f00540cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 -- 172.21.15.78:0/1969673520 >> 172.21.15.78:0/1969673520 conn(0x7f0054071dd0 msgr2=0x7f00540721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 -- 172.21.15.78:0/1969673520 shutdown_connections 2024-09-11T12:52:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.149+0000 7f005a000640 1 -- 172.21.15.78:0/1969673520 wait complete. 2024-09-11T12:52:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 Processor -- start 2024-09-11T12:52:41.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- start start 2024-09-11T12:52:41.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b3040 0x7f0054153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:41.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 0x7f00541f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:41.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00540aee60 con 0x7f00540cd770 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00540aece0 con 0x7f00540b3040 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f00540aefe0 con 0x7f00540b3420 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b3040 0x7f0054153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42906/0 (socket says 172.21.15.78:42906) 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 -- 172.21.15.78:0/2453654695 learned_addr learned my addr 172.21.15.78:0/2453654695 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0052a00640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 0x7f00541f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 -- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 msgr2=0x7f00541f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 0x7f00541f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 -- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b3040 msgr2=0x7f0054153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b3040 0x7f0054153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 -- 172.21.15.78:0/2453654695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005413e9b0 con 0x7f00540cd770 2024-09-11T12:52:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f0053e00640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f004c01c260 tx=0x7f004c00a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:41.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c0bb020 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f004c004d40 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005413ec90 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f004c0b3040 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.153+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005413f250 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.157+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005415a700 con 0x7f00540cd770 2024-09-11T12:52:41.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.157+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f004c004200 con 0x7f00540cd770 2024-09-11T12:52:41.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.157+0000 7f004b400640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:41.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.157+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f004c12c2f0 con 0x7f00540cd770 2024-09-11T12:52:41.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.157+0000 7f0053400640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:41.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.161+0000 7f0053400640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0054143b90 tx=0x7f004400b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:41.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.161+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f004c0c5060 con 0x7f00540cd770 2024-09-11T12:52:41.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.361+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0054076290 con 0x7f00540cd770 2024-09-11T12:52:41.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.985+0000 7f004b400640 1 -- 172.21.15.78:0/2453654695 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f004c017860 con 0x7f00540cd770 2024-09-11T12:52:41.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 msgr2=0x7f0034073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:41.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0054143b90 tx=0x7f004400b2e0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 msgr2=0x7f005406fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f004c01c260 tx=0x7f004c00a3d0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 shutdown_connections 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00540cd770 0x7f005406fce0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00540b3420 0x7f00541f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 --2- 172.21.15.78:0/2453654695 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00540b3040 0x7f0054153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 >> 172.21.15.78:0/2453654695 conn(0x7f0054071dd0 msgr2=0x7f00540721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 shutdown_connections 2024-09-11T12:52:41.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:41.989+0000 7f005a000640 1 -- 172.21.15.78:0/2453654695 wait complete. 2024-09-11T12:52:41.994 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:52:41.998 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:52:42.013 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:52:42.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- 172.21.15.78:0/2416681703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf280b54c0 msgr2=0x7fdf280736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/2416681703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf280b54c0 0x7fdf280736d0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009900 tx=0x7fdf2001c850 comp rx=0 tx=0).stop 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- 172.21.15.78:0/2416681703 shutdown_connections 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/2416681703 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf280b54c0 0x7fdf280736d0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- 172.21.15.78:0/2416681703 >> 172.21.15.78:0/2416681703 conn(0x7fdf2806ca80 msgr2=0x7fdf280b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:42.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- 172.21.15.78:0/2416681703 shutdown_connections 2024-09-11T12:52:42.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- 172.21.15.78:0/2416681703 wait complete. 2024-09-11T12:52:42.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 Processor -- start 2024-09-11T12:52:42.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.225+0000 7fdf2cc00640 1 -- start start 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf280b54c0 0x7fdf280af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28157ae0 0x7fdf281f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf280b48b0 con 0x7fdf28070a00 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf280b4730 con 0x7fdf28157ae0 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf280b4a30 con 0x7fdf280b54c0 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42928/0 (socket says 172.21.15.78:42928) 2024-09-11T12:52:42.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 -- 172.21.15.78:0/1082496615 learned_addr learned my addr 172.21.15.78:0/1082496615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf26800640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28157ae0 0x7fdf281f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25400640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf280b54c0 0x7fdf280af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 -- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf280b54c0 msgr2=0x7fdf280af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf280b54c0 0x7fdf280af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 -- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28157ae0 msgr2=0x7fdf281f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28157ae0 0x7fdf281f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 -- 172.21.15.78:0/1082496615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf280adcc0 con 0x7fdf28070a00 2024-09-11T12:52:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf25e00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c750 tx=0x7fdf1800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18017070 con 0x7fdf28070a00 2024-09-11T12:52:42.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf18005070 con 0x7fdf28070a00 2024-09-11T12:52:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800ce60 con 0x7fdf28070a00 2024-09-11T12:52:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf280adfa0 con 0x7fdf28070a00 2024-09-11T12:52:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf280ae560 con 0x7fdf28070a00 2024-09-11T12:52:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf280736d0 con 0x7fdf28070a00 2024-09-11T12:52:42.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.229+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fdf1801a070 con 0x7fdf28070a00 2024-09-11T12:52:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.233+0000 7fdf1e000640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 0x7fdf08073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.233+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fdf180916f0 con 0x7fdf28070a00 2024-09-11T12:52:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.233+0000 7fdf25400640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 0x7fdf08073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.233+0000 7fdf25400640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 0x7fdf08073cf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdf280cf660 tx=0x7fdf14007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.237+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdf1805cca0 con 0x7fdf28070a00 2024-09-11T12:52:42.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.437+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf28159530 con 0x7fdf28070a00 2024-09-11T12:52:42.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.437+0000 7fdf1e000640 1 -- 172.21.15.78:0/1082496615 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5229 (secure 0 0 0) 0x7fdf180611f0 con 0x7fdf28070a00 2024-09-11T12:52:42.439 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:42.440 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":36,"btime":"2024-09-11T12:52:41:982815+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:52:42.440 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 36 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 msgr2=0x7fdf08073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 0x7fdf08073cf0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fdf280cf660 tx=0x7fdf14007e40 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 msgr2=0x7fdf2814b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800c750 tx=0x7fdf1800cc10 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 shutdown_connections 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdf08071830 0x7fdf08073cf0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf28157ae0 0x7fdf281f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf280b54c0 0x7fdf280af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 --2- 172.21.15.78:0/1082496615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf28070a00 0x7fdf2814b210 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 >> 172.21.15.78:0/1082496615 conn(0x7fdf2806ca80 msgr2=0x7fdf28158680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:42.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 shutdown_connections 2024-09-11T12:52:42.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.441+0000 7fdf2cc00640 1 -- 172.21.15.78:0/1082496615 wait complete. 2024-09-11T12:52:42.461 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:52:42.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- 172.21.15.78:0/1271347237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200cc730 msgr2=0x7fea200b0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 --2- 172.21.15.78:0/1271347237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200cc730 0x7fea200b0240 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fea0c009900 tx=0x7fea0c01c5e0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- 172.21.15.78:0/1271347237 shutdown_connections 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 --2- 172.21.15.78:0/1271347237 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea200cc730 0x7fea200b0240 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- 172.21.15.78:0/1271347237 >> 172.21.15.78:0/1271347237 conn(0x7fea200cd010 msgr2=0x7fea200cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- 172.21.15.78:0/1271347237 shutdown_connections 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- 172.21.15.78:0/1271347237 wait complete. 2024-09-11T12:52:42.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 Processor -- start 2024-09-11T12:52:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.601+0000 7fea26800640 1 -- start start 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200b0780 0x7fea20164570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200cc730 0x7fea201574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea20168ed0 con 0x7fea20140ff0 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea20168d50 con 0x7fea200b0780 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea20169050 con 0x7fea200cc730 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42952/0 (socket says 172.21.15.78:42952) 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 -- 172.21.15.78:0/218008975 learned_addr learned my addr 172.21.15.78:0/218008975 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1f400640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200b0780 0x7fea20164570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1ea00640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200cc730 0x7fea201574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 -- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200cc730 msgr2=0x7fea201574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200cc730 0x7fea201574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 -- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200b0780 msgr2=0x7fea20164570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200b0780 0x7fea20164570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 -- 172.21.15.78:0/218008975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea201358e0 con 0x7fea20140ff0 2024-09-11T12:52:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea1fe00640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c730 tx=0x7fea1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18003db0 con 0x7fea20140ff0 2024-09-11T12:52:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea1800ce40 con 0x7fea20140ff0 2024-09-11T12:52:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea180052d0 con 0x7fea20140ff0 2024-09-11T12:52:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea20164ab0 con 0x7fea20140ff0 2024-09-11T12:52:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea20165030 con 0x7fea20140ff0 2024-09-11T12:52:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.605+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea200b0240 con 0x7fea20140ff0 2024-09-11T12:52:42.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.609+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fea18005470 con 0x7fea20140ff0 2024-09-11T12:52:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.609+0000 7fea17400640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 0x7fea00073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.609+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fea18091eb0 con 0x7fea20140ff0 2024-09-11T12:52:42.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.609+0000 7fea1f400640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 0x7fea00073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.609+0000 7fea1f400640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 0x7fea00073cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fea10007fd0 tx=0x7fea10009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.613+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fea201358e0 con 0x7fea20140ff0 2024-09-11T12:52:42.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea200afb70 con 0x7fea20140ff0 2024-09-11T12:52:42.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea17400640 1 -- 172.21.15.78:0/218008975 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16725 (secure 0 0 0) 0x7fea1805d440 con 0x7fea20140ff0 2024-09-11T12:52:42.781 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:42.782 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":58,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:39.967859+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:52:36.041400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-11T12:52:37.321031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 msgr2=0x7fea00073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 0x7fea00073cf0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fea10007fd0 tx=0x7fea10009040 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 msgr2=0x7fea200738d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c730 tx=0x7fea1800cbf0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 shutdown_connections 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fea00071830 0x7fea00073cf0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea20140ff0 0x7fea200738d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea200cc730 0x7fea201574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 --2- 172.21.15.78:0/218008975 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea200b0780 0x7fea20164570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 >> 172.21.15.78:0/218008975 conn(0x7fea200cd010 msgr2=0x7fea20166030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:42.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 shutdown_connections 2024-09-11T12:52:42.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.781+0000 7fea26800640 1 -- 172.21.15.78:0/218008975 wait complete. 2024-09-11T12:52:42.805 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:52:42.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 -- 172.21.15.78:0/834564569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 msgr2=0x7f6ae806ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 --2- 172.21.15.78:0/834564569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae806ecf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009cb0 tx=0x7f6ae001cda0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 -- 172.21.15.78:0/834564569 shutdown_connections 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 --2- 172.21.15.78:0/834564569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae806ecf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 --2- 172.21.15.78:0/834564569 >> v2:172.21.15.157:3300/0 conn(0x7f6ae815bc60 0x7f6ae8073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 -- 172.21.15.78:0/834564569 >> 172.21.15.78:0/834564569 conn(0x7f6ae806ca80 msgr2=0x7f6ae80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 -- 172.21.15.78:0/834564569 shutdown_connections 2024-09-11T12:52:42.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 -- 172.21.15.78:0/834564569 wait complete. 2024-09-11T12:52:42.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.945+0000 7f6aefe00640 1 Processor -- start 2024-09-11T12:52:42.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- start start 2024-09-11T12:52:42.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae80b4780 0x7f6ae81f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae8146c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8168be0 con 0x7f6ae815bc60 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ae8168a60 con 0x7f6ae80b4780 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ae8168d60 con 0x7f6ae80ccb20 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42964/0 (socket says 172.21.15.78:42964) 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 -- 172.21.15.78:0/1160915484 learned_addr learned my addr 172.21.15.78:0/1160915484 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aeca00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae8146c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aed400640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae80b4780 0x7f6ae81f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 -- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 msgr2=0x7f6ae8146c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae8146c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 -- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae80b4780 msgr2=0x7f6ae81f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae80b4780 0x7f6ae81f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:42.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 -- 172.21.15.78:0/1160915484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ae8144750 con 0x7f6ae815bc60 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aede00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009bd0 tx=0x7f6ae00092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00bc070 con 0x7f6ae815bc60 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ae00042f0 con 0x7f6ae815bc60 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae00b2040 con 0x7f6ae815bc60 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ae8144a30 con 0x7f6ae815bc60 2024-09-11T12:52:42.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ae8144ff0 con 0x7f6ae815bc60 2024-09-11T12:52:42.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.949+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ae806ecf0 con 0x7f6ae815bc60 2024-09-11T12:52:42.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.953+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f6ae0004700 con 0x7f6ae815bc60 2024-09-11T12:52:42.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.953+0000 7f6ae5600640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 0x7f6ac8073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:42.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.953+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6ae012bcf0 con 0x7f6ae815bc60 2024-09-11T12:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.953+0000 7f6aed400640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 0x7f6ac8073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.953+0000 7f6aed400640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 0x7f6ac8073d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6adc007fd0 tx=0x7f6adc009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:42.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:42.957+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6ae0016990 con 0x7f6ae815bc60 2024-09-11T12:52:43.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.121+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6ae8159650 con 0x7f6ac8071860 2024-09-11T12:52:43.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.125+0000 7f6ae5600640 1 -- 172.21.15.78:0/1160915484 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+239413 (secure 0 0 0) 0x7f6ae8159650 con 0x7f6ac8071860 2024-09-11T12:52:43.128 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:43.130 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 msgr2=0x7f6ac8073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 0x7f6ac8073d20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f6adc007fd0 tx=0x7f6adc009040 comp rx=0 tx=0).stop 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 msgr2=0x7f6ae8145e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f6ae0009bd0 tx=0x7f6ae00092c0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 shutdown_connections 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ac8071860 0x7f6ac8073d20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ae815bc60 0x7f6ae8145e70 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ae80ccb20 0x7f6ae8146c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 --2- 172.21.15.78:0/1160915484 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ae80b4780 0x7f6ae81f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 >> 172.21.15.78:0/1160915484 conn(0x7f6ae806ca80 msgr2=0x7f6ae815e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 shutdown_connections 2024-09-11T12:52:43.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.129+0000 7f6aefe00640 1 -- 172.21.15.78:0/1160915484 wait complete. 2024-09-11T12:52:43.148 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-11T12:52:41.147794+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":54,"acting":54,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":247372,"kb_used_data":6140,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727348,"statfs":{"total":773094113280,"available":772840804352,"internally_reserved":0,"allocated":6287360,"data_stored":3237369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.197309"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957967+0000","last_change":"2024-09-11T12:52:36.957967+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957967+0000","last_undegraded":"2024-09-11T12:52:36.957967+0000","last_fullsized":"2024-09-11T12:52:36.957967+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953850+0000","last_change":"2024-09-11T12:52:36.953850+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953850+0000","last_undegraded":"2024-09-11T12:52:36.953850+0000","last_fullsized":"2024-09-11T12:52:36.953850+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957291+0000","last_change":"2024-09-11T12:52:36.957291+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957291+0000","last_undegraded":"2024-09-11T12:52:36.957291+0000","last_fullsized":"2024-09-11T12:52:36.957291+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.954238+0000","last_change":"2024-09-11T12:52:36.954238+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.954238+0000","last_undegraded":"2024-09-11T12:52:36.954238+0000","last_fullsized":"2024-09-11T12:52:36.954238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.956351+0000","last_change":"2024-09-11T12:52:36.956351+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.956351+0000","last_undegraded":"2024-09-11T12:52:36.956351+0000","last_fullsized":"2024-09-11T12:52:36.956351+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957822+0000","last_change":"2024-09-11T12:52:36.957822+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957822+0000","last_undegraded":"2024-09-11T12:52:36.957822+0000","last_fullsized":"2024-09-11T12:52:36.957822+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957407+0000","last_change":"2024-09-11T12:52:36.957407+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957407+0000","last_undegraded":"2024-09-11T12:52:36.957407+0000","last_fullsized":"2024-09-11T12:52:36.957407+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957111+0000","last_change":"2024-09-11T12:52:36.957111+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957111+0000","last_undegraded":"2024-09-11T12:52:36.957111+0000","last_fullsized":"2024-09-11T12:52:36.957111+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.956396+0000","last_change":"2024-09-11T12:52:36.956396+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.956396+0000","last_undegraded":"2024-09-11T12:52:36.956396+0000","last_fullsized":"2024-09-11T12:52:36.956396+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.958924+0000","last_change":"2024-09-11T12:52:36.958924+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.958924+0000","last_undegraded":"2024-09-11T12:52:36.958924+0000","last_fullsized":"2024-09-11T12:52:36.958924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.952992+0000","last_change":"2024-09-11T12:52:36.952992+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.952992+0000","last_undegraded":"2024-09-11T12:52:36.952992+0000","last_fullsized":"2024-09-11T12:52:36.952992+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.952870+0000","last_change":"2024-09-11T12:52:36.952870+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.952870+0000","last_undegraded":"2024-09-11T12:52:36.952870+0000","last_fullsized":"2024-09-11T12:52:36.952870+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.956450+0000","last_change":"2024-09-11T12:52:36.956450+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.956450+0000","last_undegraded":"2024-09-11T12:52:36.956450+0000","last_fullsized":"2024-09-11T12:52:36.956450+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957419+0000","last_change":"2024-09-11T12:52:36.957419+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957419+0000","last_undegraded":"2024-09-11T12:52:36.957419+0000","last_fullsized":"2024-09-11T12:52:36.957419+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953314+0000","last_change":"2024-09-11T12:52:36.953314+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953314+0000","last_undegraded":"2024-09-11T12:52:36.953314+0000","last_fullsized":"2024-09-11T12:52:36.953314+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.958471+0000","last_change":"2024-09-11T12:52:36.958471+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.958471+0000","last_undegraded":"2024-09-11T12:52:36.958471+0000","last_fullsized":"2024-09-11T12:52:36.958471+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953378+0000","last_change":"2024-09-11T12:52:36.953378+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953378+0000","last_undegraded":"2024-09-11T12:52:36.953378+0000","last_fullsized":"2024-09-11T12:52:36.953378+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953610+0000","last_change":"2024-09-11T12:52:36.953610+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953610+0000","last_undegraded":"2024-09-11T12:52:36.953610+0000","last_fullsized":"2024-09-11T12:52:36.953610+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953516+0000","last_change":"2024-09-11T12:52:36.953516+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953516+0000","last_undegraded":"2024-09-11T12:52:36.953516+0000","last_fullsized":"2024-09-11T12:52:36.953516+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:36.947461+0000","last_change":"2024-09-11T12:52:36.947461+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.947461+0000","last_undegraded":"2024-09-11T12:52:36.947461+0000","last_fullsized":"2024-09-11T12:52:36.947461+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953127+0000","last_change":"2024-09-11T12:52:36.953127+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953127+0000","last_undegraded":"2024-09-11T12:52:36.953127+0000","last_fullsized":"2024-09-11T12:52:36.953127+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953224+0000","last_change":"2024-09-11T12:52:36.953224+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953224+0000","last_undegraded":"2024-09-11T12:52:36.953224+0000","last_fullsized":"2024-09-11T12:52:36.953224+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953361+0000","last_change":"2024-09-11T12:52:36.953361+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953361+0000","last_undegraded":"2024-09-11T12:52:36.953361+0000","last_fullsized":"2024-09-11T12:52:36.953361+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.956793+0000","last_change":"2024-09-11T12:52:36.956793+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.956793+0000","last_undegraded":"2024-09-11T12:52:36.956793+0000","last_fullsized":"2024-09-11T12:52:36.956793+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.953311+0000","last_change":"2024-09-11T12:52:36.953311+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.953311+0000","last_undegraded":"2024-09-11T12:52:36.953311+0000","last_fullsized":"2024-09-11T12:52:36.953311+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.957325+0000","last_change":"2024-09-11T12:52:36.957325+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.957325+0000","last_undegraded":"2024-09-11T12:52:36.957325+0000","last_fullsized":"2024-09-11T12:52:36.957325+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-11T12:52:37.949954+0000","last_change":"2024-09-11T12:52:37.949954+0000","last_active":"2024-09-11T12:52:37.949954+0000","last_peered":"2024-09-11T12:52:37.949954+0000","last_clean":"2024-09-11T12:52:37.949954+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:37.949954+0000","last_undegraded":"2024-09-11T12:52:37.949954+0000","last_fullsized":"2024-09-11T12:52:37.949954+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-09-11T12:52:36.952846+0000","last_change":"2024-09-11T12:52:36.952846+0000","last_active":"2024-09-11T12:52:36.947461+0000","last_peered":"2024-09-11T12:52:36.947461+0000","last_clean":"2024-09-11T12:52:36.947461+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:52:36.952846+0000","last_undegraded":"2024-09-11T12:52:36.952846+0000","last_fullsized":"2024-09-11T12:52:36.952846+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":170,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T12:52:36.958863+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:52:36.958863+0000","last_peered":"2024-09-11T12:52:36.958863+0000","last_clean":"2024-09-11T12:52:36.958863+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:52:36.958863+0000","last_undegraded":"2024-09-11T12:52:36.958863+0000","last_fullsized":"2024-09-11T12:52:36.958863+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":52,"acting":52,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1245184,"data_stored":853715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":257530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":651264,"data_stored":263347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31820,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340020,"statfs":{"total":96636764160,"available":96604180480,"internally_reserved":0,"allocated":638976,"data_stored":257530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":257530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27656,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":638976,"data_stored":257530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31816,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340024,"statfs":{"total":96636764160,"available":96604184576,"internally_reserved":0,"allocated":638976,"data_stored":257530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542184,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32356,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339484,"statfs":{"total":96636764160,"available":96603631616,"internally_reserved":0,"allocated":1196032,"data_stored":832657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:52:43.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:52:43.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- 172.21.15.78:0/311541279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 msgr2=0x7f35f80cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- 172.21.15.78:0/311541279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f80cdd80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f35e4009840 tx=0x7f35e401c820 comp rx=0 tx=0).stop 2024-09-11T12:52:43.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- 172.21.15.78:0/311541279 shutdown_connections 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- 172.21.15.78:0/311541279 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f80b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- 172.21.15.78:0/311541279 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f80cdd80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- 172.21.15.78:0/311541279 >> 172.21.15.78:0/311541279 conn(0x7f35f806ca80 msgr2=0x7f35f80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- 172.21.15.78:0/311541279 shutdown_connections 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- 172.21.15.78:0/311541279 wait complete. 2024-09-11T12:52:43.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 Processor -- start 2024-09-11T12:52:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- start start 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f81f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806f720 con 0x7f35f8159fc0 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f35f806f5a0 con 0x7f35f80cd910 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f3600200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f35f806f8a0 con 0x7f35f80b4db0 2024-09-11T12:52:43.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f35fe200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f35fe200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42974/0 (socket says 172.21.15.78:42974) 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.333+0000 7f35fe200640 1 -- 172.21.15.78:0/2190254260 learned_addr learned my addr 172.21.15.78:0/2190254260 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fce00640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fd800640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f81f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 -- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 msgr2=0x7f35f81f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f81f8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 -- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 msgr2=0x7f35f815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 -- 172.21.15.78:0/2190254260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f81f9090 con 0x7f35f8159fc0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fd800640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f81f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35fe200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f35f000e9b0 tx=0x7f35f000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000cd80 con 0x7f35f8159fc0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35f0004020 con 0x7f35f8159fc0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0004450 con 0x7f35f8159fc0 2024-09-11T12:52:43.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f81f9370 con 0x7f35f8159fc0 2024-09-11T12:52:43.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f815d080 con 0x7f35f8159fc0 2024-09-11T12:52:43.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35f81f5330 con 0x7f35f8159fc0 2024-09-11T12:52:43.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.337+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f35f0003600 con 0x7f35f8159fc0 2024-09-11T12:52:43.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.341+0000 7f35f5600640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 0x7f35d8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.341+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f35f0092b50 con 0x7f35f8159fc0 2024-09-11T12:52:43.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.341+0000 7f35fd800640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 0x7f35d8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.341+0000 7f35fd800640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 0x7f35d8073cf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35f81478e0 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:43.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.341+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f35f005e590 con 0x7f35f8159fc0 2024-09-11T12:52:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.505+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f35f80b5240 con 0x7f35d8071830 2024-09-11T12:52:43.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.513+0000 7f35f5600640 1 -- 172.21.15.78:0/2190254260 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f35f80b5240 con 0x7f35d8071830 2024-09-11T12:52:43.516 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:43.517 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 msgr2=0x7f35d8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 0x7f35d8073cf0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f35f81478e0 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 msgr2=0x7f35f8145590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f35f000e9b0 tx=0x7f35f000ee70 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 shutdown_connections 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f35d8071830 0x7f35d8073cf0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f35f8159fc0 0x7f35f8145590 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f35f80cd910 0x7f35f815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 --2- 172.21.15.78:0/2190254260 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f35f80b4db0 0x7f35f81f8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 >> 172.21.15.78:0/2190254260 conn(0x7f35f806ca80 msgr2=0x7f35f814fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 shutdown_connections 2024-09-11T12:52:43.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.517+0000 7f3600200640 1 -- 172.21.15.78:0/2190254260 wait complete. 2024-09-11T12:52:43.536 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-09-11T12:52:43.148188+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":188,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247796,"kb_used_data":6588,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754726924,"statfs":{"total":773094113280,"available":772840370176,"internally_reserved":0,"allocated":6746112,"data_stored":3440768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001192"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.158057+0000","last_change":"2024-09-11T12:52:38.980523+0000","last_active":"2024-09-11T12:52:41.158057+0000","last_peered":"2024-09-11T12:52:41.158057+0000","last_clean":"2024-09-11T12:52:41.158057+0000","last_became_active":"2024-09-11T12:52:38.980235+0000","last_became_peered":"2024-09-11T12:52:38.980235+0000","last_unstale":"2024-09-11T12:52:41.158057+0000","last_undegraded":"2024-09-11T12:52:41.158057+0000","last_fullsized":"2024-09-11T12:52:41.158057+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:54:39.021143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868594+0000","last_change":"2024-09-11T12:52:38.987153+0000","last_active":"2024-09-11T12:52:40.868594+0000","last_peered":"2024-09-11T12:52:40.868594+0000","last_clean":"2024-09-11T12:52:40.868594+0000","last_became_active":"2024-09-11T12:52:38.986946+0000","last_became_peered":"2024-09-11T12:52:38.986946+0000","last_unstale":"2024-09-11T12:52:40.868594+0000","last_undegraded":"2024-09-11T12:52:40.868594+0000","last_fullsized":"2024-09-11T12:52:40.868594+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:23.201860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157604+0000","last_change":"2024-09-11T12:52:38.981246+0000","last_active":"2024-09-11T12:52:41.157604+0000","last_peered":"2024-09-11T12:52:41.157604+0000","last_clean":"2024-09-11T12:52:41.157604+0000","last_became_active":"2024-09-11T12:52:38.981001+0000","last_became_peered":"2024-09-11T12:52:38.981001+0000","last_unstale":"2024-09-11T12:52:41.157604+0000","last_undegraded":"2024-09-11T12:52:41.157604+0000","last_fullsized":"2024-09-11T12:52:41.157604+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:09.916822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978993+0000","last_change":"2024-09-11T12:52:38.980965+0000","last_active":"2024-09-11T12:52:39.978993+0000","last_peered":"2024-09-11T12:52:39.978993+0000","last_clean":"2024-09-11T12:52:39.978993+0000","last_became_active":"2024-09-11T12:52:38.980689+0000","last_became_peered":"2024-09-11T12:52:38.980689+0000","last_unstale":"2024-09-11T12:52:39.978993+0000","last_undegraded":"2024-09-11T12:52:39.978993+0000","last_fullsized":"2024-09-11T12:52:39.978993+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:55:42.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157582+0000","last_change":"2024-09-11T12:52:38.983463+0000","last_active":"2024-09-11T12:52:41.157582+0000","last_peered":"2024-09-11T12:52:41.157582+0000","last_clean":"2024-09-11T12:52:41.157582+0000","last_became_active":"2024-09-11T12:52:38.983170+0000","last_became_peered":"2024-09-11T12:52:38.983170+0000","last_unstale":"2024-09-11T12:52:41.157582+0000","last_undegraded":"2024-09-11T12:52:41.157582+0000","last_fullsized":"2024-09-11T12:52:41.157582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:37.062557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.968792+0000","last_change":"2024-09-11T12:52:38.982456+0000","last_active":"2024-09-11T12:52:40.968792+0000","last_peered":"2024-09-11T12:52:40.968792+0000","last_clean":"2024-09-11T12:52:40.968792+0000","last_became_active":"2024-09-11T12:52:38.982180+0000","last_became_peered":"2024-09-11T12:52:38.982180+0000","last_unstale":"2024-09-11T12:52:40.968792+0000","last_undegraded":"2024-09-11T12:52:40.968792+0000","last_fullsized":"2024-09-11T12:52:40.968792+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:00:47.905087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977764+0000","last_change":"2024-09-11T12:52:38.967491+0000","last_active":"2024-09-11T12:52:39.977764+0000","last_peered":"2024-09-11T12:52:39.977764+0000","last_clean":"2024-09-11T12:52:39.977764+0000","last_became_active":"2024-09-11T12:52:38.967206+0000","last_became_peered":"2024-09-11T12:52:38.967206+0000","last_unstale":"2024-09-11T12:52:39.977764+0000","last_undegraded":"2024-09-11T12:52:39.977764+0000","last_fullsized":"2024-09-11T12:52:39.977764+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:00.584308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868237+0000","last_change":"2024-09-11T12:52:38.980084+0000","last_active":"2024-09-11T12:52:40.868237+0000","last_peered":"2024-09-11T12:52:40.868237+0000","last_clean":"2024-09-11T12:52:40.868237+0000","last_became_active":"2024-09-11T12:52:38.979803+0000","last_became_peered":"2024-09-11T12:52:38.979803+0000","last_unstale":"2024-09-11T12:52:40.868237+0000","last_undegraded":"2024-09-11T12:52:40.868237+0000","last_fullsized":"2024-09-11T12:52:40.868237+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:50:58.531376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979148+0000","last_change":"2024-09-11T12:52:38.967088+0000","last_active":"2024-09-11T12:52:39.979148+0000","last_peered":"2024-09-11T12:52:39.979148+0000","last_clean":"2024-09-11T12:52:39.979148+0000","last_became_active":"2024-09-11T12:52:38.966799+0000","last_became_peered":"2024-09-11T12:52:38.966799+0000","last_unstale":"2024-09-11T12:52:39.979148+0000","last_undegraded":"2024-09-11T12:52:39.979148+0000","last_fullsized":"2024-09-11T12:52:39.979148+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:27.349506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981632+0000","last_change":"2024-09-11T12:52:38.983550+0000","last_active":"2024-09-11T12:52:39.981632+0000","last_peered":"2024-09-11T12:52:39.981632+0000","last_clean":"2024-09-11T12:52:39.981632+0000","last_became_active":"2024-09-11T12:52:38.983204+0000","last_became_peered":"2024-09-11T12:52:38.983204+0000","last_unstale":"2024-09-11T12:52:39.981632+0000","last_undegraded":"2024-09-11T12:52:39.981632+0000","last_fullsized":"2024-09-11T12:52:39.981632+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:49.804276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981441+0000","last_change":"2024-09-11T12:52:38.993183+0000","last_active":"2024-09-11T12:52:39.981441+0000","last_peered":"2024-09-11T12:52:39.981441+0000","last_clean":"2024-09-11T12:52:39.981441+0000","last_became_active":"2024-09-11T12:52:38.991987+0000","last_became_peered":"2024-09-11T12:52:38.991987+0000","last_unstale":"2024-09-11T12:52:39.981441+0000","last_undegraded":"2024-09-11T12:52:39.981441+0000","last_fullsized":"2024-09-11T12:52:39.981441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:18.695320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979404+0000","last_change":"2024-09-11T12:52:38.975362+0000","last_active":"2024-09-11T12:52:39.979404+0000","last_peered":"2024-09-11T12:52:39.979404+0000","last_clean":"2024-09-11T12:52:39.979404+0000","last_became_active":"2024-09-11T12:52:38.975092+0000","last_became_peered":"2024-09-11T12:52:38.975092+0000","last_unstale":"2024-09-11T12:52:39.979404+0000","last_undegraded":"2024-09-11T12:52:39.979404+0000","last_fullsized":"2024-09-11T12:52:39.979404+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:47.608597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971498+0000","last_change":"2024-09-11T12:52:38.986355+0000","last_active":"2024-09-11T12:52:40.971498+0000","last_peered":"2024-09-11T12:52:40.971498+0000","last_clean":"2024-09-11T12:52:40.971498+0000","last_became_active":"2024-09-11T12:52:38.986002+0000","last_became_peered":"2024-09-11T12:52:38.986002+0000","last_unstale":"2024-09-11T12:52:40.971498+0000","last_undegraded":"2024-09-11T12:52:40.971498+0000","last_fullsized":"2024-09-11T12:52:40.971498+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:54.998653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977844+0000","last_change":"2024-09-11T12:52:38.982383+0000","last_active":"2024-09-11T12:52:39.977844+0000","last_peered":"2024-09-11T12:52:39.977844+0000","last_clean":"2024-09-11T12:52:39.977844+0000","last_became_active":"2024-09-11T12:52:38.981770+0000","last_became_peered":"2024-09-11T12:52:38.981770+0000","last_unstale":"2024-09-11T12:52:39.977844+0000","last_undegraded":"2024-09-11T12:52:39.977844+0000","last_fullsized":"2024-09-11T12:52:39.977844+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:47.200428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977969+0000","last_change":"2024-09-11T12:52:38.984073+0000","last_active":"2024-09-11T12:52:39.977969+0000","last_peered":"2024-09-11T12:52:39.977969+0000","last_clean":"2024-09-11T12:52:39.977969+0000","last_became_active":"2024-09-11T12:52:38.983785+0000","last_became_peered":"2024-09-11T12:52:38.983785+0000","last_unstale":"2024-09-11T12:52:39.977969+0000","last_undegraded":"2024-09-11T12:52:39.977969+0000","last_fullsized":"2024-09-11T12:52:39.977969+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:17.849748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969163+0000","last_change":"2024-09-11T12:52:38.976052+0000","last_active":"2024-09-11T12:52:40.969163+0000","last_peered":"2024-09-11T12:52:40.969163+0000","last_clean":"2024-09-11T12:52:40.969163+0000","last_became_active":"2024-09-11T12:52:38.975127+0000","last_became_peered":"2024-09-11T12:52:38.975127+0000","last_unstale":"2024-09-11T12:52:40.969163+0000","last_undegraded":"2024-09-11T12:52:40.969163+0000","last_fullsized":"2024-09-11T12:52:40.969163+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:04:54.862368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868601+0000","last_change":"2024-09-11T12:52:38.978141+0000","last_active":"2024-09-11T12:52:40.868601+0000","last_peered":"2024-09-11T12:52:40.868601+0000","last_clean":"2024-09-11T12:52:40.868601+0000","last_became_active":"2024-09-11T12:52:38.977830+0000","last_became_peered":"2024-09-11T12:52:38.977830+0000","last_unstale":"2024-09-11T12:52:40.868601+0000","last_undegraded":"2024-09-11T12:52:40.868601+0000","last_fullsized":"2024-09-11T12:52:40.868601+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:55:24.358787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157680+0000","last_change":"2024-09-11T12:52:38.986754+0000","last_active":"2024-09-11T12:52:41.157680+0000","last_peered":"2024-09-11T12:52:41.157680+0000","last_clean":"2024-09-11T12:52:41.157680+0000","last_became_active":"2024-09-11T12:52:38.985131+0000","last_became_peered":"2024-09-11T12:52:38.985131+0000","last_unstale":"2024-09-11T12:52:41.157680+0000","last_undegraded":"2024-09-11T12:52:41.157680+0000","last_fullsized":"2024-09-11T12:52:41.157680+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:08.282045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157974+0000","last_change":"2024-09-11T12:52:38.979263+0000","last_active":"2024-09-11T12:52:41.157974+0000","last_peered":"2024-09-11T12:52:41.157974+0000","last_clean":"2024-09-11T12:52:41.157974+0000","last_became_active":"2024-09-11T12:52:38.978983+0000","last_became_peered":"2024-09-11T12:52:38.978983+0000","last_unstale":"2024-09-11T12:52:41.157974+0000","last_undegraded":"2024-09-11T12:52:41.157974+0000","last_fullsized":"2024-09-11T12:52:41.157974+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:27:12.103977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868420+0000","last_change":"2024-09-11T12:52:38.982801+0000","last_active":"2024-09-11T12:52:40.868420+0000","last_peered":"2024-09-11T12:52:40.868420+0000","last_clean":"2024-09-11T12:52:40.868420+0000","last_became_active":"2024-09-11T12:52:38.982331+0000","last_became_peered":"2024-09-11T12:52:38.982331+0000","last_unstale":"2024-09-11T12:52:40.868420+0000","last_undegraded":"2024-09-11T12:52:40.868420+0000","last_fullsized":"2024-09-11T12:52:40.868420+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:19:58.262630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157775+0000","last_change":"2024-09-11T12:52:38.987687+0000","last_active":"2024-09-11T12:52:41.157775+0000","last_peered":"2024-09-11T12:52:41.157775+0000","last_clean":"2024-09-11T12:52:41.157775+0000","last_became_active":"2024-09-11T12:52:38.987390+0000","last_became_peered":"2024-09-11T12:52:38.987390+0000","last_unstale":"2024-09-11T12:52:41.157775+0000","last_undegraded":"2024-09-11T12:52:41.157775+0000","last_fullsized":"2024-09-11T12:52:41.157775+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:47.608597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868030+0000","last_change":"2024-09-11T12:52:38.981061+0000","last_active":"2024-09-11T12:52:40.868030+0000","last_peered":"2024-09-11T12:52:40.868030+0000","last_clean":"2024-09-11T12:52:40.868030+0000","last_became_active":"2024-09-11T12:52:38.980613+0000","last_became_peered":"2024-09-11T12:52:38.980613+0000","last_unstale":"2024-09-11T12:52:40.868030+0000","last_undegraded":"2024-09-11T12:52:40.868030+0000","last_fullsized":"2024-09-11T12:52:40.868030+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:09:09.131825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157900+0000","last_change":"2024-09-11T12:52:38.995132+0000","last_active":"2024-09-11T12:52:41.157900+0000","last_peered":"2024-09-11T12:52:41.157900+0000","last_clean":"2024-09-11T12:52:41.157900+0000","last_became_active":"2024-09-11T12:52:38.994485+0000","last_became_peered":"2024-09-11T12:52:38.994485+0000","last_unstale":"2024-09-11T12:52:41.157900+0000","last_undegraded":"2024-09-11T12:52:41.157900+0000","last_fullsized":"2024-09-11T12:52:41.157900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:34:58.928059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971401+0000","last_change":"2024-09-11T12:52:38.985330+0000","last_active":"2024-09-11T12:52:40.971401+0000","last_peered":"2024-09-11T12:52:40.971401+0000","last_clean":"2024-09-11T12:52:40.971401+0000","last_became_active":"2024-09-11T12:52:38.985026+0000","last_became_peered":"2024-09-11T12:52:38.985026+0000","last_unstale":"2024-09-11T12:52:40.971401+0000","last_undegraded":"2024-09-11T12:52:40.971401+0000","last_fullsized":"2024-09-11T12:52:40.971401+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:51.988321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.158195+0000","last_change":"2024-09-11T12:52:38.995902+0000","last_active":"2024-09-11T12:52:41.158195+0000","last_peered":"2024-09-11T12:52:41.158195+0000","last_clean":"2024-09-11T12:52:41.158195+0000","last_became_active":"2024-09-11T12:52:38.995652+0000","last_became_peered":"2024-09-11T12:52:38.995652+0000","last_unstale":"2024-09-11T12:52:41.158195+0000","last_undegraded":"2024-09-11T12:52:41.158195+0000","last_fullsized":"2024-09-11T12:52:41.158195+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:05.701828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979152+0000","last_change":"2024-09-11T12:52:38.969443+0000","last_active":"2024-09-11T12:52:39.979152+0000","last_peered":"2024-09-11T12:52:39.979152+0000","last_clean":"2024-09-11T12:52:39.979152+0000","last_became_active":"2024-09-11T12:52:38.969114+0000","last_became_peered":"2024-09-11T12:52:38.969114+0000","last_unstale":"2024-09-11T12:52:39.979152+0000","last_undegraded":"2024-09-11T12:52:39.979152+0000","last_fullsized":"2024-09-11T12:52:39.979152+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:25.644513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.968853+0000","last_change":"2024-09-11T12:52:38.978771+0000","last_active":"2024-09-11T12:52:40.968853+0000","last_peered":"2024-09-11T12:52:40.968853+0000","last_clean":"2024-09-11T12:52:40.968853+0000","last_became_active":"2024-09-11T12:52:38.977947+0000","last_became_peered":"2024-09-11T12:52:38.977947+0000","last_unstale":"2024-09-11T12:52:40.968853+0000","last_undegraded":"2024-09-11T12:52:40.968853+0000","last_fullsized":"2024-09-11T12:52:40.968853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:23:43.682020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157964+0000","last_change":"2024-09-11T12:52:38.981641+0000","last_active":"2024-09-11T12:52:41.157964+0000","last_peered":"2024-09-11T12:52:41.157964+0000","last_clean":"2024-09-11T12:52:41.157964+0000","last_became_active":"2024-09-11T12:52:38.981286+0000","last_became_peered":"2024-09-11T12:52:38.981286+0000","last_unstale":"2024-09-11T12:52:41.157964+0000","last_undegraded":"2024-09-11T12:52:41.157964+0000","last_fullsized":"2024-09-11T12:52:41.157964+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:08:25.871245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868461+0000","last_change":"2024-09-11T12:52:38.988580+0000","last_active":"2024-09-11T12:52:40.868461+0000","last_peered":"2024-09-11T12:52:40.868461+0000","last_clean":"2024-09-11T12:52:40.868461+0000","last_became_active":"2024-09-11T12:52:38.984725+0000","last_became_peered":"2024-09-11T12:52:38.984725+0000","last_unstale":"2024-09-11T12:52:40.868461+0000","last_undegraded":"2024-09-11T12:52:40.868461+0000","last_fullsized":"2024-09-11T12:52:40.868461+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:59.515689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157343+0000","last_change":"2024-09-11T12:52:38.980036+0000","last_active":"2024-09-11T12:52:41.157343+0000","last_peered":"2024-09-11T12:52:41.157343+0000","last_clean":"2024-09-11T12:52:41.157343+0000","last_became_active":"2024-09-11T12:52:38.979788+0000","last_became_peered":"2024-09-11T12:52:38.979788+0000","last_unstale":"2024-09-11T12:52:41.157343+0000","last_undegraded":"2024-09-11T12:52:41.157343+0000","last_fullsized":"2024-09-11T12:52:41.157343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:20.144969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157246+0000","last_change":"2024-09-11T12:52:38.983551+0000","last_active":"2024-09-11T12:52:41.157246+0000","last_peered":"2024-09-11T12:52:41.157246+0000","last_clean":"2024-09-11T12:52:41.157246+0000","last_became_active":"2024-09-11T12:52:38.983292+0000","last_became_peered":"2024-09-11T12:52:38.983292+0000","last_unstale":"2024-09-11T12:52:41.157246+0000","last_undegraded":"2024-09-11T12:52:41.157246+0000","last_fullsized":"2024-09-11T12:52:41.157246+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:35:36.023709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969081+0000","last_change":"2024-09-11T12:52:38.980337+0000","last_active":"2024-09-11T12:52:40.969081+0000","last_peered":"2024-09-11T12:52:40.969081+0000","last_clean":"2024-09-11T12:52:40.969081+0000","last_became_active":"2024-09-11T12:52:38.980072+0000","last_became_peered":"2024-09-11T12:52:38.980072+0000","last_unstale":"2024-09-11T12:52:40.969081+0000","last_undegraded":"2024-09-11T12:52:40.969081+0000","last_fullsized":"2024-09-11T12:52:40.969081+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:50.225619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969315+0000","last_change":"2024-09-11T12:52:38.976575+0000","last_active":"2024-09-11T12:52:40.969315+0000","last_peered":"2024-09-11T12:52:40.969315+0000","last_clean":"2024-09-11T12:52:40.969315+0000","last_became_active":"2024-09-11T12:52:38.976280+0000","last_became_peered":"2024-09-11T12:52:38.976280+0000","last_unstale":"2024-09-11T12:52:40.969315+0000","last_undegraded":"2024-09-11T12:52:40.969315+0000","last_fullsized":"2024-09-11T12:52:40.969315+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:52:31.310250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":19,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969178+0000","last_change":"2024-09-11T12:52:37.981036+0000","last_active":"2024-09-11T12:52:40.969178+0000","last_peered":"2024-09-11T12:52:40.969178+0000","last_clean":"2024-09-11T12:52:40.969178+0000","last_became_active":"2024-09-11T12:52:37.978319+0000","last_became_peered":"2024-09-11T12:52:37.978319+0000","last_unstale":"2024-09-11T12:52:40.969178+0000","last_undegraded":"2024-09-11T12:52:40.969178+0000","last_fullsized":"2024-09-11T12:52:40.969178+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:37.925679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981754+0000","last_change":"2024-09-11T12:52:38.988677+0000","last_active":"2024-09-11T12:52:39.981754+0000","last_peered":"2024-09-11T12:52:39.981754+0000","last_clean":"2024-09-11T12:52:39.981754+0000","last_became_active":"2024-09-11T12:52:38.988413+0000","last_became_peered":"2024-09-11T12:52:38.988413+0000","last_unstale":"2024-09-11T12:52:39.981754+0000","last_undegraded":"2024-09-11T12:52:39.981754+0000","last_fullsized":"2024-09-11T12:52:39.981754+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:06:17.662410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868141+0000","last_change":"2024-09-11T12:52:37.971060+0000","last_active":"2024-09-11T12:52:40.868141+0000","last_peered":"2024-09-11T12:52:40.868141+0000","last_clean":"2024-09-11T12:52:40.868141+0000","last_became_active":"2024-09-11T12:52:37.970857+0000","last_became_peered":"2024-09-11T12:52:37.970857+0000","last_unstale":"2024-09-11T12:52:40.868141+0000","last_undegraded":"2024-09-11T12:52:40.868141+0000","last_fullsized":"2024-09-11T12:52:40.868141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:01.790559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969015+0000","last_change":"2024-09-11T12:52:38.984612+0000","last_active":"2024-09-11T12:52:40.969015+0000","last_peered":"2024-09-11T12:52:40.969015+0000","last_clean":"2024-09-11T12:52:40.969015+0000","last_became_active":"2024-09-11T12:52:38.984374+0000","last_became_peered":"2024-09-11T12:52:38.984374+0000","last_unstale":"2024-09-11T12:52:40.969015+0000","last_undegraded":"2024-09-11T12:52:40.969015+0000","last_fullsized":"2024-09-11T12:52:40.969015+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:07:23.695922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969020+0000","last_change":"2024-09-11T12:52:37.979390+0000","last_active":"2024-09-11T12:52:40.969020+0000","last_peered":"2024-09-11T12:52:40.969020+0000","last_clean":"2024-09-11T12:52:40.969020+0000","last_became_active":"2024-09-11T12:52:37.979148+0000","last_became_peered":"2024-09-11T12:52:37.979148+0000","last_unstale":"2024-09-11T12:52:40.969020+0000","last_undegraded":"2024-09-11T12:52:40.969020+0000","last_fullsized":"2024-09-11T12:52:40.969020+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:12:28.746546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.968843+0000","last_change":"2024-09-11T12:52:38.973921+0000","last_active":"2024-09-11T12:52:40.968843+0000","last_peered":"2024-09-11T12:52:40.968843+0000","last_clean":"2024-09-11T12:52:40.968843+0000","last_became_active":"2024-09-11T12:52:38.973655+0000","last_became_peered":"2024-09-11T12:52:38.973655+0000","last_unstale":"2024-09-11T12:52:40.968843+0000","last_undegraded":"2024-09-11T12:52:40.968843+0000","last_fullsized":"2024-09-11T12:52:40.968843+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:29:47.363606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979264+0000","last_change":"2024-09-11T12:52:37.973040+0000","last_active":"2024-09-11T12:52:39.979264+0000","last_peered":"2024-09-11T12:52:39.979264+0000","last_clean":"2024-09-11T12:52:39.979264+0000","last_became_active":"2024-09-11T12:52:37.972777+0000","last_became_peered":"2024-09-11T12:52:37.972777+0000","last_unstale":"2024-09-11T12:52:39.979264+0000","last_undegraded":"2024-09-11T12:52:39.979264+0000","last_fullsized":"2024-09-11T12:52:39.979264+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:13.646594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868005+0000","last_change":"2024-09-11T12:52:38.980854+0000","last_active":"2024-09-11T12:52:40.868005+0000","last_peered":"2024-09-11T12:52:40.868005+0000","last_clean":"2024-09-11T12:52:40.868005+0000","last_became_active":"2024-09-11T12:52:38.980335+0000","last_became_peered":"2024-09-11T12:52:38.980335+0000","last_unstale":"2024-09-11T12:52:40.868005+0000","last_undegraded":"2024-09-11T12:52:40.868005+0000","last_fullsized":"2024-09-11T12:52:40.868005+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:32.640008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977884+0000","last_change":"2024-09-11T12:52:37.979950+0000","last_active":"2024-09-11T12:52:39.977884+0000","last_peered":"2024-09-11T12:52:39.977884+0000","last_clean":"2024-09-11T12:52:39.977884+0000","last_became_active":"2024-09-11T12:52:37.978943+0000","last_became_peered":"2024-09-11T12:52:37.978943+0000","last_unstale":"2024-09-11T12:52:39.977884+0000","last_undegraded":"2024-09-11T12:52:39.977884+0000","last_fullsized":"2024-09-11T12:52:39.977884+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:41:59.982733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868075+0000","last_change":"2024-09-11T12:52:38.984966+0000","last_active":"2024-09-11T12:52:40.868075+0000","last_peered":"2024-09-11T12:52:40.868075+0000","last_clean":"2024-09-11T12:52:40.868075+0000","last_became_active":"2024-09-11T12:52:38.984379+0000","last_became_peered":"2024-09-11T12:52:38.984379+0000","last_unstale":"2024-09-11T12:52:40.868075+0000","last_undegraded":"2024-09-11T12:52:40.868075+0000","last_fullsized":"2024-09-11T12:52:40.868075+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:16.725816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.968733+0000","last_change":"2024-09-11T12:52:37.977251+0000","last_active":"2024-09-11T12:52:40.968733+0000","last_peered":"2024-09-11T12:52:40.968733+0000","last_clean":"2024-09-11T12:52:40.968733+0000","last_became_active":"2024-09-11T12:52:37.976951+0000","last_became_peered":"2024-09-11T12:52:37.976951+0000","last_unstale":"2024-09-11T12:52:40.968733+0000","last_undegraded":"2024-09-11T12:52:40.968733+0000","last_fullsized":"2024-09-11T12:52:40.968733+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:52:46.195771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971528+0000","last_change":"2024-09-11T12:52:38.974872+0000","last_active":"2024-09-11T12:52:40.971528+0000","last_peered":"2024-09-11T12:52:40.971528+0000","last_clean":"2024-09-11T12:52:40.971528+0000","last_became_active":"2024-09-11T12:52:38.974587+0000","last_became_peered":"2024-09-11T12:52:38.974587+0000","last_unstale":"2024-09-11T12:52:40.971528+0000","last_undegraded":"2024-09-11T12:52:40.971528+0000","last_fullsized":"2024-09-11T12:52:40.971528+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:40.126548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978567+0000","last_change":"2024-09-11T12:52:37.976700+0000","last_active":"2024-09-11T12:52:39.978567+0000","last_peered":"2024-09-11T12:52:39.978567+0000","last_clean":"2024-09-11T12:52:39.978567+0000","last_became_active":"2024-09-11T12:52:37.976437+0000","last_became_peered":"2024-09-11T12:52:37.976437+0000","last_unstale":"2024-09-11T12:52:39.978567+0000","last_undegraded":"2024-09-11T12:52:39.978567+0000","last_fullsized":"2024-09-11T12:52:39.978567+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:16.847254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868266+0000","last_change":"2024-09-11T12:52:38.978260+0000","last_active":"2024-09-11T12:52:40.868266+0000","last_peered":"2024-09-11T12:52:40.868266+0000","last_clean":"2024-09-11T12:52:40.868266+0000","last_became_active":"2024-09-11T12:52:38.977597+0000","last_became_peered":"2024-09-11T12:52:38.977597+0000","last_unstale":"2024-09-11T12:52:40.868266+0000","last_undegraded":"2024-09-11T12:52:40.868266+0000","last_fullsized":"2024-09-11T12:52:40.868266+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:01.618164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977716+0000","last_change":"2024-09-11T12:52:37.977806+0000","last_active":"2024-09-11T12:52:39.977716+0000","last_peered":"2024-09-11T12:52:39.977716+0000","last_clean":"2024-09-11T12:52:39.977716+0000","last_became_active":"2024-09-11T12:52:37.977379+0000","last_became_peered":"2024-09-11T12:52:37.977379+0000","last_unstale":"2024-09-11T12:52:39.977716+0000","last_undegraded":"2024-09-11T12:52:39.977716+0000","last_fullsized":"2024-09-11T12:52:39.977716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:39.124054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979037+0000","last_change":"2024-09-11T12:52:38.988206+0000","last_active":"2024-09-11T12:52:39.979037+0000","last_peered":"2024-09-11T12:52:39.979037+0000","last_clean":"2024-09-11T12:52:39.979037+0000","last_became_active":"2024-09-11T12:52:38.987902+0000","last_became_peered":"2024-09-11T12:52:38.987902+0000","last_unstale":"2024-09-11T12:52:39.979037+0000","last_undegraded":"2024-09-11T12:52:39.979037+0000","last_fullsized":"2024-09-11T12:52:39.979037+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:59:29.682954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157951+0000","last_change":"2024-09-11T12:52:37.974216+0000","last_active":"2024-09-11T12:52:41.157951+0000","last_peered":"2024-09-11T12:52:41.157951+0000","last_clean":"2024-09-11T12:52:41.157951+0000","last_became_active":"2024-09-11T12:52:37.973935+0000","last_became_peered":"2024-09-11T12:52:37.973935+0000","last_unstale":"2024-09-11T12:52:41.157951+0000","last_undegraded":"2024-09-11T12:52:41.157951+0000","last_fullsized":"2024-09-11T12:52:41.157951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:49:48.801783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.158035+0000","last_change":"2024-09-11T12:52:38.983841+0000","last_active":"2024-09-11T12:52:41.158035+0000","last_peered":"2024-09-11T12:52:41.158035+0000","last_clean":"2024-09-11T12:52:41.158035+0000","last_became_active":"2024-09-11T12:52:38.983575+0000","last_became_peered":"2024-09-11T12:52:38.983575+0000","last_unstale":"2024-09-11T12:52:41.158035+0000","last_undegraded":"2024-09-11T12:52:41.158035+0000","last_fullsized":"2024-09-11T12:52:41.158035+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:09:10.643870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868268+0000","last_change":"2024-09-11T12:52:37.981826+0000","last_active":"2024-09-11T12:52:40.868268+0000","last_peered":"2024-09-11T12:52:40.868268+0000","last_clean":"2024-09-11T12:52:40.868268+0000","last_became_active":"2024-09-11T12:52:37.981573+0000","last_became_peered":"2024-09-11T12:52:37.981573+0000","last_unstale":"2024-09-11T12:52:40.868268+0000","last_undegraded":"2024-09-11T12:52:40.868268+0000","last_fullsized":"2024-09-11T12:52:40.868268+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:08.914329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981222+0000","last_change":"2024-09-11T12:52:38.993876+0000","last_active":"2024-09-11T12:52:39.981222+0000","last_peered":"2024-09-11T12:52:39.981222+0000","last_clean":"2024-09-11T12:52:39.981222+0000","last_became_active":"2024-09-11T12:52:38.993228+0000","last_became_peered":"2024-09-11T12:52:38.993228+0000","last_unstale":"2024-09-11T12:52:39.981222+0000","last_undegraded":"2024-09-11T12:52:39.981222+0000","last_fullsized":"2024-09-11T12:52:39.981222+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:54.998653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157685+0000","last_change":"2024-09-11T12:52:37.974700+0000","last_active":"2024-09-11T12:52:41.157685+0000","last_peered":"2024-09-11T12:52:41.157685+0000","last_clean":"2024-09-11T12:52:41.157685+0000","last_became_active":"2024-09-11T12:52:37.972115+0000","last_became_peered":"2024-09-11T12:52:37.972115+0000","last_unstale":"2024-09-11T12:52:41.157685+0000","last_undegraded":"2024-09-11T12:52:41.157685+0000","last_fullsized":"2024-09-11T12:52:41.157685+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:06:16.659916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157614+0000","last_change":"2024-09-11T12:52:38.985418+0000","last_active":"2024-09-11T12:52:41.157614+0000","last_peered":"2024-09-11T12:52:41.157614+0000","last_clean":"2024-09-11T12:52:41.157614+0000","last_became_active":"2024-09-11T12:52:38.985149+0000","last_became_peered":"2024-09-11T12:52:38.985149+0000","last_unstale":"2024-09-11T12:52:41.157614+0000","last_undegraded":"2024-09-11T12:52:41.157614+0000","last_fullsized":"2024-09-11T12:52:41.157614+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:55:42.972135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978169+0000","last_change":"2024-09-11T12:52:37.979218+0000","last_active":"2024-09-11T12:52:39.978169+0000","last_peered":"2024-09-11T12:52:39.978169+0000","last_clean":"2024-09-11T12:52:39.978169+0000","last_became_active":"2024-09-11T12:52:37.978903+0000","last_became_peered":"2024-09-11T12:52:37.978903+0000","last_unstale":"2024-09-11T12:52:39.978169+0000","last_undegraded":"2024-09-11T12:52:39.978169+0000","last_fullsized":"2024-09-11T12:52:39.978169+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:50.985827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978488+0000","last_change":"2024-09-11T12:52:38.990579+0000","last_active":"2024-09-11T12:52:39.978488+0000","last_peered":"2024-09-11T12:52:39.978488+0000","last_clean":"2024-09-11T12:52:39.978488+0000","last_became_active":"2024-09-11T12:52:38.990058+0000","last_became_peered":"2024-09-11T12:52:38.990058+0000","last_unstale":"2024-09-11T12:52:39.978488+0000","last_undegraded":"2024-09-11T12:52:39.978488+0000","last_fullsized":"2024-09-11T12:52:39.978488+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:09.916822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868077+0000","last_change":"2024-09-11T12:52:37.981060+0000","last_active":"2024-09-11T12:52:40.868077+0000","last_peered":"2024-09-11T12:52:40.868077+0000","last_clean":"2024-09-11T12:52:40.868077+0000","last_became_active":"2024-09-11T12:52:37.980809+0000","last_became_peered":"2024-09-11T12:52:37.980809+0000","last_unstale":"2024-09-11T12:52:40.868077+0000","last_undegraded":"2024-09-11T12:52:40.868077+0000","last_fullsized":"2024-09-11T12:52:40.868077+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:19.142476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157416+0000","last_change":"2024-09-11T12:52:38.979221+0000","last_active":"2024-09-11T12:52:41.157416+0000","last_peered":"2024-09-11T12:52:41.157416+0000","last_clean":"2024-09-11T12:52:41.157416+0000","last_became_active":"2024-09-11T12:52:38.978867+0000","last_became_peered":"2024-09-11T12:52:38.978867+0000","last_unstale":"2024-09-11T12:52:41.157416+0000","last_undegraded":"2024-09-11T12:52:41.157416+0000","last_fullsized":"2024-09-11T12:52:41.157416+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:02.793052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971510+0000","last_change":"2024-09-11T12:52:37.974259+0000","last_active":"2024-09-11T12:52:40.971510+0000","last_peered":"2024-09-11T12:52:40.971510+0000","last_clean":"2024-09-11T12:52:40.971510+0000","last_became_active":"2024-09-11T12:52:37.973999+0000","last_became_peered":"2024-09-11T12:52:37.973999+0000","last_unstale":"2024-09-11T12:52:40.971510+0000","last_undegraded":"2024-09-11T12:52:40.971510+0000","last_fullsized":"2024-09-11T12:52:40.971510+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:11:59.774647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978156+0000","last_change":"2024-09-11T12:52:38.976088+0000","last_active":"2024-09-11T12:52:39.978156+0000","last_peered":"2024-09-11T12:52:39.978156+0000","last_clean":"2024-09-11T12:52:39.978156+0000","last_became_active":"2024-09-11T12:52:38.975788+0000","last_became_peered":"2024-09-11T12:52:38.975788+0000","last_unstale":"2024-09-11T12:52:39.978156+0000","last_undegraded":"2024-09-11T12:52:39.978156+0000","last_fullsized":"2024-09-11T12:52:39.978156+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:08.660560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978679+0000","last_change":"2024-09-11T12:52:37.972031+0000","last_active":"2024-09-11T12:52:39.978679+0000","last_peered":"2024-09-11T12:52:39.978679+0000","last_clean":"2024-09-11T12:52:39.978679+0000","last_became_active":"2024-09-11T12:52:37.971725+0000","last_became_peered":"2024-09-11T12:52:37.971725+0000","last_unstale":"2024-09-11T12:52:39.978679+0000","last_undegraded":"2024-09-11T12:52:39.978679+0000","last_fullsized":"2024-09-11T12:52:39.978679+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:14:46.197934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157343+0000","last_change":"2024-09-11T12:52:38.994843+0000","last_active":"2024-09-11T12:52:41.157343+0000","last_peered":"2024-09-11T12:52:41.157343+0000","last_clean":"2024-09-11T12:52:41.157343+0000","last_became_active":"2024-09-11T12:52:38.994510+0000","last_became_peered":"2024-09-11T12:52:38.994510+0000","last_unstale":"2024-09-11T12:52:41.157343+0000","last_undegraded":"2024-09-11T12:52:41.157343+0000","last_fullsized":"2024-09-11T12:52:41.157343+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:34:05.546415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":23,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157104+0000","last_change":"2024-09-11T12:52:37.974338+0000","last_active":"2024-09-11T12:52:41.157104+0000","last_peered":"2024-09-11T12:52:41.157104+0000","last_clean":"2024-09-11T12:52:41.157104+0000","last_became_active":"2024-09-11T12:52:37.973805+0000","last_became_peered":"2024-09-11T12:52:37.973805+0000","last_unstale":"2024-09-11T12:52:41.157104+0000","last_undegraded":"2024-09-11T12:52:41.157104+0000","last_fullsized":"2024-09-11T12:52:41.157104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:17.692826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157720+0000","last_change":"2024-09-11T12:52:38.991174+0000","last_active":"2024-09-11T12:52:41.157720+0000","last_peered":"2024-09-11T12:52:41.157720+0000","last_clean":"2024-09-11T12:52:41.157720+0000","last_became_active":"2024-09-11T12:52:38.990853+0000","last_became_peered":"2024-09-11T12:52:38.990853+0000","last_unstale":"2024-09-11T12:52:41.157720+0000","last_undegraded":"2024-09-11T12:52:41.157720+0000","last_fullsized":"2024-09-11T12:52:41.157720+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:03.873076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868486+0000","last_change":"2024-09-11T12:52:37.979890+0000","last_active":"2024-09-11T12:52:40.868486+0000","last_peered":"2024-09-11T12:52:40.868486+0000","last_clean":"2024-09-11T12:52:40.868486+0000","last_became_active":"2024-09-11T12:52:37.979656+0000","last_became_peered":"2024-09-11T12:52:37.979656+0000","last_unstale":"2024-09-11T12:52:40.868486+0000","last_undegraded":"2024-09-11T12:52:40.868486+0000","last_fullsized":"2024-09-11T12:52:40.868486+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:55:41.969642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979359+0000","last_change":"2024-09-11T12:52:38.990762+0000","last_active":"2024-09-11T12:52:39.979359+0000","last_peered":"2024-09-11T12:52:39.979359+0000","last_clean":"2024-09-11T12:52:39.979359+0000","last_became_active":"2024-09-11T12:52:38.990470+0000","last_became_peered":"2024-09-11T12:52:38.990470+0000","last_unstale":"2024-09-11T12:52:39.979359+0000","last_undegraded":"2024-09-11T12:52:39.979359+0000","last_fullsized":"2024-09-11T12:52:39.979359+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:08.282045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969092+0000","last_change":"2024-09-11T12:52:37.980086+0000","last_active":"2024-09-11T12:52:40.969092+0000","last_peered":"2024-09-11T12:52:40.969092+0000","last_clean":"2024-09-11T12:52:40.969092+0000","last_became_active":"2024-09-11T12:52:37.979564+0000","last_became_peered":"2024-09-11T12:52:37.979564+0000","last_unstale":"2024-09-11T12:52:40.969092+0000","last_undegraded":"2024-09-11T12:52:40.969092+0000","last_fullsized":"2024-09-11T12:52:40.969092+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:34:21.903931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868547+0000","last_change":"2024-09-11T12:52:38.991662+0000","last_active":"2024-09-11T12:52:40.868547+0000","last_peered":"2024-09-11T12:52:40.868547+0000","last_clean":"2024-09-11T12:52:40.868547+0000","last_became_active":"2024-09-11T12:52:38.990538+0000","last_became_peered":"2024-09-11T12:52:38.990538+0000","last_unstale":"2024-09-11T12:52:40.868547+0000","last_undegraded":"2024-09-11T12:52:40.868547+0000","last_fullsized":"2024-09-11T12:52:40.868547+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:01:25.859071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979033+0000","last_change":"2024-09-11T12:52:37.974300+0000","last_active":"2024-09-11T12:52:39.979033+0000","last_peered":"2024-09-11T12:52:39.979033+0000","last_clean":"2024-09-11T12:52:39.979033+0000","last_became_active":"2024-09-11T12:52:37.974028+0000","last_became_peered":"2024-09-11T12:52:37.974028+0000","last_unstale":"2024-09-11T12:52:39.979033+0000","last_undegraded":"2024-09-11T12:52:39.979033+0000","last_fullsized":"2024-09-11T12:52:39.979033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:07.658067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157820+0000","last_change":"2024-09-11T12:52:38.995552+0000","last_active":"2024-09-11T12:52:41.157820+0000","last_peered":"2024-09-11T12:52:41.157820+0000","last_clean":"2024-09-11T12:52:41.157820+0000","last_became_active":"2024-09-11T12:52:38.995139+0000","last_became_peered":"2024-09-11T12:52:38.995139+0000","last_unstale":"2024-09-11T12:52:41.157820+0000","last_undegraded":"2024-09-11T12:52:41.157820+0000","last_fullsized":"2024-09-11T12:52:41.157820+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:54.568538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969125+0000","last_change":"2024-09-11T12:52:37.975487+0000","last_active":"2024-09-11T12:52:40.969125+0000","last_peered":"2024-09-11T12:52:40.969125+0000","last_clean":"2024-09-11T12:52:40.969125+0000","last_became_active":"2024-09-11T12:52:37.975237+0000","last_became_peered":"2024-09-11T12:52:37.975237+0000","last_unstale":"2024-09-11T12:52:40.969125+0000","last_undegraded":"2024-09-11T12:52:40.969125+0000","last_fullsized":"2024-09-11T12:52:40.969125+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:15:26.321342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978488+0000","last_change":"2024-09-11T12:52:38.990318+0000","last_active":"2024-09-11T12:52:39.978488+0000","last_peered":"2024-09-11T12:52:39.978488+0000","last_clean":"2024-09-11T12:52:39.978488+0000","last_became_active":"2024-09-11T12:52:38.989725+0000","last_became_peered":"2024-09-11T12:52:38.989725+0000","last_unstale":"2024-09-11T12:52:39.978488+0000","last_undegraded":"2024-09-11T12:52:39.978488+0000","last_fullsized":"2024-09-11T12:52:39.978488+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:35:02.793052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868241+0000","last_change":"2024-09-11T12:52:37.982929+0000","last_active":"2024-09-11T12:52:40.868241+0000","last_peered":"2024-09-11T12:52:40.868241+0000","last_clean":"2024-09-11T12:52:40.868241+0000","last_became_active":"2024-09-11T12:52:37.982667+0000","last_became_peered":"2024-09-11T12:52:37.982667+0000","last_unstale":"2024-09-11T12:52:40.868241+0000","last_undegraded":"2024-09-11T12:52:40.868241+0000","last_fullsized":"2024-09-11T12:52:40.868241+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:24.642019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969288+0000","last_change":"2024-09-11T12:52:38.977485+0000","last_active":"2024-09-11T12:52:40.969288+0000","last_peered":"2024-09-11T12:52:40.969288+0000","last_clean":"2024-09-11T12:52:40.969288+0000","last_became_active":"2024-09-11T12:52:38.977223+0000","last_became_peered":"2024-09-11T12:52:38.977223+0000","last_unstale":"2024-09-11T12:52:40.969288+0000","last_undegraded":"2024-09-11T12:52:40.969288+0000","last_fullsized":"2024-09-11T12:52:40.969288+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:27.508739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981076+0000","last_change":"2024-09-11T12:52:37.979101+0000","last_active":"2024-09-11T12:52:39.981076+0000","last_peered":"2024-09-11T12:52:39.981076+0000","last_clean":"2024-09-11T12:52:39.981076+0000","last_became_active":"2024-09-11T12:52:37.978838+0000","last_became_peered":"2024-09-11T12:52:37.978838+0000","last_unstale":"2024-09-11T12:52:39.981076+0000","last_undegraded":"2024-09-11T12:52:39.981076+0000","last_fullsized":"2024-09-11T12:52:39.981076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:31:35.312684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.968858+0000","last_change":"2024-09-11T12:52:38.976657+0000","last_active":"2024-09-11T12:52:40.968858+0000","last_peered":"2024-09-11T12:52:40.968858+0000","last_clean":"2024-09-11T12:52:40.968858+0000","last_became_active":"2024-09-11T12:52:38.976385+0000","last_became_peered":"2024-09-11T12:52:38.976385+0000","last_unstale":"2024-09-11T12:52:40.968858+0000","last_undegraded":"2024-09-11T12:52:40.968858+0000","last_fullsized":"2024-09-11T12:52:40.968858+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:55.798955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.978251+0000","last_change":"2024-09-11T12:52:37.965386+0000","last_active":"2024-09-11T12:52:39.978251+0000","last_peered":"2024-09-11T12:52:39.978251+0000","last_clean":"2024-09-11T12:52:39.978251+0000","last_became_active":"2024-09-11T12:52:37.965087+0000","last_became_peered":"2024-09-11T12:52:37.965087+0000","last_unstale":"2024-09-11T12:52:39.978251+0000","last_undegraded":"2024-09-11T12:52:39.978251+0000","last_fullsized":"2024-09-11T12:52:39.978251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:32.505935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979374+0000","last_change":"2024-09-11T12:52:38.980880+0000","last_active":"2024-09-11T12:52:39.979374+0000","last_peered":"2024-09-11T12:52:39.979374+0000","last_clean":"2024-09-11T12:52:39.979374+0000","last_became_active":"2024-09-11T12:52:38.980568+0000","last_became_peered":"2024-09-11T12:52:38.980568+0000","last_unstale":"2024-09-11T12:52:39.979374+0000","last_undegraded":"2024-09-11T12:52:39.979374+0000","last_fullsized":"2024-09-11T12:52:39.979374+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:03.873076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981711+0000","last_change":"2024-09-11T12:52:37.974554+0000","last_active":"2024-09-11T12:52:39.981711+0000","last_peered":"2024-09-11T12:52:39.981711+0000","last_clean":"2024-09-11T12:52:39.981711+0000","last_became_active":"2024-09-11T12:52:37.974283+0000","last_became_peered":"2024-09-11T12:52:37.974283+0000","last_unstale":"2024-09-11T12:52:39.981711+0000","last_undegraded":"2024-09-11T12:52:39.981711+0000","last_fullsized":"2024-09-11T12:52:39.981711+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:59.798942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979405+0000","last_change":"2024-09-11T12:52:38.976969+0000","last_active":"2024-09-11T12:52:39.979405+0000","last_peered":"2024-09-11T12:52:39.979405+0000","last_clean":"2024-09-11T12:52:39.979405+0000","last_became_active":"2024-09-11T12:52:38.976698+0000","last_became_peered":"2024-09-11T12:52:38.976698+0000","last_unstale":"2024-09-11T12:52:39.979405+0000","last_undegraded":"2024-09-11T12:52:39.979405+0000","last_fullsized":"2024-09-11T12:52:39.979405+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:54.568538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971249+0000","last_change":"2024-09-11T12:52:37.977256+0000","last_active":"2024-09-11T12:52:40.971249+0000","last_peered":"2024-09-11T12:52:40.971249+0000","last_clean":"2024-09-11T12:52:40.971249+0000","last_became_active":"2024-09-11T12:52:37.977012+0000","last_became_peered":"2024-09-11T12:52:37.977012+0000","last_unstale":"2024-09-11T12:52:40.971249+0000","last_undegraded":"2024-09-11T12:52:40.971249+0000","last_fullsized":"2024-09-11T12:52:40.971249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:03:33.766634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.981283+0000","last_change":"2024-09-11T12:52:38.994099+0000","last_active":"2024-09-11T12:52:39.981283+0000","last_peered":"2024-09-11T12:52:39.981283+0000","last_clean":"2024-09-11T12:52:39.981283+0000","last_became_active":"2024-09-11T12:52:38.993615+0000","last_became_peered":"2024-09-11T12:52:38.993615+0000","last_unstale":"2024-09-11T12:52:39.981283+0000","last_undegraded":"2024-09-11T12:52:39.981283+0000","last_fullsized":"2024-09-11T12:52:39.981283+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:56:40.126548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971441+0000","last_change":"2024-09-11T12:52:37.979269+0000","last_active":"2024-09-11T12:52:40.971441+0000","last_peered":"2024-09-11T12:52:40.971441+0000","last_clean":"2024-09-11T12:52:40.971441+0000","last_became_active":"2024-09-11T12:52:37.978857+0000","last_became_peered":"2024-09-11T12:52:37.978857+0000","last_unstale":"2024-09-11T12:52:40.971441+0000","last_undegraded":"2024-09-11T12:52:40.971441+0000","last_fullsized":"2024-09-11T12:52:40.971441+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:59.798942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.969100+0000","last_change":"2024-09-11T12:52:38.967506+0000","last_active":"2024-09-11T12:52:40.969100+0000","last_peered":"2024-09-11T12:52:40.969100+0000","last_clean":"2024-09-11T12:52:40.969100+0000","last_became_active":"2024-09-11T12:52:38.967236+0000","last_became_peered":"2024-09-11T12:52:38.967236+0000","last_unstale":"2024-09-11T12:52:40.969100+0000","last_undegraded":"2024-09-11T12:52:40.969100+0000","last_fullsized":"2024-09-11T12:52:40.969100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:04.602127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.979420+0000","last_change":"2024-09-11T12:52:37.977366+0000","last_active":"2024-09-11T12:52:39.979420+0000","last_peered":"2024-09-11T12:52:39.979420+0000","last_clean":"2024-09-11T12:52:39.979420+0000","last_became_active":"2024-09-11T12:52:37.977101+0000","last_became_peered":"2024-09-11T12:52:37.977101+0000","last_unstale":"2024-09-11T12:52:39.979420+0000","last_undegraded":"2024-09-11T12:52:39.979420+0000","last_fullsized":"2024-09-11T12:52:39.979420+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:59.581815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868441+0000","last_change":"2024-09-11T12:52:38.986034+0000","last_active":"2024-09-11T12:52:40.868441+0000","last_peered":"2024-09-11T12:52:40.868441+0000","last_clean":"2024-09-11T12:52:40.868441+0000","last_became_active":"2024-09-11T12:52:38.985769+0000","last_became_peered":"2024-09-11T12:52:38.985769+0000","last_unstale":"2024-09-11T12:52:40.868441+0000","last_undegraded":"2024-09-11T12:52:40.868441+0000","last_fullsized":"2024-09-11T12:52:40.868441+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:41.006865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.980953+0000","last_change":"2024-09-11T12:52:37.975976+0000","last_active":"2024-09-11T12:52:39.980953+0000","last_peered":"2024-09-11T12:52:39.980953+0000","last_clean":"2024-09-11T12:52:39.980953+0000","last_became_active":"2024-09-11T12:52:37.973137+0000","last_became_peered":"2024-09-11T12:52:37.973137+0000","last_unstale":"2024-09-11T12:52:39.980953+0000","last_undegraded":"2024-09-11T12:52:39.980953+0000","last_fullsized":"2024-09-11T12:52:39.980953+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:33:41.317888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157655+0000","last_change":"2024-09-11T12:52:38.991583+0000","last_active":"2024-09-11T12:52:41.157655+0000","last_peered":"2024-09-11T12:52:41.157655+0000","last_clean":"2024-09-11T12:52:41.157655+0000","last_became_active":"2024-09-11T12:52:38.991287+0000","last_became_peered":"2024-09-11T12:52:38.991287+0000","last_unstale":"2024-09-11T12:52:41.157655+0000","last_undegraded":"2024-09-11T12:52:41.157655+0000","last_fullsized":"2024-09-11T12:52:41.157655+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:32:25.644513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.977930+0000","last_change":"2024-09-11T12:52:37.981807+0000","last_active":"2024-09-11T12:52:39.977930+0000","last_peered":"2024-09-11T12:52:39.977930+0000","last_clean":"2024-09-11T12:52:39.977930+0000","last_became_active":"2024-09-11T12:52:37.981578+0000","last_became_peered":"2024-09-11T12:52:37.981578+0000","last_unstale":"2024-09-11T12:52:39.977930+0000","last_undegraded":"2024-09-11T12:52:39.977930+0000","last_fullsized":"2024-09-11T12:52:39.977930+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:53.996160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.980959+0000","last_change":"2024-09-11T12:52:38.974052+0000","last_active":"2024-09-11T12:52:39.980959+0000","last_peered":"2024-09-11T12:52:39.980959+0000","last_clean":"2024-09-11T12:52:39.980959+0000","last_became_active":"2024-09-11T12:52:38.973802+0000","last_became_peered":"2024-09-11T12:52:38.973802+0000","last_unstale":"2024-09-11T12:52:39.980959+0000","last_undegraded":"2024-09-11T12:52:39.980959+0000","last_fullsized":"2024-09-11T12:52:39.980959+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:51.988321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:39.980970+0000","last_change":"2024-09-11T12:52:37.974581+0000","last_active":"2024-09-11T12:52:39.980970+0000","last_peered":"2024-09-11T12:52:39.980970+0000","last_clean":"2024-09-11T12:52:39.980970+0000","last_became_active":"2024-09-11T12:52:37.974311+0000","last_became_peered":"2024-09-11T12:52:37.974311+0000","last_unstale":"2024-09-11T12:52:39.980970+0000","last_undegraded":"2024-09-11T12:52:39.980970+0000","last_fullsized":"2024-09-11T12:52:39.980970+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:11:59.774647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868132+0000","last_change":"2024-09-11T12:52:38.979569+0000","last_active":"2024-09-11T12:52:40.868132+0000","last_peered":"2024-09-11T12:52:40.868132+0000","last_clean":"2024-09-11T12:52:40.868132+0000","last_became_active":"2024-09-11T12:52:38.979310+0000","last_became_peered":"2024-09-11T12:52:38.979310+0000","last_unstale":"2024-09-11T12:52:40.868132+0000","last_undegraded":"2024-09-11T12:52:40.868132+0000","last_fullsized":"2024-09-11T12:52:40.868132+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:24.373028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.868104+0000","last_change":"2024-09-11T12:52:37.967585+0000","last_active":"2024-09-11T12:52:40.868104+0000","last_peered":"2024-09-11T12:52:40.868104+0000","last_clean":"2024-09-11T12:52:40.868104+0000","last_became_active":"2024-09-11T12:52:37.967286+0000","last_became_peered":"2024-09-11T12:52:37.967286+0000","last_unstale":"2024-09-11T12:52:40.868104+0000","last_undegraded":"2024-09-11T12:52:40.868104+0000","last_fullsized":"2024-09-11T12:52:40.868104+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:07.279552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.867944+0000","last_change":"2024-09-11T12:52:38.984166+0000","last_active":"2024-09-11T12:52:40.867944+0000","last_peered":"2024-09-11T12:52:40.867944+0000","last_clean":"2024-09-11T12:52:40.867944+0000","last_became_active":"2024-09-11T12:52:38.983881+0000","last_became_peered":"2024-09-11T12:52:38.983881+0000","last_unstale":"2024-09-11T12:52:40.867944+0000","last_undegraded":"2024-09-11T12:52:40.867944+0000","last_fullsized":"2024-09-11T12:52:40.867944+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:37.949954+0000","last_clean_scrub_stamp":"2024-09-11T12:52:37.949954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:41:46.015210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:40.971271+0000","last_change":"2024-09-11T12:52:37.973456+0000","last_active":"2024-09-11T12:52:40.971271+0000","last_peered":"2024-09-11T12:52:40.971271+0000","last_clean":"2024-09-11T12:52:40.971271+0000","last_became_active":"2024-09-11T12:52:37.973163+0000","last_became_peered":"2024-09-11T12:52:37.973163+0000","last_unstale":"2024-09-11T12:52:40.971271+0000","last_undegraded":"2024-09-11T12:52:40.971271+0000","last_fullsized":"2024-09-11T12:52:40.971271+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:52:36.947461+0000","last_clean_scrub_stamp":"2024-09-11T12:52:36.947461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:31:35.312684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":176,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-11T12:52:41.157378+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:52:41.157378+0000","last_peered":"2024-09-11T12:52:41.157378+0000","last_clean":"2024-09-11T12:52:41.157378+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:52:41.157378+0000","last_undegraded":"2024-09-11T12:52:41.157378+0000","last_fullsized":"2024-09-11T12:52:41.157378+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509481,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1290240,"data_stored":873217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509481,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":688128,"data_stored":282689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509481,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31876,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339964,"statfs":{"total":96636764160,"available":96604123136,"internally_reserved":0,"allocated":696320,"data_stored":283341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31880,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339960,"statfs":{"total":96636764160,"available":96604119040,"internally_reserved":0,"allocated":700416,"data_stored":284301,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509481,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":282745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509481,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":696320,"data_stored":282745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":692224,"data_stored":282779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542185,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32452,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339388,"statfs":{"total":96636764160,"available":96603533312,"internally_reserved":0,"allocated":1286144,"data_stored":868951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:52:43.540 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:52:43.540 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:52:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.725+0000 7f61c4e00640 1 -- 172.21.15.78:0/962734346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61a4004810 msgr2=0x7f61a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.725+0000 7f61c4e00640 1 --2- 172.21.15.78:0/962734346 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61a4004810 0x7f61a4004c90 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006fd0 tx=0x7f61ac01f6a0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.725+0000 7f61be800640 1 -- 172.21.15.78:0/962734346 reap_dead start 2024-09-11T12:52:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- 172.21.15.78:0/962734346 shutdown_connections 2024-09-11T12:52:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- 172.21.15.78:0/962734346 >> 172.21.15.78:0/962734346 conn(0x7f61c0074bd0 msgr2=0x7f61c006e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- 172.21.15.78:0/962734346 shutdown_connections 2024-09-11T12:52:43.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- 172.21.15.78:0/962734346 wait complete. 2024-09-11T12:52:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 Processor -- start 2024-09-11T12:52:43.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- start start 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 0x7f61c0144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 0x7f61c01f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f61c00b4520 con 0x7f61c01f7fd0 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f61c00cef20 con 0x7f61c01420e0 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f61c00b46a0 con 0x7f61a4004810 2024-09-11T12:52:43.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42986/0 (socket says 172.21.15.78:42986) 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 -- 172.21.15.78:0/4025856273 learned_addr learned my addr 172.21.15.78:0/4025856273 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61bde00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 0x7f61c0144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61bd400640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 0x7f61c01f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 -- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 msgr2=0x7f61c0144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 0x7f61c0144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 -- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 msgr2=0x7f61c01f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 0x7f61c01f7a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 -- 172.21.15.78:0/4025856273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c0144080 con 0x7f61c01f7fd0 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61bde00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 0x7f61c0144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61bd400640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 0x7f61c01f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:43.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61be800640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f61a4004790 tx=0x7f61b8001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8011630 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61b8018070 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b8011a70 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.729+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c0144360 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c0145380 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c0070480 con 0x7f61c01f7fd0 2024-09-11T12:52:43.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f61b8011c10 con 0x7f61c01f7fd0 2024-09-11T12:52:43.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61b6000640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 0x7f61a0073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:43.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f61b80927b0 con 0x7f61c01f7fd0 2024-09-11T12:52:43.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.733+0000 7f61bde00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 0x7f61a0073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:43.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.737+0000 7f61bde00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 0x7f61a0073cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61b0011490 tx=0x7f61b0009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:43.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.737+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f61b805e1f0 con 0x7f61c01f7fd0 2024-09-11T12:52:43.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.961+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f61c015a640 con 0x7f61c01f7fd0 2024-09-11T12:52:43.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.965+0000 7f61b6000640 1 -- 172.21.15.78:0/4025856273 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f61b805daf0 con 0x7f61c01f7fd0 2024-09-11T12:52:43.967 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:52:43.967 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:52:43.967 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:52:43.967 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:52:43.968 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:52:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:52:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 msgr2=0x7f61a0073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 0x7f61a0073cf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f61b0011490 tx=0x7f61b0009500 comp rx=0 tx=0).stop 2024-09-11T12:52:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 msgr2=0x7f61c01f4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:43.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f61a4004790 tx=0x7f61b8001640 comp rx=0 tx=0).stop 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 shutdown_connections 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f61a0071830 0x7f61a0073cf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f61c01f7fd0 0x7f61c01f4340 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f61c01420e0 0x7f61c01f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 --2- 172.21.15.78:0/4025856273 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f61a4004810 0x7f61c0144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 >> 172.21.15.78:0/4025856273 conn(0x7f61c0074bd0 msgr2=0x7f61c0159a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 shutdown_connections 2024-09-11T12:52:43.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:43.969+0000 7f61c4e00640 1 -- 172.21.15.78:0/4025856273 wait complete. 2024-09-11T12:52:43.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:52:44.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.129+0000 7f47bc200640 1 -- 172.21.15.78:0/2164053235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f479c003fe0 msgr2=0x7f479c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.129+0000 7f47bc200640 1 --2- 172.21.15.78:0/2164053235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f479c003fe0 0x7f479c004810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009b90 tx=0x7f47a001d2b0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.129+0000 7f47ba200640 1 -- 172.21.15.78:0/2164053235 reap_dead start 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- 172.21.15.78:0/2164053235 shutdown_connections 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- 172.21.15.78:0/2164053235 >> 172.21.15.78:0/2164053235 conn(0x7f47b40ceab0 msgr2=0x7f47b40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- 172.21.15.78:0/2164053235 shutdown_connections 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- 172.21.15.78:0/2164053235 wait complete. 2024-09-11T12:52:44.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 Processor -- start 2024-09-11T12:52:44.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- start start 2024-09-11T12:52:44.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f479c003fe0 0x7f47b41f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 0x7f47b413e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47b40d1350 con 0x7f47b40b0950 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47b415e4d0 con 0x7f47b413e480 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47bc200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47b40d14d0 con 0x7f479c003fe0 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52408/0 (socket says 172.21.15.78:52408) 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 -- 172.21.15.78:0/3318504609 learned_addr learned my addr 172.21.15.78:0/3318504609 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 -- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f479c003fe0 msgr2=0x7f47b41f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47ba200640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 0x7f47b413e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f479c003fe0 0x7f47b41f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 -- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 msgr2=0x7f47b413e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 0x7f47b413e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 -- 172.21.15.78:0/3318504609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40b1110 con 0x7f47b40b0950 2024-09-11T12:52:44.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47ba200640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 0x7f47b413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:44.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b8e00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009b90 tx=0x7f47a0005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.133+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a00b2070 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47a00b3040 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0009730 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47bc200640 1 -- 172.21.15.78:0/3318504609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b41554c0 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47bc200640 1 -- 172.21.15.78:0/3318504609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b4155a80 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4143d50 con 0x7f47b40b0950 2024-09-11T12:52:44.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f47a00046d0 con 0x7f47b40b0950 2024-09-11T12:52:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47b1600640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 0x7f4794073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.137+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f47a012d060 con 0x7f47b40b0950 2024-09-11T12:52:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.141+0000 7f47b9800640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 0x7f4794073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.141+0000 7f47b9800640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 0x7f4794073d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009e00 tx=0x7f47a8005e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.141+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f47a0017000 con 0x7f47b40b0950 2024-09-11T12:52:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.345+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47b40b2880 con 0x7f47b40b0950 2024-09-11T12:52:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.345+0000 7f47b1600640 1 -- 172.21.15.78:0/3318504609 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5229 (secure 0 0 0) 0x7f47a00f8c40 con 0x7f47b40b0950 2024-09-11T12:52:44.347 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:44.347 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":36,"btime":"2024-09-11T12:52:41:982815+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:52:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 36 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 msgr2=0x7f4794073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 0x7f4794073d20 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009e00 tx=0x7f47a8005e40 comp rx=0 tx=0).stop 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 msgr2=0x7f47b41f7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009b90 tx=0x7f47a0005590 comp rx=0 tx=0).stop 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 shutdown_connections 2024-09-11T12:52:44.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4794071860 0x7f4794073d20 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b413e480 0x7f47b413e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b40b0950 0x7f47b41f7c70 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 --2- 172.21.15.78:0/3318504609 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f479c003fe0 0x7f47b41f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 >> 172.21.15.78:0/3318504609 conn(0x7f47b40ceab0 msgr2=0x7f47b4142d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 shutdown_connections 2024-09-11T12:52:44.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.349+0000 7f47a6a00640 1 -- 172.21.15.78:0/3318504609 wait complete. 2024-09-11T12:52:44.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:52:38.352038+0000', 'modified': '2024-09-11T12:52:41.364437+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5745}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5745': {'gid': 5745, 'name': 'c', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6834/2934440940', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 2934440940}, {'type': 'v1', 'addr': '172.21.15.78:6834', 'nonce': 2934440940}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5745, 'qdb_cluster': [5745]} 2024-09-11T12:52:44.772 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:52:44.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-11T12:52:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 -- 172.21.15.78:0/3160649527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815b3e0 msgr2=0x7f4b580730a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 --2- 172.21.15.78:0/3160649527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815b3e0 0x7f4b580730a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009840 tx=0x7f4b4401c6b0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 -- 172.21.15.78:0/3160649527 shutdown_connections 2024-09-11T12:52:44.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 --2- 172.21.15.78:0/3160649527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b5815b3e0 0x7f4b580730a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 -- 172.21.15.78:0/3160649527 >> 172.21.15.78:0/3160649527 conn(0x7f4b580cfeb0 msgr2=0x7f4b580d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 -- 172.21.15.78:0/3160649527 shutdown_connections 2024-09-11T12:52:44.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.913+0000 7f4b5de00640 1 -- 172.21.15.78:0/3160649527 wait complete. 2024-09-11T12:52:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 Processor -- start 2024-09-11T12:52:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- start start 2024-09-11T12:52:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b58070040 0x7f4b581fb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b581437b0 0x7f4b5813dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5806fd30 con 0x7f4b5815b3e0 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b5806fbb0 con 0x7f4b58070040 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58167e50 con 0x7f4b581437b0 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52410/0 (socket says 172.21.15.78:52410) 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 -- 172.21.15.78:0/1895703718 learned_addr learned my addr 172.21.15.78:0/1895703718 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b57e00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b581437b0 0x7f4b5813dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b58070040 0x7f4b581fb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 -- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b581437b0 msgr2=0x7f4b5813dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b581437b0 0x7f4b5813dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 -- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b58070040 msgr2=0x7f4b581fb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b58070040 0x7f4b581fb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 -- 172.21.15.78:0/1895703718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b581f8300 con 0x7f4b5815b3e0 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5d400640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000c4d0 tx=0x7f4b5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b54c00640 1 -- 172.21.15.78:0/1895703718 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b500052d0 con 0x7f4b5815b3e0 2024-09-11T12:52:44.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b54c00640 1 -- 172.21.15.78:0/1895703718 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b50005470 con 0x7f4b5815b3e0 2024-09-11T12:52:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b54c00640 1 -- 172.21.15.78:0/1895703718 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50012870 con 0x7f4b5815b3e0 2024-09-11T12:52:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b581f7780 con 0x7f4b5815b3e0 2024-09-11T12:52:44.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.917+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b581f7d40 con 0x7f4b5815b3e0 2024-09-11T12:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b54c00640 1 -- 172.21.15.78:0/1895703718 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f4b50003600 con 0x7f4b5815b3e0 2024-09-11T12:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b54c00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 0x7f4b38073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 Processor -- start 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b54c00640 1 -- 172.21.15.78:0/1895703718 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4b50091c40 con 0x7f4b5815b3e0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 -- start start 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 0x7f4b38073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe2527e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe24db340 0x558fe24db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fe2338270 con 0x558fe25283c0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558fe2327770 con 0x558fe24db340 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.921+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x558fe232b7b0 con 0x558fe2527aa0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 0x7f4b38073cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b480015b0 tx=0x7f4b48009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b5ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe2527e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52420/0 (socket says 172.21.15.78:52420) 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 -- 172.21.15.78:0/2015140651 learned_addr learned my addr 172.21.15.78:0/2015140651 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe24db340 0x558fe24db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 -- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 msgr2=0x558fe2527e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe2527e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 -- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe24db340 msgr2=0x558fe24db720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe24db340 0x558fe24db720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 -- 172.21.15.78:0/2015140651 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe22c6f90 con 0x558fe25283c0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009790 tx=0x7f4b4401aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2888075b534aae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b4d600640 1 -- 172.21.15.78:0/2015140651 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44005240 con 0x558fe25283c0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b4d600640 1 -- 172.21.15.78:0/2015140651 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b440053e0 con 0x558fe25283c0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b4d600640 1 -- 172.21.15.78:0/2015140651 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004020 con 0x558fe25283c0 2024-09-11T12:52:44.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- 172.21.15.78:0/2015140651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 msgr2=0x558fe24dae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009790 tx=0x7f4b4401aef0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- 172.21.15.78:0/2015140651 shutdown_connections 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe24db340 0x558fe24db720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe24dae00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 --2- 172.21.15.78:0/2015140651 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe2527e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- 172.21.15.78:0/2015140651 >> 172.21.15.78:0/2015140651 conn(0x558fe2508b60 msgr2=0x558fe250af50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- 172.21.15.78:0/2015140651 shutdown_connections 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- 172.21.15.78:0/2015140651 wait complete. 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 -- start start 2024-09-11T12:52:44.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 Processor -- start 2024-09-11T12:52:44.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.925+0000 7f4b62804000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe24de070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 0x558fe24de5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fe23f7290 con 0x558fe24db340 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558fe22d3ab0 con 0x558fe25283c0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x558fe22d1170 con 0x558fe2527aa0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 10 client.0 _ll_register_callbacks cb 0x558fe24c8830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52426/0 (socket says 172.21.15.78:52426) 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 -- 172.21.15.78:0/668105335 learned_addr learned my addr 172.21.15.78:0/668105335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b57e00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe24de070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5d400640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 0x558fe24de5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 -- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 msgr2=0x558fe24de070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe24de070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 -- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 msgr2=0x558fe24de5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 0x558fe24de5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 -- 172.21.15.78:0/668105335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe23ac6f0 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5d400640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 0x558fe24de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4b480156e0 tx=0x7f4b48001600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c084dd2e2399cf43 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48025020 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4b48010040 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fe22c0f10 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b4802f050 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 10 client.5865 fetch_fsmap learned FSMap version 36 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558fe2383c40 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558fe22c0f10 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f4b48032020 con 0x558fe24db340 2024-09-11T12:52:44.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 10 client.5865 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 10 client.5865 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 20 client.5865 populate_metadata read hostname 'smithi078' 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe24defc0 0x558fe2539b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 4 client.5865 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 --> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fe233a1c0 con 0x558fe24defc0 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b57e00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe24defc0 0x558fe2539b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b57e00640 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe24defc0 0x558fe2539b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f04d30bcf657d1f server_cookie=97be595244d2fc in_seq=0 out_seq=0 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 10 client.5865 ms_handle_connect on v2:172.21.15.78:6832/2934440940 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 1 -- 172.21.15.78:0/668105335 <== mds.0 v2:172.21.15.78:6832/2934440940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558fe233a1c0 con 0x558fe24defc0 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 10 client.5865 handle_command_reply: tid=0 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 client.5865 shutdown 2024-09-11T12:52:44.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 msgr2=0x558fe24ddb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4b480156e0 tx=0x7f4b48001600 comp rx=0 tx=0).stop 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 shutdown_connections 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe24defc0 0x558fe2539b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe25283c0 0x558fe24de5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe2527aa0 0x558fe24de070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b42000640 0 client.5865 ms_handle_reset on v2:172.21.15.78:6832/2934440940 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 --2- 172.21.15.78:0/668105335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe24ddb30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.929+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 >> 172.21.15.78:0/668105335 conn(0x558fe2508b60 msgr2=0x558fe24da500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.933+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 shutdown_connections 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.933+0000 7f4b62804000 1 -- 172.21.15.78:0/668105335 wait complete. 2024-09-11T12:52:44.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.933+0000 7f4b62804000 20 client.5865 trim_cache size 0 max 16384 2024-09-11T12:52:44.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 Processor -- start 2024-09-11T12:52:44.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 -- start start 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe2597990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 0x558fe2597ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b58202d40 con 0x558fe24db340 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b580b2310 con 0x558fe2527aa0 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b58153c70 con 0x558fe25283c0 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52436/0 (socket says 172.21.15.78:52436) 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 -- 172.21.15.78:0/2340813929 learned_addr learned my addr 172.21.15.78:0/2340813929 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 0x558fe2597ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 -- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 msgr2=0x558fe2597ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe2597990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 0x558fe2597ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 -- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 msgr2=0x558fe2597990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe2597990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 -- 172.21.15.78:0/2340813929 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fe22dce70 con 0x558fe24db340 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 0x558fe2597ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe2597990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b48001750 tx=0x7f4b4800ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39839d8379a690ce server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b4cc00640 1 -- 172.21.15.78:0/2340813929 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48025070 con 0x558fe24db340 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b4cc00640 1 -- 172.21.15.78:0/2340813929 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b48013830 con 0x558fe24db340 2024-09-11T12:52:44.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.949+0000 7f4b4cc00640 1 -- 172.21.15.78:0/2340813929 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48003980 con 0x558fe24db340 2024-09-11T12:52:44.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2340813929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 msgr2=0x558fe2597450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4b48001750 tx=0x7f4b4800ea50 comp rx=0 tx=0).stop 2024-09-11T12:52:44.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2340813929 shutdown_connections 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe25283c0 0x558fe2597ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe2597990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- 172.21.15.78:0/2340813929 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe24db340 0x558fe2597450 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2340813929 >> 172.21.15.78:0/2340813929 conn(0x558fe2508b60 msgr2=0x558fe2534800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2340813929 shutdown_connections 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2340813929 wait complete. 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- start start 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 Processor -- start 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe24db340 0x558fe259ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe259b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x558fe232fde0 con 0x558fe25283c0 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x558fe232aac0 con 0x558fe2527aa0 2024-09-11T12:52:44.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x558fe233aea0 con 0x558fe24db340 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 10 client.0 _ll_register_callbacks cb 0x7f4b4800ef10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52438/0 (socket says 172.21.15.78:52438) 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 -- 172.21.15.78:0/2700991267 learned_addr learned my addr 172.21.15.78:0/2700991267 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b57e00640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe259b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe24db340 0x558fe259ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 -- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe24db340 msgr2=0x558fe259ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe24db340 0x558fe259ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 -- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 msgr2=0x558fe259b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe259b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 -- 172.21.15.78:0/2700991267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fe22b1bf0 con 0x558fe25283c0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b5d400640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4b5005d0b0 tx=0x7f4b50059050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3305fcc2d0139101 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5008c060 con 0x558fe25283c0 2024-09-11T12:52:44.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4b50018b10 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558fe2335c60 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4b5005d120 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 10 client.5871 fetch_fsmap learned FSMap version 36 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x558fe22cdc60 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x558fe2335c60 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.953+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f4b50058a90 con 0x558fe25283c0 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 10 client.5871 fetch_fsmap finished waiting for FSMap version 36 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 10 client.5871 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 20 client.5871 populate_metadata read hostname 'smithi078' 2024-09-11T12:52:44.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe2557370 0x558fe2556430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:44.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 4 client.5871 mds_command: new command op to 5745 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:52:44.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 --> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] -- command(tid 0: {"prefix": "status"}) -- 0x558fe23333a0 con 0x558fe2557370 2024-09-11T12:52:44.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe2557370 0x558fe2556430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5ca00640 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe2557370 0x558fe2556430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ea8c370f8cc153c server_cookie=d6f1a458c8f679b2 in_seq=0 out_seq=0 2024-09-11T12:52:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b42000640 10 client.5871 ms_handle_connect on v2:172.21.15.78:6832/2934440940 2024-09-11T12:52:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b42000640 1 -- 172.21.15.78:0/2700991267 <== mds.0 v2:172.21.15.78:6832/2934440940 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x558fe23333a0 con 0x558fe2557370 2024-09-11T12:52:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b42000640 10 client.5871 handle_command_reply: tid=0 2024-09-11T12:52:44.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 client.5871 shutdown 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 msgr2=0x558fe259b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4b5005d0b0 tx=0x7f4b50059050 comp rx=0 tx=0).stop 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 shutdown_connections 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:6832/2934440940,v1:172.21.15.78:6834/2934440940] conn(0x558fe2557370 0x558fe2556430 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x558fe25283c0 0x558fe259b890 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x558fe2527aa0 0x558fe259b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b42000640 0 client.5871 ms_handle_reset on v2:172.21.15.78:6832/2934440940 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 --2- 172.21.15.78:0/2700991267 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x558fe24db340 0x558fe259ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 >> 172.21.15.78:0/2700991267 conn(0x558fe2508b60 msgr2=0x558fe250d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 shutdown_connections 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 1 -- 172.21.15.78:0/2700991267 wait complete. 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b62804000 20 client.5871 trim_cache size 0 max 16384 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 5745, 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:52:44.963 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 6.0000335109999998, 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 36, 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 58, 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 58, 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.495108965 2024-09-11T12:52:44.964 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 msgr2=0x7f4b38073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 0x7f4b38073cf0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b480015b0 tx=0x7f4b48009040 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 msgr2=0x7f4b5813f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4b5000c4d0 tx=0x7f4b5000c990 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 shutdown_connections 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4b38071830 0x7f4b38073cf0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b5815b3e0 0x7f4b5813f020 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b581437b0 0x7f4b5813dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 --2- 172.21.15.78:0/1895703718 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b58070040 0x7f4b581fb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 >> 172.21.15.78:0/1895703718 conn(0x7f4b580cfeb0 msgr2=0x7f4b580732b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:44.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 shutdown_connections 2024-09-11T12:52:44.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:44.957+0000 7f4b5de00640 1 -- 172.21.15.78:0/1895703718 wait complete. 2024-09-11T12:52:44.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:52:45.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- 172.21.15.78:0/309790511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 msgr2=0x7fa21815cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- 172.21.15.78:0/309790511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa21815cf00 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa2180b2c60 tx=0x7fa20801c770 comp rx=0 tx=0).stop 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- 172.21.15.78:0/309790511 shutdown_connections 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- 172.21.15.78:0/309790511 >> v2:172.21.15.157:3300/0 conn(0x7fa21815c5f0 0x7fa2180b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- 172.21.15.78:0/309790511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa21815cf00 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- 172.21.15.78:0/309790511 >> 172.21.15.78:0/309790511 conn(0x7fa21806ca80 msgr2=0x7fa2180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- 172.21.15.78:0/309790511 shutdown_connections 2024-09-11T12:52:45.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- 172.21.15.78:0/309790511 wait complete. 2024-09-11T12:52:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 Processor -- start 2024-09-11T12:52:45.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- start start 2024-09-11T12:52:45.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2180b4520 0x7fa218071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:45.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21815c5f0 0x7fa218155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:45.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2180cd380 con 0x7fa21815eda0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2180cd200 con 0x7fa21815c5f0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2180cd500 con 0x7fa2180b4520 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52450/0 (socket says 172.21.15.78:52450) 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 -- 172.21.15.78:0/1395520621 learned_addr learned my addr 172.21.15.78:0/1395520621 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa217e00640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2180b4520 0x7fa218071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa217400640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21815c5f0 0x7fa218155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 -- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2180b4520 msgr2=0x7fa218071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2180b4520 0x7fa218071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 -- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21815c5f0 msgr2=0x7fa218155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21815c5f0 0x7fa218155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.121+0000 7fa21d200640 1 -- 172.21.15.78:0/1395520621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa218073140 con 0x7fa21815eda0 2024-09-11T12:52:45.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa21d200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b970 tx=0x7fa21000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa210019070 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa21000cc50 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2180733e0 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2100146a0 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2180739a0 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2180b4950 con 0x7fa21815eda0 2024-09-11T12:52:45.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fa21001c070 con 0x7fa21815eda0 2024-09-11T12:52:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 0x7fa1f8073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa210091cf0 con 0x7fa21815eda0 2024-09-11T12:52:45.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.125+0000 7fa217e00640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 0x7fa1f8073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:45.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.129+0000 7fa217e00640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 0x7fa1f8073cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2080af000 tx=0x7fa20801afb0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:45.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.129+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa21005d3b0 con 0x7fa21815eda0 2024-09-11T12:52:45.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa218070820 con 0x7fa21815eda0 2024-09-11T12:52:45.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa20fe00640 1 -- 172.21.15.78:0/1395520621 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7fa210061900 con 0x7fa21815eda0 2024-09-11T12:52:45.334 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 msgr2=0x7fa1f8073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 0x7fa1f8073cf0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa2080af000 tx=0x7fa20801afb0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 msgr2=0x7fa2181507b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa21000b970 tx=0x7fa21000be30 comp rx=0 tx=0).stop 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 shutdown_connections 2024-09-11T12:52:45.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa1f8071830 0x7fa1f8073cf0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa21815eda0 0x7fa2181507b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa21815c5f0 0x7fa218155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 --2- 172.21.15.78:0/1395520621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2180b4520 0x7fa218071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.333+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 >> 172.21.15.78:0/1395520621 conn(0x7fa21806ca80 msgr2=0x7fa2180cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.337+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 shutdown_connections 2024-09-11T12:52:45.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45.337+0000 7fa21f200640 1 -- 172.21.15.78:0/1395520621 wait complete. 2024-09-11T12:52:45.354 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:52:45.354 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:52:45.355 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:52:45.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.355 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.365 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42269sec preferred_lft 42269sec 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.366 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:52:45.367 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:52:45.368 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:45.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.368 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:52:45.429 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:52:45.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.431 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:52:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:52:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:52:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:52:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:52:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:45.483 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:52:45.483 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:52:45.483 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:52:45.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:52:45.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:52:45.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:52:45.484 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:52:45.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:52:45.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:52:45.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.616 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:52:45.625 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:52:45.625 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:52:45.625 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:52:45.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.626 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:52:45.687 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:52:45.687 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:52:45.736 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:52:45.736 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:52:45.736 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:52:45.787 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:52:45.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.787 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:45.876 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:45.877 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:52:45.890 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:45.890 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:52:45.890 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:52:45.890 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:52:45.890 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:52:45.893 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:52:45.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:45.894 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:52:45.913 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:52:46.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:52:46.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:52:46.022 INFO:teuthology.orchestra.run.smithi078.stdout:client5318 2024-09-11T12:52:46.022 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:52:46.022 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:46.031 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:52:46.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 -- 172.21.15.78:0/181708634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 msgr2=0x7f255006ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 --2- 172.21.15.78:0/181708634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f255006ecf0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f254c009c80 tx=0x7f254c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 -- 172.21.15.78:0/181708634 shutdown_connections 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 --2- 172.21.15.78:0/181708634 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f255006ecf0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 --2- 172.21.15.78:0/181708634 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500b4780 0x7f2550073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 -- 172.21.15.78:0/181708634 >> 172.21.15.78:0/181708634 conn(0x7f255006ca80 msgr2=0x7f25500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 -- 172.21.15.78:0/181708634 shutdown_connections 2024-09-11T12:52:46.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.213+0000 7f2558800640 1 -- 172.21.15.78:0/181708634 wait complete. 2024-09-11T12:52:46.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 Processor -- start 2024-09-11T12:52:46.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- start start 2024-09-11T12:52:46.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25500b4780 0x7f2550146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500ccb20 0x7f2550158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2550168c40 con 0x7f255015bc60 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2550168ac0 con 0x7f25500b4780 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2550168dc0 con 0x7f25500ccb20 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52468/0 (socket says 172.21.15.78:52468) 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 -- 172.21.15.78:0/3018411623 learned_addr learned my addr 172.21.15.78:0/3018411623 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2555400640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500ccb20 0x7f2550158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2555e00640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25500b4780 0x7f2550146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 -- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500ccb20 msgr2=0x7f2550158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500ccb20 0x7f2550158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 -- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25500b4780 msgr2=0x7f2550146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:46.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25500b4780 0x7f2550146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 -- 172.21.15.78:0/3018411623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25501f96d0 con 0x7f255015bc60 2024-09-11T12:52:46.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2556800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f254c009bd0 tx=0x7f254c005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:46.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c0bc070 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f254c004130 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25501f8910 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c0b2040 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.217+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25501f8ed0 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2550073f70 con 0x7f255015bc60 2024-09-11T12:52:46.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f254c0bf070 con 0x7f255015bc60 2024-09-11T12:52:46.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f254a000640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 0x7f2534073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:46.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f254c12bfc0 con 0x7f255015bc60 2024-09-11T12:52:46.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f2555e00640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 0x7f2534073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:46.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.221+0000 7f2555e00640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 0x7f2534073cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2544009b70 tx=0x7f25440091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:46.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.225+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f254c016990 con 0x7f255015bc60 2024-09-11T12:52:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f255014bc90 con 0x7f255015bc60 2024-09-11T12:52:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f254a000640 1 -- 172.21.15.78:0/3018411623 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f254c0f7ba0 con 0x7f255015bc60 2024-09-11T12:52:46.425 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:52:46.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 msgr2=0x7f2534073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 0x7f2534073cf0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2544009b70 tx=0x7f25440091e0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 msgr2=0x7f25501592f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f254c009bd0 tx=0x7f254c005ae0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 shutdown_connections 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2534071830 0x7f2534073cf0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f255015bc60 0x7f25501592f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f25500ccb20 0x7f2550158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 --2- 172.21.15.78:0/3018411623 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f25500b4780 0x7f2550146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 >> 172.21.15.78:0/3018411623 conn(0x7f255006ca80 msgr2=0x7f25500701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:46.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 shutdown_connections 2024-09-11T12:52:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:46.425+0000 7f2558800640 1 -- 172.21.15.78:0/3018411623 wait complete. 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:52:46.447 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:52:46.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:52:46.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:52:46.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.448 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:52:46.458 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:52:46.458 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:52:46.458 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:52:46.458 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.458 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:52:46.459 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:52:46.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:52:46.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:52:46.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.462 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:52:46.527 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:52:46.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.530 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:52:46.584 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:52:46.584 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:52:46.584 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:52:46.585 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:52:46.585 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:52:46.585 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:52:46.585 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:52:46.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:52:46.586 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:52:46.586 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:52:46.684 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:52:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:52:46.727 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:52:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:52:46.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.734 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:52:46.743 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:52:46.743 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:52:46.744 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:52:46.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:52:46.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:52:46.808 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:52:46.864 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:52:46.864 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:52:46.864 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:52:46.915 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:52:46.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:46.916 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:47.009 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:47.009 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:52:47.021 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:52:47.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:52:47.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:52:47.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:52:47.022 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:52:47.025 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:52:47.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:47.026 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:52:47.044 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:52:47.134 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:52:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:52:47.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:52:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:52:47.144 INFO:teuthology.orchestra.run.smithi157.stdout:client5686 2024-09-11T12:52:47.144 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:52:47.144 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:52:47.153 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:52:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 -- 172.21.15.78:0/1171392032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054151050 msgr2=0x7f0054073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 --2- 172.21.15.78:0/1171392032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054151050 0x7f0054073950 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f00541581d0 tx=0x7f004401da80 comp rx=0 tx=0).stop 2024-09-11T12:52:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 -- 172.21.15.78:0/1171392032 shutdown_connections 2024-09-11T12:52:47.300 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 --2- 172.21.15.78:0/1171392032 >> v2:172.21.15.157:3300/0 conn(0x7f0054072520 0x7f005406f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 --2- 172.21.15.78:0/1171392032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054151050 0x7f0054073950 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 -- 172.21.15.78:0/1171392032 >> 172.21.15.78:0/1171392032 conn(0x7f00540b3050 msgr2=0x7f00540b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 -- 172.21.15.78:0/1171392032 shutdown_connections 2024-09-11T12:52:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.297+0000 7f005ae00640 1 -- 172.21.15.78:0/1171392032 wait complete. 2024-09-11T12:52:47.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 Processor -- start 2024-09-11T12:52:47.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 -- start start 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0054073e90 0x7f00541f8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054151050 0x7f00540b02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00540b58c0 con 0x7f0054072520 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00540b5740 con 0x7f0054151050 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f00540b5a40 con 0x7f0054073e90 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52480/0 (socket says 172.21.15.78:52480) 2024-09-11T12:52:47.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 -- 172.21.15.78:0/2528834147 learned_addr learned my addr 172.21.15.78:0/2528834147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053400640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0054073e90 0x7f00541f8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0058e00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054151050 0x7f00540b02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 -- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0054073e90 msgr2=0x7f00541f8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0054073e90 0x7f00541f8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 -- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054151050 msgr2=0x7f00540b02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054151050 0x7f00540b02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 -- 172.21.15.78:0/2528834147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00541f90d0 con 0x7f0054072520 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f0053e00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f004401b6b0 tx=0x7f0044003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00440b3070 con 0x7f0054072520 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0044003e10 con 0x7f0054072520 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0044005d60 con 0x7f0054072520 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.301+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00540ce870 con 0x7f0054072520 2024-09-11T12:52:47.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00540ada80 con 0x7f0054072520 2024-09-11T12:52:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00540ad870 con 0x7f0054072520 2024-09-11T12:52:47.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f00440be060 con 0x7f0054072520 2024-09-11T12:52:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f004be00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f004412c670 con 0x7f0054072520 2024-09-11T12:52:47.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f0053400640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:47.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.305+0000 7f0053400640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0040004620 tx=0x7f004000c040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:47.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.309+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0044017700 con 0x7f0054072520 2024-09-11T12:52:47.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:47.509+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f00540ced90 con 0x7f0054072520 2024-09-11T12:52:48.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.009+0000 7f004be00640 1 -- 172.21.15.78:0/2528834147 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f00440f8250 con 0x7f0054072520 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 msgr2=0x7f0034073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0040004620 tx=0x7f004000c040 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 msgr2=0x7f0054149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f004401b6b0 tx=0x7f0044003c20 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 shutdown_connections 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0034071830 0x7f0034073cf0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0054151050 0x7f00540b02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0054073e90 0x7f00541f8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 --2- 172.21.15.78:0/2528834147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0054072520 0x7f0054149890 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 >> 172.21.15.78:0/2528834147 conn(0x7f00540b3050 msgr2=0x7f00541e8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 shutdown_connections 2024-09-11T12:52:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.013+0000 7f005ae00640 1 -- 172.21.15.78:0/2528834147 wait complete. 2024-09-11T12:52:48.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 -- 172.21.15.78:0/4044142509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 msgr2=0x7f2a4013efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 --2- 172.21.15.78:0/4044142509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a4013efa0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f2a400adf30 tx=0x7f2a3001c830 comp rx=0 tx=0).stop 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 -- 172.21.15.78:0/4044142509 shutdown_connections 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 --2- 172.21.15.78:0/4044142509 >> v2:172.21.15.157:3300/0 conn(0x7f2a40160180 0x7f2a401413d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 --2- 172.21.15.78:0/4044142509 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a4013efa0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 -- 172.21.15.78:0/4044142509 >> 172.21.15.78:0/4044142509 conn(0x7f2a400746c0 msgr2=0x7f2a400ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 -- 172.21.15.78:0/4044142509 shutdown_connections 2024-09-11T12:52:48.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.173+0000 7f2a3f200640 1 -- 172.21.15.78:0/4044142509 wait complete. 2024-09-11T12:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 Processor -- start 2024-09-11T12:52:48.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- start start 2024-09-11T12:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 0x7f2a4013be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 0x7f2a40156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a40168850 con 0x7f2a4015f930 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a401686d0 con 0x7f2a40160180 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a401689d0 con 0x7f2a4015fd10 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52496/0 (socket says 172.21.15.78:52496) 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 -- 172.21.15.78:0/2585391046 learned_addr learned my addr 172.21.15.78:0/2585391046 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3e800640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 0x7f2a40156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a35400640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 0x7f2a4013be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 -- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 msgr2=0x7f2a4013be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 0x7f2a4013be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 -- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 msgr2=0x7f2a40156e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 0x7f2a40156e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 -- 172.21.15.78:0/2585391046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a40159490 con 0x7f2a4015f930 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3e800640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 0x7f2a40156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a35400640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 0x7f2a4013be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3de00640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a30019440 tx=0x7f2a300a6770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30004200 con 0x7f2a4015f930 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a300bc070 con 0x7f2a4015f930 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a401566b0 con 0x7f2a4015f930 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a300044f0 con 0x7f2a4015f930 2024-09-11T12:52:48.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.177+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a401591b0 con 0x7f2a4015f930 2024-09-11T12:52:48.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a401413d0 con 0x7f2a4015f930 2024-09-11T12:52:48.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f2a300052d0 con 0x7f2a4015f930 2024-09-11T12:52:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a36a00640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 0x7f2a1c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2a3012cad0 con 0x7f2a4015f930 2024-09-11T12:52:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a35400640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 0x7f2a1c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.181+0000 7f2a35400640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 0x7f2a1c073cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006db40 tx=0x7f2a28009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.185+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2a300164b0 con 0x7f2a4015f930 2024-09-11T12:52:48.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.373+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f2a401407c0 con 0x7f2a1c071830 2024-09-11T12:52:48.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.373+0000 7f2a36a00640 1 -- 172.21.15.78:0/2585391046 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f2a401407c0 con 0x7f2a1c071830 2024-09-11T12:52:48.375 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:52:48.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:48.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:52:48.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:48.376 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 msgr2=0x7f2a1c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 0x7f2a1c073cf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2a4006db40 tx=0x7f2a28009040 comp rx=0 tx=0).stop 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 msgr2=0x7f2a40158880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f2a30019440 tx=0x7f2a300a6770 comp rx=0 tx=0).stop 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 shutdown_connections 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a1c071830 0x7f2a1c073cf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a40160180 0x7f2a40156e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a4015fd10 0x7f2a4013be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 --2- 172.21.15.78:0/2585391046 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a4015f930 0x7f2a40158880 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 >> 172.21.15.78:0/2585391046 conn(0x7f2a400746c0 msgr2=0x7f2a400d0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 shutdown_connections 2024-09-11T12:52:48.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.377+0000 7f2a3f200640 1 -- 172.21.15.78:0/2585391046 wait complete. 2024-09-11T12:52:48.397 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- 172.21.15.78:0/2659876839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440aa240 msgr2=0x7f74440702a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- 172.21.15.78:0/2659876839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440aa240 0x7f74440702a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f74440b1a70 tx=0x7f743c01ccc0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- 172.21.15.78:0/2659876839 shutdown_connections 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- 172.21.15.78:0/2659876839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74440aa240 0x7f74440702a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- 172.21.15.78:0/2659876839 >> v2:172.21.15.157:3300/0 conn(0x7f74440d0bd0 0x7f74440a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- 172.21.15.78:0/2659876839 >> 172.21.15.78:0/2659876839 conn(0x7f7444075610 msgr2=0x7f7444075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- 172.21.15.78:0/2659876839 shutdown_connections 2024-09-11T12:52:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- 172.21.15.78:0/2659876839 wait complete. 2024-09-11T12:52:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 Processor -- start 2024-09-11T12:52:48.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- start start 2024-09-11T12:52:48.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440b12f0 0x7f7444075bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74440d0bd0 0x7f74441fbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7444169010 con 0x7f7444150250 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7444168e90 con 0x7f74440d0bd0 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7444169190 con 0x7f74440b12f0 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52516/0 (socket says 172.21.15.78:52516) 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 -- 172.21.15.78:0/2906032880 learned_addr learned my addr 172.21.15.78:0/2906032880 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f7449a00640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440b12f0 0x7f7444075bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f7449000640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74440d0bd0 0x7f74441fbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 -- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440b12f0 msgr2=0x7f7444075bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440b12f0 0x7f7444075bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 -- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74440d0bd0 msgr2=0x7f74441fbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74440d0bd0 0x7f74441fbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.541+0000 7f744a400640 1 -- 172.21.15.78:0/2906032880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74441fc1b0 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f744a400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f743c009bd0 tx=0x7f743c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0bb070 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f743c004e00 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74441f7d80 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0b2040 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441f8340 con 0x7f7444150250 2024-09-11T12:52:48.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74440b1940 con 0x7f7444150250 2024-09-11T12:52:48.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f743c0be070 con 0x7f7444150250 2024-09-11T12:52:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 0x7f7424073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f743c12bd30 con 0x7f7444150250 2024-09-11T12:52:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.545+0000 7f7449a00640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 0x7f7424073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:48.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.549+0000 7f7449a00640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 0x7f7424073d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7438007fd0 tx=0x7f7438007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:48.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.549+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f74441fc1b0 con 0x7f7444150250 2024-09-11T12:52:48.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f74440702a0 con 0x7f7444150250 2024-09-11T12:52:48.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f7441600640 1 -- 172.21.15.78:0/2906032880 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f743c016970 con 0x7f7444150250 2024-09-11T12:52:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 msgr2=0x7f7424073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 0x7f7424073d20 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7438007fd0 tx=0x7f7438007e40 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 msgr2=0x7f744406d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f743c009bd0 tx=0x7f743c003f60 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 shutdown_connections 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7424071860 0x7f7424073d20 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7444150250 0x7f744406d6a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74440d0bd0 0x7f74441fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 --2- 172.21.15.78:0/2906032880 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74440b12f0 0x7f7444075bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 >> 172.21.15.78:0/2906032880 conn(0x7f7444075610 msgr2=0x7f744415bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 shutdown_connections 2024-09-11T12:52:48.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:48.717+0000 7f744c400640 1 -- 172.21.15.78:0/2906032880 wait complete. 2024-09-11T12:52:48.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:48.739 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol94520) 2024-09-11T12:52:48.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:48.766 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:52:48.817 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:52:48.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:48.818 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol94520) 2024-09-11T12:52:48.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol18661 2024-09-11T12:52:49.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 -- 172.21.15.78:0/2617913340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a54004810 msgr2=0x7f2a54004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 --2- 172.21.15.78:0/2617913340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a5800c2e0 tx=0x7f2a5801cf20 comp rx=0 tx=0).stop 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 -- 172.21.15.78:0/2617913340 shutdown_connections 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 --2- 172.21.15.78:0/2617913340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a54004810 0x7f2a54004c90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 -- 172.21.15.78:0/2617913340 >> 172.21.15.78:0/2617913340 conn(0x7f2a70071dd0 msgr2=0x7f2a700721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 -- 172.21.15.78:0/2617913340 shutdown_connections 2024-09-11T12:52:49.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 -- 172.21.15.78:0/2617913340 wait complete. 2024-09-11T12:52:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.081+0000 7f2a77000640 1 Processor -- start 2024-09-11T12:52:49.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- start start 2024-09-11T12:52:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a54004810 0x7f2a701f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a701448f0 0x7f2a70154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70141f00 con 0x7f2a70143720 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2a70141d80 con 0x7f2a701448f0 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2a700b1a10 con 0x7f2a54004810 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52520/0 (socket says 172.21.15.78:52520) 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 -- 172.21.15.78:0/2610789589 learned_addr learned my addr 172.21.15.78:0/2610789589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a6fe00640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a54004810 0x7f2a701f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a67e00640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a701448f0 0x7f2a70154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 -- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a54004810 msgr2=0x7f2a701f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a54004810 0x7f2a701f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 -- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a701448f0 msgr2=0x7f2a70154f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a701448f0 0x7f2a70154f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 -- 172.21.15.78:0/2610789589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a701f8710 con 0x7f2a70143720 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a75000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a70153010 tx=0x7f2a68010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ebd0 con 0x7f2a70143720 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a6800ee60 con 0x7f2a70143720 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a701530e0 con 0x7f2a70143720 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a680126b0 con 0x7f2a70143720 2024-09-11T12:52:49.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a701536a0 con 0x7f2a70143720 2024-09-11T12:52:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.085+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a700b1f70 con 0x7f2a70143720 2024-09-11T12:52:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.089+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f2a6801a020 con 0x7f2a70143720 2024-09-11T12:52:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.089+0000 7f2a6cc00640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 0x7f2a4c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.089+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2a68092b20 con 0x7f2a70143720 2024-09-11T12:52:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.089+0000 7f2a6fe00640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 0x7f2a4c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.089+0000 7f2a6fe00640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 0x7f2a4c073cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a60006fd0 tx=0x7f2a60006b80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:49.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.093+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2a701f8710 con 0x7f2a70143720 2024-09-11T12:52:49.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.281+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol18661", "target": ["mon-mgr", ""]}) -- 0x7f2a701440e0 con 0x7f2a4c071830 2024-09-11T12:52:49.290 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:49.286+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:49.290 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:49.286+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:49.290 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:49.286+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:49.290 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:49.286+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:49.290 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:49.286+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:49.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.325+0000 7f2a6cc00640 1 -- 172.21.15.78:0/2610789589 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2a600126e0 con 0x7f2a4c071830 2024-09-11T12:52:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 msgr2=0x7f2a4c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 0x7f2a4c073cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2a60006fd0 tx=0x7f2a60006b80 comp rx=0 tx=0).stop 2024-09-11T12:52:49.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 msgr2=0x7f2a70143ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2a70153010 tx=0x7f2a68010ce0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 shutdown_connections 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2a4c071830 0x7f2a4c073cf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2a70143720 0x7f2a70143ba0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2a701448f0 0x7f2a70154f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 --2- 172.21.15.78:0/2610789589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2a54004810 0x7f2a701f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 >> 172.21.15.78:0/2610789589 conn(0x7f2a70071dd0 msgr2=0x7f2a70069180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 shutdown_connections 2024-09-11T12:52:49.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.329+0000 7f2a77000640 1 -- 172.21.15.78:0/2610789589 wait complete. 2024-09-11T12:52:49.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:49.349 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol18661/.meta ./volumes/_nogroup/subvol94520) 2024-09-11T12:52:49.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol94520 2024-09-11T12:52:49.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- 172.21.15.78:0/2406209137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53f4003c20 msgr2=0x7f53f4004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 --2- 172.21.15.78:0/2406209137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53f4003c20 0x7f53f4004020 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5400001f50 tx=0x7f540001cc40 comp rx=0 tx=0).stop 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f540e800640 1 -- 172.21.15.78:0/2406209137 reap_dead start 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- 172.21.15.78:0/2406209137 shutdown_connections 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- 172.21.15.78:0/2406209137 >> 172.21.15.78:0/2406209137 conn(0x7f5410071dd0 msgr2=0x7f54100721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- 172.21.15.78:0/2406209137 shutdown_connections 2024-09-11T12:52:49.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- 172.21.15.78:0/2406209137 wait complete. 2024-09-11T12:52:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 Processor -- start 2024-09-11T12:52:49.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- start start 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 0x7f5410137510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5410134610 0x7f541006f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54100aeeb0 con 0x7f541006f900 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54100aed30 con 0x7f53f4003c20 2024-09-11T12:52:49.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.621+0000 7f5415000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54100af030 con 0x7f5410134610 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52544/0 (socket says 172.21.15.78:52544) 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 -- 172.21.15.78:0/2071681520 learned_addr learned my addr 172.21.15.78:0/2071681520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540de00640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 0x7f5410137510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540d400640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5410134610 0x7f541006f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 -- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5410134610 msgr2=0x7f541006f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5410134610 0x7f541006f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 -- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 msgr2=0x7f5410137510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 0x7f5410137510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 -- 172.21.15.78:0/2071681520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54101fb190 con 0x7f541006f900 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540de00640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 0x7f5410137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:49.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f540e800640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f54100af9c0 tx=0x7f540800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5408004300 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54080044a0 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54101fb470 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f540800daf0 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54101fba30 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54100cd770 con 0x7f541006f900 2024-09-11T12:52:49.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.625+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7f540800dc90 con 0x7f541006f900 2024-09-11T12:52:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.629+0000 7f5406000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 0x7f53f0073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.629+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5408092700 con 0x7f541006f900 2024-09-11T12:52:49.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.629+0000 7f540de00640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 0x7f53f0073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:49.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.629+0000 7f540de00640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 0x7f53f0073d20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f54000a9000 tx=0x7f5400019a80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:49.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.629+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f540805e140 con 0x7f541006f900 2024-09-11T12:52:49.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.821+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol94520", "target": ["mon-mgr", ""]}) -- 0x7f54100cc830 con 0x7f53f0071860 2024-09-11T12:52:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5406000640 1 -- 172.21.15.78:0/2071681520 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f54100cc830 con 0x7f53f0071860 2024-09-11T12:52:49.858 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol94520 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 msgr2=0x7f53f0073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 0x7f53f0073d20 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f54000a9000 tx=0x7f5400019a80 comp rx=0 tx=0).stop 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 msgr2=0x7f5410155eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f54100af9c0 tx=0x7f540800ee80 comp rx=0 tx=0).stop 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 shutdown_connections 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f53f0071860 0x7f53f0073d20 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f541006f900 0x7f5410155eb0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5410134610 0x7f541006f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 --2- 172.21.15.78:0/2071681520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f53f4003c20 0x7f5410137510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 >> 172.21.15.78:0/2071681520 conn(0x7f5410071dd0 msgr2=0x7f54100adcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 shutdown_connections 2024-09-11T12:52:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:49.857+0000 7f5415000640 1 -- 172.21.15.78:0/2071681520 wait complete. 2024-09-11T12:52:49.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol94520 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- 172.21.15.78:0/3309220149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0101526f0 msgr2=0x7fa0100cff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 --2- 172.21.15.78:0/3309220149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0101526f0 0x7fa0100cff30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa000009900 tx=0x7fa00001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- 172.21.15.78:0/3309220149 shutdown_connections 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 --2- 172.21.15.78:0/3309220149 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0101526f0 0x7fa0100cff30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- 172.21.15.78:0/3309220149 >> 172.21.15.78:0/3309220149 conn(0x7fa010075d50 msgr2=0x7fa0100743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:50.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- 172.21.15.78:0/3309220149 shutdown_connections 2024-09-11T12:52:50.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- 172.21.15.78:0/3309220149 wait complete. 2024-09-11T12:52:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 Processor -- start 2024-09-11T12:52:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.025+0000 7fa015a00640 1 -- start start 2024-09-11T12:52:50.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0101526f0 0x7fa01013b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 0x7fa01014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa01016a0c0 con 0x7fa0100a9b70 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa010169f40 con 0x7fa0101526f0 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa01016a240 con 0x7fa0101f84c0 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52572/0 (socket says 172.21.15.78:52572) 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 -- 172.21.15.78:0/1738937147 learned_addr learned my addr 172.21.15.78:0/1738937147 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa007e00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0101526f0 0x7fa01013b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015000640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 0x7fa01014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:50.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 -- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 msgr2=0x7fa01014d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 0x7fa01014d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 -- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0101526f0 msgr2=0x7fa01013b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0101526f0 0x7fa01013b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 -- 172.21.15.78:0/1738937147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0101fbcc0 con 0x7fa0100a9b70 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015000640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 0x7fa01014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00fe00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa000009850 tx=0x7fa0000a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0000b2040 con 0x7fa0100a9b70 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0000044b0 con 0x7fa0100a9b70 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa000003860 con 0x7fa0100a9b70 2024-09-11T12:52:50.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa01014b1e0 con 0x7fa0100a9b70 2024-09-11T12:52:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0101f7cc0 con 0x7fa0100a9b70 2024-09-11T12:52:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.029+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0101f77d0 con 0x7fa0100a9b70 2024-09-11T12:52:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.033+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 8) ==== 95085+0+0 (secure 0 0 0) 0x7fa000003a00 con 0x7fa0100a9b70 2024-09-11T12:52:50.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.033+0000 7fa00cc00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 0x7f9fec073d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.033+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa00012be30 con 0x7fa0100a9b70 2024-09-11T12:52:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.033+0000 7fa007e00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 0x7f9fec073d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:50.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.033+0000 7fa007e00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 0x7f9fec073d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa01014ade0 tx=0x7f9ff8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:50.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.037+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa0000be060 con 0x7fa0100a9b70 2024-09-11T12:52:50.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.233+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol94520", "target": ["mon-mgr", ""]}) -- 0x7fa0100b1b50 con 0x7f9fec071860 2024-09-11T12:52:50.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.237+0000 7fa00cc00640 1 -- 172.21.15.78:0/1738937147 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fa0100b1b50 con 0x7f9fec071860 2024-09-11T12:52:50.238 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol94520 2024-09-11T12:52:50.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.237+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 msgr2=0x7f9fec073d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.237+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 0x7f9fec073d20 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa01014ade0 tx=0x7f9ff8009040 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.237+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 msgr2=0x7fa0101fb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.237+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa000009850 tx=0x7fa0000a5750 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 shutdown_connections 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9fec071860 0x7f9fec073d20 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0101f84c0 0x7fa01014d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0101526f0 0x7fa01013b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 --2- 172.21.15.78:0/1738937147 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0100a9b70 0x7fa0101fb780 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 >> 172.21.15.78:0/1738937147 conn(0x7fa01006fbd0 msgr2=0x7fa0100b0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 shutdown_connections 2024-09-11T12:52:50.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:50.241+0000 7fa015a00640 1 -- 172.21.15.78:0/1738937147 wait complete. 2024-09-11T12:52:50.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:52:50.260 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/1e4eee37cd194de7df819d44947fb59f.meta) 2024-09-11T12:52:54.473 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:52:54.473 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:52:54.473 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:52:54.473 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol94520 2024-09-11T12:52:54.473 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:52:54.474 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:54.476 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol94520 alice 2024-09-11T12:52:54.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- 172.21.15.78:0/911744279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca540051d0 msgr2=0x7fca5400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 --2- 172.21.15.78:0/911744279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca540051d0 0x7fca5400b9d0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fca68009b40 tx=0x7fca6801d290 comp rx=0 tx=0).stop 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- 172.21.15.78:0/911744279 shutdown_connections 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 --2- 172.21.15.78:0/911744279 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca540051d0 0x7fca5400b9d0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- 172.21.15.78:0/911744279 >> 172.21.15.78:0/911744279 conn(0x7fca700b3050 msgr2=0x7fca700b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- 172.21.15.78:0/911744279 shutdown_connections 2024-09-11T12:52:54.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- 172.21.15.78:0/911744279 wait complete. 2024-09-11T12:52:54.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 Processor -- start 2024-09-11T12:52:54.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 -- start start 2024-09-11T12:52:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.632+0000 7fca76c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca540051d0 0x7fca700b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:54.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca70155d20 0x7fca701f7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca700cfb10 con 0x7fca700af6a0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca700cf990 con 0x7fca540051d0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca700cfc90 con 0x7fca70155d20 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49408/0 (socket says 172.21.15.78:49408) 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 -- 172.21.15.78:0/2394012113 learned_addr learned my addr 172.21.15.78:0/2394012113 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca6fe00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca540051d0 0x7fca700b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca74c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca70155d20 0x7fca701f7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 -- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca70155d20 msgr2=0x7fca701f7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca70155d20 0x7fca701f7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 -- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca540051d0 msgr2=0x7fca700b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca540051d0 0x7fca700b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 -- 172.21.15.78:0/2394012113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca701f5310 con 0x7fca700af6a0 2024-09-11T12:52:54.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca67400640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e510 tx=0x7fca5800e9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58015070 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca58010980 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca58010d90 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca700afc20 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca700b01e0 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca700b5840 con 0x7fca700af6a0 2024-09-11T12:52:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.636+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 95136+0+0 (secure 0 0 0) 0x7fca58018070 con 0x7fca700af6a0 2024-09-11T12:52:54.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.640+0000 7fca6cc00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 0x7fca4c073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.640+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fca580924f0 con 0x7fca700af6a0 2024-09-11T12:52:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.640+0000 7fca6fe00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 0x7fca4c073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:54.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.640+0000 7fca6fe00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 0x7fca4c073db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca60000f90 tx=0x7fca60006f00 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:54.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.644+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fca701f5310 con 0x7fca700af6a0 2024-09-11T12:52:54.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.832+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol94520", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7fca7015dd70 con 0x7fca4c0718f0 2024-09-11T12:52:54.845 INFO:tasks.ceph.mgr.x.smithi157.stderr:Creating meta for ID alice with tenant None 2024-09-11T12:52:54.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.884+0000 7fca6cc00640 1 -- 172.21.15.78:0/2394012113 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca7015dd70 con 0x7fca4c0718f0 2024-09-11T12:52:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 msgr2=0x7fca4c073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:54.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 0x7fca4c073db0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fca60000f90 tx=0x7fca60006f00 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 msgr2=0x7fca701f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fca5800e510 tx=0x7fca5800e9d0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 shutdown_connections 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca4c0718f0 0x7fca4c073db0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca70155d20 0x7fca701f7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca700af6a0 0x7fca701f4dd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 --2- 172.21.15.78:0/2394012113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca540051d0 0x7fca700b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 >> 172.21.15.78:0/2394012113 conn(0x7fca700b3050 msgr2=0x7fca700cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 shutdown_connections 2024-09-11T12:52:54.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:54.888+0000 7fca76c00640 1 -- 172.21.15.78:0/2394012113 wait complete. 2024-09-11T12:52:54.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-11T12:52:55.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- 172.21.15.78:0/631447378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa26415e3e0 msgr2=0x7fa26415b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- 172.21.15.78:0/631447378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa26415e3e0 0x7fa26415b420 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009900 tx=0x7fa24c01c820 comp rx=0 tx=0).stop 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- 172.21.15.78:0/631447378 shutdown_connections 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- 172.21.15.78:0/631447378 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa26415e3e0 0x7fa26415b420 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- 172.21.15.78:0/631447378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640cf520 0x7fa264071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- 172.21.15.78:0/631447378 >> 172.21.15.78:0/631447378 conn(0x7fa2640cfeb0 msgr2=0x7fa2640d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- 172.21.15.78:0/631447378 shutdown_connections 2024-09-11T12:52:55.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- 172.21.15.78:0/631447378 wait complete. 2024-09-11T12:52:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 Processor -- start 2024-09-11T12:52:55.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- start start 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26415b960 0x7fa264155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26415e3e0 0x7fa2640af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa264151240 con 0x7fa2640cf520 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2641510c0 con 0x7fa26415b960 2024-09-11T12:52:55.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa269000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2641513c0 con 0x7fa26415e3e0 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49426/0 (socket says 172.21.15.78:49426) 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 -- 172.21.15.78:0/2043576238 learned_addr learned my addr 172.21.15.78:0/2043576238 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa262a00640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26415b960 0x7fa264155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263e00640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26415e3e0 0x7fa2640af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 -- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26415e3e0 msgr2=0x7fa2640af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26415e3e0 0x7fa2640af3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 -- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26415b960 msgr2=0x7fa264155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26415b960 0x7fa264155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 -- 172.21.15.78:0/2043576238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2641558b0 con 0x7fa2640cf520 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.056+0000 7fa263400640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa25400ba80 tx=0x7fa25400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254019070 con 0x7fa2640cf520 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa25400cd10 con 0x7fa2640cf520 2024-09-11T12:52:55.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2540146a0 con 0x7fa2640cf520 2024-09-11T12:52:55.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2641529a0 con 0x7fa2640cf520 2024-09-11T12:52:55.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa264152f60 con 0x7fa2640cf520 2024-09-11T12:52:55.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa264071d60 con 0x7fa2640cf520 2024-09-11T12:52:55.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 95136+0+0 (secure 0 0 0) 0x7fa25401c070 con 0x7fa2640cf520 2024-09-11T12:52:55.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 0x7fa240073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fa2540920d0 con 0x7fa2640cf520 2024-09-11T12:52:55.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.060+0000 7fa262a00640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 0x7fa240073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.064+0000 7fa262a00640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 0x7fa240073d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0af000 tx=0x7fa24c01b060 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.064+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa25405db40 con 0x7fa2640cf520 2024-09-11T12:52:55.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.288+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7fa264069ab0 con 0x7fa2640cf520 2024-09-11T12:52:55.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa25b400640 1 -- 172.21.15.78:0/2043576238 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v16) ==== 107+0+280 (secure 0 0 0) 0x7fa264069ab0 con 0x7fa2640cf520 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.alice", 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCmkuFm1rZtMxAAfCphqJqGylhLBu2J2ZZ5sg==", 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol94520", 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:52:55.293 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:52:55.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 msgr2=0x7fa240073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 0x7fa240073d80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0af000 tx=0x7fa24c01b060 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 msgr2=0x7fa2641fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa25400ba80 tx=0x7fa25400bf40 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 shutdown_connections 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa2400718c0 0x7fa240073d80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa26415e3e0 0x7fa2640af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa26415b960 0x7fa264155370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 --2- 172.21.15.78:0/2043576238 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640cf520 0x7fa2641fb160 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 >> 172.21.15.78:0/2043576238 conn(0x7fa2640cfeb0 msgr2=0x7fa264158e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 shutdown_connections 2024-09-11T12:52:55.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.292+0000 7fa269000640 1 -- 172.21.15.78:0/2043576238 wait complete. 2024-09-11T12:52:55.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol94520 2024-09-11T12:52:55.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- 172.21.15.78:0/2597196349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3140051d0 msgr2=0x7fd31400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 --2- 172.21.15.78:0/2597196349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3140051d0 0x7fd31400b9d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd3280016f0 tx=0x7fd32801c7b0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- 172.21.15.78:0/2597196349 shutdown_connections 2024-09-11T12:52:55.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 --2- 172.21.15.78:0/2597196349 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3140051d0 0x7fd31400b9d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- 172.21.15.78:0/2597196349 >> 172.21.15.78:0/2597196349 conn(0x7fd330071dd0 msgr2=0x7fd3300721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:55.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- 172.21.15.78:0/2597196349 shutdown_connections 2024-09-11T12:52:55.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- 172.21.15.78:0/2597196349 wait complete. 2024-09-11T12:52:55.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 Processor -- start 2024-09-11T12:52:55.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.456+0000 7fd335000640 1 -- start start 2024-09-11T12:52:55.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3140051d0 0x7fd3301f1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 0x7fd3300adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b1b90 con 0x7fd3301502c0 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd3300b1a10 con 0x7fd3300adb10 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd3300b1d10 con 0x7fd3140051d0 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49448/0 (socket says 172.21.15.78:49448) 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 -- 172.21.15.78:0/2457433077 learned_addr learned my addr 172.21.15.78:0/2457433077 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32d400640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 0x7fd3300adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32de00640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3140051d0 0x7fd3301f1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 -- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3140051d0 msgr2=0x7fd3301f1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3140051d0 0x7fd3301f1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 -- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 msgr2=0x7fd3300adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 0x7fd3300adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 -- 172.21.15.78:0/2457433077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd330070b00 con 0x7fd3301502c0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32d400640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 0x7fd3300adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd32e800640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd32801d050 tx=0x7fd3280a6be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328001310 con 0x7fd3301502c0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd328003810 con 0x7fd3301502c0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd328006930 con 0x7fd3301502c0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3301f4640 con 0x7fd3301502c0 2024-09-11T12:52:55.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3301f4c00 con 0x7fd3301502c0 2024-09-11T12:52:55.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.460+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3300ae4d0 con 0x7fd3301502c0 2024-09-11T12:52:55.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.464+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 95136+0+0 (secure 0 0 0) 0x7fd328003c20 con 0x7fd3301502c0 2024-09-11T12:52:55.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.464+0000 7fd326000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 0x7fd310073d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.464+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd32812c4a0 con 0x7fd3301502c0 2024-09-11T12:52:55.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.464+0000 7fd32de00640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 0x7fd310073d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.464+0000 7fd32de00640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 0x7fd310073d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd33015a610 tx=0x7fd320009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.468+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd328016510 con 0x7fd3301502c0 2024-09-11T12:52:55.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.656+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol94520", "target": ["mon-mgr", ""]}) -- 0x7fd330150cb0 con 0x7fd3100718c0 2024-09-11T12:52:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd326000640 1 -- 172.21.15.78:0/2457433077 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd320004610 con 0x7fd3100718c0 2024-09-11T12:52:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 msgr2=0x7fd310073d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 0x7fd310073d80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd33015a610 tx=0x7fd320009500 comp rx=0 tx=0).stop 2024-09-11T12:52:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 msgr2=0x7fd330150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd32801d050 tx=0x7fd3280a6be0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 shutdown_connections 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fd3100718c0 0x7fd310073d80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3301502c0 0x7fd330150770 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300adb10 0x7fd3300adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 --2- 172.21.15.78:0/2457433077 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3140051d0 0x7fd3301f1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 >> 172.21.15.78:0/2457433077 conn(0x7fd330071dd0 msgr2=0x7fd3300703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 shutdown_connections 2024-09-11T12:52:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.688+0000 7fd335000640 1 -- 172.21.15.78:0/2457433077 wait complete. 2024-09-11T12:52:55.693 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.690+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.693 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.690+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.694 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.690+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.694 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.690+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.694 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.690+0000 7fa58e800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.710 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.706+0000 7fa58d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.710 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.706+0000 7fa58d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.710 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.706+0000 7fa58d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.710 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.706+0000 7fa58d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.710 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:52:55.706+0000 7fa58d400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:52:55.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol18661 2024-09-11T12:52:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- 172.21.15.78:0/3518349587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad815ca30 msgr2=0x7f1ad81416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 --2- 172.21.15.78:0/3518349587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad815ca30 0x7f1ad81416e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0009bc0 tx=0x7f1ad001cae0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- 172.21.15.78:0/3518349587 shutdown_connections 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 --2- 172.21.15.78:0/3518349587 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad815ca30 0x7f1ad81416e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 --2- 172.21.15.78:0/3518349587 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc5d0 0x7f1ad80b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- 172.21.15.78:0/3518349587 >> 172.21.15.78:0/3518349587 conn(0x7f1ad8071dd0 msgr2=0x7f1ad80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- 172.21.15.78:0/3518349587 shutdown_connections 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- 172.21.15.78:0/3518349587 wait complete. 2024-09-11T12:52:55.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 Processor -- start 2024-09-11T12:52:55.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.856+0000 7f1ade000640 1 -- start start 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 0x7f1ad8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 0x7f1ad80cd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8168ef0 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1ad8168d70 con 0x7f1ad815ca30 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1ad8169070 con 0x7f1ad80cc1f0 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49466/0 (socket says 172.21.15.78:49466) 2024-09-11T12:52:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 -- 172.21.15.78:0/4274909454 learned_addr learned my addr 172.21.15.78:0/4274909454 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ad7400640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 0x7f1ad8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ad7e00640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 0x7f1ad80cd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 -- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 msgr2=0x7f1ad8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 0x7f1ad8151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 -- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 msgr2=0x7f1ad80cd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 0x7f1ad80cd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 -- 172.21.15.78:0/4274909454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ad81f9650 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ad7400640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 0x7f1ad8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ad7e00640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 0x7f1ad80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acfe00640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e940 tx=0x7f1ac000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac000cd80 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac0005020 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ad8153b90 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0005450 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ad8154150 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.860+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ad8154350 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.864+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 9) ==== 95136+0+0 (secure 0 0 0) 0x7f1ac00047b0 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.864+0000 7f1acf400640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 0x7f1ab4073db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:52:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.864+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1ac0092c90 con 0x7f1ad80cc5d0 2024-09-11T12:52:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.864+0000 7f1ad7400640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 0x7f1ab4073db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:52:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.864+0000 7f1ad7400640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 0x7f1ab4073db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8000f70 tx=0x7f1ac8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:52:55.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:55.868+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1ad8154350 con 0x7f1ad80cc5d0 2024-09-11T12:52:56.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:56.060+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol18661", "target": ["mon-mgr", ""]}) -- 0x7f1ad80b2de0 con 0x7f1ab40718f0 2024-09-11T12:52:58.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:52:58.052+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f1ac005a2b0 con 0x7f1ad80cc5d0 2024-09-11T12:53:04.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.480+0000 7f1acf400640 1 -- 172.21.15.78:0/4274909454 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ad80b2de0 con 0x7f1ab40718f0 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 msgr2=0x7f1ab4073db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 0x7f1ab4073db0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8000f70 tx=0x7f1ac8009040 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 msgr2=0x7f1ad80ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000e940 tx=0x7f1ac000ee00 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 shutdown_connections 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1ab40718f0 0x7f1ab4073db0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1ad815ca30 0x7f1ad80cd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1ad80cc5d0 0x7f1ad80ccc80 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 --2- 172.21.15.78:0/4274909454 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1ad80cc1f0 0x7f1ad8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 >> 172.21.15.78:0/4274909454 conn(0x7f1ad8071dd0 msgr2=0x7f1ad80ce4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:04.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 shutdown_connections 2024-09-11T12:53:04.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.484+0000 7f1ade000640 1 -- 172.21.15.78:0/4274909454 wait complete. 2024-09-11T12:53:04.513 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:53:04.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:53:04.569 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:53:04.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:04.576 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:53:04.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:04.666 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:04.747 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:53:04.747 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:04.747 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:04.747 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:04.756 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:04.756 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:04.808 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 -- 172.21.15.78:0/783344515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 msgr2=0x7fe20c15b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 --2- 172.21.15.78:0/783344515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c15b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe204009840 tx=0x7fe20401c820 comp rx=0 tx=0).stop 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 -- 172.21.15.78:0/783344515 shutdown_connections 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 --2- 172.21.15.78:0/783344515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c15b350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 --2- 172.21.15.78:0/783344515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c06fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 -- 172.21.15.78:0/783344515 >> 172.21.15.78:0/783344515 conn(0x7fe20c06ca80 msgr2=0x7fe20c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 -- 172.21.15.78:0/783344515 shutdown_connections 2024-09-11T12:53:04.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.992+0000 7fe210c00640 1 -- 172.21.15.78:0/783344515 wait complete. 2024-09-11T12:53:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 Processor -- start 2024-09-11T12:53:04.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- start start 2024-09-11T12:53:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:04.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe20c0741e0 0x7fe20c1fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c0cccc0 con 0x7fe20c070190 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe20c15beb0 con 0x7fe20c0741e0 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe20c167eb0 con 0x7fe20c070910 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60194/0 (socket says 172.21.15.78:60194) 2024-09-11T12:53:04.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 -- 172.21.15.78:0/2437558001 learned_addr learned my addr 172.21.15.78:0/2437558001 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:05.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 -- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 msgr2=0x7fe20c140770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209400640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe20a800640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe20c0741e0 0x7fe20c1fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c140770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 -- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe20c0741e0 msgr2=0x7fe20c1fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe20c0741e0 0x7fe20c1fab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 -- 172.21.15.78:0/2437558001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe20c14d3b0 con 0x7fe20c070190 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209400640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe209e00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00cb40 tx=0x7fe1fc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc016020 con 0x7fe20c070190 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1fc00ab60 con 0x7fe20c070190 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc012420 con 0x7fe20c070190 2024-09-11T12:53:05.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe20c14d690 con 0x7fe20c070190 2024-09-11T12:53:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:04.996+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe20c14dc50 con 0x7fe20c070190 2024-09-11T12:53:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.000+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe20c15b350 con 0x7fe20c070190 2024-09-11T12:53:05.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.000+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fe1fc019020 con 0x7fe20c070190 2024-09-11T12:53:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.000+0000 7fe202000640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 0x7fe1ec073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.000+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fe1fc092680 con 0x7fe20c070190 2024-09-11T12:53:05.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.000+0000 7fe209400640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 0x7fe1ec073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.004+0000 7fe209400640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 0x7fe1ec073ea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8005fd0 tx=0x7fe1f8005e80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.004+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe1fc020020 con 0x7fe20c070190 2024-09-11T12:53:05.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe20c0b3850 con 0x7fe20c070190 2024-09-11T12:53:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe202000640 1 -- 172.21.15.78:0/2437558001 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fe1fc05e0a0 con 0x7fe20c070190 2024-09-11T12:53:05.185 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 msgr2=0x7fe1ec073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 0x7fe1ec073ea0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8005fd0 tx=0x7fe1f8005e80 comp rx=0 tx=0).stop 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 msgr2=0x7fe20c0b1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00cb40 tx=0x7fe1fc00cc30 comp rx=0 tx=0).stop 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 shutdown_connections 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fe1ec0719e0 0x7fe1ec073ea0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe20c0741e0 0x7fe20c1fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe20c070910 0x7fe20c140770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 --2- 172.21.15.78:0/2437558001 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe20c070190 0x7fe20c0b1a20 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 >> 172.21.15.78:0/2437558001 conn(0x7fe20c06ca80 msgr2=0x7fe20c06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:05.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 shutdown_connections 2024-09-11T12:53:05.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.184+0000 7fe210c00640 1 -- 172.21.15.78:0/2437558001 wait complete. 2024-09-11T12:53:05.205 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:53:05.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:05.206 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:05.268 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:53:05.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:05.269 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:53:05.284 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:05.284 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:05.284 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:05.294 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:05.294 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:05.344 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 -- 172.21.15.78:0/302445320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0b4db0 msgr2=0x7fdd0c0b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 --2- 172.21.15.78:0/302445320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0b4db0 0x7fdd0c0b5220 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009bc0 tx=0x7fdd0401cb10 comp rx=0 tx=0).stop 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 -- 172.21.15.78:0/302445320 shutdown_connections 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 --2- 172.21.15.78:0/302445320 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c0b4db0 0x7fdd0c0b5220 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 --2- 172.21.15.78:0/302445320 >> v2:172.21.15.157:3300/0 conn(0x7fdd0c070a00 0x7fdd0c0736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 -- 172.21.15.78:0/302445320 >> 172.21.15.78:0/302445320 conn(0x7fdd0c06ca80 msgr2=0x7fdd0c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:05.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 -- 172.21.15.78:0/302445320 shutdown_connections 2024-09-11T12:53:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.484+0000 7fdd12000640 1 -- 172.21.15.78:0/302445320 wait complete. 2024-09-11T12:53:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 Processor -- start 2024-09-11T12:53:05.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 -- start start 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c070a00 0x7fdd0c1367b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c157ae0 0x7fdd0c14da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c06f280 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c06f100 con 0x7fdd0c070a00 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd12000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdd0c06f400 con 0x7fdd0c157ae0 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60210/0 (socket says 172.21.15.78:60210) 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 -- 172.21.15.78:0/2000054592 learned_addr learned my addr 172.21.15.78:0/2000054592 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:05.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0b400640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c070a00 0x7fdd0c1367b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0be00640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c157ae0 0x7fdd0c14da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 -- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c157ae0 msgr2=0x7fdd0c14da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c157ae0 0x7fdd0c14da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 -- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c070a00 msgr2=0x7fdd0c1367b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c070a00 0x7fdd0c1367b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 -- 172.21.15.78:0/2000054592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c149610 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd0aa00640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800ea20 tx=0x7fdcf800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800cd80 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.488+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcf8010430 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1498f0 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8010860 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c0b0910 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c0736d0 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fdcf8010a00 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd03400640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 0x7fdcec073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fdcf8092db0 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd0b400640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 0x7fdcec073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.492+0000 7fdd0b400640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 0x7fdcec073ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc001870 tx=0x7fdcfc001380 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.496+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdcf8097070 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.672+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdd0c0cdb40 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.672+0000 7fdd03400640 1 -- 172.21.15.78:0/2000054592 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fdcf805e100 con 0x7fdd0c0b4db0 2024-09-11T12:53:05.676 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 msgr2=0x7fdcec073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 0x7fdcec073ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdcfc001870 tx=0x7fdcfc001380 comp rx=0 tx=0).stop 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 msgr2=0x7fdd0c14d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800ea20 tx=0x7fdcf800eee0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 shutdown_connections 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdcec071a10 0x7fdcec073ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd0c157ae0 0x7fdd0c14da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd0c0b4db0 0x7fdd0c14d4e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 --2- 172.21.15.78:0/2000054592 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd0c070a00 0x7fdd0c1367b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 >> 172.21.15.78:0/2000054592 conn(0x7fdd0c06ca80 msgr2=0x7fdd0c159100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 shutdown_connections 2024-09-11T12:53:05.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.676+0000 7fdd12000640 1 -- 172.21.15.78:0/2000054592 wait complete. 2024-09-11T12:53:05.697 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:53:05.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:05.697 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:05.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:53:05.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:05.726 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:53:05.781 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:05.781 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:05.781 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:05.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:05.790 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:05.790 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:05.790 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:05.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:05.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:05.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- 172.21.15.78:0/1449321429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400b14b0 msgr2=0x7fca4015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- 172.21.15.78:0/1449321429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400b14b0 0x7fca4015c100 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fca4015d270 tx=0x7fca3001c800 comp rx=0 tx=0).stop 2024-09-11T12:53:05.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- 172.21.15.78:0/1449321429 shutdown_connections 2024-09-11T12:53:05.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- 172.21.15.78:0/1449321429 >> v2:172.21.15.157:3300/0 conn(0x7fca4006fdf0 0x7fca400a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- 172.21.15.78:0/1449321429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400b14b0 0x7fca4015c100 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- 172.21.15.78:0/1449321429 >> 172.21.15.78:0/1449321429 conn(0x7fca40074b90 msgr2=0x7fca400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- 172.21.15.78:0/1449321429 shutdown_connections 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- 172.21.15.78:0/1449321429 wait complete. 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 Processor -- start 2024-09-11T12:53:05.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- start start 2024-09-11T12:53:05.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca4006fdf0 0x7fca40142490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 0x7fca4006c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca401692e0 con 0x7fca400d0bf0 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca40169160 con 0x7fca400b14b0 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca46600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca40169460 con 0x7fca4006fdf0 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60234/0 (socket says 172.21.15.78:60234) 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 -- 172.21.15.78:0/1075549553 learned_addr learned my addr 172.21.15.78:0/1075549553 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 -- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca4006fdf0 msgr2=0x7fca40142490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca4006fdf0 0x7fca40142490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 -- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 msgr2=0x7fca4006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3ea00640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 0x7fca4006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 0x7fca4006c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3fe00640 1 -- 172.21.15.78:0/1075549553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca401437f0 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.976+0000 7fca3ea00640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 0x7fca4006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca3fe00640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c730 tx=0x7fca3800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38003b80 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca38003d40 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca40143ad0 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380052d0 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca401429d0 con 0x7fca400d0bf0 2024-09-11T12:53:05.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca400a9d00 con 0x7fca400d0bf0 2024-09-11T12:53:05.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fca38005680 con 0x7fca400d0bf0 2024-09-11T12:53:05.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 0x7fca20073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:05.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fca38092410 con 0x7fca400d0bf0 2024-09-11T12:53:05.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.980+0000 7fca3f400640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 0x7fca20073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:05.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.984+0000 7fca3f400640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 0x7fca20073ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca300af000 tx=0x7fca300195a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:05.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:05.984+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fca3805ddc0 con 0x7fca400d0bf0 2024-09-11T12:53:06.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca400758f0 con 0x7fca400d0bf0 2024-09-11T12:53:06.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca37400640 1 -- 172.21.15.78:0/1075549553 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7fca3805d6c0 con 0x7fca400d0bf0 2024-09-11T12:53:06.186 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:06.187 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"btime":"2024-09-11T12:52:48:003645+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:53:06.187 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-09-11T12:53:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 msgr2=0x7fca20073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 0x7fca20073ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fca300af000 tx=0x7fca300195a0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 msgr2=0x7fca401445b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fca3800c730 tx=0x7fca3800cbf0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 shutdown_connections 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fca20071a10 0x7fca20073ed0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca400d0bf0 0x7fca401445b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca400b14b0 0x7fca4006c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 --2- 172.21.15.78:0/1075549553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca4006fdf0 0x7fca40142490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.184+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 >> 172.21.15.78:0/1075549553 conn(0x7fca40074b90 msgr2=0x7fca400b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:06.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.188+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 shutdown_connections 2024-09-11T12:53:06.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.188+0000 7fca46600640 1 -- 172.21.15.78:0/1075549553 wait complete. 2024-09-11T12:53:06.208 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 -- 172.21.15.78:0/1260494733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 msgr2=0x7f8174071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 --2- 172.21.15.78:0/1260494733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f8174071d60 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f816c009bc0 tx=0x7f816c01cb70 comp rx=0 tx=0).stop 2024-09-11T12:53:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 -- 172.21.15.78:0/1260494733 shutdown_connections 2024-09-11T12:53:06.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 --2- 172.21.15.78:0/1260494733 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f8174071d60 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 -- 172.21.15.78:0/1260494733 >> 172.21.15.78:0/1260494733 conn(0x7f817406ca80 msgr2=0x7f81740b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 -- 172.21.15.78:0/1260494733 shutdown_connections 2024-09-11T12:53:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.348+0000 7f817b800640 1 -- 172.21.15.78:0/1260494733 wait complete. 2024-09-11T12:53:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 Processor -- start 2024-09-11T12:53:06.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- start start 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8174074240 0x7f81740b1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817415ba80 0x7f81740b0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8174168ae0 con 0x7f8174070360 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8174168960 con 0x7f8174074240 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8174168c60 con 0x7f817415ba80 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60248/0 (socket says 172.21.15.78:60248) 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 -- 172.21.15.78:0/2309792131 learned_addr learned my addr 172.21.15.78:0/2309792131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8179800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817415ba80 0x7f81740b0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8173e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8174074240 0x7f81740b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 -- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817415ba80 msgr2=0x7f81740b0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817415ba80 0x7f81740b0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 -- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8174074240 msgr2=0x7f81740b1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8174074240 0x7f81740b1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 -- 172.21.15.78:0/2309792131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f817414f910 con 0x7f8174070360 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8178e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f816400c4b0 tx=0x7f816400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81640052d0 con 0x7f8174070360 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8164005470 con 0x7f8174070360 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81740ac350 con 0x7f8174070360 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8164012870 con 0x7f8174070360 2024-09-11T12:53:06.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.352+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81740acbd0 con 0x7f8174070360 2024-09-11T12:53:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.356+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8174071d60 con 0x7f8174070360 2024-09-11T12:53:06.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.356+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f8164003600 con 0x7f8174070360 2024-09-11T12:53:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.356+0000 7f8170c00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 0x7f8154073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.356+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8164091f50 con 0x7f8174070360 2024-09-11T12:53:06.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.356+0000 7f8173e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 0x7f8154073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.360+0000 7f8173e00640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 0x7f8154073ea0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f81740abff0 tx=0x7f8160009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:06.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.360+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f816405d900 con 0x7f8174070360 2024-09-11T12:53:06.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.568+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81740cf170 con 0x7f8174070360 2024-09-11T12:53:06.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.568+0000 7f8170c00640 1 -- 172.21.15.78:0/2309792131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f816405d200 con 0x7f8174070360 2024-09-11T12:53:06.571 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:06.571 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"btime":"2024-09-11T12:52:48:003645+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:53:06.571 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 msgr2=0x7f8154073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 0x7f8154073ea0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f81740abff0 tx=0x7f8160009040 comp rx=0 tx=0).stop 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 msgr2=0x7f817414f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f816400c4b0 tx=0x7f816400c970 comp rx=0 tx=0).stop 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 shutdown_connections 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f81540719e0 0x7f8154073ea0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f817415ba80 0x7f81740b0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8174074240 0x7f81740b1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 --2- 172.21.15.78:0/2309792131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8174070360 0x7f817414f3d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 >> 172.21.15.78:0/2309792131 conn(0x7f817406ca80 msgr2=0x7f817406f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:06.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 shutdown_connections 2024-09-11T12:53:06.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.572+0000 7f817b800640 1 -- 172.21.15.78:0/2309792131 wait complete. 2024-09-11T12:53:06.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- 172.21.15.78:0/1814853919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4b60 msgr2=0x7f358c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- 172.21.15.78:0/1814853919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4b60 0x7f358c15ba90 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f358c074d30 tx=0x7f358401cd80 comp rx=0 tx=0).stop 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- 172.21.15.78:0/1814853919 shutdown_connections 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- 172.21.15.78:0/1814853919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4b60 0x7f358c15ba90 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- 172.21.15.78:0/1814853919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0ccb20 0x7f358c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- 172.21.15.78:0/1814853919 >> v2:172.21.15.157:3300/0 conn(0x7f358c0b4780 0x7f358c073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- 172.21.15.78:0/1814853919 >> 172.21.15.78:0/1814853919 conn(0x7f358c06ca80 msgr2=0x7f358c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:06.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- 172.21.15.78:0/1814853919 shutdown_connections 2024-09-11T12:53:06.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- 172.21.15.78:0/1814853919 wait complete. 2024-09-11T12:53:06.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 Processor -- start 2024-09-11T12:53:06.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- start start 2024-09-11T12:53:06.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0b4b60 0x7f358c1f5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c0ccb20 0x7f358c14aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f358c06f080 con 0x7f358c0b4780 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f358c06ef00 con 0x7f358c0ccb20 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3592a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f358c06f200 con 0x7f358c0b4b60 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f358be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f358be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60266/0 (socket says 172.21.15.78:60266) 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f358be00640 1 -- 172.21.15.78:0/2010254655 learned_addr learned my addr 172.21.15.78:0/2010254655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f3590a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c0ccb20 0x7f358c14aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.732+0000 7f358b400640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0b4b60 0x7f358c1f5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 -- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0b4b60 msgr2=0x7f358c1f5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0b4b60 0x7f358c1f5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 -- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c0ccb20 msgr2=0x7f358c14aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c0ccb20 0x7f358c14aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 -- 172.21.15.78:0/2010254655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f358c14b490 con 0x7f358c0b4780 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358be00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d770 tx=0x7f357c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c017070 con 0x7f358c0b4780 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f357c009d20 con 0x7f358c0b4780 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f357c0106c0 con 0x7f358c0b4780 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f358c15c0f0 con 0x7f358c0b4780 2024-09-11T12:53:06.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f358c0afeb0 con 0x7f358c0b4780 2024-09-11T12:53:06.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f358c073f70 con 0x7f358c0b4780 2024-09-11T12:53:06.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f357c01a070 con 0x7f358c0b4780 2024-09-11T12:53:06.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 0x7f356c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:06.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f357c092510 con 0x7f358c0b4780 2024-09-11T12:53:06.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.736+0000 7f358b400640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 0x7f356c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:06.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.740+0000 7f358b400640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 0x7f356c073ea0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f35780058d0 tx=0x7f3578005860 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:06.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.740+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f357c05d760 con 0x7f358c0b4780 2024-09-11T12:53:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.908+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f358c0af750 con 0x7f358c0b4780 2024-09-11T12:53:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.908+0000 7f3583e00640 1 -- 172.21.15.78:0/2010254655 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16725 (secure 0 0 0) 0x7f357c061cb0 con 0x7f358c0b4780 2024-09-11T12:53:06.910 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:06.910 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":58,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:39.967859+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:52:36.041400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-11T12:52:37.321031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:06.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 msgr2=0x7f356c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 0x7f356c073ea0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f35780058d0 tx=0x7f3578005860 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 msgr2=0x7f358c15d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f357c00d770 tx=0x7f357c00dc30 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 shutdown_connections 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f356c0719e0 0x7f356c073ea0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f358c0ccb20 0x7f358c14aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f358c0b4b60 0x7f358c1f5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 --2- 172.21.15.78:0/2010254655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f358c0b4780 0x7f358c15d380 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 >> 172.21.15.78:0/2010254655 conn(0x7f358c06ca80 msgr2=0x7f358c165c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 shutdown_connections 2024-09-11T12:53:06.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:06.912+0000 7f3592a00640 1 -- 172.21.15.78:0/2010254655 wait complete. 2024-09-11T12:53:06.932 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:53:06.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- 172.21.15.78:0/676878924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8070360 msgr2=0x7f6ee815e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- 172.21.15.78:0/676878924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8070360 0x7f6ee815e9f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009900 tx=0x7f6ee001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- 172.21.15.78:0/676878924 shutdown_connections 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- 172.21.15.78:0/676878924 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8070360 0x7f6ee815e9f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- 172.21.15.78:0/676878924 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee8074240 0x7f6ee815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- 172.21.15.78:0/676878924 >> 172.21.15.78:0/676878924 conn(0x7f6ee806ca80 msgr2=0x7f6ee80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:07.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- 172.21.15.78:0/676878924 shutdown_connections 2024-09-11T12:53:07.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- 172.21.15.78:0/676878924 wait complete. 2024-09-11T12:53:07.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 Processor -- start 2024-09-11T12:53:07.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- start start 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 0x7f6ee81600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee815e5b0 0x7f6ee806f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee8167740 con 0x7f6ee8074240 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee80728c0 con 0x7f6ee8070360 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6eed000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ee81678c0 con 0x7f6ee815e5b0 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60288/0 (socket says 172.21.15.78:60288) 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5400640 1 -- 172.21.15.78:0/4176505156 learned_addr learned my addr 172.21.15.78:0/4176505156 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee6800640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee815e5b0 0x7f6ee806f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5e00640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 0x7f6ee81600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5400640 1 -- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee815e5b0 msgr2=0x7f6ee806f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.076+0000 7f6ee5400640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee815e5b0 0x7f6ee806f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ee5400640 1 -- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 msgr2=0x7f6ee81600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ee5400640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 0x7f6ee81600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ee5400640 1 -- 172.21.15.78:0/4176505156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee81fad40 con 0x7f6ee8074240 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ee5e00640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 0x7f6ee81600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:07.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ee5400640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c3f0 tx=0x7f6ed400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4017070 con 0x7f6ee8074240 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ed40053c0 con 0x7f6ee8074240 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee81f7d30 con 0x7f6ee8074240 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4003860 con 0x7f6ee8074240 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee81f82f0 con 0x7f6ee8074240 2024-09-11T12:53:07.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee815e9f0 con 0x7f6ee8074240 2024-09-11T12:53:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f6ed401a070 con 0x7f6ee8074240 2024-09-11T12:53:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 0x7f6ec8073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.080+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6ed4091580 con 0x7f6ee8074240 2024-09-11T12:53:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.084+0000 7f6ee5e00640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 0x7f6ec8073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.084+0000 7f6ee5e00640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 0x7f6ec8073ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af5b0 tx=0x7f6ed8009520 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.084+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6ed405c9e0 con 0x7f6ee8074240 2024-09-11T12:53:07.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.288+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6ee81faf10 con 0x7f6ee8074240 2024-09-11T12:53:07.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.288+0000 7f6ede000640 1 -- 172.21.15.78:0/4176505156 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f6ed4065270 con 0x7f6ee8074240 2024-09-11T12:53:07.291 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:07.291 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"btime":"2024-09-11T12:52:48:003645+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:53:07.291 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 msgr2=0x7f6ec8073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 0x7f6ec8073ea0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6ee80af5b0 tx=0x7f6ed8009520 comp rx=0 tx=0).stop 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 msgr2=0x7f6ee8160f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c3f0 tx=0x7f6ed400c8b0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 shutdown_connections 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6ec80719e0 0x7f6ec8073ea0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ee815e5b0 0x7f6ee806f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ee8074240 0x7f6ee8160f50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 --2- 172.21.15.78:0/4176505156 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ee8070360 0x7f6ee81600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 >> 172.21.15.78:0/4176505156 conn(0x7f6ee806ca80 msgr2=0x7f6ee80b2ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:07.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 shutdown_connections 2024-09-11T12:53:07.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.292+0000 7f6eed000640 1 -- 172.21.15.78:0/4176505156 wait complete. 2024-09-11T12:53:07.313 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 -- 172.21.15.78:0/2159264470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f900b14b0 msgr2=0x7f2f9015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 --2- 172.21.15.78:0/2159264470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f900b14b0 0x7f2f9015c100 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2f9015d270 tx=0x7f2f8001cd80 comp rx=0 tx=0).stop 2024-09-11T12:53:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 -- 172.21.15.78:0/2159264470 shutdown_connections 2024-09-11T12:53:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 --2- 172.21.15.78:0/2159264470 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 0x7f2f900d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 --2- 172.21.15.78:0/2159264470 >> v2:172.21.15.157:3300/0 conn(0x7f2f9006fdf0 0x7f2f900a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 --2- 172.21.15.78:0/2159264470 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f900b14b0 0x7f2f9015c100 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 -- 172.21.15.78:0/2159264470 >> 172.21.15.78:0/2159264470 conn(0x7f2f90074b90 msgr2=0x7f2f900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:07.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 -- 172.21.15.78:0/2159264470 shutdown_connections 2024-09-11T12:53:07.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.452+0000 7f2f96600640 1 -- 172.21.15.78:0/2159264470 wait complete. 2024-09-11T12:53:07.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 Processor -- start 2024-09-11T12:53:07.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- start start 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f900b14b0 0x7f2f90148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 0x7f2f9015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9006ed80 con 0x7f2f9006fdf0 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9006ec00 con 0x7f2f900b14b0 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9006ef00 con 0x7f2f900d0bf0 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60310/0 (socket says 172.21.15.78:60310) 2024-09-11T12:53:07.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 -- 172.21.15.78:0/2201183035 learned_addr learned my addr 172.21.15.78:0/2201183035 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8ea00640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f900b14b0 0x7f2f90148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8fe00640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 0x7f2f9015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 -- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 msgr2=0x7f2f9015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 0x7f2f9015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 -- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f900b14b0 msgr2=0x7f2f90148890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f900b14b0 0x7f2f90148890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 -- 172.21.15.78:0/2201183035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9006ca10 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f8f400640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f80000f90 tx=0x7f2f800093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800bb070 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f80004e00 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9006ccf0 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f800b2040 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9006d2b0 con 0x7f2f9006fdf0 2024-09-11T12:53:07.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.456+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9006f3d0 con 0x7f2f9006fdf0 2024-09-11T12:53:07.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.460+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f2f800be070 con 0x7f2f9006fdf0 2024-09-11T12:53:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.460+0000 7f2f87400640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 0x7f2f70073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.460+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f2f8012c190 con 0x7f2f9006fdf0 2024-09-11T12:53:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.460+0000 7f2f8ea00640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 0x7f2f70073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.460+0000 7f2f8ea00640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 0x7f2f70073ea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005fd0 tx=0x7f2f7c005e60 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.464+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2f80016970 con 0x7f2f9006fdf0 2024-09-11T12:53:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.660+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f90069070 con 0x7f2f9006fdf0 2024-09-11T12:53:07.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.660+0000 7f2f87400640 1 -- 172.21.15.78:0/2201183035 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5229 (secure 0 0 0) 0x7f2f800f7ab0 con 0x7f2f9006fdf0 2024-09-11T12:53:07.664 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:07.664 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":37,"btime":"2024-09-11T12:52:48:003645+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:52:38.352038+0000","modified":"2024-09-11T12:52:41.364437+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5745},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5745":{"gid":5745,"name":"c","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6834/2934440940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2934440940},{"type":"v1","addr":"172.21.15.78:6834","nonce":2934440940}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5745,"qdb_cluster":[5745]},"id":4}]} 2024-09-11T12:53:07.664 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 37 2024-09-11T12:53:07.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 msgr2=0x7f2f70073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 0x7f2f70073ea0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005fd0 tx=0x7f2f7c005e60 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 msgr2=0x7f2f9006dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2f80000f90 tx=0x7f2f800093f0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 shutdown_connections 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f700719e0 0x7f2f70073ea0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f900d0bf0 0x7f2f9015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f900b14b0 0x7f2f90148890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 --2- 172.21.15.78:0/2201183035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f9006fdf0 0x7f2f9006dab0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 >> 172.21.15.78:0/2201183035 conn(0x7f2f90074b90 msgr2=0x7f2f900706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 shutdown_connections 2024-09-11T12:53:07.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.664+0000 7f2f96600640 1 -- 172.21.15.78:0/2201183035 wait complete. 2024-09-11T12:53:07.685 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 -- 172.21.15.78:0/1719891947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55740aca00 msgr2=0x7f5574140c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 --2- 172.21.15.78:0/1719891947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55740aca00 0x7f5574140c30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f556c009c80 tx=0x7f556c01cd40 comp rx=0 tx=0).stop 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 -- 172.21.15.78:0/1719891947 shutdown_connections 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 --2- 172.21.15.78:0/1719891947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55740aca00 0x7f5574140c30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 --2- 172.21.15.78:0/1719891947 >> v2:172.21.15.157:3300/0 conn(0x7f557415f3d0 0x7f55741406f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 -- 172.21.15.78:0/1719891947 >> 172.21.15.78:0/1719891947 conn(0x7f5574074490 msgr2=0x7f5574074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 -- 172.21.15.78:0/1719891947 shutdown_connections 2024-09-11T12:53:07.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.824+0000 7f5573e00640 1 -- 172.21.15.78:0/1719891947 wait complete. 2024-09-11T12:53:07.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 Processor -- start 2024-09-11T12:53:07.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- start start 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55740aca00 0x7f557413b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557415f3d0 0x7f557413bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55741686e0 con 0x7f55740ac620 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5574168560 con 0x7f55740aca00 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5574168860 con 0x7f557415f3d0 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60318/0 (socket says 172.21.15.78:60318) 2024-09-11T12:53:07.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 -- 172.21.15.78:0/424181256 learned_addr learned my addr 172.21.15.78:0/424181256 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572000640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55740aca00 0x7f557413b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573400640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557415f3d0 0x7f557413bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 -- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557415f3d0 msgr2=0x7f557413bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557415f3d0 0x7f557413bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 -- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55740aca00 msgr2=0x7f557413b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55740aca00 0x7f557413b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 -- 172.21.15.78:0/424181256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741f8450 con 0x7f55740ac620 2024-09-11T12:53:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5572a00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f556400c490 tx=0x7f556400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55640052d0 con 0x7f55740ac620 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5564005490 con 0x7f55740ac620 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55741f8730 con 0x7f55740ac620 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564012850 con 0x7f55740ac620 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557406ec30 con 0x7f55740ac620 2024-09-11T12:53:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.828+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f557415bde0 con 0x7f55740ac620 2024-09-11T12:53:07.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.832+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f55640129f0 con 0x7f55740ac620 2024-09-11T12:53:07.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.832+0000 7f556aa00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 0x7f5554073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:07.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.832+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5564091f70 con 0x7f55740ac620 2024-09-11T12:53:07.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.832+0000 7f5572000640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 0x7f5554073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:07.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.832+0000 7f5572000640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 0x7f5554073ea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f557413dc60 tx=0x7f55600091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:07.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:07.836+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f556405d9e0 con 0x7f55740ac620 2024-09-11T12:53:08.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.004+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f557413f8a0 con 0x7f55740ac620 2024-09-11T12:53:08.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.004+0000 7f556aa00640 1 -- 172.21.15.78:0/424181256 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16725 (secure 0 0 0) 0x7f556405d2e0 con 0x7f55740ac620 2024-09-11T12:53:08.007 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:08.007 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":58,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:52:39.967859+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:52:36.041400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-11T12:52:37.321031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 msgr2=0x7f5554073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 0x7f5554073ea0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f557413dc60 tx=0x7f55600091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 msgr2=0x7f5574151ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f556400c490 tx=0x7f556400c950 comp rx=0 tx=0).stop 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 shutdown_connections 2024-09-11T12:53:08.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f55540719e0 0x7f5554073ea0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f557415f3d0 0x7f557413bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55740aca00 0x7f557413b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 --2- 172.21.15.78:0/424181256 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55740ac620 0x7f5574151ae0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 >> 172.21.15.78:0/424181256 conn(0x7f5574074490 msgr2=0x7f557413e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 shutdown_connections 2024-09-11T12:53:08.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.008+0000 7f5573e00640 1 -- 172.21.15.78:0/424181256 wait complete. 2024-09-11T12:53:08.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- 172.21.15.78:0/2095029599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc34006fc50 msgr2=0x7fc3400700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- 172.21.15.78:0/2095029599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc34006fc50 0x7fc3400700c0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fc3400afa30 tx=0x7fc33801cd50 comp rx=0 tx=0).stop 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- 172.21.15.78:0/2095029599 shutdown_connections 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- 172.21.15.78:0/2095029599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc34006fc50 0x7fc3400700c0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- 172.21.15.78:0/2095029599 >> v2:172.21.15.157:3300/0 conn(0x7fc34015e5b0 0x7fc34015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- 172.21.15.78:0/2095029599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc340074240 0x7fc34015ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- 172.21.15.78:0/2095029599 >> 172.21.15.78:0/2095029599 conn(0x7fc34006ca80 msgr2=0x7fc3400b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:08.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- 172.21.15.78:0/2095029599 shutdown_connections 2024-09-11T12:53:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- 172.21.15.78:0/2095029599 wait complete. 2024-09-11T12:53:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 Processor -- start 2024-09-11T12:53:08.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- start start 2024-09-11T12:53:08.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc34006fc50 0x7fc34014dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc34015e5b0 0x7fc3401f8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc340072900 con 0x7fc340074240 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc340072780 con 0x7fc34006fc50 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc346200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3400b4520 con 0x7fc34015e5b0 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc33ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc33ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60336/0 (socket says 172.21.15.78:60336) 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc33ea00640 1 -- 172.21.15.78:0/1919228258 learned_addr learned my addr 172.21.15.78:0/1919228258 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc33f400640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc34006fc50 0x7fc34014dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.172+0000 7fc33fe00640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc34015e5b0 0x7fc3401f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 -- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc34015e5b0 msgr2=0x7fc3401f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc34015e5b0 0x7fc3401f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 -- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc34006fc50 msgr2=0x7fc34014dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc34006fc50 0x7fc34014dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 -- 172.21.15.78:0/1919228258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc340150360 con 0x7fc340074240 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc33ea00640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0058d0 tx=0x7fc32c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c017070 con 0x7fc340074240 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc32c009d20 con 0x7fc340074240 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc340150640 con 0x7fc340074240 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0106c0 con 0x7fc340074240 2024-09-11T12:53:08.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc340150c00 con 0x7fc340074240 2024-09-11T12:53:08.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc34015ba80 con 0x7fc340074240 2024-09-11T12:53:08.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.176+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fc32c01a070 con 0x7fc340074240 2024-09-11T12:53:08.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.180+0000 7fc337400640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 0x7fc320073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.180+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc32c092530 con 0x7fc340074240 2024-09-11T12:53:08.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.180+0000 7fc33f400640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 0x7fc320073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.180+0000 7fc33f400640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 0x7fc320073ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc330004610 tx=0x7fc33000b2e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:08.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.184+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc32c05d730 con 0x7fc340074240 2024-09-11T12:53:08.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.380+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc340164af0 con 0x7fc340074240 2024-09-11T12:53:08.384 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:53:08.384+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:53:08.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.384+0000 7fc337400640 1 -- 172.21.15.78:0/1919228258 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7fc32c061c80 con 0x7fc340074240 2024-09-11T12:53:08.387 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 msgr2=0x7fc320073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 0x7fc320073ea0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc330004610 tx=0x7fc33000b2e0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 msgr2=0x7fc34013f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc32c0058d0 tx=0x7fc32c00dde0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 shutdown_connections 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc3200719e0 0x7fc320073ea0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc34015e5b0 0x7fc3401f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc340074240 0x7fc34013f590 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 --2- 172.21.15.78:0/1919228258 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc34006fc50 0x7fc34014dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 >> 172.21.15.78:0/1919228258 conn(0x7fc34006ca80 msgr2=0x7fc340151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 shutdown_connections 2024-09-11T12:53:08.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.388+0000 7fc346200640 1 -- 172.21.15.78:0/1919228258 wait complete. 2024-09-11T12:53:08.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:53:08.452 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- 172.21.15.78:0/309977676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4006f130 msgr2=0x7fdb4006f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- 172.21.15.78:0/309977676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4006f130 0x7fdb4006f560 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009c80 tx=0x7fdb3801cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- 172.21.15.78:0/309977676 shutdown_connections 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- 172.21.15.78:0/309977676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb4006f130 0x7fdb4006f560 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- 172.21.15.78:0/309977676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb401515c0 0x7fdb4015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- 172.21.15.78:0/309977676 >> 172.21.15.78:0/309977676 conn(0x7fdb40075a10 msgr2=0x7fdb400734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- 172.21.15.78:0/309977676 shutdown_connections 2024-09-11T12:53:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- 172.21.15.78:0/309977676 wait complete. 2024-09-11T12:53:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 Processor -- start 2024-09-11T12:53:08.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- start start 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb4006f130 0x7fdb4015d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb401515c0 0x7fdb40141d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400b57f0 con 0x7fdb401519a0 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb400b5670 con 0x7fdb401515c0 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb46800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdb400b5970 con 0x7fdb4006f130 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb3fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60362/0 (socket says 172.21.15.78:60362) 2024-09-11T12:53:08.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb3fe00640 1 -- 172.21.15.78:0/362437498 learned_addr learned my addr 172.21.15.78:0/362437498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb3f400640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb4006f130 0x7fdb4015d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.556+0000 7fdb3ea00640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb401515c0 0x7fdb40141d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 -- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb4006f130 msgr2=0x7fdb4015d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb4006f130 0x7fdb4015d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 -- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb401515c0 msgr2=0x7fdb40141d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb401515c0 0x7fdb40141d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 -- 172.21.15.78:0/362437498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb401f68e0 con 0x7fdb401519a0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb3fe00640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009bd0 tx=0x7fdb380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380bb070 con 0x7fdb401519a0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb380047d0 con 0x7fdb401519a0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4015dd20 con 0x7fdb401519a0 2024-09-11T12:53:08.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb380b2040 con 0x7fdb401519a0 2024-09-11T12:53:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb40144ea0 con 0x7fdb401519a0 2024-09-11T12:53:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb4006f560 con 0x7fdb401519a0 2024-09-11T12:53:08.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fdb380be070 con 0x7fdb401519a0 2024-09-11T12:53:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 0x7fdb20073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.560+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fdb3812c360 con 0x7fdb401519a0 2024-09-11T12:53:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.564+0000 7fdb3f400640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 0x7fdb20073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:08.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.564+0000 7fdb3f400640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 0x7fdb20073ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b0160 tx=0x7fdb3000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:08.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.564+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdb401f68e0 con 0x7fdb401519a0 2024-09-11T12:53:08.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:08.764+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdb401505f0 con 0x7fdb401519a0 2024-09-11T12:53:09.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb37400640 1 -- 172.21.15.78:0/362437498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7fdb38016990 con 0x7fdb401519a0 2024-09-11T12:53:09.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 msgr2=0x7fdb20073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:09.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 0x7fdb20073ea0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdb400b0160 tx=0x7fdb3000b390 comp rx=0 tx=0).stop 2024-09-11T12:53:09.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 msgr2=0x7fdb401f63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:09.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb38009bd0 tx=0x7fdb380045e0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 shutdown_connections 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdb200719e0 0x7fdb20073ea0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb401519a0 0x7fdb401f63a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb401515c0 0x7fdb40141d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 --2- 172.21.15.78:0/362437498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb4006f130 0x7fdb4015d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 >> 172.21.15.78:0/362437498 conn(0x7fdb40075a10 msgr2=0x7fdb400692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 shutdown_connections 2024-09-11T12:53:09.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.392+0000 7fdb46800640 1 -- 172.21.15.78:0/362437498 wait complete. 2024-09-11T12:53:09.418 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:53:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- 172.21.15.78:0/618505648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984003a10 msgr2=0x7fc984003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 --2- 172.21.15.78:0/618505648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984003a10 0x7fc984003df0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fc990001ec0 tx=0x7fc99001cc20 comp rx=0 tx=0).stop 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- 172.21.15.78:0/618505648 shutdown_connections 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 --2- 172.21.15.78:0/618505648 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc984003a10 0x7fc984003df0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- 172.21.15.78:0/618505648 >> 172.21.15.78:0/618505648 conn(0x7fc99c0b3050 msgr2=0x7fc99c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- 172.21.15.78:0/618505648 shutdown_connections 2024-09-11T12:53:09.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- 172.21.15.78:0/618505648 wait complete. 2024-09-11T12:53:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 Processor -- start 2024-09-11T12:53:09.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.608+0000 7fc9a4600640 1 -- start start 2024-09-11T12:53:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc984003a10 0x7fc99c143450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c15d0e0 0x7fc99c15d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:09.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c06ffe0 con 0x7fc99c1599f0 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc99c06fe60 con 0x7fc99c15d0e0 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc99c070160 con 0x7fc984003a10 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60388/0 (socket says 172.21.15.78:60388) 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 -- 172.21.15.78:0/4134501230 learned_addr learned my addr 172.21.15.78:0/4134501230 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1c00640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc984003a10 0x7fc99c143450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a2600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c15d0e0 0x7fc99c15d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 -- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc984003a10 msgr2=0x7fc99c143450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc984003a10 0x7fc99c143450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 -- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c15d0e0 msgr2=0x7fc99c15d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:09.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c15d0e0 0x7fc99c15d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 -- 172.21.15.78:0/4134501230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc99c143990 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a1200640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc988001000 tx=0x7fc9880039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988015070 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc98800f960 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988006bb0 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc99c15de10 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc99c146090 con 0x7fc99c1599f0 2024-09-11T12:53:09.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc99c0ca480 con 0x7fc99c1599f0 2024-09-11T12:53:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.612+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fc98800fd30 con 0x7fc99c1599f0 2024-09-11T12:53:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.616+0000 7fc999600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 0x7fc97c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.616+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fc9880930b0 con 0x7fc99c1599f0 2024-09-11T12:53:09.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.616+0000 7fc9a1c00640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 0x7fc97c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:09.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.616+0000 7fc9a1c00640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 0x7fc97c073ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9900ac000 tx=0x7fc9900b0040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:09.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.620+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc98805e8f0 con 0x7fc99c1599f0 2024-09-11T12:53:09.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:09.784+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc99c0706c0 con 0x7fc99c1599f0 2024-09-11T12:53:10.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.392+0000 7fc999600640 1 -- 172.21.15.78:0/4134501230 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7fc98805e1f0 con 0x7fc99c1599f0 2024-09-11T12:53:10.395 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:53:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 msgr2=0x7fc97c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 0x7fc97c073ea0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc9900ac000 tx=0x7fc9900b0040 comp rx=0 tx=0).stop 2024-09-11T12:53:10.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 msgr2=0x7fc99c159e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc988001000 tx=0x7fc9880039d0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 shutdown_connections 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fc97c0719e0 0x7fc97c073ea0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc99c15d0e0 0x7fc99c15d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc99c1599f0 0x7fc99c159e70 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 --2- 172.21.15.78:0/4134501230 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc984003a10 0x7fc99c143450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 >> 172.21.15.78:0/4134501230 conn(0x7fc99c0b3050 msgr2=0x7fc99c1520e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 shutdown_connections 2024-09-11T12:53:10.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.396+0000 7fc9a4600640 1 -- 172.21.15.78:0/4134501230 wait complete. 2024-09-11T12:53:10.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:53:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 -- 172.21.15.78:0/3917626863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 msgr2=0x7faf400cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 --2- 172.21.15.78:0/3917626863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf400cf990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7faf400af670 tx=0x7faf2c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 -- 172.21.15.78:0/3917626863 shutdown_connections 2024-09-11T12:53:10.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 --2- 172.21.15.78:0/3917626863 >> v2:172.21.15.157:3300/0 conn(0x7faf400b5530 0x7faf400b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 --2- 172.21.15.78:0/3917626863 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf400cf990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 -- 172.21.15.78:0/3917626863 >> 172.21.15.78:0/3917626863 conn(0x7faf400b3050 msgr2=0x7faf400b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 -- 172.21.15.78:0/3917626863 shutdown_connections 2024-09-11T12:53:10.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.568+0000 7faf45000640 1 -- 172.21.15.78:0/3917626863 wait complete. 2024-09-11T12:53:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 Processor -- start 2024-09-11T12:53:10.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- start start 2024-09-11T12:53:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b5530 0x7faf401fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:10.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf40152990 0x7faf4015ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faf40168870 con 0x7faf400cf520 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faf401686f0 con 0x7faf400b5530 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faf401689f0 con 0x7faf40152990 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60410/0 (socket says 172.21.15.78:60410) 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 -- 172.21.15.78:0/74326573 learned_addr learned my addr 172.21.15.78:0/74326573 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3de00640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b5530 0x7faf401fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:10.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3e800640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf40152990 0x7faf4015ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 -- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf40152990 msgr2=0x7faf4015ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf40152990 0x7faf4015ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 -- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b5530 msgr2=0x7faf401fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b5530 0x7faf401fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 -- 172.21.15.78:0/74326573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf401fbbe0 con 0x7faf400cf520 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf3d400640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0194d0 tx=0x7faf2c0a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0b2040 con 0x7faf400cf520 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf2c0044a0 con 0x7faf400cf520 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c005290 con 0x7faf400cf520 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf4015be70 con 0x7faf400cf520 2024-09-11T12:53:10.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.572+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf4015c430 con 0x7faf400cf520 2024-09-11T12:53:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf400b59a0 con 0x7faf400cf520 2024-09-11T12:53:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7faf2c005430 con 0x7faf400cf520 2024-09-11T12:53:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf36000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 0x7faf20073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:10.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7faf2c12c060 con 0x7faf400cf520 2024-09-11T12:53:10.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf3de00640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 0x7faf20073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:10.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.576+0000 7faf3de00640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 0x7faf20073ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf4015a7c0 tx=0x7faf30009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:10.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.580+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7faf2c0c4060 con 0x7faf400cf520 2024-09-11T12:53:10.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:10.748+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7faf4015b060 con 0x7faf400cf520 2024-09-11T12:53:11.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.400+0000 7faf36000640 1 -- 172.21.15.78:0/74326573 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7faf2c0164b0 con 0x7faf400cf520 2024-09-11T12:53:11.403 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 msgr2=0x7faf20073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 0x7faf20073ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7faf4015a7c0 tx=0x7faf30009040 comp rx=0 tx=0).stop 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 msgr2=0x7faf4014a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0194d0 tx=0x7faf2c0a57b0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 shutdown_connections 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7faf200719e0 0x7faf20073ea0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faf40152990 0x7faf4015ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faf400cf520 0x7faf4014a550 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 --2- 172.21.15.78:0/74326573 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faf400b5530 0x7faf401fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 >> 172.21.15.78:0/74326573 conn(0x7faf400b3050 msgr2=0x7faf4014d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:11.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 shutdown_connections 2024-09-11T12:53:11.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.404+0000 7faf45000640 1 -- 172.21.15.78:0/74326573 wait complete. 2024-09-11T12:53:11.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:53:11.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 -- 172.21.15.78:0/2832061930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0b14d0 msgr2=0x7f62ac166c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 --2- 172.21.15.78:0/2832061930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0b14d0 0x7f62ac166c10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009bc0 tx=0x7f62a401cae0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 -- 172.21.15.78:0/2832061930 shutdown_connections 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 --2- 172.21.15.78:0/2832061930 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0b14d0 0x7f62ac166c10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 -- 172.21.15.78:0/2832061930 >> 172.21.15.78:0/2832061930 conn(0x7f62ac0cf830 msgr2=0x7f62ac0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 -- 172.21.15.78:0/2832061930 shutdown_connections 2024-09-11T12:53:11.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.568+0000 7f62b0a00640 1 -- 172.21.15.78:0/2832061930 wait complete. 2024-09-11T12:53:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 Processor -- start 2024-09-11T12:53:11.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- start start 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 0x7f62ac14f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 0x7f62ac157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac16a1d0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62ac16a050 con 0x7f62ac0b14d0 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62ac16a350 con 0x7f62ac15f080 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60436/0 (socket says 172.21.15.78:60436) 2024-09-11T12:53:11.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 -- 172.21.15.78:0/3398880348 learned_addr learned my addr 172.21.15.78:0/3398880348 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 -- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 msgr2=0x7f62ac157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62aa800640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 0x7f62ac157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9400640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 0x7f62ac14f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 0x7f62ac157e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 -- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 msgr2=0x7f62ac14f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 0x7f62ac14f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 -- 172.21.15.78:0/3398880348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ac1607b0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62aa800640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 0x7f62ac157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9400640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 0x7f62ac14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a9e00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c3c0 tx=0x7f629c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:11.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c017070 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f629c0053c0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c003860 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ac160a90 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.572+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ac1604c0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ac0d15a0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f629c01a070 con 0x7f62ac0a75d0 2024-09-11T12:53:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62a2000640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 0x7f628c073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f629c0912d0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62a9400640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 0x7f628c073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.576+0000 7f62a9400640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 0x7f628c073e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f62ac15f990 tx=0x7f6298009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:11.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.580+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f629c05d110 con 0x7f62ac0a75d0 2024-09-11T12:53:11.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.748+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f62ac0b0da0 con 0x7f62ac0a75d0 2024-09-11T12:53:11.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.752+0000 7f62a2000640 1 -- 172.21.15.78:0/3398880348 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f629c065350 con 0x7f62ac0a75d0 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 msgr2=0x7f628c073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 0x7f628c073e40 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f62ac15f990 tx=0x7f6298009040 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 msgr2=0x7f62ac15ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c3c0 tx=0x7f629c00c880 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 shutdown_connections 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f628c071980 0x7f628c073e40 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62ac15f080 0x7f62ac157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62ac0b14d0 0x7f62ac14f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 --2- 172.21.15.78:0/3398880348 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62ac0a75d0 0x7f62ac15ff80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 >> 172.21.15.78:0/3398880348 conn(0x7f62ac0cf830 msgr2=0x7f62ac15c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 shutdown_connections 2024-09-11T12:53:11.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.756+0000 7f62b0a00640 1 -- 172.21.15.78:0/3398880348 wait complete. 2024-09-11T12:53:11.776 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:53:11.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 -- 172.21.15.78:0/3690660662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 msgr2=0x7f9b8c004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 --2- 172.21.15.78:0/3690660662 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9b8c004020 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9b98001860 tx=0x7f9b9801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9ba9800640 1 -- 172.21.15.78:0/3690660662 reap_dead start 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 -- 172.21.15.78:0/3690660662 shutdown_connections 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 -- 172.21.15.78:0/3690660662 >> 172.21.15.78:0/3690660662 conn(0x7f9ba40ceab0 msgr2=0x7f9ba40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 -- 172.21.15.78:0/3690660662 shutdown_connections 2024-09-11T12:53:11.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.916+0000 7f9bab800640 1 -- 172.21.15.78:0/3690660662 wait complete. 2024-09-11T12:53:11.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 Processor -- start 2024-09-11T12:53:11.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- start start 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ba4164250 0x7f9ba4157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ba4155b50 0x7f9ba41678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4169f40 con 0x7f9b8c003c20 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba4169dc0 con 0x7f9ba4155b50 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba416a0c0 con 0x7f9ba4164250 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60442/0 (socket says 172.21.15.78:60442) 2024-09-11T12:53:11.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 -- 172.21.15.78:0/2912478866 learned_addr learned my addr 172.21.15.78:0/2912478866 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:11.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba3e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ba4164250 0x7f9ba4157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba9800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ba4155b50 0x7f9ba41678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 -- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ba4164250 msgr2=0x7f9ba4157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ba4164250 0x7f9ba4157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 -- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ba4155b50 msgr2=0x7f9ba41678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:11.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ba4155b50 0x7f9ba41678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:11.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 -- 172.21.15.78:0/2912478866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba41563d0 con 0x7f9b8c003c20 2024-09-11T12:53:11.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba8e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9b98005040 tx=0x7f9b980a5ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b98001480 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b980b4030 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b980068d0 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba41568c0 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.920+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba4156e80 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba40d0ec0 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f9b98003b50 con 0x7f9b8c003c20 2024-09-11T12:53:11.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9ba0c00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 0x7f9b84073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:11.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f9b9812c190 con 0x7f9b8c003c20 2024-09-11T12:53:11.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9ba3e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 0x7f9b84073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:11.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.924+0000 7f9ba3e00640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 0x7f9b84073ed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4143a40 tx=0x7f9b900091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:11.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:11.928+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9b98016690 con 0x7f9b8c003c20 2024-09-11T12:53:12.048 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:53:12.053 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:53:12.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.156+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f9ba4169210 con 0x7f9b8c003c20 2024-09-11T12:53:12.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9ba0c00640 1 -- 172.21.15.78:0/2912478866 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v195) ==== 161+0+0 (secure 0 0 0) 0x7f9b981000f0 con 0x7f9b8c003c20 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 msgr2=0x7f9b84073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 0x7f9b84073ed0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9ba4143a40 tx=0x7f9b900091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 msgr2=0x7f9ba40703c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9b98005040 tx=0x7f9b980a5ca0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 shutdown_connections 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b84071a10 0x7f9b84073ed0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ba4155b50 0x7f9ba41678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ba4164250 0x7f9ba4157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 --2- 172.21.15.78:0/2912478866 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9b8c003c20 0x7f9ba40703c0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 >> 172.21.15.78:0/2912478866 conn(0x7f9ba40ceab0 msgr2=0x7f9ba40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 shutdown_connections 2024-09-11T12:53:12.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.404+0000 7f9bab800640 1 -- 172.21.15.78:0/2912478866 wait complete. 2024-09-11T12:53:12.426 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-11T12:53:12.426 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:53:12.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- 172.21.15.78:0/1061853961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 msgr2=0x7f1f9006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1061853961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9006ec50 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009900 tx=0x7f1f7c01c850 comp rx=0 tx=0).stop 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- 172.21.15.78:0/1061853961 shutdown_connections 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1061853961 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f900b3250 0x7f1f900b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1061853961 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9006ec50 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- 172.21.15.78:0/1061853961 >> 172.21.15.78:0/1061853961 conn(0x7f1f900cfeb0 msgr2=0x7f1f900d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:12.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- 172.21.15.78:0/1061853961 shutdown_connections 2024-09-11T12:53:12.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- 172.21.15.78:0/1061853961 wait complete. 2024-09-11T12:53:12.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 Processor -- start 2024-09-11T12:53:12.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.568+0000 7f1f8f200640 1 -- start start 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f900b3250 0x7f1f901f86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 0x7f1f90135090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901679a0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90167820 con 0x7f1f900b3250 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1f90167b20 con 0x7f1f9015a6d0 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60454/0 (socket says 172.21.15.78:60454) 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 -- 172.21.15.78:0/1700474872 learned_addr learned my addr 172.21.15.78:0/1700474872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:12.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8de00640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f900b3250 0x7f1f901f86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8e800640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 0x7f1f90135090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 -- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 msgr2=0x7f1f90135090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 0x7f1f90135090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 -- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f900b3250 msgr2=0x7f1f901f86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f900b3250 0x7f1f901f86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 -- 172.21.15.78:0/1700474872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f901f8bf0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8e800640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 0x7f1f90135090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8d400640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009850 tx=0x7f1f7c0a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:12.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c0b2040 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f7c004400 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f900b17c0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f7c005290 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f900b1d80 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f900b36e0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.572+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f1f7c005430 con 0x7f1f900cd9b0 2024-09-11T12:53:12.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.576+0000 7f1f86000640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 0x7f1f70073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.576+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f1f7c12bc90 con 0x7f1f900cd9b0 2024-09-11T12:53:12.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.576+0000 7f1f8de00640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 0x7f1f70073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.576+0000 7f1f8de00640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 0x7f1f70073ea0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f80001650 tx=0x7f1f80009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:12.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.580+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1f7c0164d0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.752+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1f9006ec50 con 0x7f1f900cd9b0 2024-09-11T12:53:12.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.752+0000 7f1f86000640 1 -- 172.21.15.78:0/1700474872 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13032 (secure 0 0 0) 0x7f1f7c0f78b0 con 0x7f1f900cd9b0 2024-09-11T12:53:12.755 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:12.755 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":61,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:11.396642+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6834/2934440940":"2024-09-12T12:53:08.386036+0000","172.21.15.78:6832/2934440940":"2024-09-12T12:53:08.386036+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:12.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 msgr2=0x7f1f70073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 0x7f1f70073ea0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1f80001650 tx=0x7f1f80009040 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 msgr2=0x7f1f9014d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c009850 tx=0x7f1f7c0a5710 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 shutdown_connections 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1f700719e0 0x7f1f70073ea0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1f9015a6d0 0x7f1f90135090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1f900cd9b0 0x7f1f9014d520 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 --2- 172.21.15.78:0/1700474872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1f900b3250 0x7f1f901f86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 >> 172.21.15.78:0/1700474872 conn(0x7f1f900cfeb0 msgr2=0x7f1f900cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 shutdown_connections 2024-09-11T12:53:12.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.756+0000 7f1f8f200640 1 -- 172.21.15.78:0/1700474872 wait complete. 2024-09-11T12:53:12.776 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:53:12.777 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:53:12.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- 172.21.15.78:0/95405310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c0b2120 msgr2=0x7f841c0cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 --2- 172.21.15.78:0/95405310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c0b2120 0x7f841c0cdc10 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f8408009900 tx=0x7f840801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- 172.21.15.78:0/95405310 shutdown_connections 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 --2- 172.21.15.78:0/95405310 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 0x7f841c06fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 --2- 172.21.15.78:0/95405310 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c0b2120 0x7f841c0cdc10 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- 172.21.15.78:0/95405310 >> 172.21.15.78:0/95405310 conn(0x7f841c071dd0 msgr2=0x7f841c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- 172.21.15.78:0/95405310 shutdown_connections 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- 172.21.15.78:0/95405310 wait complete. 2024-09-11T12:53:12.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 Processor -- start 2024-09-11T12:53:12.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 -- start start 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b2120 0x7f841c142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.920+0000 7f8421e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 0x7f841c14c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f841c167960 con 0x7f841c1518b0 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f841c1677e0 con 0x7f841c0b2120 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f841c167ae0 con 0x7f841c0ce150 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60466/0 (socket says 172.21.15.78:60466) 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 -- 172.21.15.78:0/2579598435 learned_addr learned my addr 172.21.15.78:0/2579598435 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841aa00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b2120 0x7f841c142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841a000640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 0x7f841c14c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 -- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 msgr2=0x7f841c14c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 0x7f841c14c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 -- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b2120 msgr2=0x7f841c142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b2120 0x7f841c142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 -- 172.21.15.78:0/2579598435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f841c1426d0 con 0x7f841c1518b0 2024-09-11T12:53:12.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f841b400640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f841400d6d0 tx=0x7f841400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414017070 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8414004310 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8414009a80 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f841c1532b0 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f841c1fb150 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f841c0cdc10 con 0x7f841c1518b0 2024-09-11T12:53:12.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.924+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f8414009c20 con 0x7f841c1518b0 2024-09-11T12:53:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.928+0000 7f8413400640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 0x7f83fc073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.928+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f84140924e0 con 0x7f841c1518b0 2024-09-11T12:53:12.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.928+0000 7f841aa00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 0x7f83fc073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:12.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.928+0000 7f841aa00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 0x7f83fc073ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f841c15a020 tx=0x7f840c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:12.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:12.928+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f841405db70 con 0x7f841c1518b0 2024-09-11T12:53:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f841c0b3dd0 con 0x7f841c1518b0 2024-09-11T12:53:13.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8413400640 1 -- 172.21.15.78:0/2579598435 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7f84140620c0 con 0x7f841c1518b0 2024-09-11T12:53:13.097 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 msgr2=0x7f83fc073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 0x7f83fc073ea0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f841c15a020 tx=0x7f840c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 msgr2=0x7f841c14cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f841400d6d0 tx=0x7f841400db90 comp rx=0 tx=0).stop 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 shutdown_connections 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f83fc0719e0 0x7f83fc073ea0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f841c1518b0 0x7f841c14cd00 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f841c0ce150 0x7f841c14c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 --2- 172.21.15.78:0/2579598435 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f841c0b2120 0x7f841c142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 >> 172.21.15.78:0/2579598435 conn(0x7f841c071dd0 msgr2=0x7f841c15c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 shutdown_connections 2024-09-11T12:53:13.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.096+0000 7f8421e00640 1 -- 172.21.15.78:0/2579598435 wait complete. 2024-09-11T12:53:13.119 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- 172.21.15.78:0/3767825711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd64004d50 msgr2=0x7fdd640051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 --2- 172.21.15.78:0/3767825711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd64004d50 0x7fdd640051d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c0cfa10 tx=0x7fdd7401ce40 comp rx=0 tx=0).stop 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd81e00640 1 -- 172.21.15.78:0/3767825711 reap_dead start 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- 172.21.15.78:0/3767825711 shutdown_connections 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- 172.21.15.78:0/3767825711 >> 172.21.15.78:0/3767825711 conn(0x7fdd7c06ca80 msgr2=0x7fdd7c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- 172.21.15.78:0/3767825711 shutdown_connections 2024-09-11T12:53:13.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- 172.21.15.78:0/3767825711 wait complete. 2024-09-11T12:53:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 Processor -- start 2024-09-11T12:53:13.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- start start 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64004d50 0x7fdd7c155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c153230 0x7fdd7c151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c0b4520 con 0x7fdd7c15d360 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdd7c0ccb10 con 0x7fdd7c153230 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd83e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdd7c0b46a0 con 0x7fdd64004d50 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd81e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:13.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd81e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60482/0 (socket says 172.21.15.78:60482) 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd81e00640 1 -- 172.21.15.78:0/574712344 learned_addr learned my addr 172.21.15.78:0/574712344 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd80a00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c153230 0x7fdd7c151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.260+0000 7fdd81400640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64004d50 0x7fdd7c155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 -- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64004d50 msgr2=0x7fdd7c155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64004d50 0x7fdd7c155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 -- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c153230 msgr2=0x7fdd7c151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c153230 0x7fdd7c151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 -- 172.21.15.78:0/574712344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd7c1f59e0 con 0x7fdd7c15d360 2024-09-11T12:53:13.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd81e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdd74098a10 tx=0x7fdd74006600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74000e50 con 0x7fdd7c15d360 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd74006b50 con 0x7fdd7c15d360 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd74001500 con 0x7fdd7c15d360 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd7c1f5cc0 con 0x7fdd7c15d360 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd7c1f6280 con 0x7fdd7c15d360 2024-09-11T12:53:13.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd7c1f6660 con 0x7fdd7c15d360 2024-09-11T12:53:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fdd74003600 con 0x7fdd7c15d360 2024-09-11T12:53:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 0x7fdd5c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.264+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fdd7412b690 con 0x7fdd7c15d360 2024-09-11T12:53:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.268+0000 7fdd81400640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 0x7fdd5c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.268+0000 7fdd81400640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 0x7fdd5c073ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c14d3f0 tx=0x7fdd70009500 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.268+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdd7c1f6660 con 0x7fdd7c15d360 2024-09-11T12:53:13.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:13.496+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fdd7c15bea0 con 0x7fdd7c15d360 2024-09-11T12:53:14.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd79600640 1 -- 172.21.15.78:0/574712344 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v197) ==== 134+0+0 (secure 0 0 0) 0x7fdd740bf030 con 0x7fdd7c15d360 2024-09-11T12:53:14.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 msgr2=0x7fdd5c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 0x7fdd5c073ea0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c14d3f0 tx=0x7fdd70009500 comp rx=0 tx=0).stop 2024-09-11T12:53:14.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 msgr2=0x7fdd7c149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fdd74098a10 tx=0x7fdd74006600 comp rx=0 tx=0).stop 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 shutdown_connections 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fdd5c0719e0 0x7fdd5c073ea0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdd7c15d360 0x7fdd7c149470 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdd7c153230 0x7fdd7c151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 --2- 172.21.15.78:0/574712344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdd64004d50 0x7fdd7c155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 >> 172.21.15.78:0/574712344 conn(0x7fdd7c06ca80 msgr2=0x7fdd7c158190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 shutdown_connections 2024-09-11T12:53:14.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.412+0000 7fdd83e00640 1 -- 172.21.15.78:0/574712344 wait complete. 2024-09-11T12:53:14.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:53:14.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- 172.21.15.78:0/3157952056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac074240 msgr2=0x7f9bac15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/3157952056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac074240 0x7f9bac15ba80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009bc0 tx=0x7f9b9c01cb70 comp rx=0 tx=0).stop 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- 172.21.15.78:0/3157952056 shutdown_connections 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/3157952056 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac15e5b0 0x7f9bac15e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/3157952056 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac074240 0x7f9bac15ba80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- 172.21.15.78:0/3157952056 >> 172.21.15.78:0/3157952056 conn(0x7f9bac06ca80 msgr2=0x7f9bac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- 172.21.15.78:0/3157952056 shutdown_connections 2024-09-11T12:53:14.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- 172.21.15.78:0/3157952056 wait complete. 2024-09-11T12:53:14.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 Processor -- start 2024-09-11T12:53:14.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- start start 2024-09-11T12:53:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 0x7f9bac1ff4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 0x7f9bac155730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:14.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac070470 con 0x7f9bac15e5b0 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac167f50 con 0x7f9bac074240 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9bb0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9bac155c70 con 0x7f9bac06fc50 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60758/0 (socket says 172.21.15.78:60758) 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 -- 172.21.15.78:0/990226954 learned_addr learned my addr 172.21.15.78:0/990226954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9ba9e00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 0x7f9bac1ff4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9ba9400640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 0x7f9bac155730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 -- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 msgr2=0x7f9bac1ff4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 0x7f9bac1ff4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 -- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 msgr2=0x7f9bac155730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 0x7f9bac155730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9baa800640 1 -- 172.21.15.78:0/990226954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac1ffb60 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9ba9e00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 0x7f9bac1ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.680+0000 7f9ba9400640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 0x7f9bac155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9baa800640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400c410 tx=0x7f9ba400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4011790 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba4017070 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac1ffe40 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba4011cc0 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac152280 con 0x7f9bac15e5b0 2024-09-11T12:53:14.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bac200360 con 0x7f9bac15e5b0 2024-09-11T12:53:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f9ba401c460 con 0x7f9bac15e5b0 2024-09-11T12:53:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 0x7f9b8c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.684+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f9ba4091e60 con 0x7f9bac15e5b0 2024-09-11T12:53:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.688+0000 7f9ba9e00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 0x7f9b8c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.688+0000 7f9ba9e00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 0x7f9b8c073ea0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0af000 tx=0x7f9b9c01b3b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.688+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9bac200360 con 0x7f9bac15e5b0 2024-09-11T12:53:14.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f9bac152480 con 0x7f9bac15e5b0 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9ba2000640 1 -- 172.21.15.78:0/990226954 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7f9bac152480 con 0x7f9bac15e5b0 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:53:14.857 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:53:14.858 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:53:14.859 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 msgr2=0x7f9b8c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 0x7f9b8c073ea0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c0af000 tx=0x7f9b9c01b3b0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 msgr2=0x7f9bac1527a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9ba400c410 tx=0x7f9ba400c8d0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 shutdown_connections 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f9b8c0719e0 0x7f9b8c073ea0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bac15e5b0 0x7f9bac1527a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bac074240 0x7f9bac155730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 --2- 172.21.15.78:0/990226954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bac06fc50 0x7f9bac1ff4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 >> 172.21.15.78:0/990226954 conn(0x7f9bac06ca80 msgr2=0x7f9bac071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 shutdown_connections 2024-09-11T12:53:14.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:14.856+0000 7f9bb0a00640 1 -- 172.21.15.78:0/990226954 wait complete. 2024-09-11T12:53:14.880 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- 172.21.15.78:0/447915914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 msgr2=0x7f2e600cec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- 172.21.15.78:0/447915914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e600cec10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f2e58009c80 tx=0x7f2e5801cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- 172.21.15.78:0/447915914 shutdown_connections 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- 172.21.15.78:0/447915914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e600cec10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- 172.21.15.78:0/447915914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e600736a0 0x7f2e600ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- 172.21.15.78:0/447915914 >> 172.21.15.78:0/447915914 conn(0x7f2e600d0410 msgr2=0x7f2e600d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:15.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- 172.21.15.78:0/447915914 shutdown_connections 2024-09-11T12:53:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- 172.21.15.78:0/447915914 wait complete. 2024-09-11T12:53:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 Processor -- start 2024-09-11T12:53:15.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- start start 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e600736a0 0x7f2e60159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e60144640 0x7f2e6013f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e601690c0 con 0x7f2e60071d90 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e60168f40 con 0x7f2e600736a0 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e65e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e60169240 con 0x7f2e60144640 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e5ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e5ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60788/0 (socket says 172.21.15.78:60788) 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e5ea00640 1 -- 172.21.15.78:0/4253350922 learned_addr learned my addr 172.21.15.78:0/4253350922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:15.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e5e000640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e600736a0 0x7f2e60159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.024+0000 7f2e5f400640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e60144640 0x7f2e6013f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 -- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e60144640 msgr2=0x7f2e6013f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e60144640 0x7f2e6013f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 -- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e600736a0 msgr2=0x7f2e60159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e600736a0 0x7f2e60159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 -- 172.21.15.78:0/4253350922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e60159e50 con 0x7f2e60071d90 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e5ea00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c3c0 tx=0x7f2e5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500052d0 con 0x7f2e60071d90 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e50007020 con 0x7f2e60071d90 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e50007470 con 0x7f2e60071d90 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e6013fe40 con 0x7f2e60071d90 2024-09-11T12:53:15.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e601faee0 con 0x7f2e60071d90 2024-09-11T12:53:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e600d0990 con 0x7f2e60071d90 2024-09-11T12:53:15.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.028+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f2e500067e0 con 0x7f2e60071d90 2024-09-11T12:53:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.032+0000 7f2e57400640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 0x7f2e40073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.032+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f2e500913f0 con 0x7f2e60071d90 2024-09-11T12:53:15.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.032+0000 7f2e5e000640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 0x7f2e40073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.032+0000 7f2e5e000640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 0x7f2e40073ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e6015aa80 tx=0x7f2e4c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.036+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2e60159e50 con 0x7f2e60071d90 2024-09-11T12:53:15.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.200+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2e60072f10 con 0x7f2e60071d90 2024-09-11T12:53:15.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e57400640 1 -- 172.21.15.78:0/4253350922 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f2e50001090 con 0x7f2e60071d90 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 msgr2=0x7f2e40073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 0x7f2e40073ea0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2e6015aa80 tx=0x7f2e4c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 msgr2=0x7f2e601fb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f2e5000c3c0 tx=0x7f2e5000c880 comp rx=0 tx=0).stop 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 shutdown_connections 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2e400719e0 0x7f2e40073ea0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e60144640 0x7f2e6013f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e600736a0 0x7f2e60159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 --2- 172.21.15.78:0/4253350922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e60071d90 0x7f2e601fb9c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 >> 172.21.15.78:0/4253350922 conn(0x7f2e600d0410 msgr2=0x7f2e6015b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 shutdown_connections 2024-09-11T12:53:15.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.204+0000 7f2e65e00640 1 -- 172.21.15.78:0/4253350922 wait complete. 2024-09-11T12:53:15.226 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:53:15.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:15.314 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:15.395 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:53:15.395 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:15.395 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:15.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:15.404 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:15.404 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:15.413 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:15.413 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:15.413 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:15.455 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:15.456 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:15.456 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:15.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:15.465 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:53:15.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:15.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.636+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2899532228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c980cf460 msgr2=0x7f2c980700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.636+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2899532228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c980cf460 0x7f2c980700c0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2c98074cf0 tx=0x7f2c9001c800 comp rx=0 tx=0).stop 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.636+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2899532228 shutdown_connections 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.636+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2899532228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c980cf460 0x7f2c980700c0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.636+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2899532228 >> v2:172.21.15.157:3300/0 conn(0x7f2c9813d840 0x7f2c98140980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2899532228 >> 172.21.15.78:0/2899532228 conn(0x7f2c9806ca80 msgr2=0x7f2c980b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2899532228 shutdown_connections 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2899532228 wait complete. 2024-09-11T12:53:15.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 Processor -- start 2024-09-11T12:53:15.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- start start 2024-09-11T12:53:15.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c980cf460 0x7f2c98150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9813d840 0x7f2c980a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98167820 con 0x7f2c9806fc50 2024-09-11T12:53:15.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2c98140380 con 0x7f2c9813d840 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2c981679a0 con 0x7f2c980cf460 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60798/0 (socket says 172.21.15.78:60798) 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 -- 172.21.15.78:0/2793468769 learned_addr learned my addr 172.21.15.78:0/2793468769 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 -- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c980cf460 msgr2=0x7f2c98150ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c980cf460 0x7f2c98150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 -- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9813d840 msgr2=0x7f2c980a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9813d840 0x7f2c980a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 -- 172.21.15.78:0/2793468769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9815c230 con 0x7f2c9806fc50 2024-09-11T12:53:15.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9d000640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ea20 tx=0x7f2c8800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:15.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8800cde0 con 0x7f2c9806fc50 2024-09-11T12:53:15.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c88004c20 con 0x7f2c9806fc50 2024-09-11T12:53:15.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9815c510 con 0x7f2c9806fc50 2024-09-11T12:53:15.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.640+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c88005020 con 0x7f2c9806fc50 2024-09-11T12:53:15.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c980af660 con 0x7f2c9806fc50 2024-09-11T12:53:15.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c980700c0 con 0x7f2c9806fc50 2024-09-11T12:53:15.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f2c880051c0 con 0x7f2c9806fc50 2024-09-11T12:53:15.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c94c00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 0x7f2c78073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:15.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f2c88092ad0 con 0x7f2c9806fc50 2024-09-11T12:53:15.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.644+0000 7f2c97e00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 0x7f2c78073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:15.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.648+0000 7f2c97e00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 0x7f2c78073ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2c840046c0 tx=0x7f2c84007910 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:15.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.648+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2c88016030 con 0x7f2c9806fc50 2024-09-11T12:53:15.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c9815ed20 con 0x7f2c9806fc50 2024-09-11T12:53:15.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c94c00640 1 -- 172.21.15.78:0/2793468769 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3917 (secure 0 0 0) 0x7f2c8805e670 con 0x7f2c9806fc50 2024-09-11T12:53:15.849 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:15.850 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":39,"btime":"2024-09-11T12:53:09:389034+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5216,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2191215983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2191215983},{"type":"v1","addr":"172.21.15.157:6836","nonce":2191215983}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5222,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3936246974","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3936246974},{"type":"v1","addr":"172.21.15.78:6835","nonce":3936246974}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5569,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6837/2336528712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2336528712},{"type":"v1","addr":"172.21.15.157:6837","nonce":2336528712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3115229889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3115229889},{"type":"v1","addr":"172.21.15.78:6834","nonce":3115229889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-11T12:53:15.850 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 39 2024-09-11T12:53:15.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 msgr2=0x7f2c78073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 0x7f2c78073ea0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f2c840046c0 tx=0x7f2c84007910 comp rx=0 tx=0).stop 2024-09-11T12:53:15.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 msgr2=0x7f2c980af120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:15.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2c8800ea20 tx=0x7f2c8800eee0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 shutdown_connections 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2c780719e0 0x7f2c78073ea0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2c9813d840 0x7f2c980a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2c980cf460 0x7f2c98150ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 --2- 172.21.15.78:0/2793468769 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2c9806fc50 0x7f2c980af120 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 >> 172.21.15.78:0/2793468769 conn(0x7f2c9806ca80 msgr2=0x7f2c98072ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 shutdown_connections 2024-09-11T12:53:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:15.848+0000 7f2c9fa00640 1 -- 172.21.15.78:0/2793468769 wait complete. 2024-09-11T12:53:15.870 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:53:15.870 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:53:15.870 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:53:15.870 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:15.870 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:53:15.871 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:53:15.871 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:53:15.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:15.871 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:53:15.871 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:53:15.871 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:53:15.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:15.872 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:53:15.872 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:53:15.872 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:53:15.872 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:15.885 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:53:15.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:53:15.886 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:53:15.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:53:15.887 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:53:15.888 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:53:15.888 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:53:15.889 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:53:15.930 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:53:15.931 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:15.932 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:53:15.932 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:15.935 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:53:15.935 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:15.937 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:53:15.937 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:15.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:53:16.017 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:53:16.018 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:53:16.029 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:53:16.029 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:53:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.084+0000 7fa268e00640 1 -- 172.21.15.78:0/1005365965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa264157ae0 msgr2=0x7fa264070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.084+0000 7fa268e00640 1 --2- 172.21.15.78:0/1005365965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa264157ae0 0x7fa264070a00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa254009900 tx=0x7fa25401c850 comp rx=0 tx=0).stop 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- 172.21.15.78:0/1005365965 shutdown_connections 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 --2- 172.21.15.78:0/1005365965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa26415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 --2- 172.21.15.78:0/1005365965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa264157ae0 0x7fa264070a00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- 172.21.15.78:0/1005365965 >> 172.21.15.78:0/1005365965 conn(0x7fa26406ca80 msgr2=0x7fa2640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- 172.21.15.78:0/1005365965 shutdown_connections 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- 172.21.15.78:0/1005365965 wait complete. 2024-09-11T12:53:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 Processor -- start 2024-09-11T12:53:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- start start 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa264143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 0x7fa2641619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa264157ae0 0x7fa2640b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa264165080 con 0x7fa2640b4db0 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa264164f00 con 0x7fa264157ae0 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa264165200 con 0x7fa2640736d0 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 0x7fa2641619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 0x7fa2641619b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60840/0 (socket says 172.21.15.78:60840) 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261400640 1 -- 172.21.15.78:0/1212611415 learned_addr learned my addr 172.21.15.78:0/1212611415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa264143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa262800640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa264157ae0 0x7fa2640b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 -- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa264157ae0 msgr2=0x7fa2640b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa264157ae0 0x7fa2640b0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 -- 172.21.15.78:0/1212611415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 msgr2=0x7fa2641619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 0x7fa2641619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 -- 172.21.15.78:0/1212611415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa264143dc0 con 0x7fa2640736d0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa261e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa264143880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2540ac000 tx=0x7fa254005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2540b2040 con 0x7fa2640736d0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa254004330 con 0x7fa2640736d0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa254003860 con 0x7fa2640736d0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.088+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa264154380 con 0x7fa2640736d0 2024-09-11T12:53:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.092+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa264154d10 con 0x7fa2640736d0 2024-09-11T12:53:16.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.092+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa26415af90 con 0x7fa2640736d0 2024-09-11T12:53:16.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.092+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fa254003a00 con 0x7fa2640736d0 2024-09-11T12:53:16.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.092+0000 7fa25a000640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 0x7fa244073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.092+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fa25412bf20 con 0x7fa2640736d0 2024-09-11T12:53:16.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.096+0000 7fa261400640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 0x7fa244073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.096+0000 7fa261400640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 0x7fa244073ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26415f430 tx=0x7fa25000b480 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:16.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.096+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa2540bd060 con 0x7fa2640736d0 2024-09-11T12:53:16.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.272+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fa2641595f0 con 0x7fa2640736d0 2024-09-11T12:53:16.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.428+0000 7fa25a000640 1 -- 172.21.15.78:0/1212611415 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v63) ==== 97+0+0 (secure 0 0 0) 0x7fa2540164d0 con 0x7fa2640736d0 2024-09-11T12:53:16.432 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 msgr2=0x7fa244073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 0x7fa244073ed0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa26415f430 tx=0x7fa25000b480 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 msgr2=0x7fa264143880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa264143880 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2540ac000 tx=0x7fa254005e70 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 shutdown_connections 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fa244071a10 0x7fa244073ed0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa264157ae0 0x7fa2640b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2640b4db0 0x7fa2641619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 --2- 172.21.15.78:0/1212611415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2640736d0 0x7fa264143880 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 >> 172.21.15.78:0/1212611415 conn(0x7fa26406ca80 msgr2=0x7fa264159960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 shutdown_connections 2024-09-11T12:53:16.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.432+0000 7fa268e00640 1 -- 172.21.15.78:0/1212611415 wait complete. 2024-09-11T12:53:16.455 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:53:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- 172.21.15.78:0/810626366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380b3230 msgr2=0x7f483806ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- 172.21.15.78:0/810626366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380b3230 0x7f483806ff80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4830009bc0 tx=0x7f483001cac0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- 172.21.15.78:0/810626366 shutdown_connections 2024-09-11T12:53:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- 172.21.15.78:0/810626366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380b3230 0x7f483806ff80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- 172.21.15.78:0/810626366 >> v2:172.21.15.157:3300/0 conn(0x7f48380cee70 0x7f48380ce3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- 172.21.15.78:0/810626366 >> 172.21.15.78:0/810626366 conn(0x7f48380cfeb0 msgr2=0x7f48380d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:16.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- 172.21.15.78:0/810626366 shutdown_connections 2024-09-11T12:53:16.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- 172.21.15.78:0/810626366 wait complete. 2024-09-11T12:53:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 Processor -- start 2024-09-11T12:53:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- start start 2024-09-11T12:53:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 0x7f4838138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4838142bf0 0x7f483815a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f48381681a0 con 0x7f48380b3230 2024-09-11T12:53:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4838168020 con 0x7f4838142bf0 2024-09-11T12:53:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f483d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4838168320 con 0x7f48380cee70 2024-09-11T12:53:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f4837400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f4837400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60854/0 (socket says 172.21.15.78:60854) 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f4837400640 1 -- 172.21.15.78:0/3662777329 learned_addr learned my addr 172.21.15.78:0/3662777329 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f4837e00640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4838142bf0 0x7f483815a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.596+0000 7f482fe00640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 0x7f4838138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 -- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 msgr2=0x7f4838138690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 0x7f4838138690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 -- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4838142bf0 msgr2=0x7f483815a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4838142bf0 0x7f483815a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 -- 172.21.15.78:0/3662777329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4838159cf0 con 0x7f48380b3230 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482fe00640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 0x7f4838138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f4837400640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f482800ea20 tx=0x7f482800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800cd80 con 0x7f48380b3230 2024-09-11T12:53:16.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4828010430 con 0x7f48380b3230 2024-09-11T12:53:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828010860 con 0x7f48380b3230 2024-09-11T12:53:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f483815b160 con 0x7f48380b3230 2024-09-11T12:53:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381fb3d0 con 0x7f48380b3230 2024-09-11T12:53:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f483806ff80 con 0x7f48380b3230 2024-09-11T12:53:16.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f4828010a00 con 0x7f48380b3230 2024-09-11T12:53:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 0x7f4814073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.600+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f4828092b60 con 0x7f48380b3230 2024-09-11T12:53:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.604+0000 7f482fe00640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 0x7f4814073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.604+0000 7f482fe00640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 0x7f4814073ed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4820001ec0 tx=0x7f4820009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:16.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.604+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f482805e120 con 0x7f48380b3230 2024-09-11T12:53:16.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.828+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f48381fb7d0 con 0x7f48380b3230 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.828+0000 7f482f400640 1 -- 172.21.15.78:0/3662777329 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5617 (secure 0 0 0) 0x7f4828062670 con 0x7f48380b3230 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.832 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:53:16.835 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.836 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.836 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.836 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.838 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.840 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.842 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:16.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.844 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.alice", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCmkuFm1rZtMxAAfCphqJqGylhLBu2J2ZZ5sg==", 2024-09-11T12:53:16.845 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol94520", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.846 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:16.847 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:53:16.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 msgr2=0x7f4814073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 0x7f4814073ed0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4820001ec0 tx=0x7f4820009040 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 msgr2=0x7f48381fbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f482800ea20 tx=0x7f482800eee0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 shutdown_connections 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4814071a10 0x7f4814073ed0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4838142bf0 0x7f483815a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f48380cee70 0x7f4838138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 --2- 172.21.15.78:0/3662777329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f48380b3230 0x7f48381fbbd0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 >> 172.21.15.78:0/3662777329 conn(0x7f48380cfeb0 msgr2=0x7f48380b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 shutdown_connections 2024-09-11T12:53:16.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.832+0000 7f483d000640 1 -- 172.21.15.78:0/3662777329 wait complete. 2024-09-11T12:53:16.853 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- 172.21.15.78:0/2841557528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db00052f0 msgr2=0x7f7db000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 --2- 172.21.15.78:0/2841557528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7db00052f0 0x7f7db000b9d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c610 tx=0x7f7dc401fa60 comp rx=0 tx=0).stop 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dcb400640 1 -- 172.21.15.78:0/2841557528 reap_dead start 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- 172.21.15.78:0/2841557528 shutdown_connections 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- 172.21.15.78:0/2841557528 >> 172.21.15.78:0/2841557528 conn(0x7f7dcc06ca80 msgr2=0x7f7dcc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- 172.21.15.78:0/2841557528 shutdown_connections 2024-09-11T12:53:16.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- 172.21.15.78:0/2841557528 wait complete. 2024-09-11T12:53:16.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 Processor -- start 2024-09-11T12:53:16.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:16.996+0000 7f7dd1600640 1 -- start start 2024-09-11T12:53:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db00052f0 0x7f7dcc0cecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dcc1588e0 0x7f7dcc158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc070b80 con 0x7f7dcc152620 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dcc070a00 con 0x7f7db00052f0 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7dcc070d00 con 0x7f7dcc1588e0 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60866/0 (socket says 172.21.15.78:60866) 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 -- 172.21.15.78:0/313709390 learned_addr learned my addr 172.21.15.78:0/313709390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dcaa00640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db00052f0 0x7f7dcc0cecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dcb400640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dcc1588e0 0x7f7dcc158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 -- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dcc1588e0 msgr2=0x7f7dcc158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dcc1588e0 0x7f7dcc158d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 -- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db00052f0 msgr2=0x7f7dcc0cecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db00052f0 0x7f7dcc0cecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 -- 172.21.15.78:0/313709390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dcc1f80f0 con 0x7f7dcc152620 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dca000640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b520 tx=0x7f7db800b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8014070 con 0x7f7dcc152620 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7db8010980 con 0x7f7dcc152620 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dcc1f83d0 con 0x7f7dcc152620 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db8010dd0 con 0x7f7dcc152620 2024-09-11T12:53:17.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dcc1f8990 con 0x7f7dcc152620 2024-09-11T12:53:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dcc15af90 con 0x7f7dcc152620 2024-09-11T12:53:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.000+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f7db8017070 con 0x7f7dcc152620 2024-09-11T12:53:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.004+0000 7f7dc2a00640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 0x7f7dac073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.004+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f7db8091e40 con 0x7f7dcc152620 2024-09-11T12:53:17.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.004+0000 7f7dcaa00640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 0x7f7dac073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.004+0000 7f7dcaa00640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 0x7f7dac073ed0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc150e50 tx=0x7f7dbc005f10 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.008+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7db805d4f0 con 0x7f7dcc152620 2024-09-11T12:53:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.232+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7f7dcc0b49d0 con 0x7f7dcc152620 2024-09-11T12:53:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dc2a00640 1 -- 172.21.15.78:0/313709390 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v17) ==== 82+0+0 (secure 0 0 0) 0x7f7db8061a40 con 0x7f7dcc152620 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 msgr2=0x7f7dac073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 0x7f7dac073ed0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc150e50 tx=0x7f7dbc005f10 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 msgr2=0x7f7dcc152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f7db800b520 tx=0x7f7db800b9e0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 shutdown_connections 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dac071a10 0x7f7dac073ed0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dcc1588e0 0x7f7dcc158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dcc152620 0x7f7dcc152aa0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 --2- 172.21.15.78:0/313709390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7db00052f0 0x7f7dcc0cecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 >> 172.21.15.78:0/313709390 conn(0x7f7dcc06ca80 msgr2=0x7f7dcc0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 shutdown_connections 2024-09-11T12:53:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.236+0000 7f7dd1600640 1 -- 172.21.15.78:0/313709390 wait complete. 2024-09-11T12:53:17.258 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:53:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 -- 172.21.15.78:0/2313214040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872013d840 msgr2=0x7f8720140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 --2- 172.21.15.78:0/2313214040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872013d840 0x7f8720140980 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f8710009bc0 tx=0x7f871001cba0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 -- 172.21.15.78:0/2313214040 shutdown_connections 2024-09-11T12:53:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 --2- 172.21.15.78:0/2313214040 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872013d840 0x7f8720140980 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 -- 172.21.15.78:0/2313214040 >> 172.21.15.78:0/2313214040 conn(0x7f872006ca80 msgr2=0x7f87200b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:17.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 -- 172.21.15.78:0/2313214040 shutdown_connections 2024-09-11T12:53:17.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.396+0000 7f8725a00640 1 -- 172.21.15.78:0/2313214040 wait complete. 2024-09-11T12:53:17.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 Processor -- start 2024-09-11T12:53:17.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- start start 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87200cf460 0x7f8720154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f872013d840 0x7f87200724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8720168e30 con 0x7f872006fc50 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87201403d0 con 0x7f87200cf460 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8720168fb0 con 0x7f872013d840 2024-09-11T12:53:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60892/0 (socket says 172.21.15.78:60892) 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 -- 172.21.15.78:0/3626275119 learned_addr learned my addr 172.21.15.78:0/3626275119 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871e000640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87200cf460 0x7f8720154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871f400640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f872013d840 0x7f87200724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 -- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f872013d840 msgr2=0x7f87200724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f872013d840 0x7f87200724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 -- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87200cf460 msgr2=0x7f8720154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87200cf460 0x7f8720154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 -- 172.21.15.78:0/3626275119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87200ada80 con 0x7f872006fc50 2024-09-11T12:53:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f871ea00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8710009b10 tx=0x7f87100a5a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100bb070 con 0x7f872006fc50 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8710004b40 con 0x7f872006fc50 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87200add60 con 0x7f872006fc50 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87100b2040 con 0x7f872006fc50 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87200ae320 con 0x7f872006fc50 2024-09-11T12:53:17.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.400+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8720140980 con 0x7f872006fc50 2024-09-11T12:53:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.404+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f87100be070 con 0x7f872006fc50 2024-09-11T12:53:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.404+0000 7f8716a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 0x7f8700073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.404+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f871012bd50 con 0x7f872006fc50 2024-09-11T12:53:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.404+0000 7f871e000640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 0x7f8700073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.404+0000 7f871e000640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 0x7f8700073ea0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87200aebe0 tx=0x7f870c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.408+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8710016790 con 0x7f872006fc50 2024-09-11T12:53:17.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.576+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f872015e910 con 0x7f872006fc50 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.576+0000 7f8716a00640 1 -- 172.21.15.78:0/3626275119 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v65) ==== 84+0+65 (secure 0 0 0) 0x7f87100f7a50 con 0x7f872006fc50 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:53:17.578 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:53:17.579 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:17.579 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:53:17.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 msgr2=0x7f8700073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 0x7f8700073ea0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f87200aebe0 tx=0x7f870c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 msgr2=0x7f87200aee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8710009b10 tx=0x7f87100a5a00 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 shutdown_connections 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f87000719e0 0x7f8700073ea0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f872013d840 0x7f87200724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87200cf460 0x7f8720154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 --2- 172.21.15.78:0/3626275119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f872006fc50 0x7f87200aee60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 >> 172.21.15.78:0/3626275119 conn(0x7f872006ca80 msgr2=0x7f87200b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 shutdown_connections 2024-09-11T12:53:17.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.580+0000 7f8725a00640 1 -- 172.21.15.78:0/3626275119 wait complete. 2024-09-11T12:53:17.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:53:17.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- 172.21.15.78:0/2432345528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7118004da0 msgr2=0x7f7118005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 --2- 172.21.15.78:0/2432345528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7118004da0 0x7f7118005220 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f712c009eb0 tx=0x7f712c01d5b0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7133400640 1 -- 172.21.15.78:0/2432345528 reap_dead start 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- 172.21.15.78:0/2432345528 shutdown_connections 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- 172.21.15.78:0/2432345528 >> 172.21.15.78:0/2432345528 conn(0x7f71340b3050 msgr2=0x7f71340b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- 172.21.15.78:0/2432345528 shutdown_connections 2024-09-11T12:53:17.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- 172.21.15.78:0/2432345528 wait complete. 2024-09-11T12:53:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 Processor -- start 2024-09-11T12:53:17.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- start start 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 0x7f7134151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7134151770 0x7f71340addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f71340cfd20 con 0x7f71341e5840 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f71340cfba0 con 0x7f7134147970 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7139c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f71340cd490 con 0x7f7134151770 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60910/0 (socket says 172.21.15.78:60910) 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 -- 172.21.15.78:0/1708567431 learned_addr learned my addr 172.21.15.78:0/1708567431 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132a00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 0x7f7134151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7133400640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7134151770 0x7f71340addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 -- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7134151770 msgr2=0x7f71340addb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7134151770 0x7f71340addb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 -- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 msgr2=0x7f7134151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 0x7f7134151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132000640 1 -- 172.21.15.78:0/1708567431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7134147d50 con 0x7f71341e5840 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.748+0000 7f7132a00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 0x7f7134151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f7132000640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7120001470 tx=0x7f7120001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120015020 con 0x7f71341e5840 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7120011920 con 0x7f71341e5840 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7134155a80 con 0x7f71341e5840 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120011d70 con 0x7f71341e5840 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7134155d20 con 0x7f71341e5840 2024-09-11T12:53:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7134073710 con 0x7f71341e5840 2024-09-11T12:53:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.752+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f7120018020 con 0x7f71341e5840 2024-09-11T12:53:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.756+0000 7f712aa00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 0x7f7114073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.756+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4857+0+0 (secure 0 0 0) 0x7f71200929e0 con 0x7f71341e5840 2024-09-11T12:53:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.756+0000 7f7132a00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 0x7f7114073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:17.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.756+0000 7f7132a00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 0x7f7114073ea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c10 tx=0x7f7124001da0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:17.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.760+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f712001f020 con 0x7f71341e5840 2024-09-11T12:53:17.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:17.924+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f71340b5530 con 0x7f71341e5840 2024-09-11T12:53:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.436+0000 7f712aa00640 1 -- 172.21.15.78:0/1708567431 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v66) ==== 138+0+0 (secure 0 0 0) 0x7f7134147d50 con 0x7f71341e5840 2024-09-11T12:53:18.440 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:53:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 msgr2=0x7f7114073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:18.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 0x7f7114073ea0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f7124000c10 tx=0x7f7124001da0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 msgr2=0x7f71340ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f7120001470 tx=0x7f7120001930 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 shutdown_connections 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f71140719e0 0x7f7114073ea0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7134151770 0x7f71340addb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f71341e5840 0x7f71340ad870 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 --2- 172.21.15.78:0/1708567431 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7134147970 0x7f7134151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 >> 172.21.15.78:0/1708567431 conn(0x7f71340b3050 msgr2=0x7f7134159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 shutdown_connections 2024-09-11T12:53:18.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.440+0000 7f7139c00640 1 -- 172.21.15.78:0/1708567431 wait complete. 2024-09-11T12:53:18.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- 172.21.15.78:0/4165968759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f58003a40 msgr2=0x7f5f58003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 --2- 172.21.15.78:0/4165968759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f58003a40 0x7f5f58003e20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5f64001e60 tx=0x7f5f6401cc00 comp rx=0 tx=0).stop 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 -- 172.21.15.78:0/4165968759 reap_dead start 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- 172.21.15.78:0/4165968759 shutdown_connections 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- 172.21.15.78:0/4165968759 >> 172.21.15.78:0/4165968759 conn(0x7f5f740b2ec0 msgr2=0x7f5f740b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- 172.21.15.78:0/4165968759 shutdown_connections 2024-09-11T12:53:18.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- 172.21.15.78:0/4165968759 wait complete. 2024-09-11T12:53:18.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 Processor -- start 2024-09-11T12:53:18.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- start start 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f58003a40 0x7f5f7413be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f741fb900 0x7f5f74142310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74073a90 con 0x7f5f74142850 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5f74073910 con 0x7f5f741fb900 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f79e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5f74165e30 con 0x7f5f58003a40 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60928/0 (socket says 172.21.15.78:60928) 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 -- 172.21.15.78:0/2810954255 learned_addr learned my addr 172.21.15.78:0/2810954255 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f72000640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f741fb900 0x7f5f74142310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f72a00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f58003a40 0x7f5f7413be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:18.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 -- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f58003a40 msgr2=0x7f5f7413be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f58003a40 0x7f5f7413be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 -- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f741fb900 msgr2=0x7f5f74142310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f741fb900 0x7f5f74142310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.616+0000 7f5f73400640 1 -- 172.21.15.78:0/2810954255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f740ad740 con 0x7f5f74142850 2024-09-11T12:53:18.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f73400640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5f740af910 tx=0x7f5f6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c004240 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f6c0043e0 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00da70 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f740ada20 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f740ab2b0 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f740adf40 con 0x7f5f74142850 2024-09-11T12:53:18.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.620+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f5f6c00dc10 con 0x7f5f74142850 2024-09-11T12:53:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.624+0000 7f5f6b400640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 0x7f5f54073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.624+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 5227+0+0 (secure 0 0 0) 0x7f5f6c092790 con 0x7f5f74142850 2024-09-11T12:53:18.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.624+0000 7f5f72a00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 0x7f5f54073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:18.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.624+0000 7f5f72a00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 0x7f5f54073ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f64006fd0 tx=0x7f5f640199a0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:18.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.628+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5f6c05e210 con 0x7f5f74142850 2024-09-11T12:53:18.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:18.796+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f5f740cdb90 con 0x7f5f74142850 2024-09-11T12:53:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.444+0000 7f5f6b400640 1 -- 172.21.15.78:0/2810954255 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v67) ==== 170+0+0 (secure 0 0 0) 0x7f5f6c05db10 con 0x7f5f74142850 2024-09-11T12:53:19.447 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:53:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 msgr2=0x7f5f54073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 0x7f5f54073ed0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f5f64006fd0 tx=0x7f5f640199a0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 msgr2=0x7f5f740a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:19.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f5f740af910 tx=0x7f5f6c00ee80 comp rx=0 tx=0).stop 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 shutdown_connections 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5f54071a10 0x7f5f54073ed0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5f74142850 0x7f5f740a9460 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5f741fb900 0x7f5f74142310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 --2- 172.21.15.78:0/2810954255 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5f58003a40 0x7f5f7413be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 >> 172.21.15.78:0/2810954255 conn(0x7f5f740b2ec0 msgr2=0x7f5f740ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 shutdown_connections 2024-09-11T12:53:19.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.448+0000 7f5f79e00640 1 -- 172.21.15.78:0/2810954255 wait complete. 2024-09-11T12:53:19.485 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:53:19.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 -- 172.21.15.78:0/2771047692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 msgr2=0x7f907815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 --2- 172.21.15.78:0/2771047692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f907815a700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9068009c80 tx=0x7f906801cd10 comp rx=0 tx=0).stop 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 -- 172.21.15.78:0/2771047692 shutdown_connections 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 --2- 172.21.15.78:0/2771047692 >> v2:172.21.15.157:3300/0 conn(0x7f90780b1a80 0x7f90780b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 --2- 172.21.15.78:0/2771047692 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f907815a700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 -- 172.21.15.78:0/2771047692 >> 172.21.15.78:0/2771047692 conn(0x7f9078071dd0 msgr2=0x7f90780721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 -- 172.21.15.78:0/2771047692 shutdown_connections 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.628+0000 7f907e600640 1 -- 172.21.15.78:0/2771047692 wait complete. 2024-09-11T12:53:19.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 Processor -- start 2024-09-11T12:53:19.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- start start 2024-09-11T12:53:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:19.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90780b1ee0 0x7f90781f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f90781f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9078165e80 con 0x7f90780b1a80 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f90780af200 con 0x7f90780b1ee0 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9078166000 con 0x7f90780cf6e0 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60932/0 (socket says 172.21.15.78:60932) 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 -- 172.21.15.78:0/3335017814 learned_addr learned my addr 172.21.15.78:0/3335017814 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077e00640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f90781f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9076a00640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90780b1ee0 0x7f90781f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 -- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 msgr2=0x7f90781f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f90781f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 -- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90780b1ee0 msgr2=0x7f90781f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90780b1ee0 0x7f90781f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 -- 172.21.15.78:0/3335017814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907813c670 con 0x7f90780b1a80 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f9077400640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f906801c780 tx=0x7f9068005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680bb070 con 0x7f90780b1a80 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9068004040 con 0x7f90780b1a80 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90781f95b0 con 0x7f90780b1a80 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90680b2040 con 0x7f90780b1a80 2024-09-11T12:53:19.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.632+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90781f9b70 con 0x7f90780b1a80 2024-09-11T12:53:19.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90780b3040 con 0x7f90780b1a80 2024-09-11T12:53:19.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f9068004430 con 0x7f90780b1a80 2024-09-11T12:53:19.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f906f400640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 0x7f9058073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:19.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5609+0+0 (secure 0 0 0) 0x7f906812c110 con 0x7f90780b1a80 2024-09-11T12:53:19.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f9076a00640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 0x7f9058073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:19.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.636+0000 7f9076a00640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 0x7f9058073ea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90780aa830 tx=0x7f90640091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:19.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.640+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9068016990 con 0x7f90780b1a80 2024-09-11T12:53:19.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:19.844+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f90780affb0 con 0x7f90780b1a80 2024-09-11T12:53:20.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.460+0000 7f906f400640 1 -- 172.21.15.78:0/3335017814 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f90680f7590 con 0x7f90780b1a80 2024-09-11T12:53:20.463 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 10 and data pool 11 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.464+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 msgr2=0x7f9058073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.464+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 0x7f9058073ea0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f90780aa830 tx=0x7f90640091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.464+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 msgr2=0x7f907813c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.464+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f906801c780 tx=0x7f9068005c00 comp rx=0 tx=0).stop 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 shutdown_connections 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f90580719e0 0x7f9058073ea0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f90780cf6e0 0x7f90781f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f90780b1ee0 0x7f90781f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 --2- 172.21.15.78:0/3335017814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90780b1a80 0x7f907813c130 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 >> 172.21.15.78:0/3335017814 conn(0x7f9078071dd0 msgr2=0x7f90781593d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 shutdown_connections 2024-09-11T12:53:20.469 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.468+0000 7f907e600640 1 -- 172.21.15.78:0/3335017814 wait complete. 2024-09-11T12:53:20.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 -- 172.21.15.78:0/1256258915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a80741e0 msgr2=0x7f96a8070910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 --2- 172.21.15.78:0/1256258915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a80741e0 0x7f96a8070910 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f9698009c80 tx=0x7f969801cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 -- 172.21.15.78:0/1256258915 shutdown_connections 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 --2- 172.21.15.78:0/1256258915 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a8072230 0x7f96a806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 --2- 172.21.15.78:0/1256258915 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a80741e0 0x7f96a8070910 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 -- 172.21.15.78:0/1256258915 >> 172.21.15.78:0/1256258915 conn(0x7f96a806ca80 msgr2=0x7f96a80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:20.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 -- 172.21.15.78:0/1256258915 shutdown_connections 2024-09-11T12:53:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.640+0000 7f96ae400640 1 -- 172.21.15.78:0/1256258915 wait complete. 2024-09-11T12:53:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 Processor -- start 2024-09-11T12:53:20.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- start start 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8072230 0x7f96a81464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a80741e0 0x7f96a81469f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96a81680c0 con 0x7f96a8070190 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96a80cccc0 con 0x7f96a8072230 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96a8168240 con 0x7f96a80741e0 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60950/0 (socket says 172.21.15.78:60950) 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 -- 172.21.15.78:0/3544386018 learned_addr learned my addr 172.21.15.78:0/3544386018 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7e00640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a80741e0 0x7f96a81469f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a6a00640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8072230 0x7f96a81464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 -- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a80741e0 msgr2=0x7f96a81469f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a80741e0 0x7f96a81469f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 -- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8072230 msgr2=0x7f96a81464b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8072230 0x7f96a81464b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 -- 172.21.15.78:0/3544386018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96a80ce5a0 con 0x7f96a8070190 2024-09-11T12:53:20.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96a7400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f90 tx=0x7f9698009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980bb070 con 0x7f96a8070190 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9698004c50 con 0x7f96a8070190 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96980b2040 con 0x7f96a8070190 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96a80afda0 con 0x7f96a8070190 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.644+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96a8150c40 con 0x7f96a8070190 2024-09-11T12:53:20.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.648+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96a806fc50 con 0x7f96a8070190 2024-09-11T12:53:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.648+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f96980a5b70 con 0x7f96a8070190 2024-09-11T12:53:20.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.648+0000 7f969f400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 0x7f9688073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.648+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f969812c490 con 0x7f96a8070190 2024-09-11T12:53:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.648+0000 7f96a6a00640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 0x7f9688073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.652+0000 7f96a6a00640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 0x7f9688073ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f96a81fb390 tx=0x7f969400b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:20.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.652+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f96980c5020 con 0x7f96a8070190 2024-09-11T12:53:20.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.820+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f96a80b0250 con 0x7f96a8070190 2024-09-11T12:53:20.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.820+0000 7f969f400640 1 -- 172.21.15.78:0/3544386018 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16741 (secure 0 0 0) 0x7f9698016990 con 0x7f96a8070190 2024-09-11T12:53:20.822 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:20.822 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":68,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:19.846990+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:17.938226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:18.807993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:20.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 msgr2=0x7f9688073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 0x7f9688073ea0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f96a81fb390 tx=0x7f969400b390 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 msgr2=0x7f96a8150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9698000f90 tx=0x7f9698009360 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 shutdown_connections 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f96880719e0 0x7f9688073ea0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96a80741e0 0x7f96a81469f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96a8072230 0x7f96a81464b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 --2- 172.21.15.78:0/3544386018 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96a8070190 0x7f96a8150700 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 >> 172.21.15.78:0/3544386018 conn(0x7f96a806ca80 msgr2=0x7f96a815c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 shutdown_connections 2024-09-11T12:53:20.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.824+0000 7f96ae400640 1 -- 172.21.15.78:0/3544386018 wait complete. 2024-09-11T12:53:20.844 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:20.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- 172.21.15.78:0/3015781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0070600 msgr2=0x7f2fb0072660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3015781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0070600 0x7f2fb0072660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f2fb006d810 tx=0x7f2f9c01c5d0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- 172.21.15.78:0/3015781 shutdown_connections 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3015781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0070600 0x7f2fb0072660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3015781 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0072ba0 0x7f2fb00cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3015781 >> v2:172.21.15.157:3300/0 conn(0x7f2fb0154890 0x7f2fb0161e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- 172.21.15.78:0/3015781 >> 172.21.15.78:0/3015781 conn(0x7f2fb0073770 msgr2=0x7f2fb0073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- 172.21.15.78:0/3015781 shutdown_connections 2024-09-11T12:53:20.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- 172.21.15.78:0/3015781 wait complete. 2024-09-11T12:53:20.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 Processor -- start 2024-09-11T12:53:20.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.984+0000 7f2fafe00640 1 -- start start 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 0x7f2fb00cb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0154890 0x7f2fb014ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00c3410 con 0x7f2fb0072ba0 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb00c3290 con 0x7f2fb0070600 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb00c3590 con 0x7f2fb0154890 2024-09-11T12:53:20.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60976/0 (socket says 172.21.15.78:60976) 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 -- 172.21.15.78:0/3426739986 learned_addr learned my addr 172.21.15.78:0/3426739986 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2faea00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 0x7f2fb00cb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2faf400640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0154890 0x7f2fb014ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 -- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0154890 msgr2=0x7f2fb014ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0154890 0x7f2fb014ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 -- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 msgr2=0x7f2fb00cb8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 0x7f2fb00cb8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 -- 172.21.15.78:0/3426739986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb00cc140 con 0x7f2fb0072ba0 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2faea00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 0x7f2fb00cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:20.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fae000640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009850 tx=0x7f2f9c0a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:20.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0ac070 con 0x7f2fb0072ba0 2024-09-11T12:53:20.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f9c004340 con 0x7f2fb0072ba0 2024-09-11T12:53:20.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb00cc420 con 0x7f2fb0072ba0 2024-09-11T12:53:20.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c005320 con 0x7f2fb0072ba0 2024-09-11T12:53:20.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb00cc9e0 con 0x7f2fb0072ba0 2024-09-11T12:53:20.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.988+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb00cf6e0 con 0x7f2fb0072ba0 2024-09-11T12:53:20.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f2f9c0af070 con 0x7f2fb0072ba0 2024-09-11T12:53:20.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2fa6a00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 0x7f2f90073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:20.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7f2f9c12c8e0 con 0x7f2fb0072ba0 2024-09-11T12:53:20.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2faea00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 0x7f2f90073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:20.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2faea00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 0x7f2f90073ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bbd0 tx=0x7f2fa000b5b0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:20.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:20.992+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2f9c0164b0 con 0x7f2fb0072ba0 2024-09-11T12:53:21.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.160+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2fb0160cb0 con 0x7f2fb0072ba0 2024-09-11T12:53:21.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.160+0000 7f2fa6a00640 1 -- 172.21.15.78:0/3426739986 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) ==== 74+0+16741 (secure 0 0 0) 0x7f2f9c0f8180 con 0x7f2fb0072ba0 2024-09-11T12:53:21.163 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:21.163 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":68,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:19.846990+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:17.938226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:18.807993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:21.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 msgr2=0x7f2f90073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:21.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 0x7f2f90073ea0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bbd0 tx=0x7f2fa000b5b0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 msgr2=0x7f2fb0147ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009850 tx=0x7f2f9c0a64e0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 shutdown_connections 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f2f900719e0 0x7f2f90073ea0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2fb0154890 0x7f2fb014ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fb0072ba0 0x7f2fb0147ad0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 --2- 172.21.15.78:0/3426739986 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2fb0070600 0x7f2fb00cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 >> 172.21.15.78:0/3426739986 conn(0x7f2fb0073770 msgr2=0x7f2fb00cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 shutdown_connections 2024-09-11T12:53:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.164+0000 7f2fafe00640 1 -- 172.21.15.78:0/3426739986 wait complete. 2024-09-11T12:53:21.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:53:21.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 -- 172.21.15.78:0/3447409401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440b1a80 msgr2=0x7fb6440b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:21.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 --2- 172.21.15.78:0/3447409401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440b1a80 0x7fb6440b3040 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb630009870 tx=0x7fb63001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 -- 172.21.15.78:0/3447409401 shutdown_connections 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 --2- 172.21.15.78:0/3447409401 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440b1a80 0x7fb6440b3040 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 --2- 172.21.15.78:0/3447409401 >> v2:172.21.15.157:3300/0 conn(0x7fb6440cf6e0 0x7fb64415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 -- 172.21.15.78:0/3447409401 >> 172.21.15.78:0/3447409401 conn(0x7fb644071dd0 msgr2=0x7fb6440721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 -- 172.21.15.78:0/3447409401 shutdown_connections 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.328+0000 7fb648c00640 1 -- 172.21.15.78:0/3447409401 wait complete. 2024-09-11T12:53:21.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 Processor -- start 2024-09-11T12:53:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- start start 2024-09-11T12:53:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6440b1f10 0x7fb64413ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440cf6e0 0x7fb64413f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440af100 con 0x7fb6440b1a80 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb6440aef80 con 0x7fb6440b1f10 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb6440af280 con 0x7fb6440cf6e0 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60984/0 (socket says 172.21.15.78:60984) 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 -- 172.21.15.78:0/340097455 learned_addr learned my addr 172.21.15.78:0/340097455 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb642800640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440cf6e0 0x7fb64413f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641400640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6440b1f10 0x7fb64413ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 -- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440cf6e0 msgr2=0x7fb64413f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440cf6e0 0x7fb64413f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 -- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6440b1f10 msgr2=0x7fb64413ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6440b1f10 0x7fb64413ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:21.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 -- 172.21.15.78:0/340097455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb64413c1e0 con 0x7fb6440b1a80 2024-09-11T12:53:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb641e00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e7c0 tx=0x7fb63400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb63400ccc0 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb63400ce80 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb634005280 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb64413f910 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.332+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb64413c730 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6440b3040 con 0x7fb6440b1a80 2024-09-11T12:53:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fb634005420 con 0x7fb6440b1a80 2024-09-11T12:53:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb63a000640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 0x7fb624073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5709+0+0 (secure 0 0 0) 0x7fb634092e30 con 0x7fb6440b1a80 2024-09-11T12:53:21.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb641400640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 0x7fb624073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:21.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.336+0000 7fb641400640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 0x7fb624073ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb6300af000 tx=0x7fb63001b030 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:21.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.340+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb63405e680 con 0x7fb6440b1a80 2024-09-11T12:53:21.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:21.544+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb6441436e0 con 0x7fb6440b1a80 2024-09-11T12:53:22.440 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-09-11T12:53:22.434+0000 7f38b4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:53:22.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.472+0000 7fb63a000640 1 -- 172.21.15.78:0/340097455 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7fb63405df80 con 0x7fb6440b1a80 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 msgr2=0x7fb624073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 0x7fb624073ea0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb6300af000 tx=0x7fb63001b030 comp rx=0 tx=0).stop 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 msgr2=0x7fb644136590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e7c0 tx=0x7fb63400ec80 comp rx=0 tx=0).stop 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 shutdown_connections 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb6240719e0 0x7fb624073ea0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb6440cf6e0 0x7fb64413f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb6440b1f10 0x7fb64413ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 --2- 172.21.15.78:0/340097455 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb6440b1a80 0x7fb644136590 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 >> 172.21.15.78:0/340097455 conn(0x7fb644071dd0 msgr2=0x7fb6441597b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:22.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 shutdown_connections 2024-09-11T12:53:22.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.476+0000 7fb648c00640 1 -- 172.21.15.78:0/340097455 wait complete. 2024-09-11T12:53:22.501 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:53:22.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:53:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- 172.21.15.78:0/3140922137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780b2120 msgr2=0x7f40780cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 --2- 172.21.15.78:0/3140922137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780b2120 0x7f40780cdc10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4064009870 tx=0x7f406401c740 comp rx=0 tx=0).stop 2024-09-11T12:53:22.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- 172.21.15.78:0/3140922137 shutdown_connections 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 --2- 172.21.15.78:0/3140922137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780b2120 0x7f40780cdc10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- 172.21.15.78:0/3140922137 >> 172.21.15.78:0/3140922137 conn(0x7f4078071dd0 msgr2=0x7f40780721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- 172.21.15.78:0/3140922137 shutdown_connections 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- 172.21.15.78:0/3140922137 wait complete. 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 Processor -- start 2024-09-11T12:53:22.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.648+0000 7f407f600640 1 -- start start 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 0x7f40781f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40781518b0 0x7f4078153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4078168e90 con 0x7f40780b2120 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4078168d10 con 0x7f40780ce150 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4078169010 con 0x7f40781518b0 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60986/0 (socket says 172.21.15.78:60986) 2024-09-11T12:53:22.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 -- 172.21.15.78:0/1586824601 learned_addr learned my addr 172.21.15.78:0/1586824601 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:22.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f4077e00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 0x7f40781f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 -- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40781518b0 msgr2=0x7f4078153010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40781518b0 0x7f4078153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 -- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 msgr2=0x7f40781f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 0x7f40781f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 -- 172.21.15.78:0/1586824601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407814c650 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f4077e00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 0x7f40781f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407cc00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f40680015b0 tx=0x7f4068001fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068016020 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407814ff90 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f406800ab40 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4068012420 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4078150550 con 0x7f40780b2120 2024-09-11T12:53:22.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.652+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40780cdc10 con 0x7f40780b2120 2024-09-11T12:53:22.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.656+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f4068019020 con 0x7f40780b2120 2024-09-11T12:53:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.656+0000 7f4074c00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 0x7f4058073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.656+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f4068092890 con 0x7f40780b2120 2024-09-11T12:53:22.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.656+0000 7f4077e00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 0x7f4058073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:22.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.656+0000 7f4077e00640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 0x7f4058073ea0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f40640ad000 tx=0x7f406401af80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:22.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.660+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4068020020 con 0x7f40780b2120 2024-09-11T12:53:22.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:22.864+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f407814e9b0 con 0x7f40780b2120 2024-09-11T12:53:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.480+0000 7f4074c00640 1 -- 172.21.15.78:0/1586824601 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f406805e230 con 0x7f40780b2120 2024-09-11T12:53:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 msgr2=0x7f4058073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 0x7f4058073ea0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f40640ad000 tx=0x7f406401af80 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 msgr2=0x7f40781f8e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f40680015b0 tx=0x7f4068001fa0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 shutdown_connections 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f40580719e0 0x7f4058073ea0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f40781518b0 0x7f4078153010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f40780ce150 0x7f40781f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 --2- 172.21.15.78:0/1586824601 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f40780b2120 0x7f40781f8e50 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 >> 172.21.15.78:0/1586824601 conn(0x7f4078071dd0 msgr2=0x7f407814d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 shutdown_connections 2024-09-11T12:53:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.484+0000 7f407f600640 1 -- 172.21.15.78:0/1586824601 wait complete. 2024-09-11T12:53:23.508 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:53:23.508 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- 172.21.15.78:0/3834492637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 msgr2=0x7f01d80736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 --2- 172.21.15.78:0/3834492637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d80736d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f01c400a980 tx=0x7f01c401db00 comp rx=0 tx=0).stop 2024-09-11T12:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- 172.21.15.78:0/3834492637 shutdown_connections 2024-09-11T12:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 --2- 172.21.15.78:0/3834492637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d80736d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- 172.21.15.78:0/3834492637 >> 172.21.15.78:0/3834492637 conn(0x7f01d806ca80 msgr2=0x7f01d80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- 172.21.15.78:0/3834492637 shutdown_connections 2024-09-11T12:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- 172.21.15.78:0/3834492637 wait complete. 2024-09-11T12:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 Processor -- start 2024-09-11T12:53:23.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- start start 2024-09-11T12:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d81f6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d80b4db0 0x7f01d8147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8168150 con 0x7f01d8157ae0 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01d8167fd0 con 0x7f01d80b4db0 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dfa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01d81682d0 con 0x7f01d8070a00 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01d7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d80b4db0 0x7f01d8147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60996/0 (socket says 172.21.15.78:60996) 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dda00640 1 -- 172.21.15.78:0/2508371761 learned_addr learned my addr 172.21.15.78:0/2508371761 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.652+0000 7f01dd000640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d81f6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 -- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 msgr2=0x7f01d81f6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d81f6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 -- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d80b4db0 msgr2=0x7f01d8147500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d80b4db0 0x7f01d8147500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 -- 172.21.15.78:0/2508371761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d81f5ee0 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dda00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c410 tx=0x7f01d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011790 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01d0017070 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d8153f10 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0011ce0 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d8147a40 con 0x7f01d8157ae0 2024-09-11T12:53:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d80736d0 con 0x7f01d8157ae0 2024-09-11T12:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f01d001c460 con 0x7f01d8157ae0 2024-09-11T12:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 0x7f01b8073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.656+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5825+0+0 (secure 0 0 0) 0x7f01d0092040 con 0x7f01d8157ae0 2024-09-11T12:53:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.660+0000 7f01dd000640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 0x7f01b8073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:23.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.660+0000 7f01dd000640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 0x7f01b8073ed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f01c80014b0 tx=0x7f01c8009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:23.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.660+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f01d005d420 con 0x7f01d8157ae0 2024-09-11T12:53:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.864+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01d80b36f0 con 0x7f01d8157ae0 2024-09-11T12:53:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.864+0000 7f01d4c00640 1 -- 172.21.15.78:0/2508371761 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f01d0061970 con 0x7f01d8157ae0 2024-09-11T12:53:23.866 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:23.866 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":46,"btime":"2024-09-11T12:53:23:476400+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:23.866 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 46 2024-09-11T12:53:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.864+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 msgr2=0x7f01b8073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.864+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 0x7f01b8073ed0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f01c80014b0 tx=0x7f01c8009040 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 msgr2=0x7f01d8154660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c410 tx=0x7f01d000c8d0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 shutdown_connections 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f01b8071a10 0x7f01b8073ed0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d8157ae0 0x7f01d8154660 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d80b4db0 0x7f01d8147500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 --2- 172.21.15.78:0/2508371761 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d8070a00 0x7f01d81f6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 >> 172.21.15.78:0/2508371761 conn(0x7f01d806ca80 msgr2=0x7f01d8152450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 shutdown_connections 2024-09-11T12:53:23.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:23.868+0000 7f01dfa00640 1 -- 172.21.15.78:0/2508371761 wait complete. 2024-09-11T12:53:23.887 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 -- 172.21.15.78:0/1950121598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640707f0 msgr2=0x7f3c6415db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 --2- 172.21.15.78:0/1950121598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640707f0 0x7f3c6415db60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3c64070bd0 tx=0x7f3c4c01c800 comp rx=0 tx=0).stop 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 -- 172.21.15.78:0/1950121598 shutdown_connections 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 --2- 172.21.15.78:0/1950121598 >> v2:172.21.15.157:3300/0 conn(0x7f3c640b4db0 0x7f3c640b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 --2- 172.21.15.78:0/1950121598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640707f0 0x7f3c6415db60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 -- 172.21.15.78:0/1950121598 >> 172.21.15.78:0/1950121598 conn(0x7f3c6406ca80 msgr2=0x7f3c640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 -- 172.21.15.78:0/1950121598 shutdown_connections 2024-09-11T12:53:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.028+0000 7f3c6a200640 1 -- 172.21.15.78:0/1950121598 wait complete. 2024-09-11T12:53:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 Processor -- start 2024-09-11T12:53:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- start start 2024-09-11T12:53:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c640707f0 0x7f3c6415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 0x7f3c641f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3c64072500 con 0x7f3c640b4db0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3c64072380 con 0x7f3c64073ed0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3c64072680 con 0x7f3c640707f0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54342/0 (socket says 172.21.15.78:54342) 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 -- 172.21.15.78:0/186929462 learned_addr learned my addr 172.21.15.78:0/186929462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c5be00640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 0x7f3c641f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63400640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c640707f0 0x7f3c6415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 -- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c640707f0 msgr2=0x7f3c6415b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c640707f0 0x7f3c6415b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 -- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 msgr2=0x7f3c641f78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 0x7f3c641f78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 -- 172.21.15.78:0/186929462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c6415d110 con 0x7f3c640b4db0 2024-09-11T12:53:24.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c5be00640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 0x7f3c641f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c63e00640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00ea20 tx=0x7f3c5c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c00cde0 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c5c004c40 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5c005280 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c6415d3f0 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c641f7390 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.032+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c6415db60 con 0x7f3c640b4db0 2024-09-11T12:53:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.036+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f3c5c005420 con 0x7f3c640b4db0 2024-09-11T12:53:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.036+0000 7f3c5b400640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 0x7f3c40073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.036+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5825+0+0 (secure 0 0 0) 0x7f3c5c092e40 con 0x7f3c640b4db0 2024-09-11T12:53:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.036+0000 7f3c63400640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 0x7f3c40073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.036+0000 7f3c63400640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 0x7f3c40073ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c54000f70 tx=0x7f3c54009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.040+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3c5c016030 con 0x7f3c640b4db0 2024-09-11T12:53:24.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.204+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3c640b3200 con 0x7f3c640b4db0 2024-09-11T12:53:24.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.204+0000 7f3c5b400640 1 -- 172.21.15.78:0/186929462 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16921 (secure 0 0 0) 0x7f3c5c05e660 con 0x7f3c640b4db0 2024-09-11T12:53:24.208 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:24.208 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":70,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:23.476169+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:17.938226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:18.807993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":5}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 msgr2=0x7f3c40073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 0x7f3c40073ea0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3c54000f70 tx=0x7f3c54009040 comp rx=0 tx=0).stop 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 msgr2=0x7f3c640b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c00ea20 tx=0x7f3c5c00eee0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 shutdown_connections 2024-09-11T12:53:24.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f3c400719e0 0x7f3c40073ea0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3c640b4db0 0x7f3c640b0ec0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3c64073ed0 0x7f3c641f78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 --2- 172.21.15.78:0/186929462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3c640707f0 0x7f3c6415b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 >> 172.21.15.78:0/186929462 conn(0x7f3c6406ca80 msgr2=0x7f3c6415af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 shutdown_connections 2024-09-11T12:53:24.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.208+0000 7f3c6a200640 1 -- 172.21.15.78:0/186929462 wait complete. 2024-09-11T12:53:24.229 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:53:24.229 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 -- 172.21.15.78:0/2297983868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb90415dee0 msgr2=0x7fb9040b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 --2- 172.21.15.78:0/2297983868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb90415dee0 0x7fb9040b54b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fb9040b6dd0 tx=0x7fb8f001c740 comp rx=0 tx=0).stop 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 -- 172.21.15.78:0/2297983868 shutdown_connections 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 --2- 172.21.15.78:0/2297983868 >> v2:172.21.15.157:3300/0 conn(0x7fb9040b59f0 0x7fb90406f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 --2- 172.21.15.78:0/2297983868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb90415dee0 0x7fb9040b54b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 -- 172.21.15.78:0/2297983868 >> 172.21.15.78:0/2297983868 conn(0x7fb90406c650 msgr2=0x7fb9040d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 -- 172.21.15.78:0/2297983868 shutdown_connections 2024-09-11T12:53:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.368+0000 7fb90c200640 1 -- 172.21.15.78:0/2297983868 wait complete. 2024-09-11T12:53:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 Processor -- start 2024-09-11T12:53:24.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- start start 2024-09-11T12:53:24.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 0x7fb9041446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 0x7fb904152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb90416b310 con 0x7fb9040b59f0 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb90416b190 con 0x7fb90415dee0 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb90416b490 con 0x7fb90415db00 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54360/0 (socket says 172.21.15.78:54360) 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 -- 172.21.15.78:0/1433381268 learned_addr learned my addr 172.21.15.78:0/1433381268 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90a200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 0x7fb904152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 -- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 msgr2=0x7fb9041446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb908e00640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 0x7fb9041446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 0x7fb9041446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 -- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 msgr2=0x7fb904152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 0x7fb904152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 -- 172.21.15.78:0/1433381268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041519a0 con 0x7fb9040b59f0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90a200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 0x7fb904152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb908e00640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 0x7fb9041446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb909800640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800cba0 tx=0x7fb8f800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8016020 con 0x7fb9040b59f0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f800ab40 con 0x7fb9040b59f0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f8012420 con 0x7fb9040b59f0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb904151c80 con 0x7fb9040b59f0 2024-09-11T12:53:24.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb904152240 con 0x7fb9040b59f0 2024-09-11T12:53:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.372+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb90406f950 con 0x7fb9040b59f0 2024-09-11T12:53:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.376+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fb8f8019020 con 0x7fb9040b59f0 2024-09-11T12:53:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.376+0000 7fb901600640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 0x7fb8e4073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.376+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5825+0+0 (secure 0 0 0) 0x7fb8f8092630 con 0x7fb9040b59f0 2024-09-11T12:53:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.376+0000 7fb908e00640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 0x7fb8e4073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.376+0000 7fb908e00640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 0x7fb8e4073ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00af000 tx=0x7fb8f001af80 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.380+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb8f8020020 con 0x7fb9040b59f0 2024-09-11T12:53:24.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.548+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb9040d2460 con 0x7fb8e4071a10 2024-09-11T12:53:24.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.552+0000 7fb901600640 1 -- 172.21.15.78:0/1433381268 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7fb9040d2460 con 0x7fb8e4071a10 2024-09-11T12:53:24.557 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:24.558 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:53:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 msgr2=0x7fb8e4073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 0x7fb8e4073ed0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fb8f00af000 tx=0x7fb8f001af80 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 msgr2=0x7fb904143660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fb8f800cba0 tx=0x7fb8f800cc90 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 shutdown_connections 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb8e4071a10 0x7fb8e4073ed0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb90415dee0 0x7fb904152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb90415db00 0x7fb9041446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 --2- 172.21.15.78:0/1433381268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9040b59f0 0x7fb904143660 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 >> 172.21.15.78:0/1433381268 conn(0x7fb90406c650 msgr2=0x7fb904149ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 shutdown_connections 2024-09-11T12:53:24.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.556+0000 7fb90c200640 1 -- 172.21.15.78:0/1433381268 wait complete. 2024-09-11T12:53:24.576 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-11T12:53:24.486324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257232,"kb_used_data":7684,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717488,"statfs":{"total":773094113280,"available":772830707712,"internally_reserved":0,"allocated":7868416,"data_stored":3933768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037178"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176460+0000","last_change":"2024-09-11T12:53:20.486705+0000","last_active":"2024-09-11T12:53:22.176460+0000","last_peered":"2024-09-11T12:53:22.176460+0000","last_clean":"2024-09-11T12:53:22.176460+0000","last_became_active":"2024-09-11T12:53:20.486395+0000","last_became_peered":"2024-09-11T12:53:20.486395+0000","last_unstale":"2024-09-11T12:53:22.176460+0000","last_undegraded":"2024-09-11T12:53:22.176460+0000","last_fullsized":"2024-09-11T12:53:22.176460+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:16.633708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469738+0000","last_change":"2024-09-11T12:53:20.477869+0000","last_active":"2024-09-11T12:53:21.469738+0000","last_peered":"2024-09-11T12:53:21.469738+0000","last_clean":"2024-09-11T12:53:21.469738+0000","last_became_active":"2024-09-11T12:53:20.477558+0000","last_became_peered":"2024-09-11T12:53:20.477558+0000","last_unstale":"2024-09-11T12:53:21.469738+0000","last_undegraded":"2024-09-11T12:53:21.469738+0000","last_fullsized":"2024-09-11T12:53:21.469738+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:07.351406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276638+0000","last_change":"2024-09-11T12:53:20.463411+0000","last_active":"2024-09-11T12:53:22.276638+0000","last_peered":"2024-09-11T12:53:22.276638+0000","last_clean":"2024-09-11T12:53:22.276638+0000","last_became_active":"2024-09-11T12:53:20.463120+0000","last_became_peered":"2024-09-11T12:53:20.463120+0000","last_unstale":"2024-09-11T12:53:22.276638+0000","last_undegraded":"2024-09-11T12:53:22.276638+0000","last_fullsized":"2024-09-11T12:53:22.276638+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:25.834098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703447+0000","last_change":"2024-09-11T12:53:20.483381+0000","last_active":"2024-09-11T12:53:21.703447+0000","last_peered":"2024-09-11T12:53:21.703447+0000","last_clean":"2024-09-11T12:53:21.703447+0000","last_became_active":"2024-09-11T12:53:20.483128+0000","last_became_peered":"2024-09-11T12:53:20.483128+0000","last_unstale":"2024-09-11T12:53:21.703447+0000","last_undegraded":"2024-09-11T12:53:21.703447+0000","last_fullsized":"2024-09-11T12:53:21.703447+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:04.881869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468088+0000","last_change":"2024-09-11T12:53:20.495951+0000","last_active":"2024-09-11T12:53:21.468088+0000","last_peered":"2024-09-11T12:53:21.468088+0000","last_clean":"2024-09-11T12:53:21.468088+0000","last_became_active":"2024-09-11T12:53:20.495600+0000","last_became_peered":"2024-09-11T12:53:20.495600+0000","last_unstale":"2024-09-11T12:53:21.468088+0000","last_undegraded":"2024-09-11T12:53:21.468088+0000","last_fullsized":"2024-09-11T12:53:21.468088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:31.296611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469108+0000","last_change":"2024-09-11T12:53:20.477991+0000","last_active":"2024-09-11T12:53:21.469108+0000","last_peered":"2024-09-11T12:53:21.469108+0000","last_clean":"2024-09-11T12:53:21.469108+0000","last_became_active":"2024-09-11T12:53:20.477685+0000","last_became_peered":"2024-09-11T12:53:20.477685+0000","last_unstale":"2024-09-11T12:53:21.469108+0000","last_undegraded":"2024-09-11T12:53:21.469108+0000","last_fullsized":"2024-09-11T12:53:21.469108+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:22.499200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468931+0000","last_change":"2024-09-11T12:53:20.459176+0000","last_active":"2024-09-11T12:53:21.468931+0000","last_peered":"2024-09-11T12:53:21.468931+0000","last_clean":"2024-09-11T12:53:21.468931+0000","last_became_active":"2024-09-11T12:53:20.458424+0000","last_became_peered":"2024-09-11T12:53:20.458424+0000","last_unstale":"2024-09-11T12:53:21.468931+0000","last_undegraded":"2024-09-11T12:53:21.468931+0000","last_fullsized":"2024-09-11T12:53:21.468931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:35.639498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469866+0000","last_change":"2024-09-11T12:53:20.479100+0000","last_active":"2024-09-11T12:53:21.469866+0000","last_peered":"2024-09-11T12:53:21.469866+0000","last_clean":"2024-09-11T12:53:21.469866+0000","last_became_active":"2024-09-11T12:53:20.478366+0000","last_became_peered":"2024-09-11T12:53:20.478366+0000","last_unstale":"2024-09-11T12:53:21.469866+0000","last_undegraded":"2024-09-11T12:53:21.469866+0000","last_fullsized":"2024-09-11T12:53:21.469866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:05.851122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468488+0000","last_change":"2024-09-11T12:53:20.475722+0000","last_active":"2024-09-11T12:53:21.468488+0000","last_peered":"2024-09-11T12:53:21.468488+0000","last_clean":"2024-09-11T12:53:21.468488+0000","last_became_active":"2024-09-11T12:53:20.474176+0000","last_became_peered":"2024-09-11T12:53:20.474176+0000","last_unstale":"2024-09-11T12:53:21.468488+0000","last_undegraded":"2024-09-11T12:53:21.468488+0000","last_fullsized":"2024-09-11T12:53:21.468488+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:42.076643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276666+0000","last_change":"2024-09-11T12:53:20.471313+0000","last_active":"2024-09-11T12:53:22.276666+0000","last_peered":"2024-09-11T12:53:22.276666+0000","last_clean":"2024-09-11T12:53:22.276666+0000","last_became_active":"2024-09-11T12:53:20.470632+0000","last_became_peered":"2024-09-11T12:53:20.470632+0000","last_unstale":"2024-09-11T12:53:22.276666+0000","last_undegraded":"2024-09-11T12:53:22.276666+0000","last_fullsized":"2024-09-11T12:53:22.276666+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:04.694195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176290+0000","last_change":"2024-09-11T12:53:20.483790+0000","last_active":"2024-09-11T12:53:22.176290+0000","last_peered":"2024-09-11T12:53:22.176290+0000","last_clean":"2024-09-11T12:53:22.176290+0000","last_became_active":"2024-09-11T12:53:20.483454+0000","last_became_peered":"2024-09-11T12:53:20.483454+0000","last_unstale":"2024-09-11T12:53:22.176290+0000","last_undegraded":"2024-09-11T12:53:22.176290+0000","last_fullsized":"2024-09-11T12:53:22.176290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:24.525533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703364+0000","last_change":"2024-09-11T12:53:20.471588+0000","last_active":"2024-09-11T12:53:21.703364+0000","last_peered":"2024-09-11T12:53:21.703364+0000","last_clean":"2024-09-11T12:53:21.703364+0000","last_became_active":"2024-09-11T12:53:20.470976+0000","last_became_peered":"2024-09-11T12:53:20.470976+0000","last_unstale":"2024-09-11T12:53:21.703364+0000","last_undegraded":"2024-09-11T12:53:21.703364+0000","last_fullsized":"2024-09-11T12:53:21.703364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:04.873191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.484490+0000","last_change":"2024-09-11T12:53:20.482418+0000","last_active":"2024-09-11T12:53:20.484490+0000","last_peered":"2024-09-11T12:53:20.484490+0000","last_clean":"2024-09-11T12:53:20.484490+0000","last_became_active":"2024-09-11T12:53:20.482122+0000","last_became_peered":"2024-09-11T12:53:20.482122+0000","last_unstale":"2024-09-11T12:53:20.484490+0000","last_undegraded":"2024-09-11T12:53:20.484490+0000","last_fullsized":"2024-09-11T12:53:20.484490+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:36:49.026146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468176+0000","last_change":"2024-09-11T12:53:20.476343+0000","last_active":"2024-09-11T12:53:21.468176+0000","last_peered":"2024-09-11T12:53:21.468176+0000","last_clean":"2024-09-11T12:53:21.468176+0000","last_became_active":"2024-09-11T12:53:20.475910+0000","last_became_peered":"2024-09-11T12:53:20.475910+0000","last_unstale":"2024-09-11T12:53:21.468176+0000","last_undegraded":"2024-09-11T12:53:21.468176+0000","last_fullsized":"2024-09-11T12:53:21.468176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:06:59.154745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.476380+0000","last_change":"2024-09-11T12:53:20.471273+0000","last_active":"2024-09-11T12:53:20.476380+0000","last_peered":"2024-09-11T12:53:20.476380+0000","last_clean":"2024-09-11T12:53:20.476380+0000","last_became_active":"2024-09-11T12:53:20.470985+0000","last_became_peered":"2024-09-11T12:53:20.470985+0000","last_unstale":"2024-09-11T12:53:20.476380+0000","last_undegraded":"2024-09-11T12:53:20.476380+0000","last_fullsized":"2024-09-11T12:53:20.476380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:36.750482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276057+0000","last_change":"2024-09-11T12:53:20.473580+0000","last_active":"2024-09-11T12:53:22.276057+0000","last_peered":"2024-09-11T12:53:22.276057+0000","last_clean":"2024-09-11T12:53:22.276057+0000","last_became_active":"2024-09-11T12:53:20.473179+0000","last_became_peered":"2024-09-11T12:53:20.473179+0000","last_unstale":"2024-09-11T12:53:22.276057+0000","last_undegraded":"2024-09-11T12:53:22.276057+0000","last_fullsized":"2024-09-11T12:53:22.276057+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:40.023711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468515+0000","last_change":"2024-09-11T12:53:20.476106+0000","last_active":"2024-09-11T12:53:21.468515+0000","last_peered":"2024-09-11T12:53:21.468515+0000","last_clean":"2024-09-11T12:53:21.468515+0000","last_became_active":"2024-09-11T12:53:20.474526+0000","last_became_peered":"2024-09-11T12:53:20.474526+0000","last_unstale":"2024-09-11T12:53:21.468515+0000","last_undegraded":"2024-09-11T12:53:21.468515+0000","last_fullsized":"2024-09-11T12:53:21.468515+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:15:28.692763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703305+0000","last_change":"2024-09-11T12:53:20.482025+0000","last_active":"2024-09-11T12:53:21.703305+0000","last_peered":"2024-09-11T12:53:21.703305+0000","last_clean":"2024-09-11T12:53:21.703305+0000","last_became_active":"2024-09-11T12:53:20.481580+0000","last_became_peered":"2024-09-11T12:53:20.481580+0000","last_unstale":"2024-09-11T12:53:21.703305+0000","last_undegraded":"2024-09-11T12:53:21.703305+0000","last_fullsized":"2024-09-11T12:53:21.703305+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:09.323415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470423+0000","last_change":"2024-09-11T12:53:20.477741+0000","last_active":"2024-09-11T12:53:21.470423+0000","last_peered":"2024-09-11T12:53:21.470423+0000","last_clean":"2024-09-11T12:53:21.470423+0000","last_became_active":"2024-09-11T12:53:20.477330+0000","last_became_peered":"2024-09-11T12:53:20.477330+0000","last_unstale":"2024-09-11T12:53:21.470423+0000","last_undegraded":"2024-09-11T12:53:21.470423+0000","last_fullsized":"2024-09-11T12:53:21.470423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:12:09.716396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.468425+0000","last_change":"2024-09-11T12:53:20.466472+0000","last_active":"2024-09-11T12:53:20.468425+0000","last_peered":"2024-09-11T12:53:20.468425+0000","last_clean":"2024-09-11T12:53:20.468425+0000","last_became_active":"2024-09-11T12:53:20.466210+0000","last_became_peered":"2024-09-11T12:53:20.466210+0000","last_unstale":"2024-09-11T12:53:20.468425+0000","last_undegraded":"2024-09-11T12:53:20.468425+0000","last_fullsized":"2024-09-11T12:53:20.468425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:04.936945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276427+0000","last_change":"2024-09-11T12:53:20.461606+0000","last_active":"2024-09-11T12:53:22.276427+0000","last_peered":"2024-09-11T12:53:22.276427+0000","last_clean":"2024-09-11T12:53:22.276427+0000","last_became_active":"2024-09-11T12:53:20.461275+0000","last_became_peered":"2024-09-11T12:53:20.461275+0000","last_unstale":"2024-09-11T12:53:22.276427+0000","last_undegraded":"2024-09-11T12:53:22.276427+0000","last_fullsized":"2024-09-11T12:53:22.276427+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:22.499200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276266+0000","last_change":"2024-09-11T12:53:20.473587+0000","last_active":"2024-09-11T12:53:22.276266+0000","last_peered":"2024-09-11T12:53:22.276266+0000","last_clean":"2024-09-11T12:53:22.276266+0000","last_became_active":"2024-09-11T12:53:20.472745+0000","last_became_peered":"2024-09-11T12:53:20.472745+0000","last_unstale":"2024-09-11T12:53:22.276266+0000","last_undegraded":"2024-09-11T12:53:22.276266+0000","last_fullsized":"2024-09-11T12:53:22.276266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:43.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175963+0000","last_change":"2024-09-11T12:53:20.483727+0000","last_active":"2024-09-11T12:53:22.175963+0000","last_peered":"2024-09-11T12:53:22.175963+0000","last_clean":"2024-09-11T12:53:22.175963+0000","last_became_active":"2024-09-11T12:53:20.483444+0000","last_became_peered":"2024-09-11T12:53:20.483444+0000","last_unstale":"2024-09-11T12:53:22.175963+0000","last_undegraded":"2024-09-11T12:53:22.175963+0000","last_fullsized":"2024-09-11T12:53:22.175963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:10.768707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175902+0000","last_change":"2024-09-11T12:53:20.492821+0000","last_active":"2024-09-11T12:53:22.175902+0000","last_peered":"2024-09-11T12:53:22.175902+0000","last_clean":"2024-09-11T12:53:22.175902+0000","last_became_active":"2024-09-11T12:53:20.492483+0000","last_became_peered":"2024-09-11T12:53:20.492483+0000","last_unstale":"2024-09-11T12:53:22.175902+0000","last_undegraded":"2024-09-11T12:53:22.175902+0000","last_fullsized":"2024-09-11T12:53:22.175902+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:56.925859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276623+0000","last_change":"2024-09-11T12:53:20.480904+0000","last_active":"2024-09-11T12:53:22.276623+0000","last_peered":"2024-09-11T12:53:22.276623+0000","last_clean":"2024-09-11T12:53:22.276623+0000","last_became_active":"2024-09-11T12:53:20.480640+0000","last_became_peered":"2024-09-11T12:53:20.480640+0000","last_unstale":"2024-09-11T12:53:22.276623+0000","last_undegraded":"2024-09-11T12:53:22.276623+0000","last_fullsized":"2024-09-11T12:53:22.276623+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:07.351406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469083+0000","last_change":"2024-09-11T12:53:20.480171+0000","last_active":"2024-09-11T12:53:21.469083+0000","last_peered":"2024-09-11T12:53:21.469083+0000","last_clean":"2024-09-11T12:53:21.469083+0000","last_became_active":"2024-09-11T12:53:20.479773+0000","last_became_peered":"2024-09-11T12:53:20.479773+0000","last_unstale":"2024-09-11T12:53:21.469083+0000","last_undegraded":"2024-09-11T12:53:21.469083+0000","last_fullsized":"2024-09-11T12:53:21.469083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:56.617603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276675+0000","last_change":"2024-09-11T12:53:20.477930+0000","last_active":"2024-09-11T12:53:22.276675+0000","last_peered":"2024-09-11T12:53:22.276675+0000","last_clean":"2024-09-11T12:53:22.276675+0000","last_became_active":"2024-09-11T12:53:20.477440+0000","last_became_peered":"2024-09-11T12:53:20.477440+0000","last_unstale":"2024-09-11T12:53:22.276675+0000","last_undegraded":"2024-09-11T12:53:22.276675+0000","last_fullsized":"2024-09-11T12:53:22.276675+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:59.300032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276581+0000","last_change":"2024-09-11T12:53:20.470892+0000","last_active":"2024-09-11T12:53:22.276581+0000","last_peered":"2024-09-11T12:53:22.276581+0000","last_clean":"2024-09-11T12:53:22.276581+0000","last_became_active":"2024-09-11T12:53:20.470550+0000","last_became_peered":"2024-09-11T12:53:20.470550+0000","last_unstale":"2024-09-11T12:53:22.276581+0000","last_undegraded":"2024-09-11T12:53:22.276581+0000","last_fullsized":"2024-09-11T12:53:22.276581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:41.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276561+0000","last_change":"2024-09-11T12:53:20.469947+0000","last_active":"2024-09-11T12:53:22.276561+0000","last_peered":"2024-09-11T12:53:22.276561+0000","last_clean":"2024-09-11T12:53:22.276561+0000","last_became_active":"2024-09-11T12:53:20.469638+0000","last_became_peered":"2024-09-11T12:53:20.469638+0000","last_unstale":"2024-09-11T12:53:22.276561+0000","last_undegraded":"2024-09-11T12:53:22.276561+0000","last_fullsized":"2024-09-11T12:53:22.276561+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:11.390895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469039+0000","last_change":"2024-09-11T12:53:20.480977+0000","last_active":"2024-09-11T12:53:21.469039+0000","last_peered":"2024-09-11T12:53:21.469039+0000","last_clean":"2024-09-11T12:53:21.469039+0000","last_became_active":"2024-09-11T12:53:20.480716+0000","last_became_peered":"2024-09-11T12:53:20.480716+0000","last_unstale":"2024-09-11T12:53:21.469039+0000","last_undegraded":"2024-09-11T12:53:21.469039+0000","last_fullsized":"2024-09-11T12:53:21.469039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:03.204323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469115+0000","last_change":"2024-09-11T12:53:20.468322+0000","last_active":"2024-09-11T12:53:21.469115+0000","last_peered":"2024-09-11T12:53:21.469115+0000","last_clean":"2024-09-11T12:53:21.469115+0000","last_became_active":"2024-09-11T12:53:20.468033+0000","last_became_peered":"2024-09-11T12:53:20.468033+0000","last_unstale":"2024-09-11T12:53:21.469115+0000","last_undegraded":"2024-09-11T12:53:21.469115+0000","last_fullsized":"2024-09-11T12:53:21.469115+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:35.898218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470044+0000","last_change":"2024-09-11T12:53:20.476990+0000","last_active":"2024-09-11T12:53:21.470044+0000","last_peered":"2024-09-11T12:53:21.470044+0000","last_clean":"2024-09-11T12:53:21.470044+0000","last_became_active":"2024-09-11T12:53:20.476664+0000","last_became_peered":"2024-09-11T12:53:20.476664+0000","last_unstale":"2024-09-11T12:53:21.470044+0000","last_undegraded":"2024-09-11T12:53:21.470044+0000","last_fullsized":"2024-09-11T12:53:21.470044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:24:39.318254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176364+0000","last_change":"2024-09-11T12:53:20.491255+0000","last_active":"2024-09-11T12:53:22.176364+0000","last_peered":"2024-09-11T12:53:22.176364+0000","last_clean":"2024-09-11T12:53:22.176364+0000","last_became_active":"2024-09-11T12:53:20.490980+0000","last_became_peered":"2024-09-11T12:53:20.490980+0000","last_unstale":"2024-09-11T12:53:22.176364+0000","last_undegraded":"2024-09-11T12:53:22.176364+0000","last_fullsized":"2024-09-11T12:53:22.176364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:25.351631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176551+0000","last_change":"2024-09-11T12:53:19.475083+0000","last_active":"2024-09-11T12:53:22.176551+0000","last_peered":"2024-09-11T12:53:22.176551+0000","last_clean":"2024-09-11T12:53:22.176551+0000","last_became_active":"2024-09-11T12:53:19.474603+0000","last_became_peered":"2024-09-11T12:53:19.474603+0000","last_unstale":"2024-09-11T12:53:22.176551+0000","last_undegraded":"2024-09-11T12:53:22.176551+0000","last_fullsized":"2024-09-11T12:53:22.176551+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:21.496429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468595+0000","last_change":"2024-09-11T12:53:20.475419+0000","last_active":"2024-09-11T12:53:21.468595+0000","last_peered":"2024-09-11T12:53:21.468595+0000","last_clean":"2024-09-11T12:53:21.468595+0000","last_became_active":"2024-09-11T12:53:20.474160+0000","last_became_peered":"2024-09-11T12:53:20.474160+0000","last_unstale":"2024-09-11T12:53:21.468595+0000","last_undegraded":"2024-09-11T12:53:21.468595+0000","last_fullsized":"2024-09-11T12:53:21.468595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:59.342083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.487183+0000","last_change":"2024-09-11T12:53:19.464327+0000","last_active":"2024-09-11T12:53:20.487183+0000","last_peered":"2024-09-11T12:53:20.487183+0000","last_clean":"2024-09-11T12:53:20.487183+0000","last_became_active":"2024-09-11T12:53:19.463993+0000","last_became_peered":"2024-09-11T12:53:19.463993+0000","last_unstale":"2024-09-11T12:53:20.487183+0000","last_undegraded":"2024-09-11T12:53:20.487183+0000","last_fullsized":"2024-09-11T12:53:20.487183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:30.015624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469663+0000","last_change":"2024-09-11T12:53:20.485239+0000","last_active":"2024-09-11T12:53:21.469663+0000","last_peered":"2024-09-11T12:53:21.469663+0000","last_clean":"2024-09-11T12:53:21.469663+0000","last_became_active":"2024-09-11T12:53:20.481288+0000","last_became_peered":"2024-09-11T12:53:20.481288+0000","last_unstale":"2024-09-11T12:53:21.469663+0000","last_undegraded":"2024-09-11T12:53:21.469663+0000","last_fullsized":"2024-09-11T12:53:21.469663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:41.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276617+0000","last_change":"2024-09-11T12:53:19.474746+0000","last_active":"2024-09-11T12:53:22.276617+0000","last_peered":"2024-09-11T12:53:22.276617+0000","last_clean":"2024-09-11T12:53:22.276617+0000","last_became_active":"2024-09-11T12:53:19.474414+0000","last_became_peered":"2024-09-11T12:53:19.474414+0000","last_unstale":"2024-09-11T12:53:22.276617+0000","last_undegraded":"2024-09-11T12:53:22.276617+0000","last_fullsized":"2024-09-11T12:53:22.276617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:53.310007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176380+0000","last_change":"2024-09-11T12:53:20.478834+0000","last_active":"2024-09-11T12:53:22.176380+0000","last_peered":"2024-09-11T12:53:22.176380+0000","last_clean":"2024-09-11T12:53:22.176380+0000","last_became_active":"2024-09-11T12:53:20.478578+0000","last_became_peered":"2024-09-11T12:53:20.478578+0000","last_unstale":"2024-09-11T12:53:22.176380+0000","last_undegraded":"2024-09-11T12:53:22.176380+0000","last_fullsized":"2024-09-11T12:53:22.176380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:53.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470063+0000","last_change":"2024-09-11T12:53:19.478638+0000","last_active":"2024-09-11T12:53:21.470063+0000","last_peered":"2024-09-11T12:53:21.470063+0000","last_clean":"2024-09-11T12:53:21.470063+0000","last_became_active":"2024-09-11T12:53:19.478069+0000","last_became_peered":"2024-09-11T12:53:19.478069+0000","last_unstale":"2024-09-11T12:53:21.470063+0000","last_undegraded":"2024-09-11T12:53:21.470063+0000","last_fullsized":"2024-09-11T12:53:21.470063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:08:56.466322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469217+0000","last_change":"2024-09-11T12:53:20.462550+0000","last_active":"2024-09-11T12:53:21.469217+0000","last_peered":"2024-09-11T12:53:21.469217+0000","last_clean":"2024-09-11T12:53:21.469217+0000","last_became_active":"2024-09-11T12:53:20.462221+0000","last_became_peered":"2024-09-11T12:53:20.462221+0000","last_unstale":"2024-09-11T12:53:21.469217+0000","last_undegraded":"2024-09-11T12:53:21.469217+0000","last_fullsized":"2024-09-11T12:53:21.469217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:56.925859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276726+0000","last_change":"2024-09-11T12:53:19.472630+0000","last_active":"2024-09-11T12:53:22.276726+0000","last_peered":"2024-09-11T12:53:22.276726+0000","last_clean":"2024-09-11T12:53:22.276726+0000","last_became_active":"2024-09-11T12:53:19.472324+0000","last_became_peered":"2024-09-11T12:53:19.472324+0000","last_unstale":"2024-09-11T12:53:22.276726+0000","last_undegraded":"2024-09-11T12:53:22.276726+0000","last_fullsized":"2024-09-11T12:53:22.276726+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:23.522762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276373+0000","last_change":"2024-09-11T12:53:20.479477+0000","last_active":"2024-09-11T12:53:22.276373+0000","last_peered":"2024-09-11T12:53:22.276373+0000","last_clean":"2024-09-11T12:53:22.276373+0000","last_became_active":"2024-09-11T12:53:20.477912+0000","last_became_peered":"2024-09-11T12:53:20.477912+0000","last_unstale":"2024-09-11T12:53:22.276373+0000","last_undegraded":"2024-09-11T12:53:22.276373+0000","last_fullsized":"2024-09-11T12:53:22.276373+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:39.754965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469067+0000","last_change":"2024-09-11T12:53:19.477533+0000","last_active":"2024-09-11T12:53:21.469067+0000","last_peered":"2024-09-11T12:53:21.469067+0000","last_clean":"2024-09-11T12:53:21.469067+0000","last_became_active":"2024-09-11T12:53:19.476520+0000","last_became_peered":"2024-09-11T12:53:19.476520+0000","last_unstale":"2024-09-11T12:53:21.469067+0000","last_undegraded":"2024-09-11T12:53:21.469067+0000","last_fullsized":"2024-09-11T12:53:21.469067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:35.058102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467963+0000","last_change":"2024-09-11T12:53:20.478537+0000","last_active":"2024-09-11T12:53:21.467963+0000","last_peered":"2024-09-11T12:53:21.467963+0000","last_clean":"2024-09-11T12:53:21.467963+0000","last_became_active":"2024-09-11T12:53:20.478220+0000","last_became_peered":"2024-09-11T12:53:20.478220+0000","last_unstale":"2024-09-11T12:53:21.467963+0000","last_undegraded":"2024-09-11T12:53:21.467963+0000","last_fullsized":"2024-09-11T12:53:21.467963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:00.187655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703052+0000","last_change":"2024-09-11T12:53:19.469426+0000","last_active":"2024-09-11T12:53:21.703052+0000","last_peered":"2024-09-11T12:53:21.703052+0000","last_clean":"2024-09-11T12:53:21.703052+0000","last_became_active":"2024-09-11T12:53:19.469135+0000","last_became_peered":"2024-09-11T12:53:19.469135+0000","last_unstale":"2024-09-11T12:53:21.703052+0000","last_undegraded":"2024-09-11T12:53:21.703052+0000","last_fullsized":"2024-09-11T12:53:21.703052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:06.134077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.477718+0000","last_change":"2024-09-11T12:53:20.475076+0000","last_active":"2024-09-11T12:53:20.477718+0000","last_peered":"2024-09-11T12:53:20.477718+0000","last_clean":"2024-09-11T12:53:20.477718+0000","last_became_active":"2024-09-11T12:53:20.474772+0000","last_became_peered":"2024-09-11T12:53:20.474772+0000","last_unstale":"2024-09-11T12:53:20.477718+0000","last_undegraded":"2024-09-11T12:53:20.477718+0000","last_fullsized":"2024-09-11T12:53:20.477718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:22.785946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467953+0000","last_change":"2024-09-11T12:53:19.468223+0000","last_active":"2024-09-11T12:53:21.467953+0000","last_peered":"2024-09-11T12:53:21.467953+0000","last_clean":"2024-09-11T12:53:21.467953+0000","last_became_active":"2024-09-11T12:53:19.467919+0000","last_became_peered":"2024-09-11T12:53:19.467919+0000","last_unstale":"2024-09-11T12:53:21.467953+0000","last_undegraded":"2024-09-11T12:53:21.467953+0000","last_fullsized":"2024-09-11T12:53:21.467953+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:18.159044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469292+0000","last_change":"2024-09-11T12:53:20.479500+0000","last_active":"2024-09-11T12:53:21.469292+0000","last_peered":"2024-09-11T12:53:21.469292+0000","last_clean":"2024-09-11T12:53:21.469292+0000","last_became_active":"2024-09-11T12:53:20.479209+0000","last_became_peered":"2024-09-11T12:53:20.479209+0000","last_unstale":"2024-09-11T12:53:21.469292+0000","last_undegraded":"2024-09-11T12:53:21.469292+0000","last_fullsized":"2024-09-11T12:53:21.469292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:34.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276766+0000","last_change":"2024-09-11T12:53:19.465975+0000","last_active":"2024-09-11T12:53:22.276766+0000","last_peered":"2024-09-11T12:53:22.276766+0000","last_clean":"2024-09-11T12:53:22.276766+0000","last_became_active":"2024-09-11T12:53:19.465693+0000","last_became_peered":"2024-09-11T12:53:19.465693+0000","last_unstale":"2024-09-11T12:53:22.276766+0000","last_undegraded":"2024-09-11T12:53:22.276766+0000","last_fullsized":"2024-09-11T12:53:22.276766+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:33.022462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176459+0000","last_change":"2024-09-11T12:53:20.493540+0000","last_active":"2024-09-11T12:53:22.176459+0000","last_peered":"2024-09-11T12:53:22.176459+0000","last_clean":"2024-09-11T12:53:22.176459+0000","last_became_active":"2024-09-11T12:53:20.493204+0000","last_became_peered":"2024-09-11T12:53:20.493204+0000","last_unstale":"2024-09-11T12:53:22.176459+0000","last_undegraded":"2024-09-11T12:53:22.176459+0000","last_fullsized":"2024-09-11T12:53:22.176459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:09.965093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469924+0000","last_change":"2024-09-11T12:53:19.473467+0000","last_active":"2024-09-11T12:53:21.469924+0000","last_peered":"2024-09-11T12:53:21.469924+0000","last_clean":"2024-09-11T12:53:21.469924+0000","last_became_active":"2024-09-11T12:53:19.473208+0000","last_became_peered":"2024-09-11T12:53:19.473208+0000","last_unstale":"2024-09-11T12:53:21.469924+0000","last_undegraded":"2024-09-11T12:53:21.469924+0000","last_fullsized":"2024-09-11T12:53:21.469924+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:29.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469852+0000","last_change":"2024-09-11T12:53:20.481626+0000","last_active":"2024-09-11T12:53:21.469852+0000","last_peered":"2024-09-11T12:53:21.469852+0000","last_clean":"2024-09-11T12:53:21.469852+0000","last_became_active":"2024-09-11T12:53:20.480916+0000","last_became_peered":"2024-09-11T12:53:20.480916+0000","last_unstale":"2024-09-11T12:53:21.469852+0000","last_undegraded":"2024-09-11T12:53:21.469852+0000","last_fullsized":"2024-09-11T12:53:21.469852+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:04.694195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276901+0000","last_change":"2024-09-11T12:53:19.470649+0000","last_active":"2024-09-11T12:53:22.276901+0000","last_peered":"2024-09-11T12:53:22.276901+0000","last_clean":"2024-09-11T12:53:22.276901+0000","last_became_active":"2024-09-11T12:53:19.470345+0000","last_became_peered":"2024-09-11T12:53:19.470345+0000","last_unstale":"2024-09-11T12:53:22.276901+0000","last_undegraded":"2024-09-11T12:53:22.276901+0000","last_fullsized":"2024-09-11T12:53:22.276901+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:24.348860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468227+0000","last_change":"2024-09-11T12:53:20.470384+0000","last_active":"2024-09-11T12:53:21.468227+0000","last_peered":"2024-09-11T12:53:21.468227+0000","last_clean":"2024-09-11T12:53:21.468227+0000","last_became_active":"2024-09-11T12:53:20.470099+0000","last_became_peered":"2024-09-11T12:53:20.470099+0000","last_unstale":"2024-09-11T12:53:21.468227+0000","last_undegraded":"2024-09-11T12:53:21.468227+0000","last_fullsized":"2024-09-11T12:53:21.468227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:15.000764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703469+0000","last_change":"2024-09-11T12:53:19.473610+0000","last_active":"2024-09-11T12:53:21.703469+0000","last_peered":"2024-09-11T12:53:21.703469+0000","last_clean":"2024-09-11T12:53:21.703469+0000","last_became_active":"2024-09-11T12:53:19.473327+0000","last_became_peered":"2024-09-11T12:53:19.473327+0000","last_unstale":"2024-09-11T12:53:21.703469+0000","last_undegraded":"2024-09-11T12:53:21.703469+0000","last_fullsized":"2024-09-11T12:53:21.703469+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:44.362640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175959+0000","last_change":"2024-09-11T12:53:20.486720+0000","last_active":"2024-09-11T12:53:22.175959+0000","last_peered":"2024-09-11T12:53:22.175959+0000","last_clean":"2024-09-11T12:53:22.175959+0000","last_became_active":"2024-09-11T12:53:20.486411+0000","last_became_peered":"2024-09-11T12:53:20.486411+0000","last_unstale":"2024-09-11T12:53:22.175959+0000","last_undegraded":"2024-09-11T12:53:22.175959+0000","last_fullsized":"2024-09-11T12:53:22.175959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:34.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703265+0000","last_change":"2024-09-11T12:53:19.474347+0000","last_active":"2024-09-11T12:53:21.703265+0000","last_peered":"2024-09-11T12:53:21.703265+0000","last_clean":"2024-09-11T12:53:21.703265+0000","last_became_active":"2024-09-11T12:53:19.474070+0000","last_became_peered":"2024-09-11T12:53:19.474070+0000","last_unstale":"2024-09-11T12:53:21.703265+0000","last_undegraded":"2024-09-11T12:53:21.703265+0000","last_fullsized":"2024-09-11T12:53:21.703265+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:48.771609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469053+0000","last_change":"2024-09-11T12:53:20.463137+0000","last_active":"2024-09-11T12:53:21.469053+0000","last_peered":"2024-09-11T12:53:21.469053+0000","last_clean":"2024-09-11T12:53:21.469053+0000","last_became_active":"2024-09-11T12:53:20.462856+0000","last_became_peered":"2024-09-11T12:53:20.462856+0000","last_unstale":"2024-09-11T12:53:21.469053+0000","last_undegraded":"2024-09-11T12:53:21.469053+0000","last_fullsized":"2024-09-11T12:53:21.469053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:26.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469166+0000","last_change":"2024-09-11T12:53:19.477205+0000","last_active":"2024-09-11T12:53:21.469166+0000","last_peered":"2024-09-11T12:53:21.469166+0000","last_clean":"2024-09-11T12:53:21.469166+0000","last_became_active":"2024-09-11T12:53:19.476500+0000","last_became_peered":"2024-09-11T12:53:19.476500+0000","last_unstale":"2024-09-11T12:53:21.469166+0000","last_undegraded":"2024-09-11T12:53:21.469166+0000","last_fullsized":"2024-09-11T12:53:21.469166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:39.417623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.484595+0000","last_change":"2024-09-11T12:53:20.480798+0000","last_active":"2024-09-11T12:53:20.484595+0000","last_peered":"2024-09-11T12:53:20.484595+0000","last_clean":"2024-09-11T12:53:20.484595+0000","last_became_active":"2024-09-11T12:53:20.480545+0000","last_became_peered":"2024-09-11T12:53:20.480545+0000","last_unstale":"2024-09-11T12:53:20.484595+0000","last_undegraded":"2024-09-11T12:53:20.484595+0000","last_fullsized":"2024-09-11T12:53:20.484595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:57.756075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469007+0000","last_change":"2024-09-11T12:53:19.465316+0000","last_active":"2024-09-11T12:53:21.469007+0000","last_peered":"2024-09-11T12:53:21.469007+0000","last_clean":"2024-09-11T12:53:21.469007+0000","last_became_active":"2024-09-11T12:53:19.464985+0000","last_became_peered":"2024-09-11T12:53:19.464985+0000","last_unstale":"2024-09-11T12:53:21.469007+0000","last_undegraded":"2024-09-11T12:53:21.469007+0000","last_fullsized":"2024-09-11T12:53:21.469007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:42.107728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467959+0000","last_change":"2024-09-11T12:53:20.473900+0000","last_active":"2024-09-11T12:53:21.467959+0000","last_peered":"2024-09-11T12:53:21.467959+0000","last_clean":"2024-09-11T12:53:21.467959+0000","last_became_active":"2024-09-11T12:53:20.473646+0000","last_became_peered":"2024-09-11T12:53:20.473646+0000","last_unstale":"2024-09-11T12:53:21.467959+0000","last_undegraded":"2024-09-11T12:53:21.467959+0000","last_fullsized":"2024-09-11T12:53:21.467959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:57:21.618883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276232+0000","last_change":"2024-09-11T12:53:19.463694+0000","last_active":"2024-09-11T12:53:22.276232+0000","last_peered":"2024-09-11T12:53:22.276232+0000","last_clean":"2024-09-11T12:53:22.276232+0000","last_became_active":"2024-09-11T12:53:19.463382+0000","last_became_peered":"2024-09-11T12:53:19.463382+0000","last_unstale":"2024-09-11T12:53:22.276232+0000","last_undegraded":"2024-09-11T12:53:22.276232+0000","last_fullsized":"2024-09-11T12:53:22.276232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:29.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276714+0000","last_change":"2024-09-11T12:53:20.466203+0000","last_active":"2024-09-11T12:53:22.276714+0000","last_peered":"2024-09-11T12:53:22.276714+0000","last_clean":"2024-09-11T12:53:22.276714+0000","last_became_active":"2024-09-11T12:53:20.465218+0000","last_became_peered":"2024-09-11T12:53:20.465218+0000","last_unstale":"2024-09-11T12:53:22.276714+0000","last_undegraded":"2024-09-11T12:53:22.276714+0000","last_fullsized":"2024-09-11T12:53:22.276714+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:05.851122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470218+0000","last_change":"2024-09-11T12:53:19.477397+0000","last_active":"2024-09-11T12:53:21.470218+0000","last_peered":"2024-09-11T12:53:21.470218+0000","last_clean":"2024-09-11T12:53:21.470218+0000","last_became_active":"2024-09-11T12:53:19.477099+0000","last_became_peered":"2024-09-11T12:53:19.477099+0000","last_unstale":"2024-09-11T12:53:21.470218+0000","last_undegraded":"2024-09-11T12:53:21.470218+0000","last_fullsized":"2024-09-11T12:53:21.470218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:57.215380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176591+0000","last_change":"2024-09-11T12:53:20.483367+0000","last_active":"2024-09-11T12:53:22.176591+0000","last_peered":"2024-09-11T12:53:22.176591+0000","last_clean":"2024-09-11T12:53:22.176591+0000","last_became_active":"2024-09-11T12:53:20.483111+0000","last_became_peered":"2024-09-11T12:53:20.483111+0000","last_unstale":"2024-09-11T12:53:22.176591+0000","last_undegraded":"2024-09-11T12:53:22.176591+0000","last_fullsized":"2024-09-11T12:53:22.176591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:04.655167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276761+0000","last_change":"2024-09-11T12:53:19.473944+0000","last_active":"2024-09-11T12:53:22.276761+0000","last_peered":"2024-09-11T12:53:22.276761+0000","last_clean":"2024-09-11T12:53:22.276761+0000","last_became_active":"2024-09-11T12:53:19.473612+0000","last_became_peered":"2024-09-11T12:53:19.473612+0000","last_unstale":"2024-09-11T12:53:22.276761+0000","last_undegraded":"2024-09-11T12:53:22.276761+0000","last_fullsized":"2024-09-11T12:53:22.276761+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:08.320644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703359+0000","last_change":"2024-09-11T12:53:20.471760+0000","last_active":"2024-09-11T12:53:21.703359+0000","last_peered":"2024-09-11T12:53:21.703359+0000","last_clean":"2024-09-11T12:53:21.703359+0000","last_became_active":"2024-09-11T12:53:20.471337+0000","last_became_peered":"2024-09-11T12:53:20.471337+0000","last_unstale":"2024-09-11T12:53:21.703359+0000","last_undegraded":"2024-09-11T12:53:21.703359+0000","last_fullsized":"2024-09-11T12:53:21.703359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:34.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470552+0000","last_change":"2024-09-11T12:53:19.477104+0000","last_active":"2024-09-11T12:53:21.470552+0000","last_peered":"2024-09-11T12:53:21.470552+0000","last_clean":"2024-09-11T12:53:21.470552+0000","last_became_active":"2024-09-11T12:53:19.476748+0000","last_became_peered":"2024-09-11T12:53:19.476748+0000","last_unstale":"2024-09-11T12:53:21.470552+0000","last_undegraded":"2024-09-11T12:53:21.470552+0000","last_fullsized":"2024-09-11T12:53:21.470552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:53.310007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468386+0000","last_change":"2024-09-11T12:53:20.478508+0000","last_active":"2024-09-11T12:53:21.468386+0000","last_peered":"2024-09-11T12:53:21.468386+0000","last_clean":"2024-09-11T12:53:21.468386+0000","last_became_active":"2024-09-11T12:53:20.478246+0000","last_became_peered":"2024-09-11T12:53:20.478246+0000","last_unstale":"2024-09-11T12:53:21.468386+0000","last_undegraded":"2024-09-11T12:53:21.468386+0000","last_fullsized":"2024-09-11T12:53:21.468386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:56.141422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176130+0000","last_change":"2024-09-11T12:53:19.470725+0000","last_active":"2024-09-11T12:53:22.176130+0000","last_peered":"2024-09-11T12:53:22.176130+0000","last_clean":"2024-09-11T12:53:22.176130+0000","last_became_active":"2024-09-11T12:53:19.470451+0000","last_became_peered":"2024-09-11T12:53:19.470451+0000","last_unstale":"2024-09-11T12:53:22.176130+0000","last_undegraded":"2024-09-11T12:53:22.176130+0000","last_fullsized":"2024-09-11T12:53:22.176130+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:38.752193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276797+0000","last_change":"2024-09-11T12:53:20.464491+0000","last_active":"2024-09-11T12:53:22.276797+0000","last_peered":"2024-09-11T12:53:22.276797+0000","last_clean":"2024-09-11T12:53:22.276797+0000","last_became_active":"2024-09-11T12:53:20.464224+0000","last_became_peered":"2024-09-11T12:53:20.464224+0000","last_unstale":"2024-09-11T12:53:22.276797+0000","last_undegraded":"2024-09-11T12:53:22.276797+0000","last_fullsized":"2024-09-11T12:53:22.276797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:04:56.705817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469207+0000","last_change":"2024-09-11T12:53:19.476907+0000","last_active":"2024-09-11T12:53:21.469207+0000","last_peered":"2024-09-11T12:53:21.469207+0000","last_clean":"2024-09-11T12:53:21.469207+0000","last_became_active":"2024-09-11T12:53:19.476155+0000","last_became_peered":"2024-09-11T12:53:19.476155+0000","last_unstale":"2024-09-11T12:53:21.469207+0000","last_undegraded":"2024-09-11T12:53:21.469207+0000","last_fullsized":"2024-09-11T12:53:21.469207+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:09:51.133433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276501+0000","last_change":"2024-09-11T12:53:20.483673+0000","last_active":"2024-09-11T12:53:22.276501+0000","last_peered":"2024-09-11T12:53:22.276501+0000","last_clean":"2024-09-11T12:53:22.276501+0000","last_became_active":"2024-09-11T12:53:20.481291+0000","last_became_peered":"2024-09-11T12:53:20.481291+0000","last_unstale":"2024-09-11T12:53:22.276501+0000","last_undegraded":"2024-09-11T12:53:22.276501+0000","last_fullsized":"2024-09-11T12:53:22.276501+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:08.397000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469116+0000","last_change":"2024-09-11T12:53:19.464486+0000","last_active":"2024-09-11T12:53:21.469116+0000","last_peered":"2024-09-11T12:53:21.469116+0000","last_clean":"2024-09-11T12:53:21.469116+0000","last_became_active":"2024-09-11T12:53:19.463653+0000","last_became_peered":"2024-09-11T12:53:19.463653+0000","last_unstale":"2024-09-11T12:53:21.469116+0000","last_undegraded":"2024-09-11T12:53:21.469116+0000","last_fullsized":"2024-09-11T12:53:21.469116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:38.752193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276519+0000","last_change":"2024-09-11T12:53:20.483611+0000","last_active":"2024-09-11T12:53:22.276519+0000","last_peered":"2024-09-11T12:53:22.276519+0000","last_clean":"2024-09-11T12:53:22.276519+0000","last_became_active":"2024-09-11T12:53:20.480553+0000","last_became_peered":"2024-09-11T12:53:20.480553+0000","last_unstale":"2024-09-11T12:53:22.276519+0000","last_undegraded":"2024-09-11T12:53:22.276519+0000","last_fullsized":"2024-09-11T12:53:22.276519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:04.268363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.459280+0000","last_change":"2024-09-11T12:53:19.464453+0000","last_active":"2024-09-11T12:53:20.459280+0000","last_peered":"2024-09-11T12:53:20.459280+0000","last_clean":"2024-09-11T12:53:20.459280+0000","last_became_active":"2024-09-11T12:53:19.464107+0000","last_became_peered":"2024-09-11T12:53:19.464107+0000","last_unstale":"2024-09-11T12:53:20.459280+0000","last_undegraded":"2024-09-11T12:53:20.459280+0000","last_fullsized":"2024-09-11T12:53:20.459280+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:53.979041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.476341+0000","last_change":"2024-09-11T12:53:20.470134+0000","last_active":"2024-09-11T12:53:20.476341+0000","last_peered":"2024-09-11T12:53:20.476341+0000","last_clean":"2024-09-11T12:53:20.476341+0000","last_became_active":"2024-09-11T12:53:20.469876+0000","last_became_peered":"2024-09-11T12:53:20.469876+0000","last_unstale":"2024-09-11T12:53:20.476341+0000","last_undegraded":"2024-09-11T12:53:20.476341+0000","last_fullsized":"2024-09-11T12:53:20.476341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:12.010107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470326+0000","last_change":"2024-09-11T12:53:19.477142+0000","last_active":"2024-09-11T12:53:21.470326+0000","last_peered":"2024-09-11T12:53:21.470326+0000","last_clean":"2024-09-11T12:53:21.470326+0000","last_became_active":"2024-09-11T12:53:19.476535+0000","last_became_peered":"2024-09-11T12:53:19.476535+0000","last_unstale":"2024-09-11T12:53:21.470326+0000","last_undegraded":"2024-09-11T12:53:21.470326+0000","last_fullsized":"2024-09-11T12:53:21.470326+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:04.862591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469706+0000","last_change":"2024-09-11T12:53:20.459328+0000","last_active":"2024-09-11T12:53:21.469706+0000","last_peered":"2024-09-11T12:53:21.469706+0000","last_clean":"2024-09-11T12:53:21.469706+0000","last_became_active":"2024-09-11T12:53:20.458974+0000","last_became_peered":"2024-09-11T12:53:20.458974+0000","last_unstale":"2024-09-11T12:53:21.469706+0000","last_undegraded":"2024-09-11T12:53:21.469706+0000","last_fullsized":"2024-09-11T12:53:21.469706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:10.768707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.477025+0000","last_change":"2024-09-11T12:53:19.471151+0000","last_active":"2024-09-11T12:53:21.477025+0000","last_peered":"2024-09-11T12:53:21.477025+0000","last_clean":"2024-09-11T12:53:21.477025+0000","last_became_active":"2024-09-11T12:53:19.470300+0000","last_became_peered":"2024-09-11T12:53:19.470300+0000","last_unstale":"2024-09-11T12:53:21.477025+0000","last_undegraded":"2024-09-11T12:53:21.477025+0000","last_fullsized":"2024-09-11T12:53:21.477025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:06.348634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176208+0000","last_change":"2024-09-11T12:53:20.494957+0000","last_active":"2024-09-11T12:53:22.176208+0000","last_peered":"2024-09-11T12:53:22.176208+0000","last_clean":"2024-09-11T12:53:22.176208+0000","last_became_active":"2024-09-11T12:53:20.494687+0000","last_became_peered":"2024-09-11T12:53:20.494687+0000","last_unstale":"2024-09-11T12:53:22.176208+0000","last_undegraded":"2024-09-11T12:53:22.176208+0000","last_fullsized":"2024-09-11T12:53:22.176208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:08.328135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468332+0000","last_change":"2024-09-11T12:53:19.469592+0000","last_active":"2024-09-11T12:53:21.468332+0000","last_peered":"2024-09-11T12:53:21.468332+0000","last_clean":"2024-09-11T12:53:21.468332+0000","last_became_active":"2024-09-11T12:53:19.469249+0000","last_became_peered":"2024-09-11T12:53:19.469249+0000","last_unstale":"2024-09-11T12:53:21.468332+0000","last_undegraded":"2024-09-11T12:53:21.468332+0000","last_fullsized":"2024-09-11T12:53:21.468332+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:34:22.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176484+0000","last_change":"2024-09-11T12:53:20.489097+0000","last_active":"2024-09-11T12:53:22.176484+0000","last_peered":"2024-09-11T12:53:22.176484+0000","last_clean":"2024-09-11T12:53:22.176484+0000","last_became_active":"2024-09-11T12:53:20.488696+0000","last_became_peered":"2024-09-11T12:53:20.488696+0000","last_unstale":"2024-09-11T12:53:22.176484+0000","last_undegraded":"2024-09-11T12:53:22.176484+0000","last_fullsized":"2024-09-11T12:53:22.176484+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:01.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.460563+0000","last_change":"2024-09-11T12:53:19.463172+0000","last_active":"2024-09-11T12:53:20.460563+0000","last_peered":"2024-09-11T12:53:20.460563+0000","last_clean":"2024-09-11T12:53:20.460563+0000","last_became_active":"2024-09-11T12:53:19.462777+0000","last_became_peered":"2024-09-11T12:53:19.462777+0000","last_unstale":"2024-09-11T12:53:20.460563+0000","last_undegraded":"2024-09-11T12:53:20.460563+0000","last_fullsized":"2024-09-11T12:53:20.460563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:01.593685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468217+0000","last_change":"2024-09-11T12:53:20.474922+0000","last_active":"2024-09-11T12:53:21.468217+0000","last_peered":"2024-09-11T12:53:21.468217+0000","last_clean":"2024-09-11T12:53:21.468217+0000","last_became_active":"2024-09-11T12:53:20.473380+0000","last_became_peered":"2024-09-11T12:53:20.473380+0000","last_unstale":"2024-09-11T12:53:21.468217+0000","last_undegraded":"2024-09-11T12:53:21.468217+0000","last_fullsized":"2024-09-11T12:53:21.468217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:28:53.470915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468088+0000","last_change":"2024-09-11T12:53:19.471741+0000","last_active":"2024-09-11T12:53:21.468088+0000","last_peered":"2024-09-11T12:53:21.468088+0000","last_clean":"2024-09-11T12:53:21.468088+0000","last_became_active":"2024-09-11T12:53:19.470462+0000","last_became_peered":"2024-09-11T12:53:19.470462+0000","last_unstale":"2024-09-11T12:53:21.468088+0000","last_undegraded":"2024-09-11T12:53:21.468088+0000","last_fullsized":"2024-09-11T12:53:21.468088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:45.385051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468917+0000","last_change":"2024-09-11T12:53:20.481889+0000","last_active":"2024-09-11T12:53:21.468917+0000","last_peered":"2024-09-11T12:53:21.468917+0000","last_clean":"2024-09-11T12:53:21.468917+0000","last_became_active":"2024-09-11T12:53:20.481630+0000","last_became_peered":"2024-09-11T12:53:20.481630+0000","last_unstale":"2024-09-11T12:53:21.468917+0000","last_undegraded":"2024-09-11T12:53:21.468917+0000","last_fullsized":"2024-09-11T12:53:21.468917+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:39.754965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703070+0000","last_change":"2024-09-11T12:53:19.473753+0000","last_active":"2024-09-11T12:53:21.703070+0000","last_peered":"2024-09-11T12:53:21.703070+0000","last_clean":"2024-09-11T12:53:21.703070+0000","last_became_active":"2024-09-11T12:53:19.473468+0000","last_became_peered":"2024-09-11T12:53:19.473468+0000","last_unstale":"2024-09-11T12:53:21.703070+0000","last_undegraded":"2024-09-11T12:53:21.703070+0000","last_fullsized":"2024-09-11T12:53:21.703070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:23.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703278+0000","last_change":"2024-09-11T12:53:20.465576+0000","last_active":"2024-09-11T12:53:21.703278+0000","last_peered":"2024-09-11T12:53:21.703278+0000","last_clean":"2024-09-11T12:53:21.703278+0000","last_became_active":"2024-09-11T12:53:20.464585+0000","last_became_peered":"2024-09-11T12:53:20.464585+0000","last_unstale":"2024-09-11T12:53:21.703278+0000","last_undegraded":"2024-09-11T12:53:21.703278+0000","last_fullsized":"2024-09-11T12:53:21.703278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:30.844586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276012+0000","last_change":"2024-09-11T12:53:19.463737+0000","last_active":"2024-09-11T12:53:22.276012+0000","last_peered":"2024-09-11T12:53:22.276012+0000","last_clean":"2024-09-11T12:53:22.276012+0000","last_became_active":"2024-09-11T12:53:19.463359+0000","last_became_peered":"2024-09-11T12:53:19.463359+0000","last_unstale":"2024-09-11T12:53:22.276012+0000","last_undegraded":"2024-09-11T12:53:22.276012+0000","last_fullsized":"2024-09-11T12:53:22.276012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:08:56.466322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468855+0000","last_change":"2024-09-11T12:53:20.478214+0000","last_active":"2024-09-11T12:53:21.468855+0000","last_peered":"2024-09-11T12:53:21.468855+0000","last_clean":"2024-09-11T12:53:21.468855+0000","last_became_active":"2024-09-11T12:53:20.477191+0000","last_became_peered":"2024-09-11T12:53:20.477191+0000","last_unstale":"2024-09-11T12:53:21.468855+0000","last_undegraded":"2024-09-11T12:53:21.468855+0000","last_fullsized":"2024-09-11T12:53:21.468855+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:43.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470133+0000","last_change":"2024-09-11T12:53:19.477833+0000","last_active":"2024-09-11T12:53:21.470133+0000","last_peered":"2024-09-11T12:53:21.470133+0000","last_clean":"2024-09-11T12:53:21.470133+0000","last_became_active":"2024-09-11T12:53:19.477514+0000","last_became_peered":"2024-09-11T12:53:19.477514+0000","last_unstale":"2024-09-11T12:53:21.470133+0000","last_undegraded":"2024-09-11T12:53:21.470133+0000","last_fullsized":"2024-09-11T12:53:21.470133+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:28.098161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276582+0000","last_change":"2024-09-11T12:53:20.484966+0000","last_active":"2024-09-11T12:53:22.276582+0000","last_peered":"2024-09-11T12:53:22.276582+0000","last_clean":"2024-09-11T12:53:22.276582+0000","last_became_active":"2024-09-11T12:53:20.480486+0000","last_became_peered":"2024-09-11T12:53:20.480486+0000","last_unstale":"2024-09-11T12:53:22.276582+0000","last_undegraded":"2024-09-11T12:53:22.276582+0000","last_fullsized":"2024-09-11T12:53:22.276582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:11.453911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470208+0000","last_change":"2024-09-11T12:53:19.478268+0000","last_active":"2024-09-11T12:53:21.470208+0000","last_peered":"2024-09-11T12:53:21.470208+0000","last_clean":"2024-09-11T12:53:21.470208+0000","last_became_active":"2024-09-11T12:53:19.477853+0000","last_became_peered":"2024-09-11T12:53:19.477853+0000","last_unstale":"2024-09-11T12:53:21.470208+0000","last_undegraded":"2024-09-11T12:53:21.470208+0000","last_fullsized":"2024-09-11T12:53:21.470208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:49.096469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":195,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175780+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:53:22.175780+0000","last_peered":"2024-09-11T12:53:22.175780+0000","last_clean":"2024-09-11T12:53:22.175780+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:53:22.175780+0000","last_undegraded":"2024-09-11T12:53:22.175780+0000","last_fullsized":"2024-09-11T12:53:22.175780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1433600,"data_stored":936873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":345981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":835584,"data_stored":345979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":345959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":339701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":347501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":839680,"data_stored":341817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:53:24.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:53:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 -- 172.21.15.78:0/335278329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f28005e60 msgr2=0x7f6f280062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 --2- 172.21.15.78:0/335278329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f28005e60 0x7f6f280062e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c00c750 tx=0x7f6f3c01fa60 comp rx=0 tx=0).stop 2024-09-11T12:53:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f42800640 1 -- 172.21.15.78:0/335278329 reap_dead start 2024-09-11T12:53:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 -- 172.21.15.78:0/335278329 shutdown_connections 2024-09-11T12:53:24.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 -- 172.21.15.78:0/335278329 >> 172.21.15.78:0/335278329 conn(0x7f6f4406ca80 msgr2=0x7f6f440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 -- 172.21.15.78:0/335278329 shutdown_connections 2024-09-11T12:53:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.760+0000 7f6f48e00640 1 -- 172.21.15.78:0/335278329 wait complete. 2024-09-11T12:53:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 Processor -- start 2024-09-11T12:53:24.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- start start 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28005e60 0x7f6f44159a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 0x7f6f441f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4415e1c0 con 0x7f6f4415c7e0 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f440b4db0 con 0x7f6f28005e60 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f4415e340 con 0x7f6f44153f70 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54366/0 (socket says 172.21.15.78:54366) 2024-09-11T12:53:24.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 -- 172.21.15.78:0/3215639462 learned_addr learned my addr 172.21.15.78:0/3215639462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:24.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f41e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28005e60 0x7f6f44159a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f3be00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 0x7f6f441f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 -- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 msgr2=0x7f6f441f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 0x7f6f441f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 -- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28005e60 msgr2=0x7f6f44159a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28005e60 0x7f6f44159a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 -- 172.21.15.78:0/3215639462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f4415d140 con 0x7f6f4415c7e0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f3be00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 0x7f6f441f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f42800640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c021140 tx=0x7f6f3c01fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c00c380 con 0x7f6f4415c7e0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f3c0a9630 con 0x7f6f4415c7e0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f3c0a9a00 con 0x7f6f4415c7e0 2024-09-11T12:53:24.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f441f48c0 con 0x7f6f4415c7e0 2024-09-11T12:53:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f4415b830 con 0x7f6f4415c7e0 2024-09-11T12:53:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.764+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f440cbed0 con 0x7f6f4415c7e0 2024-09-11T12:53:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.768+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f6f3c0a9ba0 con 0x7f6f4415c7e0 2024-09-11T12:53:24.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.768+0000 7f6f3a000640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 0x7f6f20073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.768+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f6f3c1301c0 con 0x7f6f4415c7e0 2024-09-11T12:53:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.768+0000 7f6f41e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 0x7f6f20073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:24.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.768+0000 7f6f41e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 0x7f6f20073ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005fd0 tx=0x7f6f3400b260 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:24.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.772+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6f4415e700 con 0x7f6f4415c7e0 2024-09-11T12:53:24.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.936+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f4415b300 con 0x7f6f20071a10 2024-09-11T12:53:24.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.940+0000 7f6f3a000640 1 -- 172.21.15.78:0/3215639462 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252055 (secure 0 0 0) 0x7f6f4415b300 con 0x7f6f20071a10 2024-09-11T12:53:24.944 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:53:24.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 msgr2=0x7f6f20073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 0x7f6f20073ed0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f6f34005fd0 tx=0x7f6f3400b260 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 msgr2=0x7f6f4415cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c021140 tx=0x7f6f3c01fcb0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 shutdown_connections 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6f20071a10 0x7f6f20073ed0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f4415c7e0 0x7f6f4415cc00 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f44153f70 0x7f6f441f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 --2- 172.21.15.78:0/3215639462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f28005e60 0x7f6f44159a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 >> 172.21.15.78:0/3215639462 conn(0x7f6f4406ca80 msgr2=0x7f6f44072d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 shutdown_connections 2024-09-11T12:53:24.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:24.944+0000 7f6f48e00640 1 -- 172.21.15.78:0/3215639462 wait complete. 2024-09-11T12:53:24.965 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2024-09-11T12:53:24.486324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":129,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257232,"kb_used_data":7684,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717488,"statfs":{"total":773094113280,"available":772830707712,"internally_reserved":0,"allocated":7868416,"data_stored":3933768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.037178"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176460+0000","last_change":"2024-09-11T12:53:20.486705+0000","last_active":"2024-09-11T12:53:22.176460+0000","last_peered":"2024-09-11T12:53:22.176460+0000","last_clean":"2024-09-11T12:53:22.176460+0000","last_became_active":"2024-09-11T12:53:20.486395+0000","last_became_peered":"2024-09-11T12:53:20.486395+0000","last_unstale":"2024-09-11T12:53:22.176460+0000","last_undegraded":"2024-09-11T12:53:22.176460+0000","last_fullsized":"2024-09-11T12:53:22.176460+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:16.633708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469738+0000","last_change":"2024-09-11T12:53:20.477869+0000","last_active":"2024-09-11T12:53:21.469738+0000","last_peered":"2024-09-11T12:53:21.469738+0000","last_clean":"2024-09-11T12:53:21.469738+0000","last_became_active":"2024-09-11T12:53:20.477558+0000","last_became_peered":"2024-09-11T12:53:20.477558+0000","last_unstale":"2024-09-11T12:53:21.469738+0000","last_undegraded":"2024-09-11T12:53:21.469738+0000","last_fullsized":"2024-09-11T12:53:21.469738+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:07.351406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276638+0000","last_change":"2024-09-11T12:53:20.463411+0000","last_active":"2024-09-11T12:53:22.276638+0000","last_peered":"2024-09-11T12:53:22.276638+0000","last_clean":"2024-09-11T12:53:22.276638+0000","last_became_active":"2024-09-11T12:53:20.463120+0000","last_became_peered":"2024-09-11T12:53:20.463120+0000","last_unstale":"2024-09-11T12:53:22.276638+0000","last_undegraded":"2024-09-11T12:53:22.276638+0000","last_fullsized":"2024-09-11T12:53:22.276638+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:25.834098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703447+0000","last_change":"2024-09-11T12:53:20.483381+0000","last_active":"2024-09-11T12:53:21.703447+0000","last_peered":"2024-09-11T12:53:21.703447+0000","last_clean":"2024-09-11T12:53:21.703447+0000","last_became_active":"2024-09-11T12:53:20.483128+0000","last_became_peered":"2024-09-11T12:53:20.483128+0000","last_unstale":"2024-09-11T12:53:21.703447+0000","last_undegraded":"2024-09-11T12:53:21.703447+0000","last_fullsized":"2024-09-11T12:53:21.703447+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:26:04.881869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468088+0000","last_change":"2024-09-11T12:53:20.495951+0000","last_active":"2024-09-11T12:53:21.468088+0000","last_peered":"2024-09-11T12:53:21.468088+0000","last_clean":"2024-09-11T12:53:21.468088+0000","last_became_active":"2024-09-11T12:53:20.495600+0000","last_became_peered":"2024-09-11T12:53:20.495600+0000","last_unstale":"2024-09-11T12:53:21.468088+0000","last_undegraded":"2024-09-11T12:53:21.468088+0000","last_fullsized":"2024-09-11T12:53:21.468088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:31.296611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469108+0000","last_change":"2024-09-11T12:53:20.477991+0000","last_active":"2024-09-11T12:53:21.469108+0000","last_peered":"2024-09-11T12:53:21.469108+0000","last_clean":"2024-09-11T12:53:21.469108+0000","last_became_active":"2024-09-11T12:53:20.477685+0000","last_became_peered":"2024-09-11T12:53:20.477685+0000","last_unstale":"2024-09-11T12:53:21.469108+0000","last_undegraded":"2024-09-11T12:53:21.469108+0000","last_fullsized":"2024-09-11T12:53:21.469108+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:22.499200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468931+0000","last_change":"2024-09-11T12:53:20.459176+0000","last_active":"2024-09-11T12:53:21.468931+0000","last_peered":"2024-09-11T12:53:21.468931+0000","last_clean":"2024-09-11T12:53:21.468931+0000","last_became_active":"2024-09-11T12:53:20.458424+0000","last_became_peered":"2024-09-11T12:53:20.458424+0000","last_unstale":"2024-09-11T12:53:21.468931+0000","last_undegraded":"2024-09-11T12:53:21.468931+0000","last_fullsized":"2024-09-11T12:53:21.468931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:17:35.639498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469866+0000","last_change":"2024-09-11T12:53:20.479100+0000","last_active":"2024-09-11T12:53:21.469866+0000","last_peered":"2024-09-11T12:53:21.469866+0000","last_clean":"2024-09-11T12:53:21.469866+0000","last_became_active":"2024-09-11T12:53:20.478366+0000","last_became_peered":"2024-09-11T12:53:20.478366+0000","last_unstale":"2024-09-11T12:53:21.469866+0000","last_undegraded":"2024-09-11T12:53:21.469866+0000","last_fullsized":"2024-09-11T12:53:21.469866+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:05.851122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468488+0000","last_change":"2024-09-11T12:53:20.475722+0000","last_active":"2024-09-11T12:53:21.468488+0000","last_peered":"2024-09-11T12:53:21.468488+0000","last_clean":"2024-09-11T12:53:21.468488+0000","last_became_active":"2024-09-11T12:53:20.474176+0000","last_became_peered":"2024-09-11T12:53:20.474176+0000","last_unstale":"2024-09-11T12:53:21.468488+0000","last_undegraded":"2024-09-11T12:53:21.468488+0000","last_fullsized":"2024-09-11T12:53:21.468488+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:42.076643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276666+0000","last_change":"2024-09-11T12:53:20.471313+0000","last_active":"2024-09-11T12:53:22.276666+0000","last_peered":"2024-09-11T12:53:22.276666+0000","last_clean":"2024-09-11T12:53:22.276666+0000","last_became_active":"2024-09-11T12:53:20.470632+0000","last_became_peered":"2024-09-11T12:53:20.470632+0000","last_unstale":"2024-09-11T12:53:22.276666+0000","last_undegraded":"2024-09-11T12:53:22.276666+0000","last_fullsized":"2024-09-11T12:53:22.276666+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:04.694195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176290+0000","last_change":"2024-09-11T12:53:20.483790+0000","last_active":"2024-09-11T12:53:22.176290+0000","last_peered":"2024-09-11T12:53:22.176290+0000","last_clean":"2024-09-11T12:53:22.176290+0000","last_became_active":"2024-09-11T12:53:20.483454+0000","last_became_peered":"2024-09-11T12:53:20.483454+0000","last_unstale":"2024-09-11T12:53:22.176290+0000","last_undegraded":"2024-09-11T12:53:22.176290+0000","last_fullsized":"2024-09-11T12:53:22.176290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:24.525533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703364+0000","last_change":"2024-09-11T12:53:20.471588+0000","last_active":"2024-09-11T12:53:21.703364+0000","last_peered":"2024-09-11T12:53:21.703364+0000","last_clean":"2024-09-11T12:53:21.703364+0000","last_became_active":"2024-09-11T12:53:20.470976+0000","last_became_peered":"2024-09-11T12:53:20.470976+0000","last_unstale":"2024-09-11T12:53:21.703364+0000","last_undegraded":"2024-09-11T12:53:21.703364+0000","last_fullsized":"2024-09-11T12:53:21.703364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:04.873191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.484490+0000","last_change":"2024-09-11T12:53:20.482418+0000","last_active":"2024-09-11T12:53:20.484490+0000","last_peered":"2024-09-11T12:53:20.484490+0000","last_clean":"2024-09-11T12:53:20.484490+0000","last_became_active":"2024-09-11T12:53:20.482122+0000","last_became_peered":"2024-09-11T12:53:20.482122+0000","last_unstale":"2024-09-11T12:53:20.484490+0000","last_undegraded":"2024-09-11T12:53:20.484490+0000","last_fullsized":"2024-09-11T12:53:20.484490+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:36:49.026146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468176+0000","last_change":"2024-09-11T12:53:20.476343+0000","last_active":"2024-09-11T12:53:21.468176+0000","last_peered":"2024-09-11T12:53:21.468176+0000","last_clean":"2024-09-11T12:53:21.468176+0000","last_became_active":"2024-09-11T12:53:20.475910+0000","last_became_peered":"2024-09-11T12:53:20.475910+0000","last_unstale":"2024-09-11T12:53:21.468176+0000","last_undegraded":"2024-09-11T12:53:21.468176+0000","last_fullsized":"2024-09-11T12:53:21.468176+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:06:59.154745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.476380+0000","last_change":"2024-09-11T12:53:20.471273+0000","last_active":"2024-09-11T12:53:20.476380+0000","last_peered":"2024-09-11T12:53:20.476380+0000","last_clean":"2024-09-11T12:53:20.476380+0000","last_became_active":"2024-09-11T12:53:20.470985+0000","last_became_peered":"2024-09-11T12:53:20.470985+0000","last_unstale":"2024-09-11T12:53:20.476380+0000","last_undegraded":"2024-09-11T12:53:20.476380+0000","last_fullsized":"2024-09-11T12:53:20.476380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:36.750482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276057+0000","last_change":"2024-09-11T12:53:20.473580+0000","last_active":"2024-09-11T12:53:22.276057+0000","last_peered":"2024-09-11T12:53:22.276057+0000","last_clean":"2024-09-11T12:53:22.276057+0000","last_became_active":"2024-09-11T12:53:20.473179+0000","last_became_peered":"2024-09-11T12:53:20.473179+0000","last_unstale":"2024-09-11T12:53:22.276057+0000","last_undegraded":"2024-09-11T12:53:22.276057+0000","last_fullsized":"2024-09-11T12:53:22.276057+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:40.023711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468515+0000","last_change":"2024-09-11T12:53:20.476106+0000","last_active":"2024-09-11T12:53:21.468515+0000","last_peered":"2024-09-11T12:53:21.468515+0000","last_clean":"2024-09-11T12:53:21.468515+0000","last_became_active":"2024-09-11T12:53:20.474526+0000","last_became_peered":"2024-09-11T12:53:20.474526+0000","last_unstale":"2024-09-11T12:53:21.468515+0000","last_undegraded":"2024-09-11T12:53:21.468515+0000","last_fullsized":"2024-09-11T12:53:21.468515+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:15:28.692763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703305+0000","last_change":"2024-09-11T12:53:20.482025+0000","last_active":"2024-09-11T12:53:21.703305+0000","last_peered":"2024-09-11T12:53:21.703305+0000","last_clean":"2024-09-11T12:53:21.703305+0000","last_became_active":"2024-09-11T12:53:20.481580+0000","last_became_peered":"2024-09-11T12:53:20.481580+0000","last_unstale":"2024-09-11T12:53:21.703305+0000","last_undegraded":"2024-09-11T12:53:21.703305+0000","last_fullsized":"2024-09-11T12:53:21.703305+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:09.323415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470423+0000","last_change":"2024-09-11T12:53:20.477741+0000","last_active":"2024-09-11T12:53:21.470423+0000","last_peered":"2024-09-11T12:53:21.470423+0000","last_clean":"2024-09-11T12:53:21.470423+0000","last_became_active":"2024-09-11T12:53:20.477330+0000","last_became_peered":"2024-09-11T12:53:20.477330+0000","last_unstale":"2024-09-11T12:53:21.470423+0000","last_undegraded":"2024-09-11T12:53:21.470423+0000","last_fullsized":"2024-09-11T12:53:21.470423+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:12:09.716396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.468425+0000","last_change":"2024-09-11T12:53:20.466472+0000","last_active":"2024-09-11T12:53:20.468425+0000","last_peered":"2024-09-11T12:53:20.468425+0000","last_clean":"2024-09-11T12:53:20.468425+0000","last_became_active":"2024-09-11T12:53:20.466210+0000","last_became_peered":"2024-09-11T12:53:20.466210+0000","last_unstale":"2024-09-11T12:53:20.468425+0000","last_undegraded":"2024-09-11T12:53:20.468425+0000","last_fullsized":"2024-09-11T12:53:20.468425+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:04.936945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276427+0000","last_change":"2024-09-11T12:53:20.461606+0000","last_active":"2024-09-11T12:53:22.276427+0000","last_peered":"2024-09-11T12:53:22.276427+0000","last_clean":"2024-09-11T12:53:22.276427+0000","last_became_active":"2024-09-11T12:53:20.461275+0000","last_became_peered":"2024-09-11T12:53:20.461275+0000","last_unstale":"2024-09-11T12:53:22.276427+0000","last_undegraded":"2024-09-11T12:53:22.276427+0000","last_fullsized":"2024-09-11T12:53:22.276427+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:22.499200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276266+0000","last_change":"2024-09-11T12:53:20.473587+0000","last_active":"2024-09-11T12:53:22.276266+0000","last_peered":"2024-09-11T12:53:22.276266+0000","last_clean":"2024-09-11T12:53:22.276266+0000","last_became_active":"2024-09-11T12:53:20.472745+0000","last_became_peered":"2024-09-11T12:53:20.472745+0000","last_unstale":"2024-09-11T12:53:22.276266+0000","last_undegraded":"2024-09-11T12:53:22.276266+0000","last_fullsized":"2024-09-11T12:53:22.276266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:43.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175963+0000","last_change":"2024-09-11T12:53:20.483727+0000","last_active":"2024-09-11T12:53:22.175963+0000","last_peered":"2024-09-11T12:53:22.175963+0000","last_clean":"2024-09-11T12:53:22.175963+0000","last_became_active":"2024-09-11T12:53:20.483444+0000","last_became_peered":"2024-09-11T12:53:20.483444+0000","last_unstale":"2024-09-11T12:53:22.175963+0000","last_undegraded":"2024-09-11T12:53:22.175963+0000","last_fullsized":"2024-09-11T12:53:22.175963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:10.768707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175902+0000","last_change":"2024-09-11T12:53:20.492821+0000","last_active":"2024-09-11T12:53:22.175902+0000","last_peered":"2024-09-11T12:53:22.175902+0000","last_clean":"2024-09-11T12:53:22.175902+0000","last_became_active":"2024-09-11T12:53:20.492483+0000","last_became_peered":"2024-09-11T12:53:20.492483+0000","last_unstale":"2024-09-11T12:53:22.175902+0000","last_undegraded":"2024-09-11T12:53:22.175902+0000","last_fullsized":"2024-09-11T12:53:22.175902+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:56.925859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276623+0000","last_change":"2024-09-11T12:53:20.480904+0000","last_active":"2024-09-11T12:53:22.276623+0000","last_peered":"2024-09-11T12:53:22.276623+0000","last_clean":"2024-09-11T12:53:22.276623+0000","last_became_active":"2024-09-11T12:53:20.480640+0000","last_became_peered":"2024-09-11T12:53:20.480640+0000","last_unstale":"2024-09-11T12:53:22.276623+0000","last_undegraded":"2024-09-11T12:53:22.276623+0000","last_fullsized":"2024-09-11T12:53:22.276623+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:07.351406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469083+0000","last_change":"2024-09-11T12:53:20.480171+0000","last_active":"2024-09-11T12:53:21.469083+0000","last_peered":"2024-09-11T12:53:21.469083+0000","last_clean":"2024-09-11T12:53:21.469083+0000","last_became_active":"2024-09-11T12:53:20.479773+0000","last_became_peered":"2024-09-11T12:53:20.479773+0000","last_unstale":"2024-09-11T12:53:21.469083+0000","last_undegraded":"2024-09-11T12:53:21.469083+0000","last_fullsized":"2024-09-11T12:53:21.469083+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:56.617603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276675+0000","last_change":"2024-09-11T12:53:20.477930+0000","last_active":"2024-09-11T12:53:22.276675+0000","last_peered":"2024-09-11T12:53:22.276675+0000","last_clean":"2024-09-11T12:53:22.276675+0000","last_became_active":"2024-09-11T12:53:20.477440+0000","last_became_peered":"2024-09-11T12:53:20.477440+0000","last_unstale":"2024-09-11T12:53:22.276675+0000","last_undegraded":"2024-09-11T12:53:22.276675+0000","last_fullsized":"2024-09-11T12:53:22.276675+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:59.300032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276581+0000","last_change":"2024-09-11T12:53:20.470892+0000","last_active":"2024-09-11T12:53:22.276581+0000","last_peered":"2024-09-11T12:53:22.276581+0000","last_clean":"2024-09-11T12:53:22.276581+0000","last_became_active":"2024-09-11T12:53:20.470550+0000","last_became_peered":"2024-09-11T12:53:20.470550+0000","last_unstale":"2024-09-11T12:53:22.276581+0000","last_undegraded":"2024-09-11T12:53:22.276581+0000","last_fullsized":"2024-09-11T12:53:22.276581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:41.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276561+0000","last_change":"2024-09-11T12:53:20.469947+0000","last_active":"2024-09-11T12:53:22.276561+0000","last_peered":"2024-09-11T12:53:22.276561+0000","last_clean":"2024-09-11T12:53:22.276561+0000","last_became_active":"2024-09-11T12:53:20.469638+0000","last_became_peered":"2024-09-11T12:53:20.469638+0000","last_unstale":"2024-09-11T12:53:22.276561+0000","last_undegraded":"2024-09-11T12:53:22.276561+0000","last_fullsized":"2024-09-11T12:53:22.276561+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:11.390895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469039+0000","last_change":"2024-09-11T12:53:20.480977+0000","last_active":"2024-09-11T12:53:21.469039+0000","last_peered":"2024-09-11T12:53:21.469039+0000","last_clean":"2024-09-11T12:53:21.469039+0000","last_became_active":"2024-09-11T12:53:20.480716+0000","last_became_peered":"2024-09-11T12:53:20.480716+0000","last_unstale":"2024-09-11T12:53:21.469039+0000","last_undegraded":"2024-09-11T12:53:21.469039+0000","last_fullsized":"2024-09-11T12:53:21.469039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:03.204323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469115+0000","last_change":"2024-09-11T12:53:20.468322+0000","last_active":"2024-09-11T12:53:21.469115+0000","last_peered":"2024-09-11T12:53:21.469115+0000","last_clean":"2024-09-11T12:53:21.469115+0000","last_became_active":"2024-09-11T12:53:20.468033+0000","last_became_peered":"2024-09-11T12:53:20.468033+0000","last_unstale":"2024-09-11T12:53:21.469115+0000","last_undegraded":"2024-09-11T12:53:21.469115+0000","last_fullsized":"2024-09-11T12:53:21.469115+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:35.898218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470044+0000","last_change":"2024-09-11T12:53:20.476990+0000","last_active":"2024-09-11T12:53:21.470044+0000","last_peered":"2024-09-11T12:53:21.470044+0000","last_clean":"2024-09-11T12:53:21.470044+0000","last_became_active":"2024-09-11T12:53:20.476664+0000","last_became_peered":"2024-09-11T12:53:20.476664+0000","last_unstale":"2024-09-11T12:53:21.470044+0000","last_undegraded":"2024-09-11T12:53:21.470044+0000","last_fullsized":"2024-09-11T12:53:21.470044+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:24:39.318254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176364+0000","last_change":"2024-09-11T12:53:20.491255+0000","last_active":"2024-09-11T12:53:22.176364+0000","last_peered":"2024-09-11T12:53:22.176364+0000","last_clean":"2024-09-11T12:53:22.176364+0000","last_became_active":"2024-09-11T12:53:20.490980+0000","last_became_peered":"2024-09-11T12:53:20.490980+0000","last_unstale":"2024-09-11T12:53:22.176364+0000","last_undegraded":"2024-09-11T12:53:22.176364+0000","last_fullsized":"2024-09-11T12:53:22.176364+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:25.351631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"68'4","reported_seq":19,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176551+0000","last_change":"2024-09-11T12:53:19.475083+0000","last_active":"2024-09-11T12:53:22.176551+0000","last_peered":"2024-09-11T12:53:22.176551+0000","last_clean":"2024-09-11T12:53:22.176551+0000","last_became_active":"2024-09-11T12:53:19.474603+0000","last_became_peered":"2024-09-11T12:53:19.474603+0000","last_unstale":"2024-09-11T12:53:22.176551+0000","last_undegraded":"2024-09-11T12:53:22.176551+0000","last_fullsized":"2024-09-11T12:53:22.176551+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:21.496429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468595+0000","last_change":"2024-09-11T12:53:20.475419+0000","last_active":"2024-09-11T12:53:21.468595+0000","last_peered":"2024-09-11T12:53:21.468595+0000","last_clean":"2024-09-11T12:53:21.468595+0000","last_became_active":"2024-09-11T12:53:20.474160+0000","last_became_peered":"2024-09-11T12:53:20.474160+0000","last_unstale":"2024-09-11T12:53:21.468595+0000","last_undegraded":"2024-09-11T12:53:21.468595+0000","last_fullsized":"2024-09-11T12:53:21.468595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:21:59.342083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"68'1","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.487183+0000","last_change":"2024-09-11T12:53:19.464327+0000","last_active":"2024-09-11T12:53:20.487183+0000","last_peered":"2024-09-11T12:53:20.487183+0000","last_clean":"2024-09-11T12:53:20.487183+0000","last_became_active":"2024-09-11T12:53:19.463993+0000","last_became_peered":"2024-09-11T12:53:19.463993+0000","last_unstale":"2024-09-11T12:53:20.487183+0000","last_undegraded":"2024-09-11T12:53:20.487183+0000","last_fullsized":"2024-09-11T12:53:20.487183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:30.015624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469663+0000","last_change":"2024-09-11T12:53:20.485239+0000","last_active":"2024-09-11T12:53:21.469663+0000","last_peered":"2024-09-11T12:53:21.469663+0000","last_clean":"2024-09-11T12:53:21.469663+0000","last_became_active":"2024-09-11T12:53:20.481288+0000","last_became_peered":"2024-09-11T12:53:20.481288+0000","last_unstale":"2024-09-11T12:53:21.469663+0000","last_undegraded":"2024-09-11T12:53:21.469663+0000","last_fullsized":"2024-09-11T12:53:21.469663+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:13:41.008024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276617+0000","last_change":"2024-09-11T12:53:19.474746+0000","last_active":"2024-09-11T12:53:22.276617+0000","last_peered":"2024-09-11T12:53:22.276617+0000","last_clean":"2024-09-11T12:53:22.276617+0000","last_became_active":"2024-09-11T12:53:19.474414+0000","last_became_peered":"2024-09-11T12:53:19.474414+0000","last_unstale":"2024-09-11T12:53:22.276617+0000","last_undegraded":"2024-09-11T12:53:22.276617+0000","last_fullsized":"2024-09-11T12:53:22.276617+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:53.310007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176380+0000","last_change":"2024-09-11T12:53:20.478834+0000","last_active":"2024-09-11T12:53:22.176380+0000","last_peered":"2024-09-11T12:53:22.176380+0000","last_clean":"2024-09-11T12:53:22.176380+0000","last_became_active":"2024-09-11T12:53:20.478578+0000","last_became_peered":"2024-09-11T12:53:20.478578+0000","last_unstale":"2024-09-11T12:53:22.176380+0000","last_undegraded":"2024-09-11T12:53:22.176380+0000","last_fullsized":"2024-09-11T12:53:22.176380+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:53.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470063+0000","last_change":"2024-09-11T12:53:19.478638+0000","last_active":"2024-09-11T12:53:21.470063+0000","last_peered":"2024-09-11T12:53:21.470063+0000","last_clean":"2024-09-11T12:53:21.470063+0000","last_became_active":"2024-09-11T12:53:19.478069+0000","last_became_peered":"2024-09-11T12:53:19.478069+0000","last_unstale":"2024-09-11T12:53:21.470063+0000","last_undegraded":"2024-09-11T12:53:21.470063+0000","last_fullsized":"2024-09-11T12:53:21.470063+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:08:56.466322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469217+0000","last_change":"2024-09-11T12:53:20.462550+0000","last_active":"2024-09-11T12:53:21.469217+0000","last_peered":"2024-09-11T12:53:21.469217+0000","last_clean":"2024-09-11T12:53:21.469217+0000","last_became_active":"2024-09-11T12:53:20.462221+0000","last_became_peered":"2024-09-11T12:53:20.462221+0000","last_unstale":"2024-09-11T12:53:21.469217+0000","last_undegraded":"2024-09-11T12:53:21.469217+0000","last_fullsized":"2024-09-11T12:53:21.469217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:56.925859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276726+0000","last_change":"2024-09-11T12:53:19.472630+0000","last_active":"2024-09-11T12:53:22.276726+0000","last_peered":"2024-09-11T12:53:22.276726+0000","last_clean":"2024-09-11T12:53:22.276726+0000","last_became_active":"2024-09-11T12:53:19.472324+0000","last_became_peered":"2024-09-11T12:53:19.472324+0000","last_unstale":"2024-09-11T12:53:22.276726+0000","last_undegraded":"2024-09-11T12:53:22.276726+0000","last_fullsized":"2024-09-11T12:53:22.276726+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:23.522762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276373+0000","last_change":"2024-09-11T12:53:20.479477+0000","last_active":"2024-09-11T12:53:22.276373+0000","last_peered":"2024-09-11T12:53:22.276373+0000","last_clean":"2024-09-11T12:53:22.276373+0000","last_became_active":"2024-09-11T12:53:20.477912+0000","last_became_peered":"2024-09-11T12:53:20.477912+0000","last_unstale":"2024-09-11T12:53:22.276373+0000","last_undegraded":"2024-09-11T12:53:22.276373+0000","last_fullsized":"2024-09-11T12:53:22.276373+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:39.754965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469067+0000","last_change":"2024-09-11T12:53:19.477533+0000","last_active":"2024-09-11T12:53:21.469067+0000","last_peered":"2024-09-11T12:53:21.469067+0000","last_clean":"2024-09-11T12:53:21.469067+0000","last_became_active":"2024-09-11T12:53:19.476520+0000","last_became_peered":"2024-09-11T12:53:19.476520+0000","last_unstale":"2024-09-11T12:53:21.469067+0000","last_undegraded":"2024-09-11T12:53:21.469067+0000","last_fullsized":"2024-09-11T12:53:21.469067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:35.058102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467963+0000","last_change":"2024-09-11T12:53:20.478537+0000","last_active":"2024-09-11T12:53:21.467963+0000","last_peered":"2024-09-11T12:53:21.467963+0000","last_clean":"2024-09-11T12:53:21.467963+0000","last_became_active":"2024-09-11T12:53:20.478220+0000","last_became_peered":"2024-09-11T12:53:20.478220+0000","last_unstale":"2024-09-11T12:53:21.467963+0000","last_undegraded":"2024-09-11T12:53:21.467963+0000","last_fullsized":"2024-09-11T12:53:21.467963+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:00.187655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703052+0000","last_change":"2024-09-11T12:53:19.469426+0000","last_active":"2024-09-11T12:53:21.703052+0000","last_peered":"2024-09-11T12:53:21.703052+0000","last_clean":"2024-09-11T12:53:21.703052+0000","last_became_active":"2024-09-11T12:53:19.469135+0000","last_became_peered":"2024-09-11T12:53:19.469135+0000","last_unstale":"2024-09-11T12:53:21.703052+0000","last_undegraded":"2024-09-11T12:53:21.703052+0000","last_fullsized":"2024-09-11T12:53:21.703052+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:06.134077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.477718+0000","last_change":"2024-09-11T12:53:20.475076+0000","last_active":"2024-09-11T12:53:20.477718+0000","last_peered":"2024-09-11T12:53:20.477718+0000","last_clean":"2024-09-11T12:53:20.477718+0000","last_became_active":"2024-09-11T12:53:20.474772+0000","last_became_peered":"2024-09-11T12:53:20.474772+0000","last_unstale":"2024-09-11T12:53:20.477718+0000","last_undegraded":"2024-09-11T12:53:20.477718+0000","last_fullsized":"2024-09-11T12:53:20.477718+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:22.785946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467953+0000","last_change":"2024-09-11T12:53:19.468223+0000","last_active":"2024-09-11T12:53:21.467953+0000","last_peered":"2024-09-11T12:53:21.467953+0000","last_clean":"2024-09-11T12:53:21.467953+0000","last_became_active":"2024-09-11T12:53:19.467919+0000","last_became_peered":"2024-09-11T12:53:19.467919+0000","last_unstale":"2024-09-11T12:53:21.467953+0000","last_undegraded":"2024-09-11T12:53:21.467953+0000","last_fullsized":"2024-09-11T12:53:21.467953+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:18.159044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469292+0000","last_change":"2024-09-11T12:53:20.479500+0000","last_active":"2024-09-11T12:53:21.469292+0000","last_peered":"2024-09-11T12:53:21.469292+0000","last_clean":"2024-09-11T12:53:21.469292+0000","last_became_active":"2024-09-11T12:53:20.479209+0000","last_became_peered":"2024-09-11T12:53:20.479209+0000","last_unstale":"2024-09-11T12:53:21.469292+0000","last_undegraded":"2024-09-11T12:53:21.469292+0000","last_fullsized":"2024-09-11T12:53:21.469292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:34.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276766+0000","last_change":"2024-09-11T12:53:19.465975+0000","last_active":"2024-09-11T12:53:22.276766+0000","last_peered":"2024-09-11T12:53:22.276766+0000","last_clean":"2024-09-11T12:53:22.276766+0000","last_became_active":"2024-09-11T12:53:19.465693+0000","last_became_peered":"2024-09-11T12:53:19.465693+0000","last_unstale":"2024-09-11T12:53:22.276766+0000","last_undegraded":"2024-09-11T12:53:22.276766+0000","last_fullsized":"2024-09-11T12:53:22.276766+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:33.022462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176459+0000","last_change":"2024-09-11T12:53:20.493540+0000","last_active":"2024-09-11T12:53:22.176459+0000","last_peered":"2024-09-11T12:53:22.176459+0000","last_clean":"2024-09-11T12:53:22.176459+0000","last_became_active":"2024-09-11T12:53:20.493204+0000","last_became_peered":"2024-09-11T12:53:20.493204+0000","last_unstale":"2024-09-11T12:53:22.176459+0000","last_undegraded":"2024-09-11T12:53:22.176459+0000","last_fullsized":"2024-09-11T12:53:22.176459+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:09.965093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469924+0000","last_change":"2024-09-11T12:53:19.473467+0000","last_active":"2024-09-11T12:53:21.469924+0000","last_peered":"2024-09-11T12:53:21.469924+0000","last_clean":"2024-09-11T12:53:21.469924+0000","last_became_active":"2024-09-11T12:53:19.473208+0000","last_became_peered":"2024-09-11T12:53:19.473208+0000","last_unstale":"2024-09-11T12:53:21.469924+0000","last_undegraded":"2024-09-11T12:53:21.469924+0000","last_fullsized":"2024-09-11T12:53:21.469924+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:29.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469852+0000","last_change":"2024-09-11T12:53:20.481626+0000","last_active":"2024-09-11T12:53:21.469852+0000","last_peered":"2024-09-11T12:53:21.469852+0000","last_clean":"2024-09-11T12:53:21.469852+0000","last_became_active":"2024-09-11T12:53:20.480916+0000","last_became_peered":"2024-09-11T12:53:20.480916+0000","last_unstale":"2024-09-11T12:53:21.469852+0000","last_undegraded":"2024-09-11T12:53:21.469852+0000","last_fullsized":"2024-09-11T12:53:21.469852+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:04.694195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"68'3","reported_seq":18,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276901+0000","last_change":"2024-09-11T12:53:19.470649+0000","last_active":"2024-09-11T12:53:22.276901+0000","last_peered":"2024-09-11T12:53:22.276901+0000","last_clean":"2024-09-11T12:53:22.276901+0000","last_became_active":"2024-09-11T12:53:19.470345+0000","last_became_peered":"2024-09-11T12:53:19.470345+0000","last_unstale":"2024-09-11T12:53:22.276901+0000","last_undegraded":"2024-09-11T12:53:22.276901+0000","last_fullsized":"2024-09-11T12:53:22.276901+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:24.348860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468227+0000","last_change":"2024-09-11T12:53:20.470384+0000","last_active":"2024-09-11T12:53:21.468227+0000","last_peered":"2024-09-11T12:53:21.468227+0000","last_clean":"2024-09-11T12:53:21.468227+0000","last_became_active":"2024-09-11T12:53:20.470099+0000","last_became_peered":"2024-09-11T12:53:20.470099+0000","last_unstale":"2024-09-11T12:53:21.468227+0000","last_undegraded":"2024-09-11T12:53:21.468227+0000","last_fullsized":"2024-09-11T12:53:21.468227+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:15.000764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"68'6","reported_seq":21,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703469+0000","last_change":"2024-09-11T12:53:19.473610+0000","last_active":"2024-09-11T12:53:21.703469+0000","last_peered":"2024-09-11T12:53:21.703469+0000","last_clean":"2024-09-11T12:53:21.703469+0000","last_became_active":"2024-09-11T12:53:19.473327+0000","last_became_peered":"2024-09-11T12:53:19.473327+0000","last_unstale":"2024-09-11T12:53:21.703469+0000","last_undegraded":"2024-09-11T12:53:21.703469+0000","last_fullsized":"2024-09-11T12:53:21.703469+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:44.362640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175959+0000","last_change":"2024-09-11T12:53:20.486720+0000","last_active":"2024-09-11T12:53:22.175959+0000","last_peered":"2024-09-11T12:53:22.175959+0000","last_clean":"2024-09-11T12:53:22.175959+0000","last_became_active":"2024-09-11T12:53:20.486411+0000","last_became_peered":"2024-09-11T12:53:20.486411+0000","last_unstale":"2024-09-11T12:53:22.175959+0000","last_undegraded":"2024-09-11T12:53:22.175959+0000","last_fullsized":"2024-09-11T12:53:22.175959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:15:34.644407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703265+0000","last_change":"2024-09-11T12:53:19.474347+0000","last_active":"2024-09-11T12:53:21.703265+0000","last_peered":"2024-09-11T12:53:21.703265+0000","last_clean":"2024-09-11T12:53:21.703265+0000","last_became_active":"2024-09-11T12:53:19.474070+0000","last_became_peered":"2024-09-11T12:53:19.474070+0000","last_unstale":"2024-09-11T12:53:21.703265+0000","last_undegraded":"2024-09-11T12:53:21.703265+0000","last_fullsized":"2024-09-11T12:53:21.703265+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:54:48.771609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469053+0000","last_change":"2024-09-11T12:53:20.463137+0000","last_active":"2024-09-11T12:53:21.469053+0000","last_peered":"2024-09-11T12:53:21.469053+0000","last_clean":"2024-09-11T12:53:21.469053+0000","last_became_active":"2024-09-11T12:53:20.462856+0000","last_became_peered":"2024-09-11T12:53:20.462856+0000","last_unstale":"2024-09-11T12:53:21.469053+0000","last_undegraded":"2024-09-11T12:53:21.469053+0000","last_fullsized":"2024-09-11T12:53:21.469053+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:26.268928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469166+0000","last_change":"2024-09-11T12:53:19.477205+0000","last_active":"2024-09-11T12:53:21.469166+0000","last_peered":"2024-09-11T12:53:21.469166+0000","last_clean":"2024-09-11T12:53:21.469166+0000","last_became_active":"2024-09-11T12:53:19.476500+0000","last_became_peered":"2024-09-11T12:53:19.476500+0000","last_unstale":"2024-09-11T12:53:21.469166+0000","last_undegraded":"2024-09-11T12:53:21.469166+0000","last_fullsized":"2024-09-11T12:53:21.469166+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:35:39.417623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.484595+0000","last_change":"2024-09-11T12:53:20.480798+0000","last_active":"2024-09-11T12:53:20.484595+0000","last_peered":"2024-09-11T12:53:20.484595+0000","last_clean":"2024-09-11T12:53:20.484595+0000","last_became_active":"2024-09-11T12:53:20.480545+0000","last_became_peered":"2024-09-11T12:53:20.480545+0000","last_unstale":"2024-09-11T12:53:20.484595+0000","last_undegraded":"2024-09-11T12:53:20.484595+0000","last_fullsized":"2024-09-11T12:53:20.484595+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:01:57.756075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469007+0000","last_change":"2024-09-11T12:53:19.465316+0000","last_active":"2024-09-11T12:53:21.469007+0000","last_peered":"2024-09-11T12:53:21.469007+0000","last_clean":"2024-09-11T12:53:21.469007+0000","last_became_active":"2024-09-11T12:53:19.464985+0000","last_became_peered":"2024-09-11T12:53:19.464985+0000","last_unstale":"2024-09-11T12:53:21.469007+0000","last_undegraded":"2024-09-11T12:53:21.469007+0000","last_fullsized":"2024-09-11T12:53:21.469007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:42.107728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.467959+0000","last_change":"2024-09-11T12:53:20.473900+0000","last_active":"2024-09-11T12:53:21.467959+0000","last_peered":"2024-09-11T12:53:21.467959+0000","last_clean":"2024-09-11T12:53:21.467959+0000","last_became_active":"2024-09-11T12:53:20.473646+0000","last_became_peered":"2024-09-11T12:53:20.473646+0000","last_unstale":"2024-09-11T12:53:21.467959+0000","last_undegraded":"2024-09-11T12:53:21.467959+0000","last_fullsized":"2024-09-11T12:53:21.467959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:57:21.618883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"68'8","reported_seq":23,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276232+0000","last_change":"2024-09-11T12:53:19.463694+0000","last_active":"2024-09-11T12:53:22.276232+0000","last_peered":"2024-09-11T12:53:22.276232+0000","last_clean":"2024-09-11T12:53:22.276232+0000","last_became_active":"2024-09-11T12:53:19.463382+0000","last_became_peered":"2024-09-11T12:53:19.463382+0000","last_unstale":"2024-09-11T12:53:22.276232+0000","last_undegraded":"2024-09-11T12:53:22.276232+0000","last_fullsized":"2024-09-11T12:53:22.276232+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:29.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276714+0000","last_change":"2024-09-11T12:53:20.466203+0000","last_active":"2024-09-11T12:53:22.276714+0000","last_peered":"2024-09-11T12:53:22.276714+0000","last_clean":"2024-09-11T12:53:22.276714+0000","last_became_active":"2024-09-11T12:53:20.465218+0000","last_became_peered":"2024-09-11T12:53:20.465218+0000","last_unstale":"2024-09-11T12:53:22.276714+0000","last_undegraded":"2024-09-11T12:53:22.276714+0000","last_fullsized":"2024-09-11T12:53:22.276714+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:56:05.851122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470218+0000","last_change":"2024-09-11T12:53:19.477397+0000","last_active":"2024-09-11T12:53:21.470218+0000","last_peered":"2024-09-11T12:53:21.470218+0000","last_clean":"2024-09-11T12:53:21.470218+0000","last_became_active":"2024-09-11T12:53:19.477099+0000","last_became_peered":"2024-09-11T12:53:19.477099+0000","last_unstale":"2024-09-11T12:53:21.470218+0000","last_undegraded":"2024-09-11T12:53:21.470218+0000","last_fullsized":"2024-09-11T12:53:21.470218+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:57.215380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176591+0000","last_change":"2024-09-11T12:53:20.483367+0000","last_active":"2024-09-11T12:53:22.176591+0000","last_peered":"2024-09-11T12:53:22.176591+0000","last_clean":"2024-09-11T12:53:22.176591+0000","last_became_active":"2024-09-11T12:53:20.483111+0000","last_became_peered":"2024-09-11T12:53:20.483111+0000","last_unstale":"2024-09-11T12:53:22.176591+0000","last_undegraded":"2024-09-11T12:53:22.176591+0000","last_fullsized":"2024-09-11T12:53:22.176591+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:04.655167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276761+0000","last_change":"2024-09-11T12:53:19.473944+0000","last_active":"2024-09-11T12:53:22.276761+0000","last_peered":"2024-09-11T12:53:22.276761+0000","last_clean":"2024-09-11T12:53:22.276761+0000","last_became_active":"2024-09-11T12:53:19.473612+0000","last_became_peered":"2024-09-11T12:53:19.473612+0000","last_unstale":"2024-09-11T12:53:22.276761+0000","last_undegraded":"2024-09-11T12:53:22.276761+0000","last_fullsized":"2024-09-11T12:53:22.276761+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:06:08.320644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703359+0000","last_change":"2024-09-11T12:53:20.471760+0000","last_active":"2024-09-11T12:53:21.703359+0000","last_peered":"2024-09-11T12:53:21.703359+0000","last_clean":"2024-09-11T12:53:21.703359+0000","last_became_active":"2024-09-11T12:53:20.471337+0000","last_became_peered":"2024-09-11T12:53:20.471337+0000","last_unstale":"2024-09-11T12:53:21.703359+0000","last_undegraded":"2024-09-11T12:53:21.703359+0000","last_fullsized":"2024-09-11T12:53:21.703359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:27:34.025234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"68'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470552+0000","last_change":"2024-09-11T12:53:19.477104+0000","last_active":"2024-09-11T12:53:21.470552+0000","last_peered":"2024-09-11T12:53:21.470552+0000","last_clean":"2024-09-11T12:53:21.470552+0000","last_became_active":"2024-09-11T12:53:19.476748+0000","last_became_peered":"2024-09-11T12:53:19.476748+0000","last_unstale":"2024-09-11T12:53:21.470552+0000","last_undegraded":"2024-09-11T12:53:21.470552+0000","last_fullsized":"2024-09-11T12:53:21.470552+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:13:53.310007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468386+0000","last_change":"2024-09-11T12:53:20.478508+0000","last_active":"2024-09-11T12:53:21.468386+0000","last_peered":"2024-09-11T12:53:21.468386+0000","last_clean":"2024-09-11T12:53:21.468386+0000","last_became_active":"2024-09-11T12:53:20.478246+0000","last_became_peered":"2024-09-11T12:53:20.478246+0000","last_unstale":"2024-09-11T12:53:21.468386+0000","last_undegraded":"2024-09-11T12:53:21.468386+0000","last_fullsized":"2024-09-11T12:53:21.468386+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:06:56.141422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176130+0000","last_change":"2024-09-11T12:53:19.470725+0000","last_active":"2024-09-11T12:53:22.176130+0000","last_peered":"2024-09-11T12:53:22.176130+0000","last_clean":"2024-09-11T12:53:22.176130+0000","last_became_active":"2024-09-11T12:53:19.470451+0000","last_became_peered":"2024-09-11T12:53:19.470451+0000","last_unstale":"2024-09-11T12:53:22.176130+0000","last_undegraded":"2024-09-11T12:53:22.176130+0000","last_fullsized":"2024-09-11T12:53:22.176130+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:38.752193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276797+0000","last_change":"2024-09-11T12:53:20.464491+0000","last_active":"2024-09-11T12:53:22.276797+0000","last_peered":"2024-09-11T12:53:22.276797+0000","last_clean":"2024-09-11T12:53:22.276797+0000","last_became_active":"2024-09-11T12:53:20.464224+0000","last_became_peered":"2024-09-11T12:53:20.464224+0000","last_unstale":"2024-09-11T12:53:22.276797+0000","last_undegraded":"2024-09-11T12:53:22.276797+0000","last_fullsized":"2024-09-11T12:53:22.276797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:04:56.705817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469207+0000","last_change":"2024-09-11T12:53:19.476907+0000","last_active":"2024-09-11T12:53:21.469207+0000","last_peered":"2024-09-11T12:53:21.469207+0000","last_clean":"2024-09-11T12:53:21.469207+0000","last_became_active":"2024-09-11T12:53:19.476155+0000","last_became_peered":"2024-09-11T12:53:19.476155+0000","last_unstale":"2024-09-11T12:53:21.469207+0000","last_undegraded":"2024-09-11T12:53:21.469207+0000","last_fullsized":"2024-09-11T12:53:21.469207+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:09:51.133433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276501+0000","last_change":"2024-09-11T12:53:20.483673+0000","last_active":"2024-09-11T12:53:22.276501+0000","last_peered":"2024-09-11T12:53:22.276501+0000","last_clean":"2024-09-11T12:53:22.276501+0000","last_became_active":"2024-09-11T12:53:20.481291+0000","last_became_peered":"2024-09-11T12:53:20.481291+0000","last_unstale":"2024-09-11T12:53:22.276501+0000","last_undegraded":"2024-09-11T12:53:22.276501+0000","last_fullsized":"2024-09-11T12:53:22.276501+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:08.397000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469116+0000","last_change":"2024-09-11T12:53:19.464486+0000","last_active":"2024-09-11T12:53:21.469116+0000","last_peered":"2024-09-11T12:53:21.469116+0000","last_clean":"2024-09-11T12:53:21.469116+0000","last_became_active":"2024-09-11T12:53:19.463653+0000","last_became_peered":"2024-09-11T12:53:19.463653+0000","last_unstale":"2024-09-11T12:53:21.469116+0000","last_undegraded":"2024-09-11T12:53:21.469116+0000","last_fullsized":"2024-09-11T12:53:21.469116+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:38.752193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276519+0000","last_change":"2024-09-11T12:53:20.483611+0000","last_active":"2024-09-11T12:53:22.276519+0000","last_peered":"2024-09-11T12:53:22.276519+0000","last_clean":"2024-09-11T12:53:22.276519+0000","last_became_active":"2024-09-11T12:53:20.480553+0000","last_became_peered":"2024-09-11T12:53:20.480553+0000","last_unstale":"2024-09-11T12:53:22.276519+0000","last_undegraded":"2024-09-11T12:53:22.276519+0000","last_fullsized":"2024-09-11T12:53:22.276519+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:04.268363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.459280+0000","last_change":"2024-09-11T12:53:19.464453+0000","last_active":"2024-09-11T12:53:20.459280+0000","last_peered":"2024-09-11T12:53:20.459280+0000","last_clean":"2024-09-11T12:53:20.459280+0000","last_became_active":"2024-09-11T12:53:19.464107+0000","last_became_peered":"2024-09-11T12:53:19.464107+0000","last_unstale":"2024-09-11T12:53:20.459280+0000","last_undegraded":"2024-09-11T12:53:20.459280+0000","last_fullsized":"2024-09-11T12:53:20.459280+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:53.979041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.476341+0000","last_change":"2024-09-11T12:53:20.470134+0000","last_active":"2024-09-11T12:53:20.476341+0000","last_peered":"2024-09-11T12:53:20.476341+0000","last_clean":"2024-09-11T12:53:20.476341+0000","last_became_active":"2024-09-11T12:53:20.469876+0000","last_became_peered":"2024-09-11T12:53:20.469876+0000","last_unstale":"2024-09-11T12:53:20.476341+0000","last_undegraded":"2024-09-11T12:53:20.476341+0000","last_fullsized":"2024-09-11T12:53:20.476341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:47:12.010107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470326+0000","last_change":"2024-09-11T12:53:19.477142+0000","last_active":"2024-09-11T12:53:21.470326+0000","last_peered":"2024-09-11T12:53:21.470326+0000","last_clean":"2024-09-11T12:53:21.470326+0000","last_became_active":"2024-09-11T12:53:19.476535+0000","last_became_peered":"2024-09-11T12:53:19.476535+0000","last_unstale":"2024-09-11T12:53:21.470326+0000","last_undegraded":"2024-09-11T12:53:21.470326+0000","last_fullsized":"2024-09-11T12:53:21.470326+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:04.862591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.469706+0000","last_change":"2024-09-11T12:53:20.459328+0000","last_active":"2024-09-11T12:53:21.469706+0000","last_peered":"2024-09-11T12:53:21.469706+0000","last_clean":"2024-09-11T12:53:21.469706+0000","last_became_active":"2024-09-11T12:53:20.458974+0000","last_became_peered":"2024-09-11T12:53:20.458974+0000","last_unstale":"2024-09-11T12:53:21.469706+0000","last_undegraded":"2024-09-11T12:53:21.469706+0000","last_fullsized":"2024-09-11T12:53:21.469706+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:10.768707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"69'1","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.477025+0000","last_change":"2024-09-11T12:53:19.471151+0000","last_active":"2024-09-11T12:53:21.477025+0000","last_peered":"2024-09-11T12:53:21.477025+0000","last_clean":"2024-09-11T12:53:21.477025+0000","last_became_active":"2024-09-11T12:53:19.470300+0000","last_became_peered":"2024-09-11T12:53:19.470300+0000","last_unstale":"2024-09-11T12:53:21.477025+0000","last_undegraded":"2024-09-11T12:53:21.477025+0000","last_fullsized":"2024-09-11T12:53:21.477025+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:02:06.348634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176208+0000","last_change":"2024-09-11T12:53:20.494957+0000","last_active":"2024-09-11T12:53:22.176208+0000","last_peered":"2024-09-11T12:53:22.176208+0000","last_clean":"2024-09-11T12:53:22.176208+0000","last_became_active":"2024-09-11T12:53:20.494687+0000","last_became_peered":"2024-09-11T12:53:20.494687+0000","last_unstale":"2024-09-11T12:53:22.176208+0000","last_undegraded":"2024-09-11T12:53:22.176208+0000","last_fullsized":"2024-09-11T12:53:22.176208+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:08.328135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468332+0000","last_change":"2024-09-11T12:53:19.469592+0000","last_active":"2024-09-11T12:53:21.468332+0000","last_peered":"2024-09-11T12:53:21.468332+0000","last_clean":"2024-09-11T12:53:21.468332+0000","last_became_active":"2024-09-11T12:53:19.469249+0000","last_became_peered":"2024-09-11T12:53:19.469249+0000","last_unstale":"2024-09-11T12:53:21.468332+0000","last_undegraded":"2024-09-11T12:53:21.468332+0000","last_fullsized":"2024-09-11T12:53:21.468332+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:34:22.809945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.176484+0000","last_change":"2024-09-11T12:53:20.489097+0000","last_active":"2024-09-11T12:53:22.176484+0000","last_peered":"2024-09-11T12:53:22.176484+0000","last_clean":"2024-09-11T12:53:22.176484+0000","last_became_active":"2024-09-11T12:53:20.488696+0000","last_became_peered":"2024-09-11T12:53:20.488696+0000","last_unstale":"2024-09-11T12:53:22.176484+0000","last_undegraded":"2024-09-11T12:53:22.176484+0000","last_fullsized":"2024-09-11T12:53:22.176484+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:01.824312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-09-11T12:53:20.460563+0000","last_change":"2024-09-11T12:53:19.463172+0000","last_active":"2024-09-11T12:53:20.460563+0000","last_peered":"2024-09-11T12:53:20.460563+0000","last_clean":"2024-09-11T12:53:20.460563+0000","last_became_active":"2024-09-11T12:53:19.462777+0000","last_became_peered":"2024-09-11T12:53:19.462777+0000","last_unstale":"2024-09-11T12:53:20.460563+0000","last_undegraded":"2024-09-11T12:53:20.460563+0000","last_fullsized":"2024-09-11T12:53:20.460563+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:01.593685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468217+0000","last_change":"2024-09-11T12:53:20.474922+0000","last_active":"2024-09-11T12:53:21.468217+0000","last_peered":"2024-09-11T12:53:21.468217+0000","last_clean":"2024-09-11T12:53:21.468217+0000","last_became_active":"2024-09-11T12:53:20.473380+0000","last_became_peered":"2024-09-11T12:53:20.473380+0000","last_unstale":"2024-09-11T12:53:21.468217+0000","last_undegraded":"2024-09-11T12:53:21.468217+0000","last_fullsized":"2024-09-11T12:53:21.468217+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:28:53.470915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468088+0000","last_change":"2024-09-11T12:53:19.471741+0000","last_active":"2024-09-11T12:53:21.468088+0000","last_peered":"2024-09-11T12:53:21.468088+0000","last_clean":"2024-09-11T12:53:21.468088+0000","last_became_active":"2024-09-11T12:53:19.470462+0000","last_became_peered":"2024-09-11T12:53:19.470462+0000","last_unstale":"2024-09-11T12:53:21.468088+0000","last_undegraded":"2024-09-11T12:53:21.468088+0000","last_fullsized":"2024-09-11T12:53:21.468088+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:45.385051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468917+0000","last_change":"2024-09-11T12:53:20.481889+0000","last_active":"2024-09-11T12:53:21.468917+0000","last_peered":"2024-09-11T12:53:21.468917+0000","last_clean":"2024-09-11T12:53:21.468917+0000","last_became_active":"2024-09-11T12:53:20.481630+0000","last_became_peered":"2024-09-11T12:53:20.481630+0000","last_unstale":"2024-09-11T12:53:21.468917+0000","last_undegraded":"2024-09-11T12:53:21.468917+0000","last_fullsized":"2024-09-11T12:53:21.468917+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:20:39.754965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703070+0000","last_change":"2024-09-11T12:53:19.473753+0000","last_active":"2024-09-11T12:53:21.703070+0000","last_peered":"2024-09-11T12:53:21.703070+0000","last_clean":"2024-09-11T12:53:21.703070+0000","last_became_active":"2024-09-11T12:53:19.473468+0000","last_became_peered":"2024-09-11T12:53:19.473468+0000","last_unstale":"2024-09-11T12:53:21.703070+0000","last_undegraded":"2024-09-11T12:53:21.703070+0000","last_fullsized":"2024-09-11T12:53:21.703070+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:23.461699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.703278+0000","last_change":"2024-09-11T12:53:20.465576+0000","last_active":"2024-09-11T12:53:21.703278+0000","last_peered":"2024-09-11T12:53:21.703278+0000","last_clean":"2024-09-11T12:53:21.703278+0000","last_became_active":"2024-09-11T12:53:20.464585+0000","last_became_peered":"2024-09-11T12:53:20.464585+0000","last_unstale":"2024-09-11T12:53:21.703278+0000","last_undegraded":"2024-09-11T12:53:21.703278+0000","last_fullsized":"2024-09-11T12:53:21.703278+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:57:30.844586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276012+0000","last_change":"2024-09-11T12:53:19.463737+0000","last_active":"2024-09-11T12:53:22.276012+0000","last_peered":"2024-09-11T12:53:22.276012+0000","last_clean":"2024-09-11T12:53:22.276012+0000","last_became_active":"2024-09-11T12:53:19.463359+0000","last_became_peered":"2024-09-11T12:53:19.463359+0000","last_unstale":"2024-09-11T12:53:22.276012+0000","last_undegraded":"2024-09-11T12:53:22.276012+0000","last_fullsized":"2024-09-11T12:53:22.276012+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:08:56.466322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.468855+0000","last_change":"2024-09-11T12:53:20.478214+0000","last_active":"2024-09-11T12:53:21.468855+0000","last_peered":"2024-09-11T12:53:21.468855+0000","last_clean":"2024-09-11T12:53:21.468855+0000","last_became_active":"2024-09-11T12:53:20.477191+0000","last_became_peered":"2024-09-11T12:53:20.477191+0000","last_unstale":"2024-09-11T12:53:21.468855+0000","last_undegraded":"2024-09-11T12:53:21.468855+0000","last_fullsized":"2024-09-11T12:53:21.468855+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:43.110499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470133+0000","last_change":"2024-09-11T12:53:19.477833+0000","last_active":"2024-09-11T12:53:21.470133+0000","last_peered":"2024-09-11T12:53:21.470133+0000","last_clean":"2024-09-11T12:53:21.470133+0000","last_became_active":"2024-09-11T12:53:19.477514+0000","last_became_peered":"2024-09-11T12:53:19.477514+0000","last_unstale":"2024-09-11T12:53:21.470133+0000","last_undegraded":"2024-09-11T12:53:21.470133+0000","last_fullsized":"2024-09-11T12:53:21.470133+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:28.098161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.276582+0000","last_change":"2024-09-11T12:53:20.484966+0000","last_active":"2024-09-11T12:53:22.276582+0000","last_peered":"2024-09-11T12:53:22.276582+0000","last_clean":"2024-09-11T12:53:22.276582+0000","last_became_active":"2024-09-11T12:53:20.480486+0000","last_became_peered":"2024-09-11T12:53:20.480486+0000","last_unstale":"2024-09-11T12:53:22.276582+0000","last_undegraded":"2024-09-11T12:53:22.276582+0000","last_fullsized":"2024-09-11T12:53:22.276582+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:19.442289+0000","last_clean_scrub_stamp":"2024-09-11T12:53:19.442289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:11.453911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"68'2","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:21.470208+0000","last_change":"2024-09-11T12:53:19.478268+0000","last_active":"2024-09-11T12:53:21.470208+0000","last_peered":"2024-09-11T12:53:21.470208+0000","last_clean":"2024-09-11T12:53:21.470208+0000","last_became_active":"2024-09-11T12:53:19.477853+0000","last_became_peered":"2024-09-11T12:53:19.477853+0000","last_unstale":"2024-09-11T12:53:21.470208+0000","last_undegraded":"2024-09-11T12:53:21.470208+0000","last_fullsized":"2024-09-11T12:53:21.470208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:18.439518+0000","last_clean_scrub_stamp":"2024-09-11T12:53:18.439518+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:49.096469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"18'72","reported_seq":195,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-11T12:53:22.175780+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:53:22.175780+0000","last_peered":"2024-09-11T12:53:22.175780+0000","last_clean":"2024-09-11T12:53:22.175780+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:53:22.175780+0000","last_undegraded":"2024-09-11T12:53:22.175780+0000","last_fullsized":"2024-09-11T12:53:22.175780+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32596,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339244,"statfs":{"total":96636764160,"available":96603385856,"internally_reserved":0,"allocated":1433600,"data_stored":936873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":345981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":835584,"data_stored":345979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509489,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32004,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339836,"statfs":{"total":96636764160,"available":96603992064,"internally_reserved":0,"allocated":835584,"data_stored":345959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32000,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339840,"statfs":{"total":96636764160,"available":96603996160,"internally_reserved":0,"allocated":827392,"data_stored":339701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":843776,"data_stored":347501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32012,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339828,"statfs":{"total":96636764160,"available":96603983872,"internally_reserved":0,"allocated":839680,"data_stored":341817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542193,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1413120,"data_stored":929957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:53:24.969 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:53:24.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:53:25.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.156+0000 7f8854400640 1 -- 172.21.15.78:0/60246262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8834003a60 msgr2=0x7f8834003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.156+0000 7f8854400640 1 --2- 172.21.15.78:0/60246262 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8834003a60 0x7f8834003e80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8840009b40 tx=0x7f884001d250 comp rx=0 tx=0).stop 2024-09-11T12:53:25.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8852400640 1 -- 172.21.15.78:0/60246262 reap_dead start 2024-09-11T12:53:25.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- 172.21.15.78:0/60246262 shutdown_connections 2024-09-11T12:53:25.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- 172.21.15.78:0/60246262 >> 172.21.15.78:0/60246262 conn(0x7f884c0b2ec0 msgr2=0x7f884c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:25.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- 172.21.15.78:0/60246262 shutdown_connections 2024-09-11T12:53:25.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- 172.21.15.78:0/60246262 wait complete. 2024-09-11T12:53:25.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 Processor -- start 2024-09-11T12:53:25.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- start start 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834003a60 0x7f884c1f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f884c144500 0x7f884c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f884c166b20 con 0x7f884c1fb620 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f884c1669a0 con 0x7f884c144500 2024-09-11T12:53:25.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f884c166ca0 con 0x7f8834003a60 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54384/0 (socket says 172.21.15.78:54384) 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 -- 172.21.15.78:0/974852499 learned_addr learned my addr 172.21.15.78:0/974852499 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 -- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834003a60 msgr2=0x7f884c1f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834003a60 0x7f884c1f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 -- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f884c144500 msgr2=0x7f884c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f884c144500 0x7f884c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 -- 172.21.15.78:0/974852499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f884c0ce790 con 0x7f884c1fb620 2024-09-11T12:53:25.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8851000640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8838011ab0 tx=0x7f8838011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f883801e070 con 0x7f884c1fb620 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f883800fbf0 con 0x7f884c1fb620 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.160+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f884c164710 con 0x7f884c1fb620 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8838019410 con 0x7f884c1fb620 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f884c1fbab0 con 0x7f884c1fb620 2024-09-11T12:53:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f884c153130 con 0x7f884c1fb620 2024-09-11T12:53:25.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f8838021070 con 0x7f884c1fb620 2024-09-11T12:53:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8849600640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 0x7f882c073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f8838092c70 con 0x7f884c1fb620 2024-09-11T12:53:25.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.164+0000 7f8851a00640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 0x7f882c073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.168+0000 7f8851a00640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 0x7f882c073ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88400ad000 tx=0x7f8834005840 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:25.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.168+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f884c153130 con 0x7f884c1fb620 2024-09-11T12:53:25.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.392+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f884c0737c0 con 0x7f884c1fb620 2024-09-11T12:53:25.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.396+0000 7f8849600640 1 -- 172.21.15.78:0/974852499 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7f884c164710 con 0x7f884c1fb620 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:53:25.399 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:53:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 msgr2=0x7f882c073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 0x7f882c073ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f88400ad000 tx=0x7f8834005840 comp rx=0 tx=0).stop 2024-09-11T12:53:25.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 msgr2=0x7f884c1641d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8838011ab0 tx=0x7f8838011f70 comp rx=0 tx=0).stop 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 shutdown_connections 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f882c071a10 0x7f882c073ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f884c144500 0x7f884c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f884c1fb620 0x7f884c1641d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 --2- 172.21.15.78:0/974852499 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8834003a60 0x7f884c1f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 >> 172.21.15.78:0/974852499 conn(0x7f884c0b2ec0 msgr2=0x7f884c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 shutdown_connections 2024-09-11T12:53:25.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.400+0000 7f8854400640 1 -- 172.21.15.78:0/974852499 wait complete. 2024-09-11T12:53:25.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 -- 172.21.15.78:0/1592403451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c15a1c0 msgr2=0x7f5b4c0cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 --2- 172.21.15.78:0/1592403451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c0cbed0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f5b4c1512b0 tx=0x7f5b3801c740 comp rx=0 tx=0).stop 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 -- 172.21.15.78:0/1592403451 shutdown_connections 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 --2- 172.21.15.78:0/1592403451 >> v2:172.21.15.157:3300/0 conn(0x7f5b4c13d250 0x7f5b4c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 --2- 172.21.15.78:0/1592403451 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c0cbed0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 -- 172.21.15.78:0/1592403451 >> 172.21.15.78:0/1592403451 conn(0x7f5b4c06ca80 msgr2=0x7f5b4c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:25.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 -- 172.21.15.78:0/1592403451 shutdown_connections 2024-09-11T12:53:25.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.560+0000 7f5b52400640 1 -- 172.21.15.78:0/1592403451 wait complete. 2024-09-11T12:53:25.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 Processor -- start 2024-09-11T12:53:25.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- start start 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b4c0b4780 0x7f5b4c0aed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c06f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c13eb50 con 0x7f5b4c13d250 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b4c13e9d0 con 0x7f5b4c15a1c0 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b4c13ecd0 con 0x7f5b4c0b4780 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54400/0 (socket says 172.21.15.78:54400) 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 -- 172.21.15.78:0/2413139988 learned_addr learned my addr 172.21.15.78:0/2413139988 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4b400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b4c0b4780 0x7f5b4c0aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4be00640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c06f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 -- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b4c0b4780 msgr2=0x7f5b4c0aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b4c0b4780 0x7f5b4c0aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 -- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4c15a1c0 msgr2=0x7f5b4c06f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c06f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 -- 172.21.15.78:0/2413139988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4c155210 con 0x7f5b4c13d250 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b4aa00640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c10 tx=0x7f5b38004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380bb070 con 0x7f5b4c13d250 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b38005030 con 0x7f5b4c13d250 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380b2040 con 0x7f5b4c13d250 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b4c1554f0 con 0x7f5b4c13d250 2024-09-11T12:53:25.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b4c155ab0 con 0x7f5b4c13d250 2024-09-11T12:53:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.564+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4c13f190 con 0x7f5b4c13d250 2024-09-11T12:53:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.568+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f5b380053e0 con 0x7f5b4c13d250 2024-09-11T12:53:25.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.568+0000 7f5b43400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 0x7f5b2c073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.568+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f5b3812be70 con 0x7f5b4c13d250 2024-09-11T12:53:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.568+0000 7f5b4b400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 0x7f5b2c073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:25.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.568+0000 7f5b4b400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 0x7f5b2c073ea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f70 tx=0x7f5b3c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:25.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.572+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5b380163f0 con 0x7f5b4c13d250 2024-09-11T12:53:25.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b4c159b00 con 0x7f5b4c13d250 2024-09-11T12:53:25.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b43400640 1 -- 172.21.15.78:0/2413139988 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5228 (secure 0 0 0) 0x7f5b380f7810 con 0x7f5b4c13d250 2024-09-11T12:53:25.774 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:25.774 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":46,"btime":"2024-09-11T12:53:23:476400+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:25.774 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 46 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 msgr2=0x7f5b2c073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 0x7f5b2c073ea0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5b3c000f70 tx=0x7f5b3c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 msgr2=0x7f5b4c06eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5b38000c10 tx=0x7f5b38004040 comp rx=0 tx=0).stop 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 shutdown_connections 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f5b2c0719e0 0x7f5b2c073ea0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4c15a1c0 0x7f5b4c06f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b4c13d250 0x7f5b4c06eb40 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 --2- 172.21.15.78:0/2413139988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b4c0b4780 0x7f5b4c0aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:25.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.772+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 >> 172.21.15.78:0/2413139988 conn(0x7f5b4c06ca80 msgr2=0x7f5b4c0cc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:25.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.776+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 shutdown_connections 2024-09-11T12:53:25.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:25.776+0000 7f5b52400640 1 -- 172.21.15.78:0/2413139988 wait complete. 2024-09-11T12:53:26.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:53:19.846929+0000', 'modified': '2024-09-11T12:53:22.866865+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6141}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6141': {'gid': 6141, 'name': 'd', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6837/2122140049', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6835', 'nonce': 2122140049}, {'type': 'v1', 'addr': '172.21.15.157:6837', 'nonce': 2122140049}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6141, 'qdb_cluster': [6141]} 2024-09-11T12:53:26.198 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:53:26.199 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-11T12:53:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 -- 172.21.15.78:0/2265803197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073430 msgr2=0x7f427815fd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 --2- 172.21.15.78:0/2265803197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073430 0x7f427815fd90 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f42780a9e30 tx=0x7f426801c770 comp rx=0 tx=0).stop 2024-09-11T12:53:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 -- 172.21.15.78:0/2265803197 shutdown_connections 2024-09-11T12:53:26.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 --2- 172.21.15.78:0/2265803197 >> v2:172.21.15.157:3300/0 conn(0x7f4278073810 0x7f4278144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 --2- 172.21.15.78:0/2265803197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073430 0x7f427815fd90 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 -- 172.21.15.78:0/2265803197 >> 172.21.15.78:0/2265803197 conn(0x7f427806d870 msgr2=0x7f427806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 -- 172.21.15.78:0/2265803197 shutdown_connections 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.340+0000 7f4277e00640 1 -- 172.21.15.78:0/2265803197 wait complete. 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 Processor -- start 2024-09-11T12:53:26.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- start start 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4278073430 0x7f42781fad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278145300 0x7f42780b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4278168ab0 con 0x7f4278073810 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4278168930 con 0x7f4278145300 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4278168c30 con 0x7f4278073430 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54416/0 (socket says 172.21.15.78:54416) 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 -- 172.21.15.78:0/785209166 learned_addr learned my addr 172.21.15.78:0/785209166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 -- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4278073430 msgr2=0x7f42781fad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4278073430 0x7f42781fad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 -- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278145300 msgr2=0x7f42780b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278145300 0x7f42780b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 -- 172.21.15.78:0/785209166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42781f8220 con 0x7f4278073810 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426fe00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426ea00640 1 -- 172.21.15.78:0/785209166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260017070 con 0x7f4278073810 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426ea00640 1 -- 172.21.15.78:0/785209166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4260009d20 con 0x7f4278073810 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42781f8500 con 0x7f4278073810 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f426ea00640 1 -- 172.21.15.78:0/785209166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42600106c0 con 0x7f4278073810 2024-09-11T12:53:26.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.344+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42781f8ac0 con 0x7f4278073810 2024-09-11T12:53:26.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f426ea00640 1 -- 172.21.15.78:0/785209166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f4260010860 con 0x7f4278073810 2024-09-11T12:53:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f426ea00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 0x7f4254073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 Processor -- start 2024-09-11T12:53:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f426ea00640 1 -- 172.21.15.78:0/785209166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f42600928a0 con 0x7f4278073810 2024-09-11T12:53:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 -- start start 2024-09-11T12:53:26.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4276a00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 0x7f4254073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c94d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749c95240 0x55d749c95620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d749ae5270 con 0x55d749cdd500 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d749ad4770 con 0x55d749cdde20 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.348+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d749ad87b0 con 0x55d749c95240 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54422/0 (socket says 172.21.15.78:54422) 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 -- 172.21.15.78:0/695196596 learned_addr learned my addr 172.21.15.78:0/695196596 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 0x7f4254073ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42680af000 tx=0x7f4268019510 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4277400640 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749c95240 0x55d749c95620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f426fe00640 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c94d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 -- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749c95240 msgr2=0x55d749c95620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749c95240 0x55d749c95620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 -- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 msgr2=0x55d749c94d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c94d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 -- 172.21.15.78:0/695196596 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d749a73f90 con 0x55d749cdd500 2024-09-11T12:53:26.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f426801cc50 tx=0x7f42680b5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0248e877facc63 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4267400640 1 -- 172.21.15.78:0/695196596 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680cb070 con 0x55d749cdd500 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4267400640 1 -- 172.21.15.78:0/695196596 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4268005610 con 0x55d749cdd500 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4267400640 1 -- 172.21.15.78:0/695196596 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268004ba0 con 0x55d749cdd500 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- 172.21.15.78:0/695196596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 msgr2=0x55d749cdd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f426801cc50 tx=0x7f42680b5000 comp rx=0 tx=0).stop 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- 172.21.15.78:0/695196596 shutdown_connections 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749c95240 0x55d749c95620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c94d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- 172.21.15.78:0/695196596 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749cdd8e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- 172.21.15.78:0/695196596 >> 172.21.15.78:0/695196596 conn(0x55d749cb5b60 msgr2=0x55d749cb7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- 172.21.15.78:0/695196596 shutdown_connections 2024-09-11T12:53:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- 172.21.15.78:0/695196596 wait complete. 2024-09-11T12:53:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- start start 2024-09-11T12:53:26.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 Processor -- start 2024-09-11T12:53:26.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749c9e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c9e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d749ba4290 con 0x55d749c95240 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d749a80ab0 con 0x55d749cdde20 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d749a7e170 con 0x55d749cdd500 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4280dc3000 10 client.0 _ll_register_callbacks cb 0x55d749c75830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.352+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54438/0 (socket says 172.21.15.78:54438) 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 -- 172.21.15.78:0/2333458222 learned_addr learned my addr 172.21.15.78:0/2333458222 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f426fe00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749c9e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4277400640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c9e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 -- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 msgr2=0x55d749c9e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749c9e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 -- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 msgr2=0x55d749c9e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c9e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 -- 172.21.15.78:0/2333458222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d749b596f0 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4276a00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42680c9040 tx=0x7f4268004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d872f3a3641038a server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680cb030 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d749a6df10 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f42680c0050 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f42680d9050 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 10 client.6264 fetch_fsmap learned FSMap version 46 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55d749b30c40 con 0x55d749c95240 2024-09-11T12:53:26.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55d749a6df10 con 0x55d749c95240 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f42680dc020 con 0x55d749c95240 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 10 client.6264 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 10 client.6264 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 20 client.6264 populate_metadata read hostname 'smithi078' 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749c9f370 0x55d749cef7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 4 client.6264 mds_command: new command op to 6141 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 --> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d749ae71c0 con 0x55d749c9f370 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f426fe00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749c9f370 0x55d749cef7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f426fe00640 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749c9f370 0x55d749cef7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=186c0a4e5882044 server_cookie=c022e1ea63b36136 in_seq=0 out_seq=0 2024-09-11T12:53:26.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 10 client.6264 ms_handle_connect on v2:172.21.15.157:6835/2122140049 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 1 -- 172.21.15.78:0/2333458222 <== mds.0 v2:172.21.15.157:6835/2122140049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d749ae71c0 con 0x55d749c9f370 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 10 client.6264 handle_command_reply: tid=0 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 client.6264 shutdown 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 msgr2=0x55d749c9dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f42680c9040 tx=0x7f4268004f80 comp rx=0 tx=0).stop 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 shutdown_connections 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749c9f370 0x55d749cef7c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749c9e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749c9e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f425be00640 0 client.6264 ms_handle_reset on v2:172.21.15.157:6835/2122140049 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2333458222 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749c9dee0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.356+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 >> 172.21.15.78:0/2333458222 conn(0x55d749cb5b60 msgr2=0x55d749c87500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.360+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 shutdown_connections 2024-09-11T12:53:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.360+0000 7f4280dc3000 1 -- 172.21.15.78:0/2333458222 wait complete. 2024-09-11T12:53:26.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.360+0000 7f4280dc3000 20 client.6264 trim_cache size 0 max 16384 2024-09-11T12:53:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 Processor -- start 2024-09-11T12:53:26.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 -- start start 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749d4d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749d4d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f42780ac810 con 0x55d749c95240 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f42781385a0 con 0x55d749cdde20 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4278137dc0 con 0x55d749cdd500 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4276a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54448/0 (socket says 172.21.15.78:54448) 2024-09-11T12:53:26.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4276a00640 1 -- 172.21.15.78:0/2473361564 learned_addr learned my addr 172.21.15.78:0/2473361564 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4276a00640 1 -- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 msgr2=0x55d749d4d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.376+0000 7f4276a00640 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749d4d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4276a00640 1 -- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 msgr2=0x55d749d4d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4276a00640 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749d4d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4276a00640 1 -- 172.21.15.78:0/2473361564 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d749a89e70 con 0x55d749c95240 2024-09-11T12:53:26.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4276a00640 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4268019380 tx=0x7f42680cf590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e30c1668d10b1ee server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4266a00640 1 -- 172.21.15.78:0/2473361564 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680cb070 con 0x55d749c95240 2024-09-11T12:53:26.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4266a00640 1 -- 172.21.15.78:0/2473361564 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42680cf680 con 0x55d749c95240 2024-09-11T12:53:26.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4266a00640 1 -- 172.21.15.78:0/2473361564 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680cf8d0 con 0x55d749c95240 2024-09-11T12:53:26.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- 172.21.15.78:0/2473361564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 msgr2=0x55d749d4cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f4268019380 tx=0x7f42680cf590 comp rx=0 tx=0).stop 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- 172.21.15.78:0/2473361564 shutdown_connections 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749cdde20 0x55d749d4d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdd500 0x55d749d4d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- 172.21.15.78:0/2473361564 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749c95240 0x55d749d4cea0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- 172.21.15.78:0/2473361564 >> 172.21.15.78:0/2473361564 conn(0x55d749cb5b60 msgr2=0x55d749cdc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- 172.21.15.78:0/2473361564 shutdown_connections 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- 172.21.15.78:0/2473361564 wait complete. 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- start start 2024-09-11T12:53:26.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 Processor -- start 2024-09-11T12:53:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749c95240 0x55d749d50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdde20 0x55d749d517c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55d749adcde0 con 0x55d749cdd500 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55d749ad7ac0 con 0x55d749c95240 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55d749ae7ea0 con 0x55d749cdde20 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4280dc3000 10 client.0 _ll_register_callbacks cb 0x7f42680c9070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54464/0 (socket says 172.21.15.78:54464) 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 -- 172.21.15.78:0/3960714727 learned_addr learned my addr 172.21.15.78:0/3960714727 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4277400640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdde20 0x55d749d517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f4276a00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749c95240 0x55d749d50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 -- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdde20 msgr2=0x55d749d517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdde20 0x55d749d517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 -- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749c95240 msgr2=0x55d749d50d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:26.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749c95240 0x55d749d50d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 -- 172.21.15.78:0/3960714727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d749a5ebf0 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f426fe00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f426005d950 tx=0x7f426005a090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=418691f6cd989e90 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.380+0000 7f425be00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4260020040 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f4260090300 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42680cf680 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f4260077130 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 10 client.6270 fetch_fsmap learned FSMap version 46 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55d749a7ac60 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42680cf680 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f4260086070 con 0x55d749cdd500 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 10 client.6270 fetch_fsmap finished waiting for FSMap version 46 2024-09-11T12:53:26.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 10 client.6270 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 20 client.6270 populate_metadata read hostname 'smithi078' 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749cf2f70 0x55d749d59fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 4 client.6270 mds_command: new command op to 6141 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 --> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] -- command(tid 0: {"prefix": "status"}) -- 0x55d749ae2c60 con 0x55d749cf2f70 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4276a00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749cf2f70 0x55d749d59fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4276a00640 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749cf2f70 0x55d749d59fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f60e6be68165f2a8 server_cookie=c064f658c741d4eb in_seq=0 out_seq=0 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 10 client.6270 ms_handle_connect on v2:172.21.15.157:6835/2122140049 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 1 -- 172.21.15.78:0/3960714727 <== mds.0 v2:172.21.15.157:6835/2122140049 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55d749ae2c60 con 0x55d749cf2f70 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 10 client.6270 handle_command_reply: tid=0 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 client.6270 shutdown 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 msgr2=0x55d749d51280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f426005d950 tx=0x7f426005a090 comp rx=0 tx=0).stop 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 shutdown_connections 2024-09-11T12:53:26.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:6835/2122140049,v1:172.21.15.157:6837/2122140049] conn(0x55d749cf2f70 0x55d749d59fa0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55d749cdde20 0x55d749d517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55d749cdd500 0x55d749d51280 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f425be00640 0 client.6270 ms_handle_reset on v2:172.21.15.157:6835/2122140049 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 --2- 172.21.15.78:0/3960714727 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55d749c95240 0x55d749d50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 >> 172.21.15.78:0/3960714727 conn(0x55d749cb5b60 msgr2=0x55d749cdc070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 shutdown_connections 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 1 -- 172.21.15.78:0/3960714727 wait complete. 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4280dc3000 20 client.6270 trim_cache size 0 max 16384 2024-09-11T12:53:26.390 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6141, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9229750059999997, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 46, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 71, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 69, 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.353540796000001 2024-09-11T12:53:26.391 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 msgr2=0x7f4254073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 0x7f4254073ea0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42680af000 tx=0x7f4268019510 comp rx=0 tx=0).stop 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 msgr2=0x7f427815e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f426000d950 tx=0x7f426000de10 comp rx=0 tx=0).stop 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 shutdown_connections 2024-09-11T12:53:26.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f42540719e0 0x7f4254073ea0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4278145300 0x7f42780b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4278073810 0x7f427815e070 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 --2- 172.21.15.78:0/785209166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4278073430 0x7f42781fad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 >> 172.21.15.78:0/785209166 conn(0x7f427806d870 msgr2=0x7f427815b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 shutdown_connections 2024-09-11T12:53:26.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.384+0000 7f4277e00640 1 -- 172.21.15.78:0/785209166 wait complete. 2024-09-11T12:53:26.406 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- 172.21.15.78:0/3967030113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34154be0 msgr2=0x7f1d18003a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 --2- 172.21.15.78:0/3967030113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d34154be0 0x7f1d18003a60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1d24009b40 tx=0x7f1d2401d290 comp rx=0 tx=0).stop 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d32800640 1 -- 172.21.15.78:0/3967030113 reap_dead start 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- 172.21.15.78:0/3967030113 shutdown_connections 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- 172.21.15.78:0/3967030113 >> 172.21.15.78:0/3967030113 conn(0x7f1d3406f3a0 msgr2=0x7f1d340cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- 172.21.15.78:0/3967030113 shutdown_connections 2024-09-11T12:53:26.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- 172.21.15.78:0/3967030113 wait complete. 2024-09-11T12:53:26.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 Processor -- start 2024-09-11T12:53:26.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.548+0000 7f1d38c00640 1 -- start start 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 0x7f1d34152f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3415c650 0x7f1d3415cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3415ee30 con 0x7f1d34154be0 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3415ecb0 con 0x7f1d34156c60 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3415efb0 con 0x7f1d3415c650 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54476/0 (socket says 172.21.15.78:54476) 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 -- 172.21.15.78:0/1013022338 learned_addr learned my addr 172.21.15.78:0/1013022338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d32800640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3415c650 0x7f1d3415cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31400640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 0x7f1d34152f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 -- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3415c650 msgr2=0x7f1d3415cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3415c650 0x7f1d3415cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 -- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 msgr2=0x7f1d34152f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 0x7f1d34152f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 -- 172.21.15.78:0/1013022338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d34157440 con 0x7f1d34154be0 2024-09-11T12:53:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31400640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 0x7f1d34152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:26.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d31e00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1d2401d0c0 tx=0x7f1d240050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d240bb030 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d24004a50 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d24004e40 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d34157720 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d34157ce0 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d340cfc50 con 0x7f1d34154be0 2024-09-11T12:53:26.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.552+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f1d240be030 con 0x7f1d34154be0 2024-09-11T12:53:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.556+0000 7f1d2a000640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 0x7f1d14073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.556+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f1d2412da20 con 0x7f1d34154be0 2024-09-11T12:53:26.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.556+0000 7f1d31400640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 0x7f1d14073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:26.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.556+0000 7f1d31400640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 0x7f1d14073ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d20001c10 tx=0x7f1d20001ba0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:26.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.560+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1d24016fa0 con 0x7f1d34154be0 2024-09-11T12:53:26.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d3415bf50 con 0x7f1d34154be0 2024-09-11T12:53:26.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d2a000640 1 -- 172.21.15.78:0/1013022338 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f1d240f92c0 con 0x7f1d34154be0 2024-09-11T12:53:26.761 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 msgr2=0x7f1d14073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 0x7f1d14073ed0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1d20001c10 tx=0x7f1d20001ba0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 msgr2=0x7f1d341f4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1d2401d0c0 tx=0x7f1d240050a0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 shutdown_connections 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f1d14071a10 0x7f1d14073ed0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1d3415c650 0x7f1d3415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1d34156c60 0x7f1d34152f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 --2- 172.21.15.78:0/1013022338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1d34154be0 0x7f1d341f4f10 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 >> 172.21.15.78:0/1013022338 conn(0x7f1d3406f3a0 msgr2=0x7f1d340ceef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 shutdown_connections 2024-09-11T12:53:26.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:26.760+0000 7f1d38c00640 1 -- 172.21.15.78:0/1013022338 wait complete. 2024-09-11T12:53:26.782 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:53:26.782 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:53:26.783 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:26.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:26.783 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:53:26.792 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:53:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:53:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:53:26.792 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42228sec preferred_lft 42228sec 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.794 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:53:26.795 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:26.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:26.796 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:53:26.861 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:53:26.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:26.863 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:53:26.915 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:53:26.915 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:53:26.915 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:26.916 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:53:26.916 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:53:26.916 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:53:26.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:53:26.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:53:26.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:53:26.917 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:53:27.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:53:27.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:27.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.053 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:53:27.108 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:53:27.108 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:53:27.109 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:27.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:53:27.175 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:27.175 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:27.223 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:27.224 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:27.224 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:27.275 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:53:27.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.275 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:27.366 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:27.367 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:27.380 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:27.380 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:27.380 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:53:27.381 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:53:27.381 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:53:27.384 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:53:27.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:27.404 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:53:27.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:53:27.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:27.515 INFO:teuthology.orchestra.run.smithi078.stdout:client6022 2024-09-11T12:53:27.515 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:27.515 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:27.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:53:27.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- 172.21.15.78:0/3263278427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc06cfc0 msgr2=0x7f6fcc13df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:27.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/3263278427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc13df40 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc13de10 tx=0x7f6fbc01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- 172.21.15.78:0/3263278427 shutdown_connections 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/3263278427 >> v2:172.21.15.157:3300/0 conn(0x7f6fcc0b3730 0x7f6fcc0b3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/3263278427 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc13df40 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- 172.21.15.78:0/3263278427 >> 172.21.15.78:0/3263278427 conn(0x7f6fcc073940 msgr2=0x7f6fcc06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- 172.21.15.78:0/3263278427 shutdown_connections 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- 172.21.15.78:0/3263278427 wait complete. 2024-09-11T12:53:27.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 Processor -- start 2024-09-11T12:53:27.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- start start 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc1530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fcc06d3a0 0x7f6fcc1535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc1677b0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fcc167630 con 0x7f6fcc06cfc0 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6fcc167930 con 0x7f6fcc06d3a0 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:27.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54486/0 (socket says 172.21.15.78:54486) 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 -- 172.21.15.78:0/4152755939 learned_addr learned my addr 172.21.15.78:0/4152755939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fcbe00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc1530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 -- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fcc06d3a0 msgr2=0x7f6fcc1535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fcc06d3a0 0x7f6fcc1535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 -- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 msgr2=0x7f6fcc1530a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc1530a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 -- 172.21.15.78:0/4152755939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fcc150020 con 0x7f6fcc0b3730 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fcbe00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.708+0000 7f6fd1200640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b970 tx=0x7f6fc400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc4019070 con 0x7f6fcc0b3730 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc400cc50 con 0x7f6fcc0b3730 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc40146a0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fcc1f8530 con 0x7f6fcc0b3730 2024-09-11T12:53:27.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fcc1f8af0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fcc1f8cf0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f6fc401c070 con 0x7f6fcc0b3730 2024-09-11T12:53:27.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 0x7f6fac073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:27.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f6fc40921e0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.712+0000 7f6fcbe00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 0x7f6fac073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:27.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.716+0000 7f6fcbe00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 0x7f6fac073ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0af000 tx=0x7f6fbc01b100 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:27.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.716+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6fcc1f8cf0 con 0x7f6fcc0b3730 2024-09-11T12:53:27.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6fcc13df40 con 0x7f6fcc0b3730 2024-09-11T12:53:27.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fc3e00640 1 -- 172.21.15.78:0/4152755939 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f6fc405db80 con 0x7f6fcc0b3730 2024-09-11T12:53:27.918 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 msgr2=0x7f6fac073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 0x7f6fac073ea0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0af000 tx=0x7f6fbc01b100 comp rx=0 tx=0).stop 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 msgr2=0x7f6fcc14e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.916+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6fc400b970 tx=0x7f6fc400be30 comp rx=0 tx=0).stop 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 shutdown_connections 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f6fac0719e0 0x7f6fac073ea0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fcc0b3730 0x7f6fcc14e9a0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fcc06d3a0 0x7f6fcc1535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 --2- 172.21.15.78:0/4152755939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fcc06cfc0 0x7f6fcc1530a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:27.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 >> 172.21.15.78:0/4152755939 conn(0x7f6fcc073940 msgr2=0x7f6fcc154490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:27.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 shutdown_connections 2024-09-11T12:53:27.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:27.920+0000 7f6fd1c00640 1 -- 172.21.15.78:0/4152755939 wait complete. 2024-09-11T12:53:27.940 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:53:27.940 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:53:27.940 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:53:27.940 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:53:27.941 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:27.941 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:53:27.941 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:53:27.941 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:53:27.941 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:27.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.941 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:53:27.951 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:53:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:53:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:53:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.951 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.952 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42236sec preferred_lft 42236sec 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:53:27.953 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:53:27.954 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:27.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:27.955 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:53:28.016 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:53:28.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:28.019 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:53:28.071 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:28.072 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:53:28.073 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:53:28.172 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:53:28.215 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:28.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:28.223 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:53:28.231 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:53:28.231 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:53:28.232 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:28.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:28.233 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:53:28.296 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:28.296 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:28.348 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:28.348 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:28.348 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:28.399 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:53:28.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:28.399 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:28.491 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:28.491 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:53:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:53:28.505 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:53:28.509 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:53:28.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:28.509 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:28.528 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:53:28.618 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:53:28.621 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:28.627 INFO:teuthology.orchestra.run.smithi157.stdout:client5576 2024-09-11T12:53:28.627 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:28.627 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:28.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 -- 172.21.15.78:0/2849855404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550003fd0 msgr2=0x7f4550004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 --2- 172.21.15.78:0/2849855404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4550003fd0 0x7f4550004810 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4554001770 tx=0x7f455401c7a0 comp rx=0 tx=0).stop 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f456b400640 1 -- 172.21.15.78:0/2849855404 reap_dead start 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 -- 172.21.15.78:0/2849855404 shutdown_connections 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 -- 172.21.15.78:0/2849855404 >> 172.21.15.78:0/2849855404 conn(0x7f456c0b3050 msgr2=0x7f456c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 -- 172.21.15.78:0/2849855404 shutdown_connections 2024-09-11T12:53:28.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.780+0000 7f4571a00640 1 -- 172.21.15.78:0/2849855404 wait complete. 2024-09-11T12:53:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 Processor -- start 2024-09-11T12:53:28.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- start start 2024-09-11T12:53:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550003fd0 0x7f456c156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:28.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c1f8e40 0x7f456c1f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f456c166740 con 0x7f456c1570a0 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f456c1665c0 con 0x7f4550003fd0 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f456c1668c0 con 0x7f456c1f8e40 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54496/0 (socket says 172.21.15.78:54496) 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 -- 172.21.15.78:0/2657437872 learned_addr learned my addr 172.21.15.78:0/2657437872 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f456aa00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550003fd0 0x7f456c156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f456b400640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c1f8e40 0x7f456c1f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 -- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c1f8e40 msgr2=0x7f456c1f92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c1f8e40 0x7f456c1f92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 -- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550003fd0 msgr2=0x7f456c156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550003fd0 0x7f456c156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:28.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 -- 172.21.15.78:0/2657437872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456c0724b0 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4563e00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f455401cc80 tx=0x7f4554001470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554003600 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45540b7020 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554003b10 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456c156190 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f456c14ddc0 con 0x7f456c1570a0 2024-09-11T12:53:28.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.784+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f456c0722a0 con 0x7f456c1570a0 2024-09-11T12:53:28.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.788+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f4554003cb0 con 0x7f456c1570a0 2024-09-11T12:53:28.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.788+0000 7f4562a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 0x7f4548073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:28.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.788+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f455412c6d0 con 0x7f456c1570a0 2024-09-11T12:53:28.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.788+0000 7f456aa00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 0x7f4548073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:28.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.788+0000 7f456aa00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 0x7f4548073ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c60 tx=0x7f455c0091e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:28.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.792+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f45540c1030 con 0x7f456c1570a0 2024-09-11T12:53:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:28.996+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f456c070ce0 con 0x7f456c1570a0 2024-09-11T12:53:29.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.508+0000 7f4562a00640 1 -- 172.21.15.78:0/2657437872 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f45540164b0 con 0x7f456c1570a0 2024-09-11T12:53:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 msgr2=0x7f4548073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 0x7f4548073ed0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f455c009c60 tx=0x7f455c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 msgr2=0x7f456c1f8900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f455401cc80 tx=0x7f4554001470 comp rx=0 tx=0).stop 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 shutdown_connections 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4548071a10 0x7f4548073ed0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f456c1f8e40 0x7f456c1f92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f456c1570a0 0x7f456c1f8900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 --2- 172.21.15.78:0/2657437872 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4550003fd0 0x7f456c156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 >> 172.21.15.78:0/2657437872 conn(0x7f456c0b3050 msgr2=0x7f456c15e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 shutdown_connections 2024-09-11T12:53:29.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.512+0000 7f4571a00640 1 -- 172.21.15.78:0/2657437872 wait complete. 2024-09-11T12:53:29.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:53:29.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- 172.21.15.78:0/741791567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 msgr2=0x7f70841406f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- 172.21.15.78:0/741791567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70841406f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f70840a9df0 tx=0x7f707001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- 172.21.15.78:0/741791567 shutdown_connections 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- 172.21.15.78:0/741791567 >> v2:172.21.15.157:3300/0 conn(0x7f70840aca00 0x7f7084140c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- 172.21.15.78:0/741791567 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70841406f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- 172.21.15.78:0/741791567 >> 172.21.15.78:0/741791567 conn(0x7f7084074490 msgr2=0x7f7084074870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- 172.21.15.78:0/741791567 shutdown_connections 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- 172.21.15.78:0/741791567 wait complete. 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 Processor -- start 2024-09-11T12:53:29.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- start start 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70840ac620 0x7f7084153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70840aca00 0x7f708413afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7084168770 con 0x7f708415f3d0 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7084141d30 con 0x7f70840aca00 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.676+0000 7f7083200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f70841688f0 con 0x7f70840ac620 2024-09-11T12:53:29.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54524/0 (socket says 172.21.15.78:54524) 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 -- 172.21.15.78:0/3115916414 learned_addr learned my addr 172.21.15.78:0/3115916414 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7081e00640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70840ac620 0x7f7084153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7081400640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70840aca00 0x7f708413afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 -- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70840ac620 msgr2=0x7f7084153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70840ac620 0x7f7084153b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 -- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70840aca00 msgr2=0x7f708413afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70840aca00 0x7f708413afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 -- 172.21.15.78:0/3115916414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708413b720 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7082800640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f707c00ea20 tx=0x7f707c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c00cde0 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f707c004c40 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708413ba00 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707c005280 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708413bfc0 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708415bde0 con 0x7f708415f3d0 2024-09-11T12:53:29.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.680+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f707c005420 con 0x7f708415f3d0 2024-09-11T12:53:29.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.684+0000 7f707a000640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 0x7f7064073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:29.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.684+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f707c093290 con 0x7f708415f3d0 2024-09-11T12:53:29.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.684+0000 7f7081e00640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 0x7f7064073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:29.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.684+0000 7f7081e00640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 0x7f7064073ea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7074007fd0 tx=0x7f7074007e40 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:29.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.684+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f707c016030 con 0x7f708415f3d0 2024-09-11T12:53:29.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.872+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f708413f1a0 con 0x7f70640719e0 2024-09-11T12:53:29.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.872+0000 7f707a000640 1 -- 172.21.15.78:0/3115916414 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f708413f1a0 con 0x7f70640719e0 2024-09-11T12:53:29.875 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:53:29.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:29.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:53:29.876 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:29.876 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 msgr2=0x7f7064073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 0x7f7064073ea0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f7074007fd0 tx=0x7f7074007e40 comp rx=0 tx=0).stop 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 msgr2=0x7f70840afbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f707c00ea20 tx=0x7f707c00eee0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 shutdown_connections 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f70640719e0 0x7f7064073ea0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f708415f3d0 0x7f70840afbe0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70840aca00 0x7f708413afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 --2- 172.21.15.78:0/3115916414 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f70840ac620 0x7f7084153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:29.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 >> 172.21.15.78:0/3115916414 conn(0x7f7084074490 msgr2=0x7f708413f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:29.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 shutdown_connections 2024-09-11T12:53:29.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:29.876+0000 7f7083200640 1 -- 172.21.15.78:0/3115916414 wait complete. 2024-09-11T12:53:29.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- 172.21.15.78:0/1017081543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300b2de0 msgr2=0x7fb73015ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 --2- 172.21.15.78:0/1017081543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300b2de0 0x7fb73015ca30 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fb728009840 tx=0x7fb72801c790 comp rx=0 tx=0).stop 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- 172.21.15.78:0/1017081543 shutdown_connections 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 --2- 172.21.15.78:0/1017081543 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300b2de0 0x7fb73015ca30 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- 172.21.15.78:0/1017081543 >> 172.21.15.78:0/1017081543 conn(0x7fb730071dd0 msgr2=0x7fb7300721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- 172.21.15.78:0/1017081543 shutdown_connections 2024-09-11T12:53:30.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- 172.21.15.78:0/1017081543 wait complete. 2024-09-11T12:53:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 Processor -- start 2024-09-11T12:53:30.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.040+0000 7fb736600640 1 -- start start 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b2de0 0x7fb730144360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300cc1f0 0x7fb73013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb730169e50 con 0x7fb7300cc5d0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb730169cd0 con 0x7fb7300b2de0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb730169fd0 con 0x7fb7300cc1f0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54550/0 (socket says 172.21.15.78:54550) 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 -- 172.21.15.78:0/1568351494 learned_addr learned my addr 172.21.15.78:0/1568351494 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72f400640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b2de0 0x7fb730144360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72ea00640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300cc1f0 0x7fb73013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 -- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300cc1f0 msgr2=0x7fb73013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300cc1f0 0x7fb73013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 -- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b2de0 msgr2=0x7fb730144360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b2de0 0x7fb730144360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 -- 172.21.15.78:0/1568351494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb730144940 con 0x7fb7300cc5d0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb72fe00640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb728009790 tx=0x7fb7280a6f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280ac070 con 0x7fb7300cc5d0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb728004320 con 0x7fb7300cc5d0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb730144c20 con 0x7fb7300cc5d0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280052d0 con 0x7fb7300cc5d0 2024-09-11T12:53:30.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb730154790 con 0x7fb7300cc5d0 2024-09-11T12:53:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73015ca30 con 0x7fb7300cc5d0 2024-09-11T12:53:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.044+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fb7280af070 con 0x7fb7300cc5d0 2024-09-11T12:53:30.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.048+0000 7fb727400640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 0x7fb710073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.048+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7fb72812c760 con 0x7fb7300cc5d0 2024-09-11T12:53:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.048+0000 7fb72f400640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 0x7fb710073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.048+0000 7fb72f400640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 0x7fb710073e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7200021a0 tx=0x7fb720009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.052+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb728016410 con 0x7fb7300cc5d0 2024-09-11T12:53:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.216+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb73015ca30 con 0x7fb7300cc5d0 2024-09-11T12:53:30.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.216+0000 7fb727400640 1 -- 172.21.15.78:0/1568351494 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7fb728005470 con 0x7fb7300cc5d0 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 msgr2=0x7fb710073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 0x7fb710073e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fb7200021a0 tx=0x7fb720009040 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 msgr2=0x7fb73013d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fb728009790 tx=0x7fb7280a6f50 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 shutdown_connections 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fb710071950 0x7fb710073e10 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb7300cc5d0 0x7fb73013d170 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7300cc1f0 0x7fb73013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 --2- 172.21.15.78:0/1568351494 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb7300b2de0 0x7fb730144360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 >> 172.21.15.78:0/1568351494 conn(0x7fb730071dd0 msgr2=0x7fb7301416e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:30.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 shutdown_connections 2024-09-11T12:53:30.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.220+0000 7fb736600640 1 -- 172.21.15.78:0/1568351494 wait complete. 2024-09-11T12:53:30.239 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:30.239 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:30.248 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:30.248 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:30.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- 172.21.15.78:0/3756957359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 msgr2=0x7f7de8074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 --2- 172.21.15.78:0/3756957359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de8074260 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f7de8072f50 tx=0x7f7ddc01cd50 comp rx=0 tx=0).stop 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- 172.21.15.78:0/3756957359 shutdown_connections 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 --2- 172.21.15.78:0/3756957359 >> v2:172.21.15.157:3300/0 conn(0x7f7de806fc50 0x7f7de80700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 --2- 172.21.15.78:0/3756957359 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de8074260 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- 172.21.15.78:0/3756957359 >> 172.21.15.78:0/3756957359 conn(0x7f7de806ca80 msgr2=0x7f7de80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- 172.21.15.78:0/3756957359 shutdown_connections 2024-09-11T12:53:30.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- 172.21.15.78:0/3756957359 wait complete. 2024-09-11T12:53:30.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 Processor -- start 2024-09-11T12:53:30.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.488+0000 7f7df0200640 1 -- start start 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de806fc50 0x7f7de81f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de815aec0 0x7f7de81460f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7de815f370 con 0x7f7de80cf760 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7de815f1f0 con 0x7f7de806fc50 2024-09-11T12:53:30.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7de8151010 con 0x7f7de815aec0 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54576/0 (socket says 172.21.15.78:54576) 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 -- 172.21.15.78:0/3024811405 learned_addr learned my addr 172.21.15.78:0/3024811405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7ded800640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de806fc50 0x7f7de81f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dee200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de815aec0 0x7f7de81460f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 -- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de815aec0 msgr2=0x7f7de81460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de815aec0 0x7f7de81460f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 -- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de806fc50 msgr2=0x7f7de81f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de806fc50 0x7f7de81f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 -- 172.21.15.78:0/3024811405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de8150b60 con 0x7f7de80cf760 2024-09-11T12:53:30.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7dece00640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c3c0 tx=0x7f7dd400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4017070 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dd40053c0 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4003860 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de8150320 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81fa5a0 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de80700c0 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.492+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f7dd401a070 con 0x7f7de80cf760 2024-09-11T12:53:30.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.496+0000 7f7de5600640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 0x7f7dc8073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.496+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f7dd4091710 con 0x7f7de80cf760 2024-09-11T12:53:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.496+0000 7f7ded800640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 0x7f7dc8073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.496+0000 7f7ded800640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 0x7f7dc8073ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc01b590 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.500+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7de8150320 con 0x7f7de80cf760 2024-09-11T12:53:30.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.676+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7de814f2e0 con 0x7f7de80cf760 2024-09-11T12:53:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.676+0000 7f7de5600640 1 -- 172.21.15.78:0/3024811405 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f7dd405d0d0 con 0x7f7de80cf760 2024-09-11T12:53:30.679 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:53:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 msgr2=0x7f7dc8073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 0x7f7dc8073ea0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc0af000 tx=0x7f7ddc01b590 comp rx=0 tx=0).stop 2024-09-11T12:53:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 msgr2=0x7f7de80b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400c3c0 tx=0x7f7dd400c880 comp rx=0 tx=0).stop 2024-09-11T12:53:30.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 shutdown_connections 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7dc80719e0 0x7f7dc8073ea0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7de815aec0 0x7f7de81460f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7de80cf760 0x7f7de80b1330 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 --2- 172.21.15.78:0/3024811405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7de806fc50 0x7f7de81f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 >> 172.21.15.78:0/3024811405 conn(0x7f7de806ca80 msgr2=0x7f7de814a810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 shutdown_connections 2024-09-11T12:53:30.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.680+0000 7f7df0200640 1 -- 172.21.15.78:0/3024811405 wait complete. 2024-09-11T12:53:30.699 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:53:30.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:30.700 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:30.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:53:30.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:30.726 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:53:30.780 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:30.780 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:30.790 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:30.790 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:30.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:53:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.984+0000 7f0f9b600640 1 -- 172.21.15.78:0/4083358904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f94152bd0 msgr2=0x7f0f9415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.984+0000 7f0f9b600640 1 --2- 172.21.15.78:0/4083358904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f94152bd0 0x7f0f9415ca20 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009900 tx=0x7f0f8001c810 comp rx=0 tx=0).stop 2024-09-11T12:53:30.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- 172.21.15.78:0/4083358904 shutdown_connections 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 --2- 172.21.15.78:0/4083358904 >> v2:172.21.15.157:3300/0 conn(0x7f0f940b3070 0x7f0f940b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 --2- 172.21.15.78:0/4083358904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f94152bd0 0x7f0f9415ca20 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- 172.21.15.78:0/4083358904 >> 172.21.15.78:0/4083358904 conn(0x7f0f940b1c50 msgr2=0x7f0f94070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- 172.21.15.78:0/4083358904 shutdown_connections 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- 172.21.15.78:0/4083358904 wait complete. 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 Processor -- start 2024-09-11T12:53:30.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- start start 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f941527f0 0x7f0f94151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f94152bd0 0x7f0f941452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94168a70 con 0x7f0f940b3070 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941688f0 con 0x7f0f94152bd0 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f9b600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0f94168bf0 con 0x7f0f941527f0 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54600/0 (socket says 172.21.15.78:54600) 2024-09-11T12:53:30.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 -- 172.21.15.78:0/488797104 learned_addr learned my addr 172.21.15.78:0/488797104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f99600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f94152bd0 0x7f0f941452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f93e00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f941527f0 0x7f0f94151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 -- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f941527f0 msgr2=0x7f0f94151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f941527f0 0x7f0f94151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 -- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f94152bd0 msgr2=0x7f0f941452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f94152bd0 0x7f0f941452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 -- 172.21.15.78:0/488797104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f941fed70 con 0x7f0f940b3070 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.988+0000 7f0f98c00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c7c0 tx=0x7f0f8400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84016020 con 0x7f0f940b3070 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f8400ab40 con 0x7f0f940b3070 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f941fc340 con 0x7f0f940b3070 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84012420 con 0x7f0f940b3070 2024-09-11T12:53:30.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941fc620 con 0x7f0f940b3070 2024-09-11T12:53:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f940b3520 con 0x7f0f940b3070 2024-09-11T12:53:30.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f0f84019020 con 0x7f0f940b3070 2024-09-11T12:53:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 0x7f0f74073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:30.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f0f84092330 con 0x7f0f940b3070 2024-09-11T12:53:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.992+0000 7f0f93e00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 0x7f0f74073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:30.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.996+0000 7f0f93e00640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 0x7f0f74073ea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f800af000 tx=0x7f0f8001b050 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:30.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:30.996+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0f84020030 con 0x7f0f940b3070 2024-09-11T12:53:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.176+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0f94155e20 con 0x7f0f940b3070 2024-09-11T12:53:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.176+0000 7f0f90c00640 1 -- 172.21.15.78:0/488797104 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v71) ==== 80+0+0 (secure 0 0 0) 0x7f0f8405d6c0 con 0x7f0f940b3070 2024-09-11T12:53:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:53:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 msgr2=0x7f0f74073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 0x7f0f74073ea0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0f800af000 tx=0x7f0f8001b050 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 msgr2=0x7f0f941ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400c7c0 tx=0x7f0f8400cc80 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 shutdown_connections 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f740719e0 0x7f0f74073ea0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0f94152bd0 0x7f0f941452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0f941527f0 0x7f0f94151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 --2- 172.21.15.78:0/488797104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0f940b3070 0x7f0f941ff810 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 >> 172.21.15.78:0/488797104 conn(0x7f0f940b1c50 msgr2=0x7f0f94155620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 shutdown_connections 2024-09-11T12:53:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.180+0000 7f0f9b600640 1 -- 172.21.15.78:0/488797104 wait complete. 2024-09-11T12:53:31.199 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:53:31.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:31.200 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:31.230 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:53:31.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:31.230 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:53:31.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 -- 172.21.15.78:0/712561156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f7406fdf0 msgr2=0x7f4f740a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/712561156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f7406fdf0 0x7f4f740a9d00 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4f740d1780 tx=0x7f4f6001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 -- 172.21.15.78:0/712561156 shutdown_connections 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/712561156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f7406fdf0 0x7f4f740a9d00 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/712561156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f740d0bf0 0x7f4f740d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/712561156 >> v2:172.21.15.157:3300/0 conn(0x7f4f740b14b0 0x7f4f7415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 -- 172.21.15.78:0/712561156 >> 172.21.15.78:0/712561156 conn(0x7f4f74074b90 msgr2=0x7f4f740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:31.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 -- 172.21.15.78:0/712561156 shutdown_connections 2024-09-11T12:53:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.420+0000 7f4f7ae00640 1 -- 172.21.15.78:0/712561156 wait complete. 2024-09-11T12:53:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 Processor -- start 2024-09-11T12:53:31.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- start start 2024-09-11T12:53:31.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 0x7f4f7414e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 0x7f4f741551e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7406edb0 con 0x7f4f740b14b0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7406ec30 con 0x7f4f740d0bf0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7406ef30 con 0x7f4f7406fdf0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54620/0 (socket says 172.21.15.78:54620) 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 -- 172.21.15.78:0/41406379 learned_addr learned my addr 172.21.15.78:0/41406379 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f78e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 0x7f4f741551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 0x7f4f7414e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 -- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 msgr2=0x7f4f7414e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 0x7f4f7414e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 -- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 msgr2=0x7f4f741551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 0x7f4f741551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 -- 172.21.15.78:0/41406379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f74146bc0 con 0x7f4f740b14b0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f78e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 0x7f4f741551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 0x7f4f7414e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f73400640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4f600b0040 tx=0x7f4f600a56a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f600b2040 con 0x7f4f740b14b0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f600044d0 con 0x7f4f740b14b0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f74146ea0 con 0x7f4f740b14b0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f60005280 con 0x7f4f740b14b0 2024-09-11T12:53:31.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f74147460 con 0x7f4f740b14b0 2024-09-11T12:53:31.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.424+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7406f3e0 con 0x7f4f740b14b0 2024-09-11T12:53:31.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.428+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f4f60005420 con 0x7f4f740b14b0 2024-09-11T12:53:31.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.428+0000 7f4f6be00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 0x7f4f54073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.428+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f4f6012c050 con 0x7f4f740b14b0 2024-09-11T12:53:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.428+0000 7f4f73e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 0x7f4f54073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.428+0000 7f4f73e00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 0x7f4f54073ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005fd0 tx=0x7f4f64005ee0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:31.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.432+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4f600bc060 con 0x7f4f740b14b0 2024-09-11T12:53:31.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.632+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f740aa020 con 0x7f4f740b14b0 2024-09-11T12:53:31.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.632+0000 7f4f6be00640 1 -- 172.21.15.78:0/41406379 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f4f600164b0 con 0x7f4f740b14b0 2024-09-11T12:53:31.635 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:31.636 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:31.636 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 msgr2=0x7f4f54073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 0x7f4f54073ea0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4f64005fd0 tx=0x7f4f64005ee0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 msgr2=0x7f4f74146090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4f600b0040 tx=0x7f4f600a56a0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 shutdown_connections 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f4f540719e0 0x7f4f54073ea0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f740d0bf0 0x7f4f741551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f740b14b0 0x7f4f74146090 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 --2- 172.21.15.78:0/41406379 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f7406fdf0 0x7f4f7414e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 >> 172.21.15.78:0/41406379 conn(0x7f4f74074b90 msgr2=0x7f4f7406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 shutdown_connections 2024-09-11T12:53:31.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.636+0000 7f4f7ae00640 1 -- 172.21.15.78:0/41406379 wait complete. 2024-09-11T12:53:31.658 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-11T12:53:31.827 INFO:teuthology.orchestra.run.smithi157.stdout:[] 2024-09-11T12:53:31.827 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-11T12:53:31.827 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol59530 2024-09-11T12:53:31.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 -- 172.21.15.78:0/1010344626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800b3040 msgr2=0x7f89800cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 --2- 172.21.15.78:0/1010344626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800b3040 0x7f89800cf6e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8970009bf0 tx=0x7f897001cb50 comp rx=0 tx=0).stop 2024-09-11T12:53:31.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 -- 172.21.15.78:0/1010344626 shutdown_connections 2024-09-11T12:53:31.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 --2- 172.21.15.78:0/1010344626 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800b3040 0x7f89800cf6e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 -- 172.21.15.78:0/1010344626 >> 172.21.15.78:0/1010344626 conn(0x7f8980071dd0 msgr2=0x7f89800721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.968+0000 7f8984c00640 1 -- 172.21.15.78:0/1010344626 shutdown_connections 2024-09-11T12:53:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- 172.21.15.78:0/1010344626 wait complete. 2024-09-11T12:53:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 Processor -- start 2024-09-11T12:53:31.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- start start 2024-09-11T12:53:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89800b3420 0x7f8980137f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800cd770 0x7f898013c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89800aef80 con 0x7f89800b3040 2024-09-11T12:53:31.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89800aee00 con 0x7f89800b3420 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89800af100 con 0x7f89800cd770 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54630/0 (socket says 172.21.15.78:54630) 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 -- 172.21.15.78:0/4082677855 learned_addr learned my addr 172.21.15.78:0/4082677855 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897e800640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800cd770 0x7f898013c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897d400640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89800b3420 0x7f8980137f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 -- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800cd770 msgr2=0x7f898013c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800cd770 0x7f898013c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 -- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89800b3420 msgr2=0x7f8980137f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89800b3420 0x7f8980137f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:31.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 -- 172.21.15.78:0/4082677855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89801551e0 con 0x7f89800b3040 2024-09-11T12:53:31.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f897de00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8970000f90 tx=0x7f8970009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:31.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700bb070 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8970004d40 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89801554c0 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.972+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89700b2040 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8980155a80 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89800cf6e0 con 0x7f89800b3040 2024-09-11T12:53:31.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f89700be070 con 0x7f89800b3040 2024-09-11T12:53:31.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f8976000640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 0x7f8960073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:31.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f897012c030 con 0x7f89800b3040 2024-09-11T12:53:31.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.976+0000 7f897d400640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 0x7f8960073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:31.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.980+0000 7f897d400640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 0x7f8960073ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f896c007f30 tx=0x7f896c007ec0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:31.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:31.980+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f89700167d0 con 0x7f89800b3040 2024-09-11T12:53:32.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.172+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 --> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol59530", "target": ["mon-mgr", ""]}) -- 0x7f898006eda0 con 0x7f8960071a10 2024-09-11T12:53:32.180 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:53:32.178+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:32.181 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:53:32.178+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:32.181 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:53:32.178+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:32.181 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:53:32.178+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:32.181 INFO:tasks.ceph.mgr.x.smithi157.stderr:2024-09-11T12:53:32.178+0000 7fa58b600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:32.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8976000640 1 -- 172.21.15.78:0/4082677855 <== mgr.4105 v2:172.21.15.157:6800/855997071 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f898006eda0 con 0x7f8960071a10 2024-09-11T12:53:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 msgr2=0x7f8960073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 0x7f8960073ed0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f896c007f30 tx=0x7f896c007ec0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 msgr2=0x7f89801538f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8970000f90 tx=0x7f8970009220 comp rx=0 tx=0).stop 2024-09-11T12:53:32.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 shutdown_connections 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f8960071a10 0x7f8960073ed0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89800cd770 0x7f898013c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89800b3420 0x7f8980137f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 --2- 172.21.15.78:0/4082677855 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89800b3040 0x7f89801538f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 >> 172.21.15.78:0/4082677855 conn(0x7f8980071dd0 msgr2=0x7f89801685a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 shutdown_connections 2024-09-11T12:53:32.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.212+0000 7f8984c00640 1 -- 172.21.15.78:0/4082677855 wait complete. 2024-09-11T12:53:32.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- 172.21.15.78:0/2742610372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0b4db0 msgr2=0x7fee2c0b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- 172.21.15.78:0/2742610372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0b4db0 0x7fee2c0b5240 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fee24009c80 tx=0x7fee2401cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- 172.21.15.78:0/2742610372 shutdown_connections 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- 172.21.15.78:0/2742610372 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0b4db0 0x7fee2c0b5240 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- 172.21.15.78:0/2742610372 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0cd910 0x7fee2c0cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- 172.21.15.78:0/2742610372 >> 172.21.15.78:0/2742610372 conn(0x7fee2c06ca80 msgr2=0x7fee2c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- 172.21.15.78:0/2742610372 shutdown_connections 2024-09-11T12:53:32.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- 172.21.15.78:0/2742610372 wait complete. 2024-09-11T12:53:32.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 Processor -- start 2024-09-11T12:53:32.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- start start 2024-09-11T12:53:32.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0b4db0 0x7fee2c14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 0x7fee2c14d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c06f730 con 0x7fee2c159fc0 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee2c06f5b0 con 0x7fee2c0cd910 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee31e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee2c06f8b0 con 0x7fee2c0b4db0 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54636/0 (socket says 172.21.15.78:54636) 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 -- 172.21.15.78:0/2747252845 learned_addr learned my addr 172.21.15.78:0/2747252845 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2aa00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0b4db0 0x7fee2c14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2a000640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 0x7fee2c14d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 -- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0b4db0 msgr2=0x7fee2c14a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0b4db0 0x7fee2c14a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 -- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 msgr2=0x7fee2c14d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 0x7fee2c14d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2b400640 1 -- 172.21.15.78:0/2747252845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2c14e340 con 0x7fee2c159fc0 2024-09-11T12:53:32.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.376+0000 7fee2a000640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 0x7fee2c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:32.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee2b400640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fee2401c810 tx=0x7fee24005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee240b2040 con 0x7fee2c159fc0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee240a59f0 con 0x7fee2c159fc0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24004240 con 0x7fee2c159fc0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2c15ce80 con 0x7fee2c159fc0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2c15d440 con 0x7fee2c159fc0 2024-09-11T12:53:32.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee2c15d640 con 0x7fee2c159fc0 2024-09-11T12:53:32.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7fee240bd030 con 0x7fee2c159fc0 2024-09-11T12:53:32.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 0x7fee0c073ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7fee2412c3b0 con 0x7fee2c159fc0 2024-09-11T12:53:32.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.380+0000 7fee2aa00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 0x7fee0c073ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.384+0000 7fee2aa00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 0x7fee0c073ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee1c001640 tx=0x7fee1c009040 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:32.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.384+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fee2c15d640 con 0x7fee2c159fc0 2024-09-11T12:53:32.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee2c155380 con 0x7fee2c159fc0 2024-09-11T12:53:32.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee23400640 1 -- 172.21.15.78:0/2747252845 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7fee240c4020 con 0x7fee2c159fc0 2024-09-11T12:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:32.590 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:32.590 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 msgr2=0x7fee0c073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 0x7fee0c073ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee1c001640 tx=0x7fee1c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 msgr2=0x7fee2c1eca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.588+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fee2401c810 tx=0x7fee24005c40 comp rx=0 tx=0).stop 2024-09-11T12:53:32.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 shutdown_connections 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7fee0c071a10 0x7fee0c073ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee2c159fc0 0x7fee2c1eca80 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee2c0cd910 0x7fee2c14d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 --2- 172.21.15.78:0/2747252845 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee2c0b4db0 0x7fee2c14a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 >> 172.21.15.78:0/2747252845 conn(0x7fee2c06ca80 msgr2=0x7fee2c0b1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 shutdown_connections 2024-09-11T12:53:32.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.592+0000 7fee31e00640 1 -- 172.21.15.78:0/2747252845 wait complete. 2024-09-11T12:53:32.611 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-11T12:53:32.772 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":6067,"entity":{"name":{"type":"client","num":6067},"addr":{"type":"any","addr":"172.21.15.157:0","nonce":656148200}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.57599451499999998,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.971378069213298,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6067 172.21.15.157:0/656148200","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"x","hostname":"smithi157","pid":"28808","root":"/"}}] 2024-09-11T12:53:32.772 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi157", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28808", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.157:0", "nonce": 656148200, "type": "any" }, "name": { "num": 6067, "type": "client" } }, "id": 6067, "inst": "client.6067 172.21.15.157:0/656148200", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.971378069213298 }, "state": "open", "uptime": 0.575994515 } ] 2024-09-11T12:53:32.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- 172.21.15.78:0/4276197522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40af6c0 msgr2=0x7f7dd40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 --2- 172.21.15.78:0/4276197522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40af6c0 0x7f7dd40b4d20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc009c80 tx=0x7f7dcc01cd70 comp rx=0 tx=0).stop 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- 172.21.15.78:0/4276197522 shutdown_connections 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 --2- 172.21.15.78:0/4276197522 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40af6c0 0x7f7dd40b4d20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 --2- 172.21.15.78:0/4276197522 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd40759d0 0x7f7dd40cfdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- 172.21.15.78:0/4276197522 >> 172.21.15.78:0/4276197522 conn(0x7f7dd4073d80 msgr2=0x7f7dd4074190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- 172.21.15.78:0/4276197522 shutdown_connections 2024-09-11T12:53:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- 172.21.15.78:0/4276197522 wait complete. 2024-09-11T12:53:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 Processor -- start 2024-09-11T12:53:32.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.912+0000 7f7dd3200640 1 -- start start 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd40af6c0 0x7f7dd415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd415ba10 0x7f7dd415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd4168860 con 0x7f7dd40759d0 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd40b6000 con 0x7f7dd40af6c0 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7dd41689e0 con 0x7f7dd415ba10 2024-09-11T12:53:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:54656/0 (socket says 172.21.15.78:54656) 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 -- 172.21.15.78:0/3359231590 learned_addr learned my addr 172.21.15.78:0/3359231590 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1400640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd40af6c0 0x7f7dd415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd2800640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd415ba10 0x7f7dd415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 -- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd415ba10 msgr2=0x7f7dd415a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd415ba10 0x7f7dd415a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 -- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd40af6c0 msgr2=0x7f7dd415a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd40af6c0 0x7f7dd415a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 -- 172.21.15.78:0/3359231590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd415ac40 con 0x7f7dd40759d0 2024-09-11T12:53:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd1e00640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c3c0 tx=0x7f7dc400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4017070 con 0x7f7dd40759d0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dc40053c0 con 0x7f7dd40759d0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc4003860 con 0x7f7dd40759d0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd4141720 con 0x7f7dd40759d0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd4141ce0 con 0x7f7dd40759d0 2024-09-11T12:53:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.916+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd40b4d20 con 0x7f7dd40759d0 2024-09-11T12:53:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.920+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 10) ==== 95238+0+0 (secure 0 0 0) 0x7f7dc401a070 con 0x7f7dd40759d0 2024-09-11T12:53:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.920+0000 7f7dca000640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 0x7f7db4073ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.920+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f7dc4091600 con 0x7f7dd40759d0 2024-09-11T12:53:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.920+0000 7f7dd1400640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 0x7f7db4073ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.920+0000 7f7dd1400640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 0x7f7db4073ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40ce4d0 tx=0x7f7dc000b390 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:32.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:32.924+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7dc405c990 con 0x7f7dd40759d0 2024-09-11T12:53:33.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.128+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7dd40aed40 con 0x7f7dd40759d0 2024-09-11T12:53:33.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.136+0000 7f7dca000640 1 -- 172.21.15.78:0/3359231590 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v10) ==== 81+0+498969 (secure 0 0 0) 0x7f7dc4065220 con 0x7f7dd40759d0 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "epoch": 10, 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "active_gid": 4105, 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "active_name": "x", 2024-09-11T12:53:33.142 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addrs": { 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:6800", 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 855997071 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v1", 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:6801", 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 855997071 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.143 INFO:teuthology.orchestra.run.smithi078.stdout: "active_addr": "172.21.15.157:6801/855997071", 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "active_change": "2024-09-11T12:49:23.091583+0000", 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "active_mgr_features": 4540701547738038271, 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "available": true, 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "standbys": [ 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 4116, 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "y", 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr_features": 4540701547738038271, 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.144 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.145 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.146 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.148 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.149 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.150 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.151 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.152 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.153 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.154 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.155 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.156 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.157 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.158 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.159 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.160 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.161 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.162 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.163 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T12:53:33.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-11T12:53:33.166 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T12:53:33.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T12:53:33.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.169 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.171 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.172 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.173 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.174 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-11T12:53:33.175 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.176 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.177 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.178 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.179 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-11T12:53:33.180 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.181 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T12:53:33.182 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.184 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.186 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T12:53:33.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-11T12:53:33.188 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T12:53:33.189 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T12:53:33.190 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.193 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-11T12:53:33.194 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-11T12:53:33.195 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.196 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.197 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.198 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T12:53:33.199 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.200 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-11T12:53:33.201 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-11T12:53:33.202 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nginx": { 2024-09-11T12:53:33.203 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nginx", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nginx container image", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-11T12:53:33.204 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-11T12:53:33.205 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.206 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-11T12:53:33.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-11T12:53:33.208 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.212 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.217 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-11T12:53:33.219 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.220 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.221 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.222 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.223 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.224 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.227 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.228 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.229 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.230 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.231 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.232 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.233 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.234 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-11T12:53:33.235 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-11T12:53:33.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-11T12:53:33.237 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-11T12:53:33.238 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-11T12:53:33.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-11T12:53:33.240 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-11T12:53:33.241 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-11T12:53:33.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-11T12:53:33.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-11T12:53:33.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.246 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.247 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-11T12:53:33.248 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-11T12:53:33.249 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.250 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.252 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.253 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.254 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.255 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.256 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.257 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.258 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.259 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-11T12:53:33.260 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.261 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.262 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.263 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.264 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.265 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.266 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.267 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.268 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.269 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.270 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.271 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.272 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.273 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.274 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.275 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.278 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.280 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.281 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.282 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.283 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.284 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.285 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.286 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.288 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.289 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.292 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.294 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.295 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T12:53:33.296 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T12:53:33.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T12:53:33.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T12:53:33.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T12:53:33.300 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.301 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.302 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.303 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.304 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.305 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.306 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.307 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.308 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.309 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.310 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.311 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.312 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.313 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.315 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.316 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.317 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.318 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.319 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.320 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.321 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.322 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.323 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.324 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.325 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T12:53:33.328 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-11T12:53:33.329 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.330 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.332 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.333 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.334 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.335 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.336 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.337 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.338 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.339 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.340 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.341 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.342 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.343 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.344 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.345 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.346 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.352 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.353 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-11T12:53:33.354 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.355 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.357 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.358 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.359 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.360 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.361 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-11T12:53:33.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.363 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T12:53:33.365 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.366 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.367 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.368 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.369 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.370 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.371 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.372 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.373 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.374 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.375 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.376 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.377 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.378 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.379 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.380 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.381 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.383 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.384 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.387 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.388 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.389 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.390 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.391 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.392 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.393 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.394 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.395 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.396 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.398 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.399 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.400 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.401 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.403 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.404 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.405 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.407 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.408 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.409 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.410 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.411 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.412 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.413 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.414 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.415 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-11T12:53:33.416 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.417 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.418 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.419 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.420 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.421 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.422 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.423 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.424 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.425 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.426 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.427 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.428 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.429 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.430 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.432 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.433 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.434 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.435 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.436 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.437 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.438 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.439 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.440 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.443 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.444 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.445 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.447 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-11T12:53:33.449 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.450 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-11T12:53:33.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-11T12:53:33.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.453 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.454 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.455 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.456 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.457 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.458 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.459 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.460 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-11T12:53:33.466 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T12:53:33.467 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.468 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.469 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.470 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.471 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.472 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.473 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.474 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.475 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.476 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.477 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-11T12:53:33.478 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T12:53:33.479 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.480 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.481 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.482 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.483 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.484 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.486 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.487 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.488 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.489 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.490 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.491 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.492 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.493 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T12:53:33.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.495 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.497 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.498 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.499 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.500 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.501 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.502 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.503 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.504 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.505 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.506 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.507 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.508 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.509 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.510 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.511 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.512 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-11T12:53:33.514 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-11T12:53:33.515 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.517 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.518 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.519 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.520 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.521 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.522 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.523 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.524 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.525 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.526 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.527 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.531 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.532 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.533 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.534 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.535 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.537 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.538 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.539 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.541 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.544 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.545 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.546 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.547 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.548 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.549 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.550 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.551 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.552 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.553 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.554 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T12:53:33.555 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.556 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-11T12:53:33.557 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-11T12:53:33.558 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.559 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.560 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.561 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.562 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.563 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.564 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.565 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.566 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.567 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.568 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T12:53:33.569 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.570 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.571 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.572 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.573 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.574 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.575 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.577 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.578 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.579 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.580 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.581 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.582 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.583 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-11T12:53:33.584 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-11T12:53:33.585 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.586 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.588 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.589 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.590 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.591 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.593 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.594 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.595 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.596 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.597 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.598 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.599 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "modules": [ 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "iostat", 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "nfs", 2024-09-11T12:53:33.600 INFO:teuthology.orchestra.run.smithi078.stdout: "restful" 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "available_modules": [ 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "alerts", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.601 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How frequently to reexamine health status", 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.603 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.604 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.605 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.606 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_destination": { 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_destination", 2024-09-11T12:53:33.607 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email address to send alerts to", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_from_name": { 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_from_name", 2024-09-11T12:53:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "Ceph", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Email From: name", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_host": { 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_host", 2024-09-11T12:53:33.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP server", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_password": { 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_password", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.610 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Password to authenticate with", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_port": { 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_port", 2024-09-11T12:53:33.611 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "465", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP port", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_sender": { 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_sender", 2024-09-11T12:53:33.612 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SMTP envelope sender", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.613 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_ssl": { 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_ssl", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.614 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "smtp_user": { 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "smtp_user", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "User to authenticate as", 2024-09-11T12:53:33.615 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.616 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "balancer", 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "active": { 2024-09-11T12:53:33.617 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "active", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically balance PGs across cluster", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_time": { 2024-09-11T12:53:33.618 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_time", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0000", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "beginning time of day to automatically balance", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.619 INFO:teuthology.orchestra.run.smithi078.stdout: "begin_weekday": { 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "begin_weekday", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.620 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_max_iterations": { 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_max_iterations", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "25", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "250", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_metrics": { 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_metrics", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "pgs,objects,bytes", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-11T12:53:33.622 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "crush_compat_step": { 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crush_compat_step", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.5", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0.001", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "0.999", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "aggressiveness of optimization", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-11T12:53:33.623 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "end_time": { 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_time", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2359", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ending time of day to automatically balance", 2024-09-11T12:53:33.624 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "end_weekday": { 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "end_weekday", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "0", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "6", 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.625 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.626 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.627 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.628 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.629 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.630 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "min_score": { 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_score", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.631 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.632 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "upmap", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "crush-compat", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "none", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "read", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap-read" 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Balancer mode", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_ids": { 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_ids", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.634 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.635 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_deviation": { 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_deviation", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.637 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "upmap_max_optimizations": { 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "upmap_max_optimizations", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.638 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm", 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_down_multiplier": { 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_down_multiplier", 2024-09-11T12:53:33.639 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_refresh_rate": { 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_refresh_rate", 2024-09-11T12:53:33.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "20", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "agent_starting_port": { 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "agent_starting_port", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.641 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4721", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_ptrace": { 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_ptrace", 2024-09-11T12:53:33.642 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_interval": { 2024-09-11T12:53:33.643 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_interval", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.644 INFO:teuthology.orchestra.run.smithi078.stdout: "autotune_memory_target_ratio": { 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "autotune_memory_target_ratio", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0.7", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm_log_destination": { 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephadm_log_destination", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "file", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "file,syslog", 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: "syslog" 2024-09-11T12:53:33.646 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "cgroups_split": { 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cgroups_split", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.647 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "config_checks_enabled": { 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_checks_enabled", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.648 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "config_dashboard": { 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "config_dashboard", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.649 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_alertmanager": { 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_alertmanager", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.650 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_base": { 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_base", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.651 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Container image name, without the tag", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_elasticsearch": { 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_elasticsearch", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.652 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "elasticsearch container image", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_grafana": { 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_grafana", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/grafana:10.4.0", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.653 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_haproxy": { 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_haproxy", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.654 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "HAproxy container image", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_agent": { 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_agent", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-11T12:53:33.655 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger agent container image", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_collector": { 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_collector", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-11T12:53:33.656 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger collector container image", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_jaeger_query": { 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_jaeger_query", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.657 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Jaeger query container image", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_keepalived": { 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_keepalived", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-11T12:53:33.658 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Keepalived container image", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_loki": { 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_loki", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.659 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Loki container image", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nginx": { 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nginx", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nginx:1.26.1", 2024-09-11T12:53:33.660 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nginx container image", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_node_exporter": { 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_node_exporter", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-11T12:53:33.661 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_nvmeof": { 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_nvmeof", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-11T12:53:33.662 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Nvme-of container image", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_prometheus": { 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_prometheus", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.663 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Prometheus container image", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_promtail": { 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_promtail", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.664 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Promtail container image", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_samba": { 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_samba", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Samba/SMB container image", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "container_image_snmp_gateway": { 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_image_snmp_gateway", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "SNMP Gateway container image", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "container_init": { 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "container_init", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Run podman/docker with `--init`", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "daemon_cache_timeout": { 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "daemon_cache_timeout", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.668 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_cephadm_command_timeout": { 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_cephadm_command_timeout", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.669 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "900", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_registry": { 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "default_registry", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "docker.io", 2024-09-11T12:53:33.670 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "device_cache_timeout": { 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_cache_timeout", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1800", 2024-09-11T12:53:33.671 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache device inventory", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "device_enhanced_scan": { 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_enhanced_scan", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.672 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "facts_cache_timeout": { 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "facts_cache_timeout", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.673 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "seconds to cache host facts data", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "grafana_dashboards_path": { 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "grafana_dashboards_path", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "host_check_interval": { 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "host_check_interval", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to perform a host check", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "hw_monitoring": { 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hw_monitoring", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.676 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "inventory_list_all": { 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "inventory_list_all", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.677 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.678 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.679 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "log_refresh_metadata": { 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_refresh_metadata", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.680 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.681 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.682 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-11T12:53:33.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-11T12:53:33.685 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "*", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.686 INFO:teuthology.orchestra.run.smithi078.stdout: "max_count_per_host": { 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_count_per_host", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of daemons per service per host", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.687 INFO:teuthology.orchestra.run.smithi078.stdout: "max_osd_draining_count": { 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_osd_draining_count", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "migration_current": { 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "migration_current", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "internal - do not modify", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": { 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mode", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "root", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm-package", 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: "root" 2024-09-11T12:53:33.690 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "mode for remote execution of cephadm", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "oob_default_addr": { 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "oob_default_addr", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "169.254.1.1", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.691 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "prometheus_alerts_path": { 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus_alerts_path", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-11T12:53:33.692 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_insecure": { 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_insecure", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.693 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_password": { 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_password", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.694 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_url": { 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_url", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.695 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "registry_username": { 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "registry_username", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.696 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-11T12:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.699 INFO:teuthology.orchestra.run.smithi078.stdout: "secure_monitoring_stack": { 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secure_monitoring_stack", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.700 INFO:teuthology.orchestra.run.smithi078.stdout: "service_discovery_port": { 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "service_discovery_port", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8765", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "cephadm service discovery port", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.701 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "ssh_config_file": { 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssh_config_file", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-11T12:53:33.703 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "use_agent": { 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_agent", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.704 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "use_repo_digest": { 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "use_repo_digest", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.705 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_failed_host_check": { 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_failed_host_check", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.706 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if the host check fails", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_daemons": { 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_daemons", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.707 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_on_stray_hosts": { 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_on_stray_hosts", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.708 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crash", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.709 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.710 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.711 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.714 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "retain_interval": { 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retain_interval", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "31536000", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.715 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.716 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_recent_interval": { 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_recent_interval", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.717 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1209600", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dashboard", 2024-09-11T12:53:33.718 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.719 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_HOST": { 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.720 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.721 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_ENABLED": { 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_ENABLED", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.722 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.723 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "ENABLE_BROWSABLE_API": { 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.724 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.725 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-11T12:53:33.726 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-11T12:53:33.727 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.729 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.732 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.733 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_PASSWORD": { 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.734 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.735 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_URL": { 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_URL", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.736 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_API_USERNAME": { 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.738 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.739 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-11T12:53:33.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-11T12:53:33.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-11T12:53:33.742 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "[]", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "MULTICLUSTER_CONFIG": { 2024-09-11T12:53:33.743 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "{}", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.744 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_HOST": { 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.746 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.747 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.748 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.749 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-11T12:53:33.750 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-11T12:53:33.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-11T12:53:33.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_ENABLED": { 2024-09-11T12:53:33.753 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-11T12:53:33.754 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-11T12:53:33.755 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-11T12:53:33.756 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-11T12:53:33.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "45", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ACCESS_KEY": { 2024-09-11T12:53:33.758 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-11T12:53:33.759 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "admin", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SECRET_KEY": { 2024-09-11T12:53:33.760 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.761 INFO:teuthology.orchestra.run.smithi078.stdout: "RGW_API_SSL_VERIFY": { 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "UNSAFE_TLS_v1_2": { 2024-09-11T12:53:33.762 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-11T12:53:33.763 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.766 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "cross_origin_url": { 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cross_origin_url", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.767 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "crt_file": { 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "crt_file", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.768 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "debug": { 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "debug", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Enable/disable debug options", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "jwt_token_ttl": { 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "jwt_token_ttl", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "28800", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.770 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.771 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.772 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.774 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.775 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.776 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "motd": { 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "motd", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.777 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "The message of the day", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect_resolve_ip_addr": { 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "redirect_resolve_ip_addr", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.778 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.779 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.780 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8080", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.781 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.782 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl_server_port": { 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl_server_port", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.783 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8443", 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T12:53:33.784 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "redirect", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "redirect" 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "url_prefix": { 2024-09-11T12:53:33.786 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url_prefix", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_monitoring": { 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_monitoring", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.788 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "monitor device health metrics", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.789 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.790 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.791 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.792 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.793 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "mark_out_threshold": { 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mark_out_threshold", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "2419200", 2024-09-11T12:53:33.794 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_name": { 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pool_name", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "device_health_metrics", 2024-09-11T12:53:33.795 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "retention_period": { 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "retention_period", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.796 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15552000", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long to retain device health metrics", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_frequency": { 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_frequency", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "self_heal": { 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "self_heal", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.798 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.799 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how frequently to wake up and check device health", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.800 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "warn_threshold": { 2024-09-11T12:53:33.801 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "warn_threshold", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "7257600", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.802 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "diskprediction_local", 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.803 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.804 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.805 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.806 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.807 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "predict_interval": { 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predict_interval", 2024-09-11T12:53:33.808 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "86400", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "predictor_model": { 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "predictor_model", 2024-09-11T12:53:33.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "prophetstor", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.810 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "600", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.811 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.812 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "influx", 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": false, 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "influxdb python module not found", 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "batch_size": { 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "batch_size", 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.813 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5000", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "database": { 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "database", 2024-09-11T12:53:33.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "ceph", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "hostname": { 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "hostname", 2024-09-11T12:53:33.815 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server hostname", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:33.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "30", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "5", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.818 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.819 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.820 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "password": { 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "password", 2024-09-11T12:53:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "password of InfluxDB server user", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "port": { 2024-09-11T12:53:33.823 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "port", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "8086", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "InfluxDB server port", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.825 INFO:teuthology.orchestra.run.smithi078.stdout: "ssl": { 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "ssl", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "false", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "threads": { 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threads", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "32", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.827 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "username": { 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "username", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "username of InfluxDB server user", 2024-09-11T12:53:33.828 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "verify_ssl": { 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "verify_ssl", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "true", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-11T12:53:33.829 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "insights", 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.830 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.831 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.833 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.835 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "iostat", 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.836 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.837 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.838 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.839 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "localpool", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "failure_domain": { 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "failure_domain", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "host", 2024-09-11T12:53:33.843 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "failure domain for any created local pool", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.844 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.845 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.847 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "min_size": { 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "min_size", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.849 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default min_size for any created local pool", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "num_rep": { 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "num_rep", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.850 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default replica count for any created local pool", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_num": { 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_num", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "128", 2024-09-11T12:53:33.851 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "default pg_num for any created local pool", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "prefix": { 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prefix", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.852 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "name prefix for any created local pool", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.853 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "subtree": { 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subtree", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "rack", 2024-09-11T12:53:33.854 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirroring", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.855 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.856 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.857 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.858 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.859 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.860 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.861 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "nfs", 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.862 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.863 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.864 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.865 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.866 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.867 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.868 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "fail_fs": { 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fail_fs", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-11T12:53:33.869 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.870 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.871 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.872 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.873 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.874 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator": { 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "orchestrator", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "cephadm", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "rook", 2024-09-11T12:53:33.875 INFO:teuthology.orchestra.run.smithi078.stdout: "test_orchestrator" 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Orchestrator backend", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.876 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_perf_query", 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.877 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.878 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.879 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.880 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.881 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.883 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.884 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "osd_support", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.885 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.886 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.887 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.888 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.889 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.890 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "pg_autoscaler", 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.891 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.892 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.893 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.894 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.895 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.897 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.898 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "threshold": { 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "threshold", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "3.0", 2024-09-11T12:53:33.899 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1.0", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "scaling threshold", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "progress", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.900 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_pg_recovery_event": { 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_pg_recovery_event", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T12:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T12:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.905 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.906 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.907 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.908 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "max_completed_events": { 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_completed_events", 2024-09-11T12:53:33.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "50", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "number of past completed events to remember", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "sleep_interval": { 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sleep_interval", 2024-09-11T12:53:33.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "how long the module is going to sleep", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.911 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.912 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "prometheus", 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "cache": { 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cache", 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "exclude_perf_counters": { 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "exclude_perf_counters", 2024-09-11T12:53:33.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.915 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.916 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.917 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.918 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.919 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools": { 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.921 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "300", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "scrape_interval": { 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "scrape_interval", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15.0", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "::", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-11T12:53:33.924 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "9283", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-11T12:53:33.925 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.926 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "stale_cache_strategy": { 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stale_cache_strategy", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "log", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_behaviour": { 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_behaviour", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "default", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "default", 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: "error" 2024-09-11T12:53:33.928 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "standby_error_status_code": { 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "standby_error_status_code", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "500", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "400", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "599", 2024-09-11T12:53:33.929 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.930 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.931 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.932 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.933 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_snap_create": { 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_snap_create", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.936 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "mirror_snapshot_schedule": { 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "mirror_snapshot_schedule", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.937 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.938 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "trash_purge_schedule": { 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "trash_purge_schedule", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.939 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "restful", 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.940 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enable_auth": { 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enable_auth", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.941 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "key_file": { 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "key_file", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.943 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.945 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.946 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.947 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "server_addr": { 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_addr", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.948 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "server_port": { 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "server_port", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.949 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.950 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rgw", 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.951 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.952 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.954 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.956 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "secondary_zone_period_retry_limit": { 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "5", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-11T12:53:33.957 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.958 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "selftest", 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.960 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.961 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.962 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.964 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "roption1": { 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption1", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "roption2": { 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "roption2", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "xyz", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.966 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption1": { 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption1", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption2": { 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption2", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption3": { 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption3", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "float", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.969 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption4": { 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption4", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption5": { 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption5", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.971 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption6": { 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption6", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.972 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "rwoption7": { 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rwoption7", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "1", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "42", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.973 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "testkey": { 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testkey", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "testlkey": { 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testlkey", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "testnewline": { 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "testnewline", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snap_schedule", 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "allow_m_granularity": { 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "allow_m_granularity", 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "allow minute scheduled snapshots", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "dump_on_update": { 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "dump_on_update", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "dump database to debug log on update", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.981 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.983 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.985 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.986 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "stats", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.987 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.988 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.989 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "status", 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.994 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.995 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.996 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.997 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:33.998 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:33.999 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telegraf", 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "address": { 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "address", 2024-09-11T12:53:34.000 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:34.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "15", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:34.002 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.003 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.004 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.005 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.008 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "telemetry", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_basic": { 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_basic", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:34.009 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_crash": { 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_crash", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:34.010 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_device": { 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_device", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.011 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_ident": { 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_ident", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.012 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "channel_perf": { 2024-09-11T12:53:34.013 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "channel_perf", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "contact": { 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "contact", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "description": { 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "description", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "device_url": { 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "device_url", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enabled": { 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "enabled", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "24", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "8", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.019 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "last_opt_revision": { 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "last_opt_revision", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "1", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.020 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard": { 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.021 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "leaderboard_description": { 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "leaderboard_description", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.022 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.023 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.025 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.026 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.027 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.028 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "organization": { 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "organization", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.029 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "proxy": { 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "proxy", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.030 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.031 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "url": { 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "url", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-11T12:53:34.032 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "test_orchestrator", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:34.033 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.034 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.035 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.037 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.039 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:34.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.041 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.042 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.043 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:34.044 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.045 INFO:teuthology.orchestra.run.smithi078.stdout: "max_concurrent_clones": { 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "max_concurrent_clones", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "4", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.046 INFO:teuthology.orchestra.run.smithi078.stdout: "periodic_async_work": { 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "periodic_async_work", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Periodically check for async work", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.047 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_delay": { 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_delay", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot_clone_no_wait": { 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "snapshot_clone_no_wait", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "True", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.050 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "can_run": true, 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "error_string": "", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "module_options": { 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "discovery_interval": { 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "discovery_interval", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "uint", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "100", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.051 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "identifier": { 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "identifier", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.052 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "interval": { 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "interval", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "secs", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "60", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.053 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "log_level": { 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_level", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.054 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster": { 2024-09-11T12:53:34.055 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.056 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_cluster_level": { 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_cluster_level", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "info", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [ 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "critical", 2024-09-11T12:53:34.057 INFO:teuthology.orchestra.run.smithi078.stdout: "debug", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "error", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "info", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "warning" 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "log_to_file": { 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "log_to_file", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "bool", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.058 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "False", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "sqlite3_killpoint": { 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "sqlite3_killpoint", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.059 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "dev", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 1, 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "0", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_host": { 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_host", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.060 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_port": { 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_port", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "int", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "10051", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "zabbix_sender": { 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "zabbix_sender", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "str", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "level": "advanced", 2024-09-11T12:53:34.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flags": 0, 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "min": "", 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "max": "", 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "enum_allowed": [], 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "desc": "", 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "long_desc": "", 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "tags": [], 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "see_also": [] 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "services": {}, 2024-09-11T12:53:34.063 INFO:teuthology.orchestra.run.smithi078.stdout: "always_on_modules": { 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "octopus": [ 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "pacific": [ 2024-09-11T12:53:34.064 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "quincy": [ 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T12:53:34.065 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "reef": [ 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T12:53:34.066 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "squid": [ 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "balancer", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "crash", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "devicehealth", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "orchestrator", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "pg_autoscaler", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "progress", 2024-09-11T12:53:34.067 INFO:teuthology.orchestra.run.smithi078.stdout: "rbd_support", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "status", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "telemetry", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "volumes" 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "last_failure_osd_epoch": 0, 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "active_clients": [ 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "devicehealth", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.068 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3934951986 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "libcephsqlite", 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2095507505 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.069 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "rbd_support", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 8749197 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.070 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4008390568 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3248865947 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.071 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3311569373 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.072 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 4141073556 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 3924405138 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 2014298080 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "volumes", 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: "addrvec": [ 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "v2", 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: "addr": "172.21.15.157:0", 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: "nonce": 1789189662 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:34.075 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 msgr2=0x7f7db4073ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 0x7f7db4073ea0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7dd40ce4d0 tx=0x7f7dc000b390 comp rx=0 tx=0).stop 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 msgr2=0x7f7dd4140b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f7dc400c3c0 tx=0x7f7dc400c880 comp rx=0 tx=0).stop 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 shutdown_connections 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f7db40719e0 0x7f7db4073ea0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7dd415ba10 0x7f7dd415a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7dd40af6c0 0x7f7dd415a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 --2- 172.21.15.78:0/3359231590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7dd40759d0 0x7f7dd4140b10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 >> 172.21.15.78:0/3359231590 conn(0x7f7dd4073d80 msgr2=0x7f7dd40b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 shutdown_connections 2024-09-11T12:53:34.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:33.140+0000 7f7dd3200640 1 -- 172.21.15.78:0/3359231590 wait complete. 2024-09-11T12:53:34.082 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-11T12:53:34.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- 172.21.15.78:0/1878770017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc074240 msgr2=0x7f0fbc15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1878770017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc074240 0x7f0fbc15ba80 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0fac009bc0 tx=0x7f0fac01cba0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- 172.21.15.78:0/1878770017 shutdown_connections 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1878770017 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc074240 0x7f0fbc15ba80 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- 172.21.15.78:0/1878770017 >> 172.21.15.78:0/1878770017 conn(0x7f0fbc06ca80 msgr2=0x7f0fbc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- 172.21.15.78:0/1878770017 shutdown_connections 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- 172.21.15.78:0/1878770017 wait complete. 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 Processor -- start 2024-09-11T12:53:34.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- start start 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 0x7f0fbc14a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 0x7f0fbc0b0280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc1692f0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0fbc169170 con 0x7f0fbc074240 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.228+0000 7f0fc1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0fbc169470 con 0x7f0fbc070360 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51374/0 (socket says 172.21.15.78:51374) 2024-09-11T12:53:34.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 -- 172.21.15.78:0/1581440890 learned_addr learned my addr 172.21.15.78:0/1581440890 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb9400640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 0x7f0fbc0b0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb9e00640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 0x7f0fbc14a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 -- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 msgr2=0x7f0fbc14a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 0x7f0fbc14a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 -- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 msgr2=0x7f0fbc0b0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 0x7f0fbc0b0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 -- 172.21.15.78:0/1581440890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fbc14c2d0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb9e00640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 0x7f0fbc14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb9400640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 0x7f0fbc0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fba800640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c3c0 tx=0x7f0fb400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4011790 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fb4017070 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fbc14c5b0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4011cc0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fbc14cb70 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fbc15ba60 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.232+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 11) ==== 95289+0+0 (secure 0 0 0) 0x7f0fb401c460 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.236+0000 7f0fb2000640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 0x7f0f9c073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.236+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5825+0+0 (secure 0 0 0) 0x7f0fb40923b0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.236+0000 7f0fb9e00640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 0x7f0f9c073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.236+0000 7f0fb9e00640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 0x7f0f9c073f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0af000 tx=0x7f0fac01b3e0 comp rx=0 tx=0).ready entity=mgr.4105 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.236+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0fb405dc10 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.444+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7f0fbc0cc520 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.448+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v12) ==== 68+0+0 (secure 0 0 0) 0x7f0fb405d510 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.448+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mgrmap(e 12) ==== 47504+0+0 (secure 0 0 0) 0x7f0fb40597a0 con 0x7f0fbc15e5b0 2024-09-11T12:53:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fb2000640 1 -- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 msgr2=0x7f0f9c073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fb2000640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 0x7f0f9c073f30 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f0fac0af000 tx=0x7f0fac01b3e0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 msgr2=0x7f0fbc0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0fb400c3c0 tx=0x7f0fb400c880 comp rx=0 tx=0).stop 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 shutdown_connections 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:6800/855997071,v1:172.21.15.157:6801/855997071] conn(0x7f0f9c071a70 0x7f0f9c073f30 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0fbc15e5b0 0x7f0fbc0b0bc0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0fbc074240 0x7f0fbc0b0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 --2- 172.21.15.78:0/1581440890 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0fbc070360 0x7f0fbc14a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 >> 172.21.15.78:0/1581440890 conn(0x7f0fbc06ca80 msgr2=0x7f0fbc0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 shutdown_connections 2024-09-11T12:53:34.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.452+0000 7f0fc1200640 1 -- 172.21.15.78:0/1581440890 wait complete. 2024-09-11T12:53:34.489 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:34.565 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.565 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.566 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.566 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.566 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.566 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.566 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.567 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.567 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.567 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:53:34.564+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:53:34.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- 172.21.15.78:0/1098461950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa628003a60 msgr2=0x7fa628003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 --2- 172.21.15.78:0/1098461950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa628003a60 0x7fa628003e80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fa634006fa0 tx=0x7fa63401cf00 comp rx=0 tx=0).stop 2024-09-11T12:53:34.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa649c00640 1 -- 172.21.15.78:0/1098461950 reap_dead start 2024-09-11T12:53:34.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- 172.21.15.78:0/1098461950 shutdown_connections 2024-09-11T12:53:34.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- 172.21.15.78:0/1098461950 >> 172.21.15.78:0/1098461950 conn(0x7fa64406ca80 msgr2=0x7fa6440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:34.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- 172.21.15.78:0/1098461950 shutdown_connections 2024-09-11T12:53:34.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- 172.21.15.78:0/1098461950 wait complete. 2024-09-11T12:53:34.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 Processor -- start 2024-09-11T12:53:34.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 -- start start 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa628003a60 0x7fa6441f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.668+0000 7fa64bc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6440ae7d0 0x7fa64414e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440cd150 con 0x7fa6441f57c0 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa6440ccfd0 con 0x7fa628003a60 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa6440cd2d0 con 0x7fa6440ae7d0 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51480/0 (socket says 172.21.15.78:51480) 2024-09-11T12:53:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 -- 172.21.15.78:0/2394664493 learned_addr learned my addr 172.21.15.78:0/2394664493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa643e00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6440ae7d0 0x7fa64414e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649200640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa628003a60 0x7fa6441f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 -- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6440ae7d0 msgr2=0x7fa64414e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6440ae7d0 0x7fa64414e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 -- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa628003a60 msgr2=0x7fa6441f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa628003a60 0x7fa6441f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 -- 172.21.15.78:0/2394664493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6441f8d10 con 0x7fa6441f57c0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa649c00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b2c30 tx=0x7fa63c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00cc60 con 0x7fa6441f57c0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa63c003980 con 0x7fa6441f57c0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6441f8ff0 con 0x7fa6441f57c0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003dd0 con 0x7fa6441f57c0 2024-09-11T12:53:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6441f52a0 con 0x7fa6441f57c0 2024-09-11T12:53:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6441f54a0 con 0x7fa6441f57c0 2024-09-11T12:53:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 12) ==== 47504+0+0 (secure 0 0 0) 0x7fa63c01b020 con 0x7fa6441f57c0 2024-09-11T12:53:34.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.672+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7fa63c04ea70 con 0x7fa6441f57c0 2024-09-11T12:53:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.676+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa6441f54a0 con 0x7fa6441f57c0 2024-09-11T12:53:34.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.876+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa64415c390 con 0x7fa6441f57c0 2024-09-11T12:53:34.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.876+0000 7fa640c00640 1 -- 172.21.15.78:0/2394664493 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7fa63c022020 con 0x7fa6441f57c0 2024-09-11T12:53:34.879 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:34.880 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:34.880 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:34.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 msgr2=0x7fa6441f5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fa6440b2c30 tx=0x7fa63c00c2d0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 shutdown_connections 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa6441f57c0 0x7fa6441f5c20 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa6440ae7d0 0x7fa64414e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 --2- 172.21.15.78:0/2394664493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa628003a60 0x7fa6441f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 >> 172.21.15.78:0/2394664493 conn(0x7fa64406ca80 msgr2=0x7fa64406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 shutdown_connections 2024-09-11T12:53:34.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:34.880+0000 7fa64bc00640 1 -- 172.21.15.78:0/2394664493 wait complete. 2024-09-11T12:53:34.900 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json session ls 2024-09-11T12:53:35.063 INFO:teuthology.orchestra.run.smithi157.stdout:[{"id":6109,"entity":{"name":{"type":"client","num":6109},"addr":{"type":"any","addr":"172.21.15.78:0","nonce":1174449660}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.47599546999999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978201064679288,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6109 172.21.15.78:0/1174449660","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"y","hostname":"smithi078","pid":"28832","root":"/"}},{"id":6375,"entity":{"name":{"type":"client","num":6375},"addr":{"type":"any","addr":"172.21.15.78:0","nonce":1225395661}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.47599546999999998,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9978201064679288,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6375 172.21.15.78:0/1225395661","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d","ceph_version":"ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)","entity_id":"y","hostname":"smithi078","pid":"28832","root":"/"}}] 2024-09-11T12:53:35.063 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi078", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28832", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.78:0", "nonce": 1174449660, "type": "any" }, "name": { "num": 6109, "type": "client" } }, "id": 6109, "inst": "client.6109 172.21.15.78:0/1174449660", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978201064679288 }, "state": "open", "uptime": 0.47599547 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d", "ceph_version": "ceph version 19.3.0-4207-g13fc0377 (13fc0377f1c0ab47b0d88ac21e50c2cc49cb647d) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi078", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "28832", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.78:0", "nonce": 1225395661, "type": "any" }, "name": { "num": 6375, "type": "client" } }, "id": 6375, "inst": "client.6375 172.21.15.78:0/1225395661", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9978201064679288 }, "state": "open", "uptime": 0.47599547 } ] 2024-09-11T12:53:35.064 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-11T12:53:35.064 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:53:35.064 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:35.142 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:35.231 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:53:35.231 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:35.232 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:35.232 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:35.240 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:35.241 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:35.241 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:35.241 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:35.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:35.249 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:35.250 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:35.250 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:35.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:35.291 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:53:35.292 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:35.292 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:35.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:35.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:35.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- 172.21.15.78:0/3537958101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015c390 msgr2=0x7efc5015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 --2- 172.21.15.78:0/3537958101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015c390 0x7efc5015c770 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc40009840 tx=0x7efc4001c790 comp rx=0 tx=0).stop 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- 172.21.15.78:0/3537958101 shutdown_connections 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 --2- 172.21.15.78:0/3537958101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015c390 0x7efc5015c770 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- 172.21.15.78:0/3537958101 >> 172.21.15.78:0/3537958101 conn(0x7efc500b19a0 msgr2=0x7efc50070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- 172.21.15.78:0/3537958101 shutdown_connections 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- 172.21.15.78:0/3537958101 wait complete. 2024-09-11T12:53:35.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 Processor -- start 2024-09-11T12:53:35.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- start start 2024-09-11T12:53:35.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 0x7efc5006dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015ccb0 0x7efc50154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc50169b80 con 0x7efc500b34d0 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc50169a00 con 0x7efc5015ccb0 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc50169d00 con 0x7efc5015c390 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51492/0 (socket says 172.21.15.78:51492) 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 -- 172.21.15.78:0/515736233 learned_addr learned my addr 172.21.15.78:0/515736233 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc55200640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015ccb0 0x7efc50154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4f400640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 0x7efc5006dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 -- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 msgr2=0x7efc5006dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 0x7efc5006dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 -- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015ccb0 msgr2=0x7efc50154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015ccb0 0x7efc50154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4fe00640 1 -- 172.21.15.78:0/515736233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5006e450 con 0x7efc500b34d0 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.484+0000 7efc4f400640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 0x7efc5006dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:35.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc4fe00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efc40098170 tx=0x7efc400a6760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc400be070 con 0x7efc500b34d0 2024-09-11T12:53:35.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc40005250 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5006e790 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40005620 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5006efd0 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5015c770 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 47631+0+0 (secure 0 0 0) 0x7efc400046d0 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 0x7efc3003d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7efc400e90d0 con 0x7efc500b34d0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc4f400640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 0x7efc3003d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.488+0000 7efc4f400640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 0x7efc3003d0b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc50151990 tx=0x7efc3c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:35.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.492+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efc5006e790 con 0x7efc500b34d0 2024-09-11T12:53:35.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.692+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc50074e00 con 0x7efc500b34d0 2024-09-11T12:53:35.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc47e00640 1 -- 172.21.15.78:0/515736233 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7efc400ac030 con 0x7efc500b34d0 2024-09-11T12:53:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:35.697 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:35.697 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:35.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 msgr2=0x7efc3003d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 0x7efc3003d0b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efc50151990 tx=0x7efc3c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 msgr2=0x7efc5014b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7efc40098170 tx=0x7efc400a6760 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 shutdown_connections 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc3003abf0 0x7efc3003d0b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc5015ccb0 0x7efc50154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc5015c390 0x7efc5006dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 --2- 172.21.15.78:0/515736233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc500b34d0 0x7efc5014b1b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 >> 172.21.15.78:0/515736233 conn(0x7efc500b19a0 msgr2=0x7efc5006d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 shutdown_connections 2024-09-11T12:53:35.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.696+0000 7efc55c00640 1 -- 172.21.15.78:0/515736233 wait complete. 2024-09-11T12:53:35.717 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:35.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 -- 172.21.15.78:0/759402612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab58003c20 msgr2=0x7fab58004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 --2- 172.21.15.78:0/759402612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab58003c20 0x7fab58004020 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fab64009ef0 tx=0x7fab6401d5b0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab79800640 1 -- 172.21.15.78:0/759402612 reap_dead start 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 -- 172.21.15.78:0/759402612 shutdown_connections 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 -- 172.21.15.78:0/759402612 >> 172.21.15.78:0/759402612 conn(0x7fab740746c0 msgr2=0x7fab740ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 -- 172.21.15.78:0/759402612 shutdown_connections 2024-09-11T12:53:35.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.856+0000 7fab7a200640 1 -- 172.21.15.78:0/759402612 wait complete. 2024-09-11T12:53:35.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 Processor -- start 2024-09-11T12:53:35.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- start start 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab58003c20 0x7fab741388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740aa670 0x7fab741fa720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab7413ef80 con 0x7fab740aaad0 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab74141f10 con 0x7fab740aa670 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab7413f100 con 0x7fab58003c20 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51506/0 (socket says 172.21.15.78:51506) 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 -- 172.21.15.78:0/1728293045 learned_addr learned my addr 172.21.15.78:0/1728293045 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab78e00640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab58003c20 0x7fab741388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab73e00640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740aa670 0x7fab741fa720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 -- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab58003c20 msgr2=0x7fab741388a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab58003c20 0x7fab741388a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 -- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740aa670 msgr2=0x7fab741fa720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740aa670 0x7fab741fa720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 -- 172.21.15.78:0/1728293045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab740ab030 con 0x7fab740aaad0 2024-09-11T12:53:35.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab79800640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fab6c001ec0 tx=0x7fab6c00e9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:35.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c011870 con 0x7fab740aaad0 2024-09-11T12:53:35.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab6c017070 con 0x7fab740aaad0 2024-09-11T12:53:35.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6c011dc0 con 0x7fab740aaad0 2024-09-11T12:53:35.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab740ab310 con 0x7fab740aaad0 2024-09-11T12:53:35.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab740ab8d0 con 0x7fab740aaad0 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab740cebe0 con 0x7fab740aaad0 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.860+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 47631+0+0 (secure 0 0 0) 0x7fab6c01c460 con 0x7fab740aaad0 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.864+0000 7fab70c00640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 0x7fab5403d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.864+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7fab6c04f260 con 0x7fab740aaad0 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.864+0000 7fab78e00640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 0x7fab5403d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:35.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.864+0000 7fab78e00640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 0x7fab5403d0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab640ae000 tx=0x7fab6401a350 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:35.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:35.868+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fab6c019d20 con 0x7fab740aaad0 2024-09-11T12:53:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.068+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fab7413b6a0 con 0x7fab740aaad0 2024-09-11T12:53:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.068+0000 7fab70c00640 1 -- 172.21.15.78:0/1728293045 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7fab6c01cb80 con 0x7fab740aaad0 2024-09-11T12:53:36.071 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:36.071 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:36.071 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 msgr2=0x7fab5403d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 0x7fab5403d0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fab640ae000 tx=0x7fab6401a350 comp rx=0 tx=0).stop 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 msgr2=0x7fab741fac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fab6c001ec0 tx=0x7fab6c00e9a0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 shutdown_connections 2024-09-11T12:53:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab740aaad0 0x7fab741fac60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab5403abf0 0x7fab5403d0b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab740aa670 0x7fab741fa720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 --2- 172.21.15.78:0/1728293045 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab58003c20 0x7fab741388a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 >> 172.21.15.78:0/1728293045 conn(0x7fab740746c0 msgr2=0x7fab740d0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 shutdown_connections 2024-09-11T12:53:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.072+0000 7fab7a200640 1 -- 172.21.15.78:0/1728293045 wait complete. 2024-09-11T12:53:36.093 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 -- 172.21.15.78:0/3742984477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180aed20 msgr2=0x7f01180cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 --2- 172.21.15.78:0/3742984477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180aed20 0x7f01180cf280 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f01180ce550 tx=0x7f010c01c740 comp rx=0 tx=0).stop 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 -- 172.21.15.78:0/3742984477 shutdown_connections 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 --2- 172.21.15.78:0/3742984477 >> v2:172.21.15.157:3300/0 conn(0x7f011815d1e0 0x7f01180b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 --2- 172.21.15.78:0/3742984477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180aed20 0x7f01180cf280 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 -- 172.21.15.78:0/3742984477 >> 172.21.15.78:0/3742984477 conn(0x7f011806ca80 msgr2=0x7f01180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 -- 172.21.15.78:0/3742984477 shutdown_connections 2024-09-11T12:53:36.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.232+0000 7f011fe00640 1 -- 172.21.15.78:0/3742984477 wait complete. 2024-09-11T12:53:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 Processor -- start 2024-09-11T12:53:36.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- start start 2024-09-11T12:53:36.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01180aed20 0x7f011814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011815d1e0 0x7f01181fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01180cd540 con 0x7f01180cf7c0 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01180cd3c0 con 0x7f011815d1e0 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01180cd6c0 con 0x7f01180aed20 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51530/0 (socket says 172.21.15.78:51530) 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011815d1e0 0x7f01181fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 -- 172.21.15.78:0/3012778676 learned_addr learned my addr 172.21.15.78:0/3012778676 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011d400640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01180aed20 0x7f011814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 -- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01180aed20 msgr2=0x7f011814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01180aed20 0x7f011814d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 -- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011815d1e0 msgr2=0x7f01181fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011815d1e0 0x7f01181fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 -- 172.21.15.78:0/3012778676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011814dc20 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011ca00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f010400c4d0 tx=0x7f010400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104016020 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0104012920 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0104012d30 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01180b07e0 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.236+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01180b0da0 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01180cdbe0 con 0x7f01180cf7c0 2024-09-11T12:53:36.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 13) ==== 47631+0+0 (secure 0 0 0) 0x7f0104019020 con 0x7f01180cf7c0 2024-09-11T12:53:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f0115600640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 0x7f00f803d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7f010404ef10 con 0x7f01180cf7c0 2024-09-11T12:53:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f011d400640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 0x7f00f803d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.240+0000 7f011d400640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 0x7f00f803d0b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01af80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.244+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0104020020 con 0x7f01180cf7c0 2024-09-11T12:53:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.412+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f01180695e0 con 0x7f01180cf7c0 2024-09-11T12:53:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.412+0000 7f0115600640 1 -- 172.21.15.78:0/3012778676 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17739 (secure 0 0 0) 0x7f011814dc20 con 0x7f01180cf7c0 2024-09-11T12:53:36.415 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:36.415 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:34.446555+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:17.938226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:18.807993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":5}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:0/2014298080":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3924405138":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6801/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/4141073556":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6800/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/8749197":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/656148200":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3311569373":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/4008390568":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3248865947":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/2095507505":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3934951986":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/1789189662":"2024-09-12T12:53:34.446515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 msgr2=0x7f00f803d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 0x7f00f803d0b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f010c0af000 tx=0x7f010c01af80 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 msgr2=0x7f01180c5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f010400c4d0 tx=0x7f010400c990 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 shutdown_connections 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00f803abf0 0x7f00f803d0b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011815d1e0 0x7f01181fa680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01180cf7c0 0x7f01180c5da0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 --2- 172.21.15.78:0/3012778676 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01180aed20 0x7f011814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 >> 172.21.15.78:0/3012778676 conn(0x7f011806ca80 msgr2=0x7f011815d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 shutdown_connections 2024-09-11T12:53:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.416+0000 7f011fe00640 1 -- 172.21.15.78:0/3012778676 wait complete. 2024-09-11T12:53:36.437 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:53:36.437 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:36.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 -- 172.21.15.78:0/3198457031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2408152bd0 msgr2=0x7f240815ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 --2- 172.21.15.78:0/3198457031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2408152bd0 0x7f240815ca20 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f23f4009870 tx=0x7f23f401c760 comp rx=0 tx=0).stop 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 -- 172.21.15.78:0/3198457031 shutdown_connections 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 --2- 172.21.15.78:0/3198457031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2408152bd0 0x7f240815ca20 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 -- 172.21.15.78:0/3198457031 >> 172.21.15.78:0/3198457031 conn(0x7f24080b1c50 msgr2=0x7f2408070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 -- 172.21.15.78:0/3198457031 shutdown_connections 2024-09-11T12:53:36.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.588+0000 7f240fc00640 1 -- 172.21.15.78:0/3198457031 wait complete. 2024-09-11T12:53:36.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 Processor -- start 2024-09-11T12:53:36.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 -- start start 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24080b3070 0x7f240806c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 0x7f240815a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f240816a290 con 0x7f2408152bd0 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f240816a110 con 0x7f24081527f0 2024-09-11T12:53:36.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f240816a410 con 0x7f24080b3070 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f2407e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 0x7f240815a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51560/0 (socket says 172.21.15.78:51560) 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240dc00640 1 -- 172.21.15.78:0/2365763650 learned_addr learned my addr 172.21.15.78:0/2365763650 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.592+0000 7f240d200640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24080b3070 0x7f240806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 -- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24080b3070 msgr2=0x7f240806c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24080b3070 0x7f240806c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 -- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 msgr2=0x7f240815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 -- 172.21.15.78:0/2365763650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f240812e300 con 0x7f24081527f0 2024-09-11T12:53:36.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240dc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2407e00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 0x7f240815a9a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ac000 tx=0x7f23f40a5f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40bb070 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23f4004330 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f40b2040 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f240812e580 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f240812eb40 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 14) ==== 47839+0+0 (secure 0 0 0) 0x7f23f40a56f0 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f240815ca20 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 0x7f23e803d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7f23f40e8d50 con 0x7f24081527f0 2024-09-11T12:53:36.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240d200640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 0x7f23e803d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.596+0000 7f240d200640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 0x7f23e803d2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23f8001ee0 tx=0x7f23f8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.600+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f240812e300 con 0x7f24081527f0 2024-09-11T12:53:36.756 INFO:tasks.ceph.mgr.x.smithi157.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T12:53:36.757 INFO:tasks.ceph.mgr.x.smithi157.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T12:53:36.757 INFO:tasks.ceph.mgr.x.smithi157.stderr: from numpy import show_config as show_numpy_config 2024-09-11T12:53:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24080b46a0 con 0x7f24081527f0 2024-09-11T12:53:36.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f2404c00640 1 -- 172.21.15.78:0/2365763650 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f23f4016410 con 0x7f24081527f0 2024-09-11T12:53:36.802 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:36.802 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:36.802 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 msgr2=0x7f23e803d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 0x7f23e803d2f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f23f8001ee0 tx=0x7f23f8009040 comp rx=0 tx=0).stop 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 msgr2=0x7f240815a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 0x7f240815a9a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f23f40ac000 tx=0x7f23f40a5f80 comp rx=0 tx=0).stop 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 shutdown_connections 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2408152bd0 0x7f240815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f23e803ae30 0x7f23e803d2f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24081527f0 0x7f240815a9a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 --2- 172.21.15.78:0/2365763650 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24080b3070 0x7f240806c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 >> 172.21.15.78:0/2365763650 conn(0x7f24080b1c50 msgr2=0x7f240806e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 shutdown_connections 2024-09-11T12:53:36.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.800+0000 7f240fc00640 1 -- 172.21.15.78:0/2365763650 wait complete. 2024-09-11T12:53:36.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:36.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- 172.21.15.78:0/2004747289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c15cad0 msgr2=0x7f362c15cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 --2- 172.21.15.78:0/2004747289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c15cad0 0x7f362c15cf30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3618009870 tx=0x7f361801c830 comp rx=0 tx=0).stop 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- 172.21.15.78:0/2004747289 shutdown_connections 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 --2- 172.21.15.78:0/2004747289 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c15cad0 0x7f362c15cf30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- 172.21.15.78:0/2004747289 >> 172.21.15.78:0/2004747289 conn(0x7f362c06ca80 msgr2=0x7f362c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- 172.21.15.78:0/2004747289 shutdown_connections 2024-09-11T12:53:36.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- 172.21.15.78:0/2004747289 wait complete. 2024-09-11T12:53:36.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 Processor -- start 2024-09-11T12:53:36.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.964+0000 7f3634000640 1 -- start start 2024-09-11T12:53:36.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 0x7f362c153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 0x7f362c141a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 0x7f362c1505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f362c0cdc70 con 0x7f362c15cad0 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f362c0cdaf0 con 0x7f362c0aed20 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f362c0cddf0 con 0x7f362c0b4520 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3632000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 0x7f362c1505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3631600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 0x7f362c153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 0x7f362c141a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3631600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 0x7f362c153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:45966/0 (socket says 172.21.15.78:45966) 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3632000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 0x7f362c1505a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51570/0 (socket says 172.21.15.78:51570) 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3631600640 1 -- 172.21.15.78:0/1483551422 learned_addr learned my addr 172.21.15.78:0/1483551422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 -- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 msgr2=0x7f362c153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 0x7f362c153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 -- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 msgr2=0x7f362c1505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 0x7f362c1505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:36.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 -- 172.21.15.78:0/1483551422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f362c150ae0 con 0x7f362c0b4520 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3630c00640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 0x7f362c141a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36180ac000 tx=0x7f3618005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36180b2040 con 0x7f362c0b4520 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3618004330 con 0x7f362c0b4520 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618003840 con 0x7f362c0b4520 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f362c1539e0 con 0x7f362c0b4520 2024-09-11T12:53:36.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f362c150d60 con 0x7f362c0b4520 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f362c15cf30 con 0x7f362c0b4520 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 14) ==== 47839+0+0 (secure 0 0 0) 0x7f36180bd060 con 0x7f362c0b4520 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 0x7f360c03d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7f36180e8ae0 con 0x7f362c0b4520 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.968+0000 7f3631600640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 0x7f360c03d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:36.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.972+0000 7f3631600640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 0x7f360c03d2f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3620001f80 tx=0x7f3620009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:36.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:36.976+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f362c150ae0 con 0x7f362c0b4520 2024-09-11T12:53:37.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.172+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f362c14b0d0 con 0x7f362c0b4520 2024-09-11T12:53:37.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.172+0000 7f3629600640 1 -- 172.21.15.78:0/1483551422 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5228 (secure 0 0 0) 0x7f3618016450 con 0x7f362c0b4520 2024-09-11T12:53:37.175 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:37.176 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":47,"btime":"2024-09-11T12:53:29:505415+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:19.846929+0000","modified":"2024-09-11T12:53:22.866865+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6141},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6141":{"gid":6141,"name":"d","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6837/2122140049","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6835","nonce":2122140049},{"type":"v1","addr":"172.21.15.157:6837","nonce":2122140049}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6141,"qdb_cluster":[6141]},"id":5}]} 2024-09-11T12:53:37.176 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 47 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 msgr2=0x7f360c03d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 0x7f360c03d2f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3620001f80 tx=0x7f3620009040 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 msgr2=0x7f362c141a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 0x7f362c141a60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f36180ac000 tx=0x7f3618005d20 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 shutdown_connections 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f362c15cad0 0x7f362c1505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f362c0b4520 0x7f362c141a60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f360c03ae30 0x7f360c03d2f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 --2- 172.21.15.78:0/1483551422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f362c0aed20 0x7f362c153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 >> 172.21.15.78:0/1483551422 conn(0x7f362c06ca80 msgr2=0x7f362c0b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:37.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 shutdown_connections 2024-09-11T12:53:37.179 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.176+0000 7f3634000640 1 -- 172.21.15.78:0/1483551422 wait complete. 2024-09-11T12:53:37.197 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 -- 172.21.15.78:0/2944325610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a40b14b0 msgr2=0x7f94a415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 --2- 172.21.15.78:0/2944325610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a40b14b0 0x7f94a415c100 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f94a40ce790 tx=0x7f949401c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 -- 172.21.15.78:0/2944325610 shutdown_connections 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 --2- 172.21.15.78:0/2944325610 >> v2:172.21.15.157:3300/0 conn(0x7f94a40d0bf0 0x7f94a40d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 --2- 172.21.15.78:0/2944325610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a40b14b0 0x7f94a415c100 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 -- 172.21.15.78:0/2944325610 >> 172.21.15.78:0/2944325610 conn(0x7f94a4074b90 msgr2=0x7f94a40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 -- 172.21.15.78:0/2944325610 shutdown_connections 2024-09-11T12:53:37.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.336+0000 7f94aac00640 1 -- 172.21.15.78:0/2944325610 wait complete. 2024-09-11T12:53:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 Processor -- start 2024-09-11T12:53:37.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- start start 2024-09-11T12:53:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94a40b14b0 0x7f94a40b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 0x7f94a415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4168e50 con 0x7f94a406fdf0 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94a4168cd0 con 0x7f94a40d0bf0 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94a4168fd0 con 0x7f94a40b14b0 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51574/0 (socket says 172.21.15.78:51574) 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 -- 172.21.15.78:0/1315684293 learned_addr learned my addr 172.21.15.78:0/1315684293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a8c00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 0x7f94a415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3400640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94a40b14b0 0x7f94a40b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 -- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94a40b14b0 msgr2=0x7f94a40b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94a40b14b0 0x7f94a40b3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 -- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 msgr2=0x7f94a415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 0x7f94a415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 -- 172.21.15.78:0/1315684293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a41fb2d0 con 0x7f94a406fdf0 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a8c00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 0x7f94a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94a3e00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f94940194d0 tx=0x7f94940a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:37.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94940bc070 con 0x7f94a406fdf0 2024-09-11T12:53:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9494004470 con 0x7f94a406fdf0 2024-09-11T12:53:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94940b2040 con 0x7f94a406fdf0 2024-09-11T12:53:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a41fb5b0 con 0x7f94a406fdf0 2024-09-11T12:53:37.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a41466f0 con 0x7f94a406fdf0 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a415c100 con 0x7f94a406fdf0 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.340+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 14) ==== 47839+0+0 (secure 0 0 0) 0x7f9494005290 con 0x7f94a406fdf0 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.344+0000 7f949be00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 0x7f948403d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.344+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7f94940e8b40 con 0x7f94a406fdf0 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.344+0000 7f94a3400640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 0x7f948403d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.344+0000 7f94a3400640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 0x7f948403d320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94a4153fc0 tx=0x7f9490009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:37.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.348+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f94940164b0 con 0x7f94a406fdf0 2024-09-11T12:53:37.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94a41549c0 con 0x7f94a406fdf0 2024-09-11T12:53:37.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f949be00640 1 -- 172.21.15.78:0/1315684293 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+17739 (secure 0 0 0) 0x7f94940a9e30 con 0x7f94a406fdf0 2024-09-11T12:53:37.517 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:37.518 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:34.446555+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:17.938226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:18.807993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.4","mappings":[{"from":6,"to":0}]},{"pgid":"11.10","mappings":[{"from":6,"to":1}]},{"pgid":"11.1e","mappings":[{"from":6,"to":5}]},{"pgid":"11.26","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:0/2014298080":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3924405138":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6801/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/4141073556":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6800/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/8749197":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/656148200":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3311569373":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/4008390568":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3248865947":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/2095507505":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3934951986":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/1789189662":"2024-09-12T12:53:34.446515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 msgr2=0x7f948403d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 0x7f948403d320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f94a4153fc0 tx=0x7f9490009040 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 msgr2=0x7f94a41461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f94940194d0 tx=0x7f94940a5600 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 shutdown_connections 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f948403ae60 0x7f948403d320 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94a40d0bf0 0x7f94a415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94a40b14b0 0x7f94a40b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 --2- 172.21.15.78:0/1315684293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94a406fdf0 0x7f94a41461b0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.516+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 >> 172.21.15.78:0/1315684293 conn(0x7f94a4074b90 msgr2=0x7f94a414fbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:37.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.520+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 shutdown_connections 2024-09-11T12:53:37.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.520+0000 7f94aac00640 1 -- 172.21.15.78:0/1315684293 wait complete. 2024-09-11T12:53:37.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:53:37.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 -- 172.21.15.78:0/3682370548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 msgr2=0x7fc49415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 --2- 172.21.15.78:0/3682370548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc49415ba80 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc4940b2f70 tx=0x7fc48401c890 comp rx=0 tx=0).stop 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 -- 172.21.15.78:0/3682370548 shutdown_connections 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 --2- 172.21.15.78:0/3682370548 >> v2:172.21.15.157:3300/0 conn(0x7fc49415e5b0 0x7fc49415ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 --2- 172.21.15.78:0/3682370548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc49415ba80 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 -- 172.21.15.78:0/3682370548 >> 172.21.15.78:0/3682370548 conn(0x7fc49406ca80 msgr2=0x7fc4940b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 -- 172.21.15.78:0/3682370548 shutdown_connections 2024-09-11T12:53:37.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.680+0000 7fc49b800640 1 -- 172.21.15.78:0/3682370548 wait complete. 2024-09-11T12:53:37.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 Processor -- start 2024-09-11T12:53:37.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- start start 2024-09-11T12:53:37.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49406fc50 0x7fc49414be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49415e5b0 0x7fc494159a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc494165730 con 0x7fc494074240 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc494072860 con 0x7fc49406fc50 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4941658b0 con 0x7fc49415e5b0 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51590/0 (socket says 172.21.15.78:51590) 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 -- 172.21.15.78:0/110343131 learned_addr learned my addr 172.21.15.78:0/110343131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc498e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49406fc50 0x7fc49414be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc499800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49415e5b0 0x7fc494159a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 -- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49415e5b0 msgr2=0x7fc494159a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49415e5b0 0x7fc494159a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 -- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49406fc50 msgr2=0x7fc49414be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49406fc50 0x7fc49414be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 -- 172.21.15.78:0/110343131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc494154890 con 0x7fc494074240 2024-09-11T12:53:37.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc493e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc4800058d0 tx=0x7fc48000dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc480009d20 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc49414d180 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800106c0 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc494154b10 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49415ba60 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.684+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 14) ==== 47839+0+0 (secure 0 0 0) 0x7fc48001a070 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.688+0000 7fc490c00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 0x7fc47403d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.688+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6384+0+0 (secure 0 0 0) 0x7fc48004f6d0 con 0x7fc494074240 2024-09-11T12:53:37.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.688+0000 7fc498e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 0x7fc47403d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:37.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.688+0000 7fc498e00640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 0x7fc47403d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc48401b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:37.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.692+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc494154890 con 0x7fc494074240 2024-09-11T12:53:37.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.892+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc49414ddb0 con 0x7fc494074240 2024-09-11T12:53:37.892 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:53:37.892+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:53:37.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc490c00640 1 -- 172.21.15.78:0/110343131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7fc480016da0 con 0x7fc494074240 2024-09-11T12:53:37.896 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 msgr2=0x7fc47403d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 0x7fc47403d2f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc4840af000 tx=0x7fc48401b0d0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 msgr2=0x7fc494152870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc4800058d0 tx=0x7fc48000dde0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 shutdown_connections 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc49415e5b0 0x7fc494159a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc494074240 0x7fc494152870 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc47403ae30 0x7fc47403d2f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 --2- 172.21.15.78:0/110343131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc49406fc50 0x7fc49414be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:37.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 >> 172.21.15.78:0/110343131 conn(0x7fc49406ca80 msgr2=0x7fc4940b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:37.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 shutdown_connections 2024-09-11T12:53:37.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:37.896+0000 7fc49b800640 1 -- 172.21.15.78:0/110343131 wait complete. 2024-09-11T12:53:37.913 INFO:tasks.ceph.mds.d.smithi157.stderr: -8320> 2024-09-11T12:53:22.434+0000 7f38b4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:53:37.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:53:37.963 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- 172.21.15.78:0/2268828053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 msgr2=0x7f8c5c15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 --2- 172.21.15.78:0/2268828053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c15b990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c009bf0 tx=0x7f8c4c01cb50 comp rx=0 tx=0).stop 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- 172.21.15.78:0/2268828053 shutdown_connections 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 --2- 172.21.15.78:0/2268828053 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c15b990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- 172.21.15.78:0/2268828053 >> 172.21.15.78:0/2268828053 conn(0x7f8c5c075a10 msgr2=0x7f8c5c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- 172.21.15.78:0/2268828053 shutdown_connections 2024-09-11T12:53:38.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- 172.21.15.78:0/2268828053 wait complete. 2024-09-11T12:53:38.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 Processor -- start 2024-09-11T12:53:38.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- start start 2024-09-11T12:53:38.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:38.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c0af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:38.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c5c0723d0 0x7f8c5c15bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:38.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c1513a0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8c5c151220 con 0x7f8c5c0723d0 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c62400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8c5c151520 con 0x7f8c5c071ff0 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51592/0 (socket says 172.21.15.78:51592) 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 -- 172.21.15.78:0/3180389528 learned_addr learned my addr 172.21.15.78:0/3180389528 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5aa00640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c0af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5be00640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c5c0723d0 0x7f8c5c15bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 -- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 msgr2=0x7f8c5c0af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c0af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 -- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c5c0723d0 msgr2=0x7f8c5c15bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c5c0723d0 0x7f8c5c15bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.064+0000 7f8c5b400640 1 -- 172.21.15.78:0/3180389528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c5c0b0760 con 0x7f8c5c06fd40 2024-09-11T12:53:38.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c5b400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c019760 tx=0x7f8c4c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0b2040 con 0x7f8c5c06fd40 2024-09-11T12:53:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c004040 con 0x7f8c5c06fd40 2024-09-11T12:53:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c004430 con 0x7f8c5c06fd40 2024-09-11T12:53:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c5c0b0a40 con 0x7f8c5c06fd40 2024-09-11T12:53:38.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c5c0b1000 con 0x7f8c5c06fd40 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c5c147ca0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 14) ==== 47839+0+0 (secure 0 0 0) 0x7f8c4c0bd030 con 0x7f8c5c06fd40 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 0x7f8c3c03d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6470+0+0 (secure 0 0 0) 0x7f8c4c0e97d0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c5aa00640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 0x7f8c3c03d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:38.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.068+0000 7f8c5aa00640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 0x7f8c3c03d2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c1586e0 tx=0x7f8c48001380 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:38.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.072+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8c4c0c4020 con 0x7f8c5c06fd40 2024-09-11T12:53:38.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.268+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8c5c151ac0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.552+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f8c4c0ac3a0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.900+0000 7f8c53400640 1 -- 172.21.15.78:0/3180389528 <== mon.0 v2:172.21.15.78:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7f8c4c0167d0 con 0x7f8c5c06fd40 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 msgr2=0x7f8c3c03d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 0x7f8c3c03d2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c1586e0 tx=0x7f8c48001380 comp rx=0 tx=0).stop 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 msgr2=0x7f8c5c147500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c019760 tx=0x7f8c4c005b00 comp rx=0 tx=0).stop 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 shutdown_connections 2024-09-11T12:53:38.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8c3c03ae30 0x7f8c3c03d2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8c5c0723d0 0x7f8c5c15bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8c5c071ff0 0x7f8c5c0af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 --2- 172.21.15.78:0/3180389528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8c5c06fd40 0x7f8c5c147500 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 >> 172.21.15.78:0/3180389528 conn(0x7f8c5c075a10 msgr2=0x7f8c5c0cf110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 shutdown_connections 2024-09-11T12:53:38.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:38.904+0000 7f8c62400640 1 -- 172.21.15.78:0/3180389528 wait complete. 2024-09-11T12:53:38.928 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- 172.21.15.78:0/2659497220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b1350 msgr2=0x7feb7815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 --2- 172.21.15.78:0/2659497220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b1350 0x7feb7815d060 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7feb780d0330 tx=0x7feb6401c890 comp rx=0 tx=0).stop 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- 172.21.15.78:0/2659497220 shutdown_connections 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 --2- 172.21.15.78:0/2659497220 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b1350 0x7feb7815d060 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 --2- 172.21.15.78:0/2659497220 >> v2:172.21.15.157:3300/0 conn(0x7feb780b0f70 0x7feb7815c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- 172.21.15.78:0/2659497220 >> 172.21.15.78:0/2659497220 conn(0x7feb780b1a50 msgr2=0x7feb78070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- 172.21.15.78:0/2659497220 shutdown_connections 2024-09-11T12:53:39.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- 172.21.15.78:0/2659497220 wait complete. 2024-09-11T12:53:39.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 Processor -- start 2024-09-11T12:53:39.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.076+0000 7feb77e00640 1 -- start start 2024-09-11T12:53:39.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:39.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1350 0x7feb78145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:39.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 0x7feb781465a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb78168620 con 0x7feb780b0f70 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb780b3790 con 0x7feb7815d5a0 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feb781687a0 con 0x7feb780b1350 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51620/0 (socket says 172.21.15.78:51620) 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 -- 172.21.15.78:0/2756691533 learned_addr learned my addr 172.21.15.78:0/2756691533 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76000640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1350 0x7feb78145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77400640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 0x7feb781465a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 -- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1350 msgr2=0x7feb78145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1350 0x7feb78145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 -- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 msgr2=0x7feb781465a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 0x7feb781465a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 -- 172.21.15.78:0/2756691533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb78145cc0 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77400640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 0x7feb781465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb76a00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d9b0 tx=0x7feb6800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68017070 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb68009cc0 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb78146ae0 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb68005280 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb78147040 con 0x7feb780b0f70 2024-09-11T12:53:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7815c8f0 con 0x7feb780b0f70 2024-09-11T12:53:39.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.080+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7feb68005420 con 0x7feb780b0f70 2024-09-11T12:53:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.084+0000 7feb6ea00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 0x7feb58073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.084+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6470+0+0 (secure 0 0 0) 0x7feb68092570 con 0x7feb780b0f70 2024-09-11T12:53:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.084+0000 7feb76000640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 0x7feb58073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.084+0000 7feb76000640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 0x7feb58073c60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb640af000 tx=0x7feb6401b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:39.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.088+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7feb78146ae0 con 0x7feb780b0f70 2024-09-11T12:53:39.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.252+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7feb78153ba0 con 0x7feb780b0f70 2024-09-11T12:53:39.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.904+0000 7feb6ea00640 1 -- 172.21.15.78:0/2756691533 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v74) ==== 181+0+0 (secure 0 0 0) 0x7feb680660f0 con 0x7feb780b0f70 2024-09-11T12:53:39.907 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:53:39.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 msgr2=0x7feb58073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:39.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 0x7feb58073c60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb640af000 tx=0x7feb6401b0d0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 msgr2=0x7feb78070770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7feb6800d9b0 tx=0x7feb6800de70 comp rx=0 tx=0).stop 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 shutdown_connections 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb580717a0 0x7feb58073c60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb7815d5a0 0x7feb781465a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb780b1350 0x7feb78145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 --2- 172.21.15.78:0/2756691533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb780b0f70 0x7feb78070770 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 >> 172.21.15.78:0/2756691533 conn(0x7feb780b1a50 msgr2=0x7feb7814df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 shutdown_connections 2024-09-11T12:53:39.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:39.908+0000 7feb77e00640 1 -- 172.21.15.78:0/2756691533 wait complete. 2024-09-11T12:53:39.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- 172.21.15.78:0/3629296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41940b5790 msgr2=0x7f41940723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- 172.21.15.78:0/3629296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41940b5790 0x7f41940723f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f41940cd910 tx=0x7f418401cd80 comp rx=0 tx=0).stop 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- 172.21.15.78:0/3629296802 shutdown_connections 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- 172.21.15.78:0/3629296802 >> v2:172.21.15.157:3300/0 conn(0x7f4194151340 0x7f41941517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- 172.21.15.78:0/3629296802 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41940b5790 0x7f41940723f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- 172.21.15.78:0/3629296802 >> 172.21.15.78:0/3629296802 conn(0x7f41940b3050 msgr2=0x7f41940b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- 172.21.15.78:0/3629296802 shutdown_connections 2024-09-11T12:53:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- 172.21.15.78:0/3629296802 wait complete. 2024-09-11T12:53:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 Processor -- start 2024-09-11T12:53:40.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- start start 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 0x7f4194144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 0x7f4194152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4194167ad0 con 0x7f4194151340 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4194167950 con 0x7f41940b5b70 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4199c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4194167c50 con 0x7f41940b5790 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:40.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51640/0 (socket says 172.21.15.78:51640) 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 -- 172.21.15.78:0/846521789 learned_addr learned my addr 172.21.15.78:0/846521789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4192000640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 0x7f4194152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4192a00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 0x7f4194144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 -- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 msgr2=0x7f4194144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 0x7f4194144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 -- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 msgr2=0x7f4194152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.084+0000 7f4193400640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 0x7f4194152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4193400640 1 -- 172.21.15.78:0/846521789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41941f81d0 con 0x7f4194151340 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4192a00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 0x7f4194144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4192000640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 0x7f4194152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:40.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4193400640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d660 tx=0x7f418c00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:40.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c017070 con 0x7f4194151340 2024-09-11T12:53:40.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f418c005310 con 0x7f4194151340 2024-09-11T12:53:40.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f418c009820 con 0x7f4194151340 2024-09-11T12:53:40.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41941f7ef0 con 0x7f4194151340 2024-09-11T12:53:40.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f419414afe0 con 0x7f4194151340 2024-09-11T12:53:40.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41941517d0 con 0x7f4194151340 2024-09-11T12:53:40.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f418c0099c0 con 0x7f4194151340 2024-09-11T12:53:40.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 0x7f4174073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:40.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6032+0+0 (secure 0 0 0) 0x7f418c092170 con 0x7f4194151340 2024-09-11T12:53:40.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.088+0000 7f4192a00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 0x7f4174073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:40.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.092+0000 7f4192a00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 0x7f4174073c60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41840af000 tx=0x7f418401b5c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:40.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.092+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f418c05dc90 con 0x7f4194151340 2024-09-11T12:53:40.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.260+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4194072010 con 0x7f4194151340 2024-09-11T12:53:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.912+0000 7f418aa00640 1 -- 172.21.15.78:0/846521789 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v75) ==== 169+0+0 (secure 0 0 0) 0x7f418c05d590 con 0x7f4194151340 2024-09-11T12:53:40.912 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:53:40.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.912+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 msgr2=0x7f4174073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:40.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.912+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 0x7f4174073c60 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f41840af000 tx=0x7f418401b5c0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.912+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 msgr2=0x7f4194159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.912+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f418c00d660 tx=0x7f418c00db20 comp rx=0 tx=0).stop 2024-09-11T12:53:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 shutdown_connections 2024-09-11T12:53:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4194151340 0x7f4194159a00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41740717a0 0x7f4174073c60 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41940b5b70 0x7f4194152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 --2- 172.21.15.78:0/846521789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41940b5790 0x7f4194144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:40.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 >> 172.21.15.78:0/846521789 conn(0x7f41940b3050 msgr2=0x7f419414fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:40.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 shutdown_connections 2024-09-11T12:53:40.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:40.916+0000 7f4199c00640 1 -- 172.21.15.78:0/846521789 wait complete. 2024-09-11T12:53:40.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:53:41.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- 172.21.15.78:0/1250234060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 msgr2=0x7f092815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 --2- 172.21.15.78:0/1250234060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f092815aec0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f0914009930 tx=0x7f091401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- 172.21.15.78:0/1250234060 shutdown_connections 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 --2- 172.21.15.78:0/1250234060 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f092806fc50 0x7f09280700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 --2- 172.21.15.78:0/1250234060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f092815aec0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- 172.21.15.78:0/1250234060 >> 172.21.15.78:0/1250234060 conn(0x7f092806ca80 msgr2=0x7f09280b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- 172.21.15.78:0/1250234060 shutdown_connections 2024-09-11T12:53:41.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- 172.21.15.78:0/1250234060 wait complete. 2024-09-11T12:53:41.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 Processor -- start 2024-09-11T12:53:41.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.088+0000 7f092f000640 1 -- start start 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f092806fc50 0x7f092814b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09280cf760 0x7f092814cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0928151010 con 0x7f0928074260 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f092815f290 con 0x7f092806fc50 2024-09-11T12:53:41.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0928151190 con 0x7f09280cf760 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51656/0 (socket says 172.21.15.78:51656) 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 -- 172.21.15.78:0/4230326293 learned_addr learned my addr 172.21.15.78:0/4230326293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927e00640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f092806fc50 0x7f092814b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092d000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09280cf760 0x7f092814cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 -- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09280cf760 msgr2=0x7f092814cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09280cf760 0x7f092814cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 -- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f092806fc50 msgr2=0x7f092814b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f092806fc50 0x7f092814b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 -- 172.21.15.78:0/4230326293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09281f7be0 con 0x7f0928074260 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f0927400640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f09140b0040 tx=0x7f09140a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140b2040 con 0x7f0928074260 2024-09-11T12:53:41.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09140044b0 con 0x7f0928074260 2024-09-11T12:53:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09281f7900 con 0x7f0928074260 2024-09-11T12:53:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0914005280 con 0x7f0928074260 2024-09-11T12:53:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f092814c5f0 con 0x7f0928074260 2024-09-11T12:53:41.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.092+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09280700c0 con 0x7f0928074260 2024-09-11T12:53:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.096+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f09140bd060 con 0x7f0928074260 2024-09-11T12:53:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.096+0000 7f091fe00640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 0x7f0908073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.096+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f091412b750 con 0x7f0928074260 2024-09-11T12:53:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.096+0000 7f0927e00640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 0x7f0908073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.096+0000 7f0927e00640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 0x7f0908073c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0918007fd0 tx=0x7f0918009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:41.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.100+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f09140164d0 con 0x7f0928074260 2024-09-11T12:53:41.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.272+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f09280695e0 con 0x7f0928074260 2024-09-11T12:53:41.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f091fe00640 1 -- 172.21.15.78:0/4230326293 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f09140f6f20 con 0x7f0928074260 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 msgr2=0x7f0908073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 0x7f0908073c90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0918007fd0 tx=0x7f0918009040 comp rx=0 tx=0).stop 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 msgr2=0x7f0928151660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f09140b0040 tx=0x7f09140a57f0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 shutdown_connections 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09280cf760 0x7f092814cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0928074260 0x7f0928151660 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09080717d0 0x7f0908073c90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 --2- 172.21.15.78:0/4230326293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f092806fc50 0x7f092814b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 >> 172.21.15.78:0/4230326293 conn(0x7f092806ca80 msgr2=0x7f09280731f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 shutdown_connections 2024-09-11T12:53:41.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.276+0000 7f092f000640 1 -- 172.21.15.78:0/4230326293 wait complete. 2024-09-11T12:53:41.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-11T12:53:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.436+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1988817369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 msgr2=0x7fcb98074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.436+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1988817369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb98074260 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009840 tx=0x7fcb8801c7f0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1988817369 shutdown_connections 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1988817369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb98074260 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1988817369 >> 172.21.15.78:0/1988817369 conn(0x7fcb9806ca80 msgr2=0x7fcb980b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1988817369 shutdown_connections 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1988817369 wait complete. 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 Processor -- start 2024-09-11T12:53:41.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- start start 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb980b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9815aec0 0x7fcb981433f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb981519a0 con 0x7fcb9806fc50 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb98151820 con 0x7fcb980cf760 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb98151b20 con 0x7fcb9815aec0 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51660/0 (socket says 172.21.15.78:51660) 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 -- 172.21.15.78:0/1194523729 learned_addr learned my addr 172.21.15.78:0/1194523729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95400640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb980b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb96800640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9815aec0 0x7fcb981433f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 -- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9815aec0 msgr2=0x7fcb981433f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9815aec0 0x7fcb981433f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 -- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 msgr2=0x7fcb980b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb980b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 -- 172.21.15.78:0/1194523729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb98142930 con 0x7fcb9806fc50 2024-09-11T12:53:41.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb95e00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009790 tx=0x7fcb88004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb880b2040 con 0x7fcb9806fc50 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb88005020 con 0x7fcb9806fc50 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb88005430 con 0x7fcb9806fc50 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.440+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb98147e80 con 0x7fcb9806fc50 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb98147b20 con 0x7fcb9806fc50 2024-09-11T12:53:41.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb980cc390 con 0x7fcb9806fc50 2024-09-11T12:53:41.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fcb880be060 con 0x7fcb9806fc50 2024-09-11T12:53:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb8e000640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 0x7fcb78073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fcb8812b7f0 con 0x7fcb9806fc50 2024-09-11T12:53:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb95400640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 0x7fcb78073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:41.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.444+0000 7fcb95400640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 0x7fcb78073c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb98145060 tx=0x7fcb84009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:41.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.448+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcb88016410 con 0x7fcb9806fc50 2024-09-11T12:53:41.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.684+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fcb98074240 con 0x7fcb9806fc50 2024-09-11T12:53:41.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.912+0000 7fcb8e000640 1 -- 172.21.15.78:0/1194523729 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v224) ==== 131+0+0 (secure 0 0 0) 0x7fcb880f7580 con 0x7fcb9806fc50 2024-09-11T12:53:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 msgr2=0x7fcb78073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 0x7fcb78073c90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fcb98145060 tx=0x7fcb84009040 comp rx=0 tx=0).stop 2024-09-11T12:53:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 msgr2=0x7fcb981475e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fcb88009790 tx=0x7fcb88004040 comp rx=0 tx=0).stop 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 shutdown_connections 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb9815aec0 0x7fcb981433f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb780717d0 0x7fcb78073c90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb980cf760 0x7fcb980b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 --2- 172.21.15.78:0/1194523729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb9806fc50 0x7fcb981475e0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 >> 172.21.15.78:0/1194523729 conn(0x7fcb9806ca80 msgr2=0x7fcb9814a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 shutdown_connections 2024-09-11T12:53:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:41.916+0000 7fcb9cc00640 1 -- 172.21.15.78:0/1194523729 wait complete. 2024-09-11T12:53:41.937 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-11T12:53:41.937 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-11T12:53:41.937 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:53:41.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- 172.21.15.78:0/4259369664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 msgr2=0x7fa8600b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- 172.21.15.78:0/4259369664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa8600b4970 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa858009840 tx=0x7fa85801c820 comp rx=0 tx=0).stop 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- 172.21.15.78:0/4259369664 shutdown_connections 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- 172.21.15.78:0/4259369664 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa8600b4970 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- 172.21.15.78:0/4259369664 >> v2:172.21.15.157:3300/0 conn(0x7fa86015cad0 0x7fa86015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- 172.21.15.78:0/4259369664 >> 172.21.15.78:0/4259369664 conn(0x7fa86006ca80 msgr2=0x7fa8600b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- 172.21.15.78:0/4259369664 shutdown_connections 2024-09-11T12:53:42.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- 172.21.15.78:0/4259369664 wait complete. 2024-09-11T12:53:42.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 Processor -- start 2024-09-11T12:53:42.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- start start 2024-09-11T12:53:42.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600aed20 0x7fa8601f7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa860155b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600cd250 con 0x7fa86015cad0 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8600cd0d0 con 0x7fa8600aed20 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa865e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8600cd3d0 con 0x7fa8600b4520 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa85f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa85f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51672/0 (socket says 172.21.15.78:51672) 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa85f400640 1 -- 172.21.15.78:0/102337159 learned_addr learned my addr 172.21.15.78:0/102337159 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa85ea00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600aed20 0x7fa8601f7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.080+0000 7fa85e000640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa860155b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 -- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 msgr2=0x7fa860155b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa860155b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 -- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600aed20 msgr2=0x7fa8601f7af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600aed20 0x7fa8601f7af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 -- 172.21.15.78:0/102337159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86014d1e0 con 0x7fa86015cad0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa85f400640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa85801a450 tx=0x7fa8580a55d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8580b2040 con 0x7fa86015cad0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8580043b0 con 0x7fa86015cad0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86014d4c0 con 0x7fa86015cad0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa858005260 con 0x7fa86015cad0 2024-09-11T12:53:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86014da80 con 0x7fa86015cad0 2024-09-11T12:53:42.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8600b4970 con 0x7fa86015cad0 2024-09-11T12:53:42.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa8580be060 con 0x7fa86015cad0 2024-09-11T12:53:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.084+0000 7fa857400640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 0x7fa840073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.088+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fa85812b690 con 0x7fa86015cad0 2024-09-11T12:53:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.088+0000 7fa85ea00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 0x7fa840073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.088+0000 7fa85ea00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 0x7fa840073c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8500021a0 tx=0x7fa850009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.088+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa858016410 con 0x7fa86015cad0 2024-09-11T12:53:42.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:53:42.108 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:53:42.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8600b1cc0 con 0x7fa86015cad0 2024-09-11T12:53:42.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa857400640 1 -- 172.21.15.78:0/102337159 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+13854 (secure 0 0 0) 0x7fa8580f74b0 con 0x7fa86015cad0 2024-09-11T12:53:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:42.262 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":75,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:40.908461+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:0/1789189662":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3934951986":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/2095507505":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6837/2122140049":"2024-09-12T12:53:37.894363+0000","172.21.15.157:0/3248865947":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/4008390568":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3311569373":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/656148200":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/8749197":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6800/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6835/2122140049":"2024-09-12T12:53:37.894363+0000","172.21.15.157:0/4141073556":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/3924405138":"2024-09-12T12:53:34.446515+0000","172.21.15.157:6801/855997071":"2024-09-12T12:53:34.446515+0000","172.21.15.157:0/2014298080":"2024-09-12T12:53:34.446515+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 msgr2=0x7fa840073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 0x7fa840073c60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fa8500021a0 tx=0x7fa850009040 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 msgr2=0x7fa86014dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa85801a450 tx=0x7fa8580a55d0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 shutdown_connections 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa86015cad0 0x7fa86014dfa0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b4520 0x7fa860155b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa8400717a0 0x7fa840073c60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 --2- 172.21.15.78:0/102337159 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600aed20 0x7fa8601f7af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 >> 172.21.15.78:0/102337159 conn(0x7fa86006ca80 msgr2=0x7fa86006f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:42.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 shutdown_connections 2024-09-11T12:53:42.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.260+0000 7fa865e00640 1 -- 172.21.15.78:0/102337159 wait complete. 2024-09-11T12:53:42.283 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:53:42.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:53:42.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 -- 172.21.15.78:0/792165131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe040cc1f0 msgr2=0x7fbe040b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 --2- 172.21.15.78:0/792165131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe040cc1f0 0x7fbe040b2de0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4009c80 tx=0x7fbdf401cd10 comp rx=0 tx=0).stop 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 -- 172.21.15.78:0/792165131 shutdown_connections 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 --2- 172.21.15.78:0/792165131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe0415ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 --2- 172.21.15.78:0/792165131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe040cc1f0 0x7fbe040b2de0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 -- 172.21.15.78:0/792165131 >> 172.21.15.78:0/792165131 conn(0x7fbe04071dd0 msgr2=0x7fbe040721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 -- 172.21.15.78:0/792165131 shutdown_connections 2024-09-11T12:53:42.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.424+0000 7fbe08c00640 1 -- 172.21.15.78:0/792165131 wait complete. 2024-09-11T12:53:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 Processor -- start 2024-09-11T12:53:42.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- start start 2024-09-11T12:53:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 0x7fbe04151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe04137c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04168ee0 con 0x7fbe0413f000 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe04168d60 con 0x7fbe040cc1f0 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe04169060 con 0x7fbe040cc5d0 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51688/0 (socket says 172.21.15.78:51688) 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 -- 172.21.15.78:0/612720868 learned_addr learned my addr 172.21.15.78:0/612720868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe01400640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe04137c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe01e00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 0x7fbe04151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 -- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 msgr2=0x7fbe04137c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe04137c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 -- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 msgr2=0x7fbe04151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 0x7fbe04151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 -- 172.21.15.78:0/612720868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe04151de0 con 0x7fbe0413f000 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe01400640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe04137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe01e00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 0x7fbe04151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe02800640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c3c0 tx=0x7fbdfc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc017070 con 0x7fbe0413f000 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdfc0053c0 con 0x7fbe0413f000 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe04132fb0 con 0x7fbe0413f000 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc003860 con 0x7fbe0413f000 2024-09-11T12:53:42.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.428+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe04133570 con 0x7fbe0413f000 2024-09-11T12:53:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe040b2de0 con 0x7fbe0413f000 2024-09-11T12:53:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fbdfc01a070 con 0x7fbe0413f000 2024-09-11T12:53:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbdfa000640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 0x7fbde4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fbdfc091130 con 0x7fbe0413f000 2024-09-11T12:53:42.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbe01e00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 0x7fbde4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.432+0000 7fbe01e00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 0x7fbde4073c60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40af000 tx=0x7fbdf401b550 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.436+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbdfc05cf50 con 0x7fbe0413f000 2024-09-11T12:53:42.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbe04140c90 con 0x7fbe0413f000 2024-09-11T12:53:42.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbdfa000640 1 -- 172.21.15.78:0/612720868 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) ==== 93+0+10 (secure 0 0 0) 0x7fbdfc065190 con 0x7fbe0413f000 2024-09-11T12:53:42.605 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:53:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 msgr2=0x7fbde4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 0x7fbde4073c60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbdf40af000 tx=0x7fbdf401b550 comp rx=0 tx=0).stop 2024-09-11T12:53:42.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 msgr2=0x7fbe04138190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc00c3c0 tx=0x7fbdfc00c880 comp rx=0 tx=0).stop 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 shutdown_connections 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe0413f000 0x7fbe04138190 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe040cc5d0 0x7fbe04137c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbde40717a0 0x7fbde4073c60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 --2- 172.21.15.78:0/612720868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe040cc1f0 0x7fbe04151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 >> 172.21.15.78:0/612720868 conn(0x7fbe04071dd0 msgr2=0x7fbe040cdac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 shutdown_connections 2024-09-11T12:53:42.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.604+0000 7fbe08c00640 1 -- 172.21.15.78:0/612720868 wait complete. 2024-09-11T12:53:42.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- 172.21.15.78:0/3636130635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 msgr2=0x7f9cfc0700a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- 172.21.15.78:0/3636130635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc0700a0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc0cc800 tx=0x7f9cf401c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- 172.21.15.78:0/3636130635 shutdown_connections 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- 172.21.15.78:0/3636130635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc0700a0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- 172.21.15.78:0/3636130635 >> v2:172.21.15.157:3300/0 conn(0x7f9cfc074260 0x7f9cfc15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- 172.21.15.78:0/3636130635 >> 172.21.15.78:0/3636130635 conn(0x7f9cfc06ca80 msgr2=0x7f9cfc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- 172.21.15.78:0/3636130635 shutdown_connections 2024-09-11T12:53:42.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- 172.21.15.78:0/3636130635 wait complete. 2024-09-11T12:53:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 Processor -- start 2024-09-11T12:53:42.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- start start 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 0x7f9cfc14cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 0x7f9cfc159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc151010 con 0x7f9cfc06fc50 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9cfc15f340 con 0x7f9cfc0cf760 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d04200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9cfc151190 con 0x7f9cfc074260 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d01800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d01800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51704/0 (socket says 172.21.15.78:51704) 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d01800640 1 -- 172.21.15.78:0/1531517708 learned_addr learned my addr 172.21.15.78:0/1531517708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:42.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d02200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 0x7f9cfc159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.768+0000 7f9d00e00640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 0x7f9cfc14cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 -- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 msgr2=0x7f9cfc14cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 0x7f9cfc14cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 -- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 msgr2=0x7f9cfc159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 0x7f9cfc159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 -- 172.21.15.78:0/1531517708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cfc051480 con 0x7f9cfc06fc50 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d02200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 0x7f9cfc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d00e00640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 0x7f9cfc14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:42.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d01800640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000c730 tx=0x7f9cf000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf0003b80 con 0x7f9cfc06fc50 2024-09-11T12:53:42.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf0003d40 con 0x7f9cfc06fc50 2024-09-11T12:53:42.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cfc0702b0 con 0x7f9cfc06fc50 2024-09-11T12:53:42.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000aab0 con 0x7f9cfc06fc50 2024-09-11T12:53:42.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cfc14ff20 con 0x7f9cfc06fc50 2024-09-11T12:53:42.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cfc0700a0 con 0x7f9cfc06fc50 2024-09-11T12:53:42.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f9cf000ac50 con 0x7f9cfc06fc50 2024-09-11T12:53:42.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 0x7f9cdc073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:42.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f9cf0091d80 con 0x7f9cfc06fc50 2024-09-11T12:53:42.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.772+0000 7f9d00e00640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 0x7f9cdc073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:42.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.776+0000 7f9d00e00640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 0x7f9cdc073c90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8005fd0 tx=0x7f9ce8005dc0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:42.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:42.776+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9cf005dba0 con 0x7f9cfc06fc50 2024-09-11T12:53:43.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.008+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7f9cfc06f230 con 0x7f9cfc06fc50 2024-09-11T12:53:43.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.920+0000 7f9cf9600640 1 -- 172.21.15.78:0/1531517708 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v226) ==== 149+0+0 (secure 0 0 0) 0x7f9cf005d4a0 con 0x7f9cfc06fc50 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 msgr2=0x7f9cdc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 0x7f9cdc073c90 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8005fd0 tx=0x7f9ce8005dc0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 msgr2=0x7f9cfc14f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000c730 tx=0x7f9cf000cbf0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 shutdown_connections 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9cdc0717d0 0x7f9cdc073c90 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9cfc0cf760 0x7f9cfc159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9cfc074260 0x7f9cfc14cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 --2- 172.21.15.78:0/1531517708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9cfc06fc50 0x7f9cfc14f540 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:43.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 >> 172.21.15.78:0/1531517708 conn(0x7f9cfc06ca80 msgr2=0x7f9cfc15cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 shutdown_connections 2024-09-11T12:53:43.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:43.924+0000 7f9d04200640 1 -- 172.21.15.78:0/1531517708 wait complete. 2024-09-11T12:53:43.948 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:53:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 -- 172.21.15.78:0/1971473264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 msgr2=0x7f0d4c0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 --2- 172.21.15.78:0/1971473264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c0a9d00 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0d34009900 tx=0x7f0d3401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 -- 172.21.15.78:0/1971473264 shutdown_connections 2024-09-11T12:53:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 --2- 172.21.15.78:0/1971473264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c0a9d00 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 --2- 172.21.15.78:0/1971473264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 0x7f0d4c15c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 -- 172.21.15.78:0/1971473264 >> 172.21.15.78:0/1971473264 conn(0x7f0d4c074b90 msgr2=0x7f0d4c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 -- 172.21.15.78:0/1971473264 shutdown_connections 2024-09-11T12:53:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.092+0000 7f0d51200640 1 -- 172.21.15.78:0/1971473264 wait complete. 2024-09-11T12:53:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 Processor -- start 2024-09-11T12:53:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- start start 2024-09-11T12:53:44.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 0x7f0d4c14bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c1693c0 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c169240 con 0x7f0d4c06fdf0 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4c169540 con 0x7f0d4c0b14b0 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38184/0 (socket says 172.21.15.78:38184) 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 -- 172.21.15.78:0/617531655 learned_addr learned my addr 172.21.15.78:0/617531655 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d49e00640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d43e00640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 0x7f0d4c14bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 -- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 msgr2=0x7f0d4c14bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 0x7f0d4c14bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 -- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 msgr2=0x7f0d4c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 -- 172.21.15.78:0/617531655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c14a4b0 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d4a800640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c6b0 tx=0x7f0d4400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44003b80 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d44003d40 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d44005330 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c06d920 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c14a790 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.096+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d4c0a9d00 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.100+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f0d440054d0 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.100+0000 7f0d42000640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 0x7f0d28073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.100+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f0d44091b40 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.100+0000 7f0d49e00640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 0x7f0d28073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.100+0000 7f0d49e00640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 0x7f0d28073c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009be0 tx=0x7f0d3c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.104+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0d4405d310 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0d4c152950 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d42000640 1 -- 172.21.15.78:0/617531655 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7f0d44061860 con 0x7f0d4c0d0bf0 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:53:44.273 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:53:44.274 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:53:44.275 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 msgr2=0x7f0d28073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 0x7f0d28073c60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c009be0 tx=0x7f0d3c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 msgr2=0x7f0d4c150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c6b0 tx=0x7f0d4400cb70 comp rx=0 tx=0).stop 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 shutdown_connections 2024-09-11T12:53:44.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0d4c0d0bf0 0x7f0d4c150f30 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0d4c0b14b0 0x7f0d4c14bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0d280717a0 0x7f0d28073c60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 --2- 172.21.15.78:0/617531655 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0d4c06fdf0 0x7f0d4c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 >> 172.21.15.78:0/617531655 conn(0x7f0d4c074b90 msgr2=0x7f0d4c0b23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 shutdown_connections 2024-09-11T12:53:44.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.272+0000 7f0d51200640 1 -- 172.21.15.78:0/617531655 wait complete. 2024-09-11T12:53:44.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:53:44.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- 172.21.15.78:0/3670084248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a40ce320 msgr2=0x7f59a41655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 --2- 172.21.15.78:0/3670084248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a40ce320 0x7f59a41655c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f599c009930 tx=0x7f599c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- 172.21.15.78:0/3670084248 shutdown_connections 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 --2- 172.21.15.78:0/3670084248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a40ce320 0x7f59a41655c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- 172.21.15.78:0/3670084248 >> 172.21.15.78:0/3670084248 conn(0x7f59a4071dd0 msgr2=0x7f59a40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- 172.21.15.78:0/3670084248 shutdown_connections 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- 172.21.15.78:0/3670084248 wait complete. 2024-09-11T12:53:44.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 Processor -- start 2024-09-11T12:53:44.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- start start 2024-09-11T12:53:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a40b2120 0x7f59a41360c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 0x7f59a41483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4168f40 con 0x7f59a41518b0 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f59a4168dc0 con 0x7f59a40b2120 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f59a41690c0 con 0x7f59a40ce320 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38198/0 (socket says 172.21.15.78:38198) 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 -- 172.21.15.78:0/757208628 learned_addr learned my addr 172.21.15.78:0/757208628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a2a00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a40b2120 0x7f59a41360c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a2000640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 0x7f59a41483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 -- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 msgr2=0x7f59a41483b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 0x7f59a41483b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 -- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a40b2120 msgr2=0x7f59a41360c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a40b2120 0x7f59a41360c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a3400640 1 -- 172.21.15.78:0/757208628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59a414ad60 con 0x7f59a41518b0 2024-09-11T12:53:44.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.440+0000 7f59a2000640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 0x7f59a41483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f59a3400640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f599c0b0040 tx=0x7f599c0a55c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c0b2040 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f599c0043d0 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f599c005290 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59a414b040 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59a4133e40 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59a40cdc10 con 0x7f59a41518b0 2024-09-11T12:53:44.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f599c005430 con 0x7f59a41518b0 2024-09-11T12:53:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.444+0000 7f599b400640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 0x7f5984073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.448+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f599c12bf10 con 0x7f59a41518b0 2024-09-11T12:53:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.448+0000 7f59a2a00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 0x7f5984073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:44.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.448+0000 7f59a2a00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 0x7f5984073c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5994001e40 tx=0x7f5994009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:44.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.448+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f599c0164d0 con 0x7f59a41518b0 2024-09-11T12:53:44.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.620+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f59a40716e0 con 0x7f59a41518b0 2024-09-11T12:53:44.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.624+0000 7f599b400640 1 -- 172.21.15.78:0/757208628 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f599c0f7ca0 con 0x7f59a41518b0 2024-09-11T12:53:44.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 msgr2=0x7f5984073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 0x7f5984073c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5994001e40 tx=0x7f5994009040 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 msgr2=0x7f59a414df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f599c0b0040 tx=0x7f599c0a55c0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 shutdown_connections 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f59a41518b0 0x7f59a414df30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f59a40ce320 0x7f59a41483b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f59840717d0 0x7f5984073c90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 --2- 172.21.15.78:0/757208628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f59a40b2120 0x7f59a41360c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 >> 172.21.15.78:0/757208628 conn(0x7f59a4071dd0 msgr2=0x7f59a41446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 shutdown_connections 2024-09-11T12:53:44.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:44.628+0000 7f59a9e00640 1 -- 172.21.15.78:0/757208628 wait complete. 2024-09-11T12:53:44.648 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:53:44.648 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:44.738 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:53:44.815 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:53:44.815 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:44.815 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:44.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:44.824 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:44.824 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:44.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:44.833 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:44.833 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:44.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:53:44.876 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:44.876 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:44.885 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:53:44.885 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:53:44.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- 172.21.15.78:0/4207245328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2415fa90 msgr2=0x7fdc2413ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 --2- 172.21.15.78:0/4207245328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2415fa90 0x7fdc2413ede0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c009c80 tx=0x7fdc1c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- 172.21.15.78:0/4207245328 shutdown_connections 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 --2- 172.21.15.78:0/4207245328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc2415fa90 0x7fdc2413ede0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 --2- 172.21.15.78:0/4207245328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2413e5d0 0x7fdc24141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- 172.21.15.78:0/4207245328 >> 172.21.15.78:0/4207245328 conn(0x7fdc24071cf0 msgr2=0x7fdc24072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- 172.21.15.78:0/4207245328 shutdown_connections 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- 172.21.15.78:0/4207245328 wait complete. 2024-09-11T12:53:45.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 Processor -- start 2024-09-11T12:53:45.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.060+0000 7fdc2b000640 1 -- start start 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2413e5d0 0x7fdc24140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 0x7fdc2413aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24168930 con 0x7fdc240a9460 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241687b0 con 0x7fdc2413e5d0 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc24168ab0 con 0x7fdc2415fa90 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38226/0 (socket says 172.21.15.78:38226) 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 -- 172.21.15.78:0/751989259 learned_addr learned my addr 172.21.15.78:0/751989259 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29200640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2413e5d0 0x7fdc24140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2a600640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 0x7fdc2413aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 -- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 msgr2=0x7fdc2413aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 0x7fdc2413aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 -- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2413e5d0 msgr2=0x7fdc24140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2413e5d0 0x7fdc24140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 -- 172.21.15.78:0/751989259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241596c0 con 0x7fdc240a9460 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2a600640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 0x7fdc2413aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:45.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc29c00640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d9b0 tx=0x7fdc1800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18017070 con 0x7fdc240a9460 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc18009cc0 con 0x7fdc240a9460 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc18005280 con 0x7fdc240a9460 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc24159100 con 0x7fdc240a9460 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc24157570 con 0x7fdc240a9460 2024-09-11T12:53:45.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.064+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc24157360 con 0x7fdc240a9460 2024-09-11T12:53:45.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.068+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fdc18005420 con 0x7fdc240a9460 2024-09-11T12:53:45.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.068+0000 7fdc21600640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 0x7fdc04073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.068+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7fdc18092050 con 0x7fdc240a9460 2024-09-11T12:53:45.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.068+0000 7fdc29200640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 0x7fdc04073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.068+0000 7fdc29200640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 0x7fdc04073c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc24154e00 tx=0x7fdc10009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:45.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.072+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdc1805d790 con 0x7fdc240a9460 2024-09-11T12:53:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.268+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc24157a70 con 0x7fdc240a9460 2024-09-11T12:53:45.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc21600640 1 -- 172.21.15.78:0/751989259 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3908 (secure 0 0 0) 0x7fdc24157a70 con 0x7fdc240a9460 2024-09-11T12:53:45.273 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:45.273 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":49,"btime":"2024-09-11T12:53:38:898463+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5480,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6836/2288189361","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2288189361},{"type":"v1","addr":"172.21.15.157:6836","nonce":2288189361}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/96277227","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":96277227},{"type":"v1","addr":"172.21.15.157:6801","nonce":96277227}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49},{"gid":5902,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3201268975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3201268975},{"type":"v1","addr":"172.21.15.78:6835","nonce":3201268975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6126,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/148513709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":148513709},{"type":"v1","addr":"172.21.15.78:6834","nonce":148513709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[]} 2024-09-11T12:53:45.273 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 49 2024-09-11T12:53:45.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 msgr2=0x7fdc04073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 0x7fdc04073c60 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fdc24154e00 tx=0x7fdc10009040 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 msgr2=0x7fdc2406fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fdc1800d9b0 tx=0x7fdc1800de70 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 shutdown_connections 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc2415fa90 0x7fdc2413aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc040717a0 0x7fdc04073c60 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc2413e5d0 0x7fdc24140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 --2- 172.21.15.78:0/751989259 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc240a9460 0x7fdc2406fd50 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 >> 172.21.15.78:0/751989259 conn(0x7fdc24071cf0 msgr2=0x7fdc240ce520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 shutdown_connections 2024-09-11T12:53:45.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.272+0000 7fdc2b000640 1 -- 172.21.15.78:0/751989259 wait complete. 2024-09-11T12:53:45.294 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:53:45.294 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:53:45.294 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:53:45.294 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:45.294 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:53:45.294 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:53:45.294 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:53:45.295 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:45.295 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:53:45.295 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:53:45.295 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:53:45.295 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:45.295 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:53:45.295 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:53:45.296 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:53:45.296 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:53:45.307 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:53:45.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:53:45.309 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:53:45.309 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:53:45.311 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:53:45.312 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:53:45.313 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:53:45.313 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:53:45.354 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:53:45.354 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:45.355 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:53:45.355 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:45.356 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:53:45.356 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:45.357 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:53:45.357 DEBUG:teuthology.parallel:result is None 2024-09-11T12:53:45.357 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:53:45.439 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:53:45.442 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:53:45.449 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:53:45.449 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 -- 172.21.15.78:0/2266003035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da40aed20 msgr2=0x7f5da40cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 --2- 172.21.15.78:0/2266003035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da40aed20 0x7f5da40cf280 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009840 tx=0x7f5d9401c7f0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 -- 172.21.15.78:0/2266003035 shutdown_connections 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 --2- 172.21.15.78:0/2266003035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 0x7f5da415cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 --2- 172.21.15.78:0/2266003035 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da40aed20 0x7f5da40cf280 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 -- 172.21.15.78:0/2266003035 >> 172.21.15.78:0/2266003035 conn(0x7f5da406ca80 msgr2=0x7f5da40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:45.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 -- 172.21.15.78:0/2266003035 shutdown_connections 2024-09-11T12:53:45.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.504+0000 7f5daa800640 1 -- 172.21.15.78:0/2266003035 wait complete. 2024-09-11T12:53:45.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 Processor -- start 2024-09-11T12:53:45.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- start start 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 0x7f5da4150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da415d010 0x7f5da41509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5da40cd0d0 con 0x7f5da40aed20 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5da4152da0 con 0x7f5da415d010 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5da40cd250 con 0x7f5da40cf7c0 2024-09-11T12:53:45.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38262/0 (socket says 172.21.15.78:38262) 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 -- 172.21.15.78:0/1766232771 learned_addr learned my addr 172.21.15.78:0/1766232771 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da2a00640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 0x7f5da4150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3e00640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da415d010 0x7f5da41509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 -- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 msgr2=0x7f5da4150470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 0x7f5da4150470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 -- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da415d010 msgr2=0x7f5da41509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da415d010 0x7f5da41509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 -- 172.21.15.78:0/1766232771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da41f7dc0 con 0x7f5da40aed20 2024-09-11T12:53:45.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5da3400640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009790 tx=0x7f5d940a57c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d940b2040 con 0x7f5da40aed20 2024-09-11T12:53:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da41f83a0 con 0x7f5da40aed20 2024-09-11T12:53:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d94004430 con 0x7f5da40aed20 2024-09-11T12:53:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d94005280 con 0x7f5da40aed20 2024-09-11T12:53:45.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.508+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da41f8960 con 0x7f5da40aed20 2024-09-11T12:53:45.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da40cf280 con 0x7f5da40aed20 2024-09-11T12:53:45.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f5d94005420 con 0x7f5da40aed20 2024-09-11T12:53:45.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5d9b400640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 0x7f5d84073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:45.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5502+0+0 (secure 0 0 0) 0x7f5d9412b4e0 con 0x7f5da40aed20 2024-09-11T12:53:45.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5da2a00640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 0x7f5d84073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:45.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.512+0000 7f5da2a00640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 0x7f5d84073c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009600 tx=0x7f5d900091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:45.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.516+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5d94016410 con 0x7f5da40aed20 2024-09-11T12:53:45.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.692+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5da40b53b0 con 0x7f5da40aed20 2024-09-11T12:53:45.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.936+0000 7f5d9b400640 1 -- 172.21.15.78:0/1766232771 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v77) ==== 97+0+0 (secure 0 0 0) 0x7f5d940f7280 con 0x7f5da40aed20 2024-09-11T12:53:45.939 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 msgr2=0x7f5d84073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 0x7f5d84073c60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5d90009600 tx=0x7f5d900091e0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 msgr2=0x7f5da415adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5d94009790 tx=0x7f5d940a57c0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 shutdown_connections 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d840717a0 0x7f5d84073c60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5da415d010 0x7f5da41509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5da40cf7c0 0x7f5da4150470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 --2- 172.21.15.78:0/1766232771 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5da40aed20 0x7f5da415adb0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 >> 172.21.15.78:0/1766232771 conn(0x7f5da406ca80 msgr2=0x7f5da4073160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 shutdown_connections 2024-09-11T12:53:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:45.940+0000 7f5daa800640 1 -- 172.21.15.78:0/1766232771 wait complete. 2024-09-11T12:53:45.965 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:53:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- 172.21.15.78:0/499889138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac15e5b0 msgr2=0x7fa8ac15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/499889138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac15e9f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac073760 tx=0x7fa89801c890 comp rx=0 tx=0).stop 2024-09-11T12:53:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- 172.21.15.78:0/499889138 shutdown_connections 2024-09-11T12:53:46.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/499889138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac15e9f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/499889138 >> v2:172.21.15.157:3300/0 conn(0x7fa8ac06fc50 0x7fa8ac0700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/499889138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 0x7fa8ac15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- 172.21.15.78:0/499889138 >> 172.21.15.78:0/499889138 conn(0x7fa8ac06ca80 msgr2=0x7fa8ac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- 172.21.15.78:0/499889138 shutdown_connections 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- 172.21.15.78:0/499889138 wait complete. 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 Processor -- start 2024-09-11T12:53:46.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- start start 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 0x7fa8ac14bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac072900 con 0x7fa8ac06fc50 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa8ac072780 con 0x7fa8ac15e5b0 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b3c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa8ac0b4520 con 0x7fa8ac074240 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.108+0000 7fa8b1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38278/0 (socket says 172.21.15.78:38278) 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 -- 172.21.15.78:0/1913515680 learned_addr learned my addr 172.21.15.78:0/1913515680 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8abe00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 0x7fa8ac14bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 -- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 msgr2=0x7fa8ac14bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 0x7fa8ac14bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 -- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac15e5b0 msgr2=0x7fa8ac153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 -- 172.21.15.78:0/1913515680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8ac14c060 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b1200640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00d980 tx=0x7fa89c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c017070 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa89c009d60 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8ac1524c0 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c005280 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8ac152a80 con 0x7fa8ac06fc50 2024-09-11T12:53:46.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8ac15ba80 con 0x7fa8ac06fc50 2024-09-11T12:53:46.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.112+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa89c005420 con 0x7fa8ac06fc50 2024-09-11T12:53:46.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.116+0000 7fa8a8c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 0x7fa88c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.116+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7fa89c091db0 con 0x7fa8ac06fc50 2024-09-11T12:53:46.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.116+0000 7fa8abe00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 0x7fa88c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.116+0000 7fa8abe00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 0x7fa88c073c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8980af000 tx=0x7fa89801b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.120+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa89c096070 con 0x7fa8ac06fc50 2024-09-11T12:53:46.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.340+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa8ac15e080 con 0x7fa8ac06fc50 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.340+0000 7fa8a8c00640 1 -- 172.21.15.78:0/1913515680 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) ==== 80+0+5304 (secure 0 0 0) 0x7fa89c05d6f0 con 0x7fa8ac06fc50 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:53:46.343 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:46.344 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.345 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.346 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.348 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.350 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.351 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:46.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:53:46.356 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:53:46.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 msgr2=0x7fa88c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 0x7fa88c073c60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa8980af000 tx=0x7fa89801b0d0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 msgr2=0x7fa8ac1f71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa89c00d980 tx=0x7fa89c00de40 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 shutdown_connections 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa88c0717a0 0x7fa88c073c60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8ac15e5b0 0x7fa8ac153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8ac074240 0x7fa8ac14bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 --2- 172.21.15.78:0/1913515680 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8ac06fc50 0x7fa8ac1f71d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 >> 172.21.15.78:0/1913515680 conn(0x7fa8ac06ca80 msgr2=0x7fa8ac150cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 shutdown_connections 2024-09-11T12:53:46.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.344+0000 7fa8b3c00640 1 -- 172.21.15.78:0/1913515680 wait complete. 2024-09-11T12:53:46.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:53:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- 172.21.15.78:0/3052020225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2024070a00 msgr2=0x7f20240736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 --2- 172.21.15.78:0/3052020225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2024070a00 0x7f20240736d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2014009c80 tx=0x7f201401cd10 comp rx=0 tx=0).stop 2024-09-11T12:53:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- 172.21.15.78:0/3052020225 shutdown_connections 2024-09-11T12:53:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 --2- 172.21.15.78:0/3052020225 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 0x7f202415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 --2- 172.21.15.78:0/3052020225 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2024070a00 0x7f20240736d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- 172.21.15.78:0/3052020225 >> 172.21.15.78:0/3052020225 conn(0x7f202406ca80 msgr2=0x7f20240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- 172.21.15.78:0/3052020225 shutdown_connections 2024-09-11T12:53:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- 172.21.15.78:0/3052020225 wait complete. 2024-09-11T12:53:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 Processor -- start 2024-09-11T12:53:46.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.504+0000 7f202ae00640 1 -- start start 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202406eb40 0x7f202414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 0x7f2024146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b51a0 con 0x7f2024070a00 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20240b5020 con 0x7f202406eb40 2024-09-11T12:53:46.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20240b5320 con 0x7f2024157ae0 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38284/0 (socket says 172.21.15.78:38284) 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 -- 172.21.15.78:0/1934516799 learned_addr learned my addr 172.21.15.78:0/1934516799 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023e00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202406eb40 0x7f202414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2028e00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 0x7f2024146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 -- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 msgr2=0x7f2024146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 0x7f2024146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 -- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202406eb40 msgr2=0x7f202414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202406eb40 0x7f202414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 -- 172.21.15.78:0/1934516799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f202414ffb0 con 0x7f2024070a00 2024-09-11T12:53:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f2023400640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f201000e7c0 tx=0x7f201000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201000ccc0 con 0x7f2024070a00 2024-09-11T12:53:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201000ce80 con 0x7f2024070a00 2024-09-11T12:53:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2010005280 con 0x7f2024070a00 2024-09-11T12:53:46.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20241f5800 con 0x7f2024070a00 2024-09-11T12:53:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20241f5dc0 con 0x7f2024070a00 2024-09-11T12:53:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.508+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20240736d0 con 0x7f2024070a00 2024-09-11T12:53:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.512+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f2010005420 con 0x7f2024070a00 2024-09-11T12:53:46.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.512+0000 7f201be00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 0x7f2004073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.512+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f2010092470 con 0x7f2024070a00 2024-09-11T12:53:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.512+0000 7f2023e00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 0x7f2004073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.512+0000 7f2023e00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 0x7f2004073c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20140af000 tx=0x7f20140a5f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.516+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f201005ddb0 con 0x7f2024070a00 2024-09-11T12:53:46.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f2024159ac0 con 0x7f2024070a00 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f201be00640 1 -- 172.21.15.78:0/1934516799 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v79) ==== 84+0+65 (secure 0 0 0) 0x7f2010062300 con 0x7f2024070a00 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:46.685 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 msgr2=0x7f2004073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 0x7f2004073c90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f20140af000 tx=0x7f20140a5f90 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 msgr2=0x7f20241516a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f201000e7c0 tx=0x7f201000ec80 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 shutdown_connections 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2024157ae0 0x7f2024146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2024070a00 0x7f20241516a0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20040717d0 0x7f2004073c90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 --2- 172.21.15.78:0/1934516799 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f202406eb40 0x7f202414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 >> 172.21.15.78:0/1934516799 conn(0x7f202406ca80 msgr2=0x7f2024072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 shutdown_connections 2024-09-11T12:53:46.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.684+0000 7f202ae00640 1 -- 172.21.15.78:0/1934516799 wait complete. 2024-09-11T12:53:46.705 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:53:46.706 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:53:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 -- 172.21.15.78:0/291434589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415ba20 msgr2=0x7f16c40c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 --2- 172.21.15.78:0/291434589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415ba20 0x7f16c40c51a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f16c406db40 tx=0x7f16b401ccc0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 -- 172.21.15.78:0/291434589 shutdown_connections 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 --2- 172.21.15.78:0/291434589 >> v2:172.21.15.157:3300/0 conn(0x7f16c415be00 0x7f16c40ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 --2- 172.21.15.78:0/291434589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415ba20 0x7f16c40c51a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 -- 172.21.15.78:0/291434589 >> 172.21.15.78:0/291434589 conn(0x7f16c406ce00 msgr2=0x7f16c40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 -- 172.21.15.78:0/291434589 shutdown_connections 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.848+0000 7f16c8e00640 1 -- 172.21.15.78:0/291434589 wait complete. 2024-09-11T12:53:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 Processor -- start 2024-09-11T12:53:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- start start 2024-09-11T12:53:46.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 0x7f16c41fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 0x7f16c415d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f16c4169370 con 0x7f16c415be00 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f16c41691f0 con 0x7f16c415ba20 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f16c41694f0 con 0x7f16c4074180 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 0x7f16c415d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 0x7f16c41fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:38060/0 (socket says 172.21.15.78:38060) 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1e00640 1 -- 172.21.15.78:0/2070322132 learned_addr learned my addr 172.21.15.78:0/2070322132 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 -- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 msgr2=0x7f16c414e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 -- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 msgr2=0x7f16c41fbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 0x7f16c41fbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 -- 172.21.15.78:0/2070322132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16c414edc0 con 0x7f16c415be00 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c1400640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 0x7f16c41fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c2800640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 0x7f16c415d7d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c3c0 tx=0x7f16bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc017070 con 0x7f16c415be00 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16bc0053c0 con 0x7f16c415be00 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16c41f84c0 con 0x7f16c415be00 2024-09-11T12:53:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16bc003860 con 0x7f16c415be00 2024-09-11T12:53:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.852+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16c41f8a80 con 0x7f16c415be00 2024-09-11T12:53:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16c40c51a0 con 0x7f16c415be00 2024-09-11T12:53:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f16bc01a070 con 0x7f16c415be00 2024-09-11T12:53:46.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16ba000640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 0x7f16a4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 4857+0+0 (secure 0 0 0) 0x7f16bc090e50 con 0x7f16c415be00 2024-09-11T12:53:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16c1e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 0x7f16a4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:46.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.856+0000 7f16c1e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 0x7f16a4073c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16b40b0040 tx=0x7f16b401b500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:46.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:46.860+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f16bc05c960 con 0x7f16c415be00 2024-09-11T12:53:47.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.032+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f16c4155140 con 0x7f16c415be00 2024-09-11T12:53:47.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.948+0000 7f16ba000640 1 -- 172.21.15.78:0/2070322132 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v80) ==== 138+0+0 (secure 0 0 0) 0x7f16bc0651f0 con 0x7f16c415be00 2024-09-11T12:53:47.952 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:53:47.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.952+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 msgr2=0x7f16a4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:47.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.952+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 0x7f16a4073c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16b40b0040 tx=0x7f16b401b500 comp rx=0 tx=0).stop 2024-09-11T12:53:47.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.952+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 msgr2=0x7f16c415d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:47.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.952+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 0x7f16c415d7d0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00c3c0 tx=0x7f16bc00c880 comp rx=0 tx=0).stop 2024-09-11T12:53:47.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 shutdown_connections 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f16c415be00 0x7f16c415d7d0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f16a40717a0 0x7f16a4073c60 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f16b40b0040 tx=0x7f16b401b500 comp rx=0 tx=0).stop 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f16c415ba20 0x7f16c41fbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 --2- 172.21.15.78:0/2070322132 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f16c4074180 0x7f16c414e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 >> 172.21.15.78:0/2070322132 conn(0x7f16c406ce00 msgr2=0x7f16c40cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 shutdown_connections 2024-09-11T12:53:47.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:47.956+0000 7f16c8e00640 1 -- 172.21.15.78:0/2070322132 wait complete. 2024-09-11T12:53:47.984 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- 172.21.15.78:0/1100023188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 msgr2=0x7ff13006faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 --2- 172.21.15.78:0/1100023188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff13006faa0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff120009840 tx=0x7ff12001c790 comp rx=0 tx=0).stop 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- 172.21.15.78:0/1100023188 shutdown_connections 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 --2- 172.21.15.78:0/1100023188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff13006faa0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- 172.21.15.78:0/1100023188 >> 172.21.15.78:0/1100023188 conn(0x7ff1300b2ec0 msgr2=0x7ff1300b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- 172.21.15.78:0/1100023188 shutdown_connections 2024-09-11T12:53:48.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- 172.21.15.78:0/1100023188 wait complete. 2024-09-11T12:53:48.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 Processor -- start 2024-09-11T12:53:48.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- start start 2024-09-11T12:53:48.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff130072e70 0x7ff130141ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:48.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff130142610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff130169050 con 0x7ff13015af70 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff130168ed0 con 0x7ff13015dfc0 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff136400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1301691d0 con 0x7ff130072e70 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38310/0 (socket says 172.21.15.78:38310) 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff12ea00640 1 -- 172.21.15.78:0/4227490571 learned_addr learned my addr 172.21.15.78:0/4227490571 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff12fe00640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff130142610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.136+0000 7ff12f400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff130072e70 0x7ff130141ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 -- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff130072e70 msgr2=0x7ff130141ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff130072e70 0x7ff130141ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 -- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 msgr2=0x7ff130142610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff130142610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 -- 172.21.15.78:0/4227490571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff130150180 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff12ea00640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c490 tx=0x7ff11c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0052d0 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff11c005490 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c012870 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff130150460 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff130150a20 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13006faa0 con 0x7ff13015af70 2024-09-11T12:53:48.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff11c012a10 con 0x7ff13015af70 2024-09-11T12:53:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.140+0000 7ff127400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 0x7ff110073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.144+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 5227+0+0 (secure 0 0 0) 0x7ff11c0919e0 con 0x7ff13015af70 2024-09-11T12:53:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.144+0000 7ff12f400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 0x7ff110073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:48.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.144+0000 7ff12f400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 0x7ff110073c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1200af000 tx=0x7ff12001afd0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:48.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.148+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff11c05d880 con 0x7ff13015af70 2024-09-11T12:53:48.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.312+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7ff13006f610 con 0x7ff13015af70 2024-09-11T12:53:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.956+0000 7ff127400640 1 -- 172.21.15.78:0/4227490571 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v81) ==== 170+0+0 (secure 0 0 0) 0x7ff11c05d180 con 0x7ff13015af70 2024-09-11T12:53:48.959 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 msgr2=0x7ff110073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 0x7ff110073c60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff1200af000 tx=0x7ff12001afd0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 msgr2=0x7ff13014ab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00c490 tx=0x7ff11c00c950 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 shutdown_connections 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1100717a0 0x7ff110073c60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff13015dfc0 0x7ff130142610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff13015af70 0x7ff13014ab10 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 --2- 172.21.15.78:0/4227490571 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff130072e70 0x7ff130141ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 >> 172.21.15.78:0/4227490571 conn(0x7ff1300b2ec0 msgr2=0x7ff1300afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 shutdown_connections 2024-09-11T12:53:48.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:48.960+0000 7ff136400640 1 -- 172.21.15.78:0/4227490571 wait complete. 2024-09-11T12:53:49.005 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:53:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- 172.21.15.78:0/4139813622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d01518b0 msgr2=0x7ff4d00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 --2- 172.21.15.78:0/4139813622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d01518b0 0x7ff4d00b1a10 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7ff4d00af400 tx=0x7ff4c001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- 172.21.15.78:0/4139813622 shutdown_connections 2024-09-11T12:53:49.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 --2- 172.21.15.78:0/4139813622 >> v2:172.21.15.157:3300/0 conn(0x7ff4d00b1f50 0x7ff4d00cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 --2- 172.21.15.78:0/4139813622 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d01518b0 0x7ff4d00b1a10 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- 172.21.15.78:0/4139813622 >> 172.21.15.78:0/4139813622 conn(0x7ff4d0071dd0 msgr2=0x7ff4d00721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- 172.21.15.78:0/4139813622 shutdown_connections 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- 172.21.15.78:0/4139813622 wait complete. 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 Processor -- start 2024-09-11T12:53:49.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.148+0000 7ff4d5400640 1 -- start start 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 0x7ff4d01531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 0x7ff4d0142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d0167960 con 0x7ff4d00ce150 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d01677e0 con 0x7ff4d01518b0 2024-09-11T12:53:49.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d0167ae0 con 0x7ff4d00b1f50 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38316/0 (socket says 172.21.15.78:38316) 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 -- 172.21.15.78:0/3412378024 learned_addr learned my addr 172.21.15.78:0/3412378024 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4cf200640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 0x7ff4d0142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4ce800640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 0x7ff4d01531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 -- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 msgr2=0x7ff4d01531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 0x7ff4d01531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 -- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 msgr2=0x7ff4d0142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 0x7ff4d0142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 -- 172.21.15.78:0/3412378024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d0153720 con 0x7ff4d00ce150 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4cf200640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 0x7ff4d0142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:49.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4ce800640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 0x7ff4d01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c7e00640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d9b0 tx=0x7ff4b800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8017070 con 0x7ff4d00ce150 2024-09-11T12:53:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4b8009cc0 con 0x7ff4d00ce150 2024-09-11T12:53:49.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d014e2c0 con 0x7ff4d00ce150 2024-09-11T12:53:49.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4b8005280 con 0x7ff4d00ce150 2024-09-11T12:53:49.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d014ecf0 con 0x7ff4d00ce150 2024-09-11T12:53:49.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.152+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d00cdc10 con 0x7ff4d00ce150 2024-09-11T12:53:49.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.156+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff4b8005420 con 0x7ff4d00ce150 2024-09-11T12:53:49.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.156+0000 7ff4c6a00640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 0x7ff4ac073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:49.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.156+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5609+0+0 (secure 0 0 0) 0x7ff4b8092650 con 0x7ff4d00ce150 2024-09-11T12:53:49.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.156+0000 7ff4ce800640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 0x7ff4ac073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:49.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.156+0000 7ff4ce800640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 0x7ff4ac073c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00af000 tx=0x7ff4c001b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:49.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.160+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff4b805e370 con 0x7ff4d00ce150 2024-09-11T12:53:49.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.360+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff4d0152090 con 0x7ff4d00ce150 2024-09-11T12:53:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.960+0000 7ff4c6a00640 1 -- 172.21.15.78:0/3412378024 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7ff4b805dc70 con 0x7ff4d00ce150 2024-09-11T12:53:49.962 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 12 and data pool 13 2024-09-11T12:53:49.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 msgr2=0x7ff4ac073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 0x7ff4ac073c60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4c00af000 tx=0x7ff4c001b100 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 msgr2=0x7ff4d014dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7ff4b800d9b0 tx=0x7ff4b800de70 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 shutdown_connections 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff4ac0717a0 0x7ff4ac073c60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4d01518b0 0x7ff4d0142c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4d00ce150 0x7ff4d014dd80 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 --2- 172.21.15.78:0/3412378024 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4d00b1f50 0x7ff4d01531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:49.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 >> 172.21.15.78:0/3412378024 conn(0x7ff4d0071dd0 msgr2=0x7ff4d015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:49.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 shutdown_connections 2024-09-11T12:53:49.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:49.964+0000 7ff4d5400640 1 -- 172.21.15.78:0/3412378024 wait complete. 2024-09-11T12:53:50.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 -- 172.21.15.78:0/2223122243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4000d0bf0 msgr2=0x7ff4000d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 --2- 172.21.15.78:0/2223122243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4000d0bf0 0x7ff4000d10a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009c80 tx=0x7ff3f801cda0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 -- 172.21.15.78:0/2223122243 shutdown_connections 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 --2- 172.21.15.78:0/2223122243 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4000d0bf0 0x7ff4000d10a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 --2- 172.21.15.78:0/2223122243 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000b14b0 0x7ff40015c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 -- 172.21.15.78:0/2223122243 >> 172.21.15.78:0/2223122243 conn(0x7ff400074b90 msgr2=0x7ff4000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 -- 172.21.15.78:0/2223122243 shutdown_connections 2024-09-11T12:53:50.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.164+0000 7ff407400640 1 -- 172.21.15.78:0/2223122243 wait complete. 2024-09-11T12:53:50.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 Processor -- start 2024-09-11T12:53:50.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- start start 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4000b14b0 0x7ff400149000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000d0bf0 0x7ff4001481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff400168f00 con 0x7ff40006fdf0 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff400168d80 con 0x7ff4000b14b0 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff400169080 con 0x7ff4000d0bf0 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38334/0 (socket says 172.21.15.78:38334) 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 -- 172.21.15.78:0/407483548 learned_addr learned my addr 172.21.15.78:0/407483548 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff3ffe00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4000b14b0 0x7ff400149000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff405400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000d0bf0 0x7ff4001481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 -- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000d0bf0 msgr2=0x7ff4001481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000d0bf0 0x7ff4001481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 -- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4000b14b0 msgr2=0x7ff400149000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4000b14b0 0x7ff400149000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 -- 172.21.15.78:0/407483548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff400149770 con 0x7ff40006fdf0 2024-09-11T12:53:50.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff404a00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e970 tx=0x7ff3f400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f400cd80 con 0x7ff40006fdf0 2024-09-11T12:53:50.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3f4005020 con 0x7ff40006fdf0 2024-09-11T12:53:50.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff400149a50 con 0x7ff40006fdf0 2024-09-11T12:53:50.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4005450 con 0x7ff40006fdf0 2024-09-11T12:53:50.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff40014a010 con 0x7ff40006fdf0 2024-09-11T12:53:50.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.168+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40015c100 con 0x7ff40006fdf0 2024-09-11T12:53:50.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.172+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff3f40047b0 con 0x7ff40006fdf0 2024-09-11T12:53:50.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.172+0000 7ff3fcc00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 0x7ff3e0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.172+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7ff3f4092a60 con 0x7ff40006fdf0 2024-09-11T12:53:50.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.172+0000 7ff3ffe00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 0x7ff3e0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.172+0000 7ff3ffe00640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 0x7ff3e0073c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff40006eb90 tx=0x7ff3ec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.176+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff3f405e740 con 0x7ff40006fdf0 2024-09-11T12:53:50.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.344+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4000ce570 con 0x7ff40006fdf0 2024-09-11T12:53:50.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.344+0000 7ff3fcc00640 1 -- 172.21.15.78:0/407483548 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7ff3f405e040 con 0x7ff40006fdf0 2024-09-11T12:53:50.348 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:50.348 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:49.362232+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:47.045831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:48.326392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 msgr2=0x7ff3e0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 0x7ff3e0073c60 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff40006eb90 tx=0x7ff3ec009040 comp rx=0 tx=0).stop 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 msgr2=0x7ff4001fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400e970 tx=0x7ff3f400ee30 comp rx=0 tx=0).stop 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 shutdown_connections 2024-09-11T12:53:50.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff4000d0bf0 0x7ff4001481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3e00717a0 0x7ff3e0073c60 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff4000b14b0 0x7ff400149000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 --2- 172.21.15.78:0/407483548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff40006fdf0 0x7ff4001fbb80 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 >> 172.21.15.78:0/407483548 conn(0x7ff400074b90 msgr2=0x7ff40006ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 shutdown_connections 2024-09-11T12:53:50.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.348+0000 7ff407400640 1 -- 172.21.15.78:0/407483548 wait complete. 2024-09-11T12:53:50.370 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:50.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- 172.21.15.78:0/3340229908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a44070910 msgr2=0x7f3a4406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 --2- 172.21.15.78:0/3340229908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a44070910 0x7f3a4406fc50 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f3a30009900 tx=0x7f3a3001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- 172.21.15.78:0/3340229908 shutdown_connections 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 --2- 172.21.15.78:0/3340229908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a44070910 0x7f3a4406fc50 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- 172.21.15.78:0/3340229908 >> 172.21.15.78:0/3340229908 conn(0x7f3a4406ca80 msgr2=0x7f3a440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- 172.21.15.78:0/3340229908 shutdown_connections 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- 172.21.15.78:0/3340229908 wait complete. 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 Processor -- start 2024-09-11T12:53:50.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- start start 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a44070190 0x7f3a4413d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a44070910 0x7f3a4413da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44169040 con 0x7f3a440741e0 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44168ec0 con 0x7f3a44070190 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a441691c0 con 0x7f3a44070910 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.512+0000 7f3a4a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38346/0 (socket says 172.21.15.78:38346) 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a49a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a44070190 0x7f3a4413d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 -- 172.21.15.78:0/891539847 learned_addr learned my addr 172.21.15.78:0/891539847 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:50.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a49000640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a44070910 0x7f3a4413da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 -- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a44070910 msgr2=0x7f3a4413da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a44070910 0x7f3a4413da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 -- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a44070190 msgr2=0x7f3a4413d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a44070190 0x7f3a4413d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 -- 172.21.15.78:0/891539847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a441517d0 con 0x7f3a440741e0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4a400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c4d0 tx=0x7f3a3c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c0052d0 con 0x7f3a440741e0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a3c005470 con 0x7f3a440741e0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c012870 con 0x7f3a440741e0 2024-09-11T12:53:50.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a44151ab0 con 0x7f3a440741e0 2024-09-11T12:53:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a44152070 con 0x7f3a440741e0 2024-09-11T12:53:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4406fc50 con 0x7f3a440741e0 2024-09-11T12:53:50.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.516+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f3a3c003600 con 0x7f3a440741e0 2024-09-11T12:53:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.520+0000 7f3a41600640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 0x7f3a24073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.520+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f3a3c091940 con 0x7f3a440741e0 2024-09-11T12:53:50.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.520+0000 7f3a49a00640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 0x7f3a24073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.520+0000 7f3a49a00640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 0x7f3a24073c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a380012f0 tx=0x7f3a38009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.520+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3a3c05d040 con 0x7f3a440741e0 2024-09-11T12:53:50.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.688+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3a440cfcc0 con 0x7f3a440741e0 2024-09-11T12:53:50.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.688+0000 7f3a41600640 1 -- 172.21.15.78:0/891539847 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+16733 (secure 0 0 0) 0x7f3a3c061590 con 0x7f3a440741e0 2024-09-11T12:53:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:50.690 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:49.362232+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:47.045831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:48.326392+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:50.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 msgr2=0x7f3a24073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 0x7f3a24073c60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3a380012f0 tx=0x7f3a38009040 comp rx=0 tx=0).stop 2024-09-11T12:53:50.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 msgr2=0x7f3a4413dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c00c4d0 tx=0x7f3a3c00c990 comp rx=0 tx=0).stop 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 shutdown_connections 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a440741e0 0x7f3a4413dfc0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a44070910 0x7f3a4413da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a240717a0 0x7f3a24073c60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 --2- 172.21.15.78:0/891539847 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a44070190 0x7f3a4413d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 >> 172.21.15.78:0/891539847 conn(0x7f3a4406ca80 msgr2=0x7f3a440b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 shutdown_connections 2024-09-11T12:53:50.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.692+0000 7f3a4c400640 1 -- 172.21.15.78:0/891539847 wait complete. 2024-09-11T12:53:50.711 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:53:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.852+0000 7f158fe00640 1 -- 172.21.15.78:0/2795192478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 msgr2=0x7f15900a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.852+0000 7f158fe00640 1 --2- 172.21.15.78:0/2795192478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15900a9460 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f1578009840 tx=0x7f157801c7f0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- 172.21.15.78:0/2795192478 shutdown_connections 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 --2- 172.21.15.78:0/2795192478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 0x7f15900700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 --2- 172.21.15.78:0/2795192478 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15900a9460 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- 172.21.15.78:0/2795192478 >> 172.21.15.78:0/2795192478 conn(0x7f15900742a0 msgr2=0x7f15900746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- 172.21.15.78:0/2795192478 shutdown_connections 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- 172.21.15.78:0/2795192478 wait complete. 2024-09-11T12:53:50.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 Processor -- start 2024-09-11T12:53:50.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- start start 2024-09-11T12:53:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f159006fcb0 0x7f1590165160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 0x7f159014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15901685c0 con 0x7f1590152690 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1590168440 con 0x7f159006fcb0 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1590168740 con 0x7f15900cebc0 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38366/0 (socket says 172.21.15.78:38366) 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 -- 172.21.15.78:0/2915958127 learned_addr learned my addr 172.21.15.78:0/2915958127 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158ea00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f159006fcb0 0x7f1590165160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158e000640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 0x7f159014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 -- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 msgr2=0x7f159014c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 0x7f159014c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 -- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f159006fcb0 msgr2=0x7f1590165160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f159006fcb0 0x7f1590165160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 -- 172.21.15.78:0/2915958127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15901656a0 con 0x7f1590152690 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158f400640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f158800cd10 tx=0x7f158800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800f450 con 0x7f1590152690 2024-09-11T12:53:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f158800f6e0 con 0x7f1590152690 2024-09-11T12:53:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f159006da00 con 0x7f1590152690 2024-09-11T12:53:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.856+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158800fb30 con 0x7f1590152690 2024-09-11T12:53:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1590153ce0 con 0x7f1590152690 2024-09-11T12:53:50.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15900700f0 con 0x7f1590152690 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f158800fcd0 con 0x7f1590152690 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f1586a00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 0x7f156c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5709+0+0 (secure 0 0 0) 0x7f1588092a90 con 0x7f1590152690 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f158ea00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 0x7f156c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.860+0000 7f158ea00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 0x7f156c073c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15780af000 tx=0x7f157801b030 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:50.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:50.864+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1588014030 con 0x7f1590152690 2024-09-11T12:53:51.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.068+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f159014c660 con 0x7f1590152690 2024-09-11T12:53:51.949 INFO:tasks.ceph.mds.a.smithi078.stderr:2024-09-11T12:53:51.948+0000 7fd31a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:53:51.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.984+0000 7f1586a00640 1 -- 172.21.15.78:0/2915958127 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) ==== 118+0+0 (secure 0 0 0) 0x7f158805e6d0 con 0x7f1590152690 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 msgr2=0x7f156c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 0x7f156c073c60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f15780af000 tx=0x7f157801b030 comp rx=0 tx=0).stop 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 msgr2=0x7f15901f72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f158800cd10 tx=0x7f158800a620 comp rx=0 tx=0).stop 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 shutdown_connections 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1590152690 0x7f15901f72c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15900cebc0 0x7f159014c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:51.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f156c0717a0 0x7f156c073c60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:51.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 --2- 172.21.15.78:0/2915958127 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f159006fcb0 0x7f1590165160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:51.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 >> 172.21.15.78:0/2915958127 conn(0x7f15900742a0 msgr2=0x7f159014f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:51.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 shutdown_connections 2024-09-11T12:53:51.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:51.988+0000 7f158fe00640 1 -- 172.21.15.78:0/2915958127 wait complete. 2024-09-11T12:53:52.012 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:53:52.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- 172.21.15.78:0/3791946464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7680d1740 msgr2=0x7fe7680aaba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- 172.21.15.78:0/3791946464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7680d1740 0x7fe7680aaba0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fe768071000 tx=0x7fe76001c770 comp rx=0 tx=0).stop 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- 172.21.15.78:0/3791946464 shutdown_connections 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- 172.21.15.78:0/3791946464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7680d1740 0x7fe7680aaba0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- 172.21.15.78:0/3791946464 >> v2:172.21.15.157:3300/0 conn(0x7fe76815f030 0x7fe7680aa660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- 172.21.15.78:0/3791946464 >> 172.21.15.78:0/3791946464 conn(0x7fe768075610 msgr2=0x7fe768075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- 172.21.15.78:0/3791946464 shutdown_connections 2024-09-11T12:53:52.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- 172.21.15.78:0/3791946464 wait complete. 2024-09-11T12:53:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 Processor -- start 2024-09-11T12:53:52.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- start start 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7680d1740 0x7fe76814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76815ec50 0x7fe768156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe76815d470 con 0x7fe76815f030 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe76815d2f0 con 0x7fe7680d1740 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe76e800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe76815d5f0 con 0x7fe76815ec50 2024-09-11T12:53:52.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe767e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe767e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38386/0 (socket says 172.21.15.78:38386) 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe767e00640 1 -- 172.21.15.78:0/1261328420 learned_addr learned my addr 172.21.15.78:0/1261328420 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe767400640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7680d1740 0x7fe76814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.160+0000 7fe75ea00640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76815ec50 0x7fe768156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 -- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76815ec50 msgr2=0x7fe768156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76815ec50 0x7fe768156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 -- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7680d1740 msgr2=0x7fe76814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7680d1740 0x7fe76814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 -- 172.21.15.78:0/1261328420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe768156cc0 con 0x7fe76815f030 2024-09-11T12:53:52.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe767e00640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe760009790 tx=0x7fe760004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600bb070 con 0x7fe76815f030 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe760005030 con 0x7fe76815f030 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7600b2040 con 0x7fe76815f030 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe768159330 con 0x7fe76815f030 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe768158fe0 con 0x7fe76815f030 2024-09-11T12:53:52.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe76815c8c0 con 0x7fe76815f030 2024-09-11T12:53:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fe760005480 con 0x7fe76815f030 2024-09-11T12:53:52.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.164+0000 7fe75fe00640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 0x7fe744073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.168+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fe76012b940 con 0x7fe76815f030 2024-09-11T12:53:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.168+0000 7fe767400640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 0x7fe744073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:52.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.168+0000 7fe767400640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 0x7fe744073c60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe758001850 tx=0x7fe7580017e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:52.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.168+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe7600163f0 con 0x7fe76815f030 2024-09-11T12:53:52.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.372+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe7680d04b0 con 0x7fe76815f030 2024-09-11T12:53:52.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.988+0000 7fe75fe00640 1 -- 172.21.15.78:0/1261328420 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) ==== 122+0+0 (secure 0 0 0) 0x7fe7600f7040 con 0x7fe76815f030 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 msgr2=0x7fe744073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 0x7fe744073c60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe758001850 tx=0x7fe7580017e0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 msgr2=0x7fe768156fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe760009790 tx=0x7fe760004270 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 shutdown_connections 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe76815f030 0x7fe768156fe0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe76815ec50 0x7fe768156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7440717a0 0x7fe744073c60 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 --2- 172.21.15.78:0/1261328420 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7680d1740 0x7fe76814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:52.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 >> 172.21.15.78:0/1261328420 conn(0x7fe768075610 msgr2=0x7fe76815da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 shutdown_connections 2024-09-11T12:53:52.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:52.992+0000 7fe76e800640 1 -- 172.21.15.78:0/1261328420 wait complete. 2024-09-11T12:53:53.015 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:53:53.016 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 -- 172.21.15.78:0/3582113700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b14b0 msgr2=0x7fd06415c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 --2- 172.21.15.78:0/3582113700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b14b0 0x7fd06415c100 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd058009840 tx=0x7fd05801c820 comp rx=0 tx=0).stop 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 -- 172.21.15.78:0/3582113700 shutdown_connections 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 --2- 172.21.15.78:0/3582113700 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd06406fdf0 0x7fd0640a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 --2- 172.21.15.78:0/3582113700 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0640b14b0 0x7fd06415c100 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 -- 172.21.15.78:0/3582113700 >> 172.21.15.78:0/3582113700 conn(0x7fd064074b90 msgr2=0x7fd0640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 -- 172.21.15.78:0/3582113700 shutdown_connections 2024-09-11T12:53:53.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.156+0000 7fd06c800640 1 -- 172.21.15.78:0/3582113700 wait complete. 2024-09-11T12:53:53.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 Processor -- start 2024-09-11T12:53:53.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- start start 2024-09-11T12:53:53.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06406fdf0 0x7fd064142170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640d0bf0 0x7fd06415ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd064169390 con 0x7fd0640b14b0 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd064169210 con 0x7fd06406fdf0 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd064169510 con 0x7fd0640d0bf0 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38388/0 (socket says 172.21.15.78:38388) 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 -- 172.21.15.78:0/2969755188 learned_addr learned my addr 172.21.15.78:0/2969755188 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069e00640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06406fdf0 0x7fd064142170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06a800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640d0bf0 0x7fd06415ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 -- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640d0bf0 msgr2=0x7fd06415ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640d0bf0 0x7fd06415ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 -- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06406fdf0 msgr2=0x7fd064142170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06406fdf0 0x7fd064142170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 -- 172.21.15.78:0/2969755188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd06414f710 con 0x7fd0640b14b0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd069400640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9c0 tx=0x7fd05400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd05400cd80 con 0x7fd0640b14b0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd054004030 con 0x7fd0640b14b0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0641fb8f0 con 0x7fd0640b14b0 2024-09-11T12:53:53.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd054004480 con 0x7fd0640b14b0 2024-09-11T12:53:53.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.160+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0641fbeb0 con 0x7fd0640b14b0 2024-09-11T12:53:53.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0641fc0b0 con 0x7fd0640b14b0 2024-09-11T12:53:53.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd054003600 con 0x7fd0640b14b0 2024-09-11T12:53:53.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd05e000640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 0x7fd048073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fd054092aa0 con 0x7fd0640b14b0 2024-09-11T12:53:53.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd069e00640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 0x7fd048073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.164+0000 7fd069e00640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 0x7fd048073c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0580af000 tx=0x7fd05801b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.168+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd0641fc0b0 con 0x7fd0640b14b0 2024-09-11T12:53:53.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.368+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd0640a9d00 con 0x7fd0640b14b0 2024-09-11T12:53:53.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.368+0000 7fd05e000640 1 -- 172.21.15.78:0/2969755188 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7fd05405e6e0 con 0x7fd0640b14b0 2024-09-11T12:53:53.370 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:53.370 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":56,"btime":"2024-09-11T12:53:52:987817+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6537,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3066636859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3066636859},{"type":"v1","addr":"172.21.15.78:6833","nonce":3066636859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:53:53.371 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 56 2024-09-11T12:53:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 msgr2=0x7fd048073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 0x7fd048073c60 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd0580af000 tx=0x7fd05801b060 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 msgr2=0x7fd06414ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fd05400e9c0 tx=0x7fd05400ee80 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 shutdown_connections 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0640d0bf0 0x7fd06415ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0640b14b0 0x7fd06414ec50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0480717a0 0x7fd048073c60 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 --2- 172.21.15.78:0/2969755188 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd06406fdf0 0x7fd064142170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 >> 172.21.15.78:0/2969755188 conn(0x7fd064074b90 msgr2=0x7fd0640b2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 shutdown_connections 2024-09-11T12:53:53.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.372+0000 7fd06c800640 1 -- 172.21.15.78:0/2969755188 wait complete. 2024-09-11T12:53:53.391 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:53:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- 172.21.15.78:0/918616213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 msgr2=0x7f3cc0070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 --2- 172.21.15.78:0/918616213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc0070a00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f3cc00b2c60 tx=0x7f3cb001ccf0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- 172.21.15.78:0/918616213 shutdown_connections 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 --2- 172.21.15.78:0/918616213 >> v2:172.21.15.157:3300/0 conn(0x7f3cc00736d0 0x7f3cc015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 --2- 172.21.15.78:0/918616213 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc0070a00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- 172.21.15.78:0/918616213 >> 172.21.15.78:0/918616213 conn(0x7f3cc006ca80 msgr2=0x7f3cc00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- 172.21.15.78:0/918616213 shutdown_connections 2024-09-11T12:53:53.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- 172.21.15.78:0/918616213 wait complete. 2024-09-11T12:53:53.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 Processor -- start 2024-09-11T12:53:53.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.532+0000 7f3cc6800640 1 -- start start 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cc00736d0 0x7f3cc0151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 0x7f3cc01535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0167720 con 0x7f3cc0157ae0 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc00b5730 con 0x7f3cc00b4db0 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc01678a0 con 0x7f3cc00736d0 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:38410/0 (socket says 172.21.15.78:38410) 2024-09-11T12:53:53.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 -- 172.21.15.78:0/2236374557 learned_addr learned my addr 172.21.15.78:0/2236374557 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 -- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cc00736d0 msgr2=0x7f3cc0151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cc00736d0 0x7f3cc0151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 -- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 msgr2=0x7f3cc01535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbea00640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 0x7f3cc01535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 0x7f3cc01535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 -- 172.21.15.78:0/2236374557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc0152580 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbea00640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 0x7f3cc01535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cbfe00640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c3a0 tx=0x7f3cb800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8017070 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cb8003600 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb8003a50 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc0152820 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01530c0 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc015af70 con 0x7f3cc0157ae0 2024-09-11T12:53:53.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.536+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f3cb801a070 con 0x7f3cc0157ae0 2024-09-11T12:53:53.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.540+0000 7f3cb7400640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 0x7f3ca0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.540+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f3cb8091280 con 0x7f3cc0157ae0 2024-09-11T12:53:53.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.540+0000 7f3cbf400640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 0x7f3ca0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.540+0000 7f3cbf400640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 0x7f3ca0073c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00af000 tx=0x7f3cb0019a90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.544+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3cb805cfd0 con 0x7f3cc0157ae0 2024-09-11T12:53:53.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3cc006edc0 con 0x7f3cc0157ae0 2024-09-11T12:53:53.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cb7400640 1 -- 172.21.15.78:0/2236374557 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+16720 (secure 0 0 0) 0x7f3cb8065210 con 0x7f3cc0157ae0 2024-09-11T12:53:53.713 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:53.714 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":83,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:53:50.971936+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:47.045831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:48.326392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 msgr2=0x7f3ca0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 0x7f3ca0073c60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00af000 tx=0x7f3cb0019a90 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 msgr2=0x7f3cc00b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f3cb800c3a0 tx=0x7f3cb800c860 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 shutdown_connections 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3cc0157ae0 0x7f3cc00b07e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3ca00717a0 0x7f3ca0073c60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3cc00b4db0 0x7f3cc01535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 --2- 172.21.15.78:0/2236374557 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3cc00736d0 0x7f3cc0151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 >> 172.21.15.78:0/2236374557 conn(0x7f3cc006ca80 msgr2=0x7f3cc015c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 shutdown_connections 2024-09-11T12:53:53.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.712+0000 7f3cc6800640 1 -- 172.21.15.78:0/2236374557 wait complete. 2024-09-11T12:53:53.735 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:53:53.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:53:53.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- 172.21.15.78:0/133975692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 msgr2=0x7f6888004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 --2- 172.21.15.78:0/133975692 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f6888004c90 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f689000bb10 tx=0x7f689001cde0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a2800640 1 -- 172.21.15.78:0/133975692 reap_dead start 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- 172.21.15.78:0/133975692 shutdown_connections 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- 172.21.15.78:0/133975692 >> 172.21.15.78:0/133975692 conn(0x7f68a406ca80 msgr2=0x7f68a40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- 172.21.15.78:0/133975692 shutdown_connections 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- 172.21.15.78:0/133975692 wait complete. 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 Processor -- start 2024-09-11T12:53:53.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.876+0000 7f68a8c00640 1 -- start start 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40b0ff0 0x7f68a41f9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a4158d20 0x7f68a41591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f68a406f0c0 con 0x7f6888004810 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f68a406ef40 con 0x7f68a40b0ff0 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f68a406f240 con 0x7f68a4158d20 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44394/0 (socket says 172.21.15.78:44394) 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 -- 172.21.15.78:0/304805079 learned_addr learned my addr 172.21.15.78:0/304805079 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1400640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40b0ff0 0x7f68a41f9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a2800640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a4158d20 0x7f68a41591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 -- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a4158d20 msgr2=0x7f68a41591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a4158d20 0x7f68a41591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 -- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40b0ff0 msgr2=0x7f68a41f9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40b0ff0 0x7f68a41f9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 -- 172.21.15.78:0/304805079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68a40b1540 con 0x7f6888004810 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a1e00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f689400e600 tx=0x7f689400eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6894015070 con 0x7f6888004810 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f689400ca30 con 0x7f6888004810 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689400ce20 con 0x7f6888004810 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68a414a360 con 0x7f6888004810 2024-09-11T12:53:53.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68a414a920 con 0x7f6888004810 2024-09-11T12:53:53.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.880+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68a40736d0 con 0x7f6888004810 2024-09-11T12:53:53.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.884+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f6894018070 con 0x7f6888004810 2024-09-11T12:53:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.884+0000 7f689a000640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 0x7f6884073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.884+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f68940924e0 con 0x7f6888004810 2024-09-11T12:53:53.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.884+0000 7f68a1400640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 0x7f6884073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:53.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.884+0000 7f68a1400640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 0x7f6884073c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f68900b0000 tx=0x7f68900a7040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:53.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:53.888+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f689405d9c0 con 0x7f6888004810 2024-09-11T12:53:54.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.052+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68a40b1760 con 0x7f68840717a0 2024-09-11T12:53:54.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.056+0000 7f689a000640 1 -- 172.21.15.78:0/304805079 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f68a40b1760 con 0x7f68840717a0 2024-09-11T12:53:54.062 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:54.063 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:53:54.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 msgr2=0x7f6884073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 0x7f6884073c60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f68900b0000 tx=0x7f68900a7040 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 msgr2=0x7f68a4149640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f689400e600 tx=0x7f689400eac0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 shutdown_connections 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f68a4158d20 0x7f68a41591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f68840717a0 0x7f6884073c60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f68a40b0ff0 0x7f68a41f9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 --2- 172.21.15.78:0/304805079 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6888004810 0x7f68a4149640 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 >> 172.21.15.78:0/304805079 conn(0x7f68a406ca80 msgr2=0x7f68a415af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 shutdown_connections 2024-09-11T12:53:54.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.060+0000 7f68a8c00640 1 -- 172.21.15.78:0/304805079 wait complete. 2024-09-11T12:53:54.081 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:53:52.471967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258420,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716300,"statfs":{"total":773094113280,"available":772829491200,"internally_reserved":0,"allocated":9113600,"data_stored":4541060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.517731"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000338+0000","last_change":"2024-09-11T12:53:49.994111+0000","last_active":"2024-09-11T12:53:50.000338+0000","last_peered":"2024-09-11T12:53:50.000338+0000","last_clean":"2024-09-11T12:53:50.000338+0000","last_became_active":"2024-09-11T12:53:49.993553+0000","last_became_peered":"2024-09-11T12:53:49.993553+0000","last_unstale":"2024-09-11T12:53:50.000338+0000","last_undegraded":"2024-09-11T12:53:50.000338+0000","last_fullsized":"2024-09-11T12:53:50.000338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:48:32.688884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999733+0000","last_change":"2024-09-11T12:53:49.994797+0000","last_active":"2024-09-11T12:53:49.999733+0000","last_peered":"2024-09-11T12:53:49.999733+0000","last_clean":"2024-09-11T12:53:49.999733+0000","last_became_active":"2024-09-11T12:53:49.994087+0000","last_became_peered":"2024-09-11T12:53:49.994087+0000","last_unstale":"2024-09-11T12:53:49.999733+0000","last_undegraded":"2024-09-11T12:53:49.999733+0000","last_fullsized":"2024-09-11T12:53:49.999733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:37.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991897+0000","last_change":"2024-09-11T12:53:49.987897+0000","last_active":"2024-09-11T12:53:49.991897+0000","last_peered":"2024-09-11T12:53:49.991897+0000","last_clean":"2024-09-11T12:53:49.991897+0000","last_became_active":"2024-09-11T12:53:49.987580+0000","last_became_peered":"2024-09-11T12:53:49.987580+0000","last_unstale":"2024-09-11T12:53:49.991897+0000","last_undegraded":"2024-09-11T12:53:49.991897+0000","last_fullsized":"2024-09-11T12:53:49.991897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:45.722422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005287+0000","last_change":"2024-09-11T12:53:49.997395+0000","last_active":"2024-09-11T12:53:50.005287+0000","last_peered":"2024-09-11T12:53:50.005287+0000","last_clean":"2024-09-11T12:53:50.005287+0000","last_became_active":"2024-09-11T12:53:49.997068+0000","last_became_peered":"2024-09-11T12:53:49.997068+0000","last_unstale":"2024-09-11T12:53:50.005287+0000","last_undegraded":"2024-09-11T12:53:50.005287+0000","last_fullsized":"2024-09-11T12:53:50.005287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:14.877164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995368+0000","last_change":"2024-09-11T12:53:49.983511+0000","last_active":"2024-09-11T12:53:49.995368+0000","last_peered":"2024-09-11T12:53:49.995368+0000","last_clean":"2024-09-11T12:53:49.995368+0000","last_became_active":"2024-09-11T12:53:49.982983+0000","last_became_peered":"2024-09-11T12:53:49.982983+0000","last_unstale":"2024-09-11T12:53:49.995368+0000","last_undegraded":"2024-09-11T12:53:49.995368+0000","last_fullsized":"2024-09-11T12:53:49.995368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:10.111906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.988462+0000","last_change":"2024-09-11T12:53:49.985655+0000","last_active":"2024-09-11T12:53:49.988462+0000","last_peered":"2024-09-11T12:53:49.988462+0000","last_clean":"2024-09-11T12:53:49.988462+0000","last_became_active":"2024-09-11T12:53:49.985363+0000","last_became_peered":"2024-09-11T12:53:49.985363+0000","last_unstale":"2024-09-11T12:53:49.988462+0000","last_undegraded":"2024-09-11T12:53:49.988462+0000","last_fullsized":"2024-09-11T12:53:49.988462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:40.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.998892+0000","last_change":"2024-09-11T12:53:49.990010+0000","last_active":"2024-09-11T12:53:49.998892+0000","last_peered":"2024-09-11T12:53:49.998892+0000","last_clean":"2024-09-11T12:53:49.998892+0000","last_became_active":"2024-09-11T12:53:49.989731+0000","last_became_peered":"2024-09-11T12:53:49.989731+0000","last_unstale":"2024-09-11T12:53:49.998892+0000","last_undegraded":"2024-09-11T12:53:49.998892+0000","last_fullsized":"2024-09-11T12:53:49.998892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:36.648601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995972+0000","last_change":"2024-09-11T12:53:49.991096+0000","last_active":"2024-09-11T12:53:49.995972+0000","last_peered":"2024-09-11T12:53:49.995972+0000","last_clean":"2024-09-11T12:53:49.995972+0000","last_became_active":"2024-09-11T12:53:49.990845+0000","last_became_peered":"2024-09-11T12:53:49.990845+0000","last_unstale":"2024-09-11T12:53:49.995972+0000","last_undegraded":"2024-09-11T12:53:49.995972+0000","last_fullsized":"2024-09-11T12:53:49.995972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.986044+0000","last_change":"2024-09-11T12:53:49.984485+0000","last_active":"2024-09-11T12:53:49.986044+0000","last_peered":"2024-09-11T12:53:49.986044+0000","last_clean":"2024-09-11T12:53:49.986044+0000","last_became_active":"2024-09-11T12:53:49.984207+0000","last_became_peered":"2024-09-11T12:53:49.984207+0000","last_unstale":"2024-09-11T12:53:49.986044+0000","last_undegraded":"2024-09-11T12:53:49.986044+0000","last_fullsized":"2024-09-11T12:53:49.986044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:33.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995992+0000","last_change":"2024-09-11T12:53:49.987810+0000","last_active":"2024-09-11T12:53:49.995992+0000","last_peered":"2024-09-11T12:53:49.995992+0000","last_clean":"2024-09-11T12:53:49.995992+0000","last_became_active":"2024-09-11T12:53:49.987501+0000","last_became_peered":"2024-09-11T12:53:49.987501+0000","last_unstale":"2024-09-11T12:53:49.995992+0000","last_undegraded":"2024-09-11T12:53:49.995992+0000","last_fullsized":"2024-09-11T12:53:49.995992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991466+0000","last_change":"2024-09-11T12:53:49.980601+0000","last_active":"2024-09-11T12:53:49.991466+0000","last_peered":"2024-09-11T12:53:49.991466+0000","last_clean":"2024-09-11T12:53:49.991466+0000","last_became_active":"2024-09-11T12:53:49.980282+0000","last_became_peered":"2024-09-11T12:53:49.980282+0000","last_unstale":"2024-09-11T12:53:49.991466+0000","last_undegraded":"2024-09-11T12:53:49.991466+0000","last_fullsized":"2024-09-11T12:53:49.991466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:26.129355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996457+0000","last_change":"2024-09-11T12:53:49.988000+0000","last_active":"2024-09-11T12:53:49.996457+0000","last_peered":"2024-09-11T12:53:49.996457+0000","last_clean":"2024-09-11T12:53:49.996457+0000","last_became_active":"2024-09-11T12:53:49.987724+0000","last_became_peered":"2024-09-11T12:53:49.987724+0000","last_unstale":"2024-09-11T12:53:49.996457+0000","last_undegraded":"2024-09-11T12:53:49.996457+0000","last_fullsized":"2024-09-11T12:53:49.996457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:41.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995481+0000","last_change":"2024-09-11T12:53:49.991124+0000","last_active":"2024-09-11T12:53:49.995481+0000","last_peered":"2024-09-11T12:53:49.995481+0000","last_clean":"2024-09-11T12:53:49.995481+0000","last_became_active":"2024-09-11T12:53:49.990800+0000","last_became_peered":"2024-09-11T12:53:49.990800+0000","last_unstale":"2024-09-11T12:53:49.995481+0000","last_undegraded":"2024-09-11T12:53:49.995481+0000","last_fullsized":"2024-09-11T12:53:49.995481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:23.205971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005879+0000","last_change":"2024-09-11T12:53:49.999885+0000","last_active":"2024-09-11T12:53:50.005879+0000","last_peered":"2024-09-11T12:53:50.005879+0000","last_clean":"2024-09-11T12:53:50.005879+0000","last_became_active":"2024-09-11T12:53:49.999408+0000","last_became_peered":"2024-09-11T12:53:49.999408+0000","last_unstale":"2024-09-11T12:53:50.005879+0000","last_undegraded":"2024-09-11T12:53:50.005879+0000","last_fullsized":"2024-09-11T12:53:50.005879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.647957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992210+0000","last_change":"2024-09-11T12:53:49.989420+0000","last_active":"2024-09-11T12:53:49.992210+0000","last_peered":"2024-09-11T12:53:49.992210+0000","last_clean":"2024-09-11T12:53:49.992210+0000","last_became_active":"2024-09-11T12:53:49.989158+0000","last_became_peered":"2024-09-11T12:53:49.989158+0000","last_unstale":"2024-09-11T12:53:49.992210+0000","last_undegraded":"2024-09-11T12:53:49.992210+0000","last_fullsized":"2024-09-11T12:53:49.992210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:13.311775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.990014+0000","last_change":"2024-09-11T12:53:49.980155+0000","last_active":"2024-09-11T12:53:49.990014+0000","last_peered":"2024-09-11T12:53:49.990014+0000","last_clean":"2024-09-11T12:53:49.990014+0000","last_became_active":"2024-09-11T12:53:49.979606+0000","last_became_peered":"2024-09-11T12:53:49.979606+0000","last_unstale":"2024-09-11T12:53:49.990014+0000","last_undegraded":"2024-09-11T12:53:49.990014+0000","last_fullsized":"2024-09-11T12:53:49.990014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:34.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997806+0000","last_change":"2024-09-11T12:53:49.991908+0000","last_active":"2024-09-11T12:53:49.997806+0000","last_peered":"2024-09-11T12:53:49.997806+0000","last_clean":"2024-09-11T12:53:49.997806+0000","last_became_active":"2024-09-11T12:53:49.991616+0000","last_became_peered":"2024-09-11T12:53:49.991616+0000","last_unstale":"2024-09-11T12:53:49.997806+0000","last_undegraded":"2024-09-11T12:53:49.997806+0000","last_fullsized":"2024-09-11T12:53:49.997806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:40.902648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.998916+0000","last_change":"2024-09-11T12:53:49.991414+0000","last_active":"2024-09-11T12:53:49.998916+0000","last_peered":"2024-09-11T12:53:49.998916+0000","last_clean":"2024-09-11T12:53:49.998916+0000","last_became_active":"2024-09-11T12:53:49.991141+0000","last_became_peered":"2024-09-11T12:53:49.991141+0000","last_unstale":"2024-09-11T12:53:49.998916+0000","last_undegraded":"2024-09-11T12:53:49.998916+0000","last_fullsized":"2024-09-11T12:53:49.998916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:19.286133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.003600+0000","last_change":"2024-09-11T12:53:49.994210+0000","last_active":"2024-09-11T12:53:50.003600+0000","last_peered":"2024-09-11T12:53:50.003600+0000","last_clean":"2024-09-11T12:53:50.003600+0000","last_became_active":"2024-09-11T12:53:49.993930+0000","last_became_peered":"2024-09-11T12:53:49.993930+0000","last_unstale":"2024-09-11T12:53:50.003600+0000","last_undegraded":"2024-09-11T12:53:50.003600+0000","last_fullsized":"2024-09-11T12:53:50.003600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:12.949102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005396+0000","last_change":"2024-09-11T12:53:49.998011+0000","last_active":"2024-09-11T12:53:50.005396+0000","last_peered":"2024-09-11T12:53:50.005396+0000","last_clean":"2024-09-11T12:53:50.005396+0000","last_became_active":"2024-09-11T12:53:49.997653+0000","last_became_peered":"2024-09-11T12:53:49.997653+0000","last_unstale":"2024-09-11T12:53:50.005396+0000","last_undegraded":"2024-09-11T12:53:50.005396+0000","last_fullsized":"2024-09-11T12:53:50.005396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:58.612685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009226+0000","last_change":"2024-09-11T12:53:50.005651+0000","last_active":"2024-09-11T12:53:50.009226+0000","last_peered":"2024-09-11T12:53:50.009226+0000","last_clean":"2024-09-11T12:53:50.009226+0000","last_became_active":"2024-09-11T12:53:50.005350+0000","last_became_peered":"2024-09-11T12:53:50.005350+0000","last_unstale":"2024-09-11T12:53:50.009226+0000","last_undegraded":"2024-09-11T12:53:50.009226+0000","last_fullsized":"2024-09-11T12:53:50.009226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:19.610993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.004768+0000","last_change":"2024-09-11T12:53:49.996315+0000","last_active":"2024-09-11T12:53:50.004768+0000","last_peered":"2024-09-11T12:53:50.004768+0000","last_clean":"2024-09-11T12:53:50.004768+0000","last_became_active":"2024-09-11T12:53:49.996007+0000","last_became_peered":"2024-09-11T12:53:49.996007+0000","last_unstale":"2024-09-11T12:53:50.004768+0000","last_undegraded":"2024-09-11T12:53:50.004768+0000","last_fullsized":"2024-09-11T12:53:50.004768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:54.037285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999911+0000","last_change":"2024-09-11T12:53:49.998031+0000","last_active":"2024-09-11T12:53:49.999911+0000","last_peered":"2024-09-11T12:53:49.999911+0000","last_clean":"2024-09-11T12:53:49.999911+0000","last_became_active":"2024-09-11T12:53:49.997689+0000","last_became_peered":"2024-09-11T12:53:49.997689+0000","last_unstale":"2024-09-11T12:53:49.999911+0000","last_undegraded":"2024-09-11T12:53:49.999911+0000","last_fullsized":"2024-09-11T12:53:49.999911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:59.620107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005611+0000","last_change":"2024-09-11T12:53:49.998056+0000","last_active":"2024-09-11T12:53:50.005611+0000","last_peered":"2024-09-11T12:53:50.005611+0000","last_clean":"2024-09-11T12:53:50.005611+0000","last_became_active":"2024-09-11T12:53:49.997757+0000","last_became_peered":"2024-09-11T12:53:49.997757+0000","last_unstale":"2024-09-11T12:53:50.005611+0000","last_undegraded":"2024-09-11T12:53:50.005611+0000","last_fullsized":"2024-09-11T12:53:50.005611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:05.572626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991639+0000","last_change":"2024-09-11T12:53:49.983507+0000","last_active":"2024-09-11T12:53:49.991639+0000","last_peered":"2024-09-11T12:53:49.991639+0000","last_clean":"2024-09-11T12:53:49.991639+0000","last_became_active":"2024-09-11T12:53:49.983229+0000","last_became_peered":"2024-09-11T12:53:49.983229+0000","last_unstale":"2024-09-11T12:53:49.991639+0000","last_undegraded":"2024-09-11T12:53:49.991639+0000","last_fullsized":"2024-09-11T12:53:49.991639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:37:26.437611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001447+0000","last_change":"2024-09-11T12:53:49.999888+0000","last_active":"2024-09-11T12:53:50.001447+0000","last_peered":"2024-09-11T12:53:50.001447+0000","last_clean":"2024-09-11T12:53:50.001447+0000","last_became_active":"2024-09-11T12:53:49.999564+0000","last_became_peered":"2024-09-11T12:53:49.999564+0000","last_unstale":"2024-09-11T12:53:50.001447+0000","last_undegraded":"2024-09-11T12:53:50.001447+0000","last_fullsized":"2024-09-11T12:53:50.001447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.149519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995680+0000","last_change":"2024-09-11T12:53:49.986916+0000","last_active":"2024-09-11T12:53:49.995680+0000","last_peered":"2024-09-11T12:53:49.995680+0000","last_clean":"2024-09-11T12:53:49.995680+0000","last_became_active":"2024-09-11T12:53:49.986485+0000","last_became_peered":"2024-09-11T12:53:49.986485+0000","last_unstale":"2024-09-11T12:53:49.995680+0000","last_undegraded":"2024-09-11T12:53:49.995680+0000","last_fullsized":"2024-09-11T12:53:49.995680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:37.908752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000145+0000","last_change":"2024-09-11T12:53:49.998436+0000","last_active":"2024-09-11T12:53:50.000145+0000","last_peered":"2024-09-11T12:53:50.000145+0000","last_clean":"2024-09-11T12:53:50.000145+0000","last_became_active":"2024-09-11T12:53:49.998073+0000","last_became_peered":"2024-09-11T12:53:49.998073+0000","last_unstale":"2024-09-11T12:53:50.000145+0000","last_undegraded":"2024-09-11T12:53:50.000145+0000","last_fullsized":"2024-09-11T12:53:50.000145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:45.722422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009092+0000","last_change":"2024-09-11T12:53:50.001373+0000","last_active":"2024-09-11T12:53:50.009092+0000","last_peered":"2024-09-11T12:53:50.009092+0000","last_clean":"2024-09-11T12:53:50.009092+0000","last_became_active":"2024-09-11T12:53:49.996893+0000","last_became_peered":"2024-09-11T12:53:49.996893+0000","last_unstale":"2024-09-11T12:53:50.009092+0000","last_undegraded":"2024-09-11T12:53:50.009092+0000","last_fullsized":"2024-09-11T12:53:50.009092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:23.270789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996197+0000","last_change":"2024-09-11T12:53:49.986827+0000","last_active":"2024-09-11T12:53:49.996197+0000","last_peered":"2024-09-11T12:53:49.996197+0000","last_clean":"2024-09-11T12:53:49.996197+0000","last_became_active":"2024-09-11T12:53:49.986399+0000","last_became_peered":"2024-09-11T12:53:49.986399+0000","last_unstale":"2024-09-11T12:53:49.996197+0000","last_undegraded":"2024-09-11T12:53:49.996197+0000","last_fullsized":"2024-09-11T12:53:49.996197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:04.156160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997753+0000","last_change":"2024-09-11T12:53:49.995428+0000","last_active":"2024-09-11T12:53:49.997753+0000","last_peered":"2024-09-11T12:53:49.997753+0000","last_clean":"2024-09-11T12:53:49.997753+0000","last_became_active":"2024-09-11T12:53:49.995145+0000","last_became_peered":"2024-09-11T12:53:49.995145+0000","last_unstale":"2024-09-11T12:53:49.997753+0000","last_undegraded":"2024-09-11T12:53:49.997753+0000","last_fullsized":"2024-09-11T12:53:49.997753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997283+0000","last_change":"2024-09-11T12:53:49.994112+0000","last_active":"2024-09-11T12:53:49.997283+0000","last_peered":"2024-09-11T12:53:49.997283+0000","last_clean":"2024-09-11T12:53:49.997283+0000","last_became_active":"2024-09-11T12:53:49.993589+0000","last_became_peered":"2024-09-11T12:53:49.993589+0000","last_unstale":"2024-09-11T12:53:49.997283+0000","last_undegraded":"2024-09-11T12:53:49.997283+0000","last_fullsized":"2024-09-11T12:53:49.997283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:38.895803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996475+0000","last_change":"2024-09-11T12:53:49.992127+0000","last_active":"2024-09-11T12:53:49.996475+0000","last_peered":"2024-09-11T12:53:49.996475+0000","last_clean":"2024-09-11T12:53:49.996475+0000","last_became_active":"2024-09-11T12:53:49.989888+0000","last_became_peered":"2024-09-11T12:53:49.989888+0000","last_unstale":"2024-09-11T12:53:49.996475+0000","last_undegraded":"2024-09-11T12:53:49.996475+0000","last_fullsized":"2024-09-11T12:53:49.996475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:40.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.007197+0000","last_change":"2024-09-11T12:53:48.989674+0000","last_active":"2024-09-11T12:53:50.007197+0000","last_peered":"2024-09-11T12:53:50.007197+0000","last_clean":"2024-09-11T12:53:50.007197+0000","last_became_active":"2024-09-11T12:53:48.989340+0000","last_became_peered":"2024-09-11T12:53:48.989340+0000","last_unstale":"2024-09-11T12:53:50.007197+0000","last_undegraded":"2024-09-11T12:53:50.007197+0000","last_fullsized":"2024-09-11T12:53:50.007197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:19.587352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000319+0000","last_change":"2024-09-11T12:53:49.993875+0000","last_active":"2024-09-11T12:53:50.000319+0000","last_peered":"2024-09-11T12:53:50.000319+0000","last_clean":"2024-09-11T12:53:50.000319+0000","last_became_active":"2024-09-11T12:53:49.992356+0000","last_became_peered":"2024-09-11T12:53:49.992356+0000","last_unstale":"2024-09-11T12:53:50.000319+0000","last_undegraded":"2024-09-11T12:53:50.000319+0000","last_fullsized":"2024-09-11T12:53:50.000319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:15.858073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994950+0000","last_change":"2024-09-11T12:53:48.976033+0000","last_active":"2024-09-11T12:53:49.994950+0000","last_peered":"2024-09-11T12:53:49.994950+0000","last_clean":"2024-09-11T12:53:49.994950+0000","last_became_active":"2024-09-11T12:53:48.975625+0000","last_became_peered":"2024-09-11T12:53:48.975625+0000","last_unstale":"2024-09-11T12:53:49.994950+0000","last_undegraded":"2024-09-11T12:53:49.994950+0000","last_fullsized":"2024-09-11T12:53:49.994950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:30.333424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997242+0000","last_change":"2024-09-11T12:53:49.989424+0000","last_active":"2024-09-11T12:53:49.997242+0000","last_peered":"2024-09-11T12:53:49.997242+0000","last_clean":"2024-09-11T12:53:49.997242+0000","last_became_active":"2024-09-11T12:53:49.989145+0000","last_became_peered":"2024-09-11T12:53:49.989145+0000","last_unstale":"2024-09-11T12:53:49.997242+0000","last_undegraded":"2024-09-11T12:53:49.997242+0000","last_fullsized":"2024-09-11T12:53:49.997242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:16.491656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.003876+0000","last_change":"2024-09-11T12:53:48.988295+0000","last_active":"2024-09-11T12:53:50.003876+0000","last_peered":"2024-09-11T12:53:50.003876+0000","last_clean":"2024-09-11T12:53:50.003876+0000","last_became_active":"2024-09-11T12:53:48.987797+0000","last_became_peered":"2024-09-11T12:53:48.987797+0000","last_unstale":"2024-09-11T12:53:50.003876+0000","last_undegraded":"2024-09-11T12:53:50.003876+0000","last_fullsized":"2024-09-11T12:53:50.003876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:04:24.610208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995006+0000","last_change":"2024-09-11T12:53:49.990699+0000","last_active":"2024-09-11T12:53:49.995006+0000","last_peered":"2024-09-11T12:53:49.995006+0000","last_clean":"2024-09-11T12:53:49.995006+0000","last_became_active":"2024-09-11T12:53:49.990224+0000","last_became_peered":"2024-09-11T12:53:49.990224+0000","last_unstale":"2024-09-11T12:53:49.995006+0000","last_undegraded":"2024-09-11T12:53:49.995006+0000","last_fullsized":"2024-09-11T12:53:49.995006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:16.076036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.002549+0000","last_change":"2024-09-11T12:53:48.988534+0000","last_active":"2024-09-11T12:53:50.002549+0000","last_peered":"2024-09-11T12:53:50.002549+0000","last_clean":"2024-09-11T12:53:50.002549+0000","last_became_active":"2024-09-11T12:53:48.988129+0000","last_became_peered":"2024-09-11T12:53:48.988129+0000","last_unstale":"2024-09-11T12:53:50.002549+0000","last_undegraded":"2024-09-11T12:53:50.002549+0000","last_fullsized":"2024-09-11T12:53:50.002549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:35:21.266551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.006351+0000","last_change":"2024-09-11T12:53:50.000099+0000","last_active":"2024-09-11T12:53:50.006351+0000","last_peered":"2024-09-11T12:53:50.006351+0000","last_clean":"2024-09-11T12:53:50.006351+0000","last_became_active":"2024-09-11T12:53:49.987102+0000","last_became_peered":"2024-09-11T12:53:49.987102+0000","last_unstale":"2024-09-11T12:53:50.006351+0000","last_undegraded":"2024-09-11T12:53:50.006351+0000","last_fullsized":"2024-09-11T12:53:50.006351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:54.863383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001874+0000","last_change":"2024-09-11T12:53:48.981414+0000","last_active":"2024-09-11T12:53:50.001874+0000","last_peered":"2024-09-11T12:53:50.001874+0000","last_clean":"2024-09-11T12:53:50.001874+0000","last_became_active":"2024-09-11T12:53:48.981095+0000","last_became_peered":"2024-09-11T12:53:48.981095+0000","last_unstale":"2024-09-11T12:53:50.001874+0000","last_undegraded":"2024-09-11T12:53:50.001874+0000","last_fullsized":"2024-09-11T12:53:50.001874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:45.142820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996177+0000","last_change":"2024-09-11T12:53:49.991481+0000","last_active":"2024-09-11T12:53:49.996177+0000","last_peered":"2024-09-11T12:53:49.996177+0000","last_clean":"2024-09-11T12:53:49.996177+0000","last_became_active":"2024-09-11T12:53:49.991206+0000","last_became_peered":"2024-09-11T12:53:49.991206+0000","last_unstale":"2024-09-11T12:53:49.996177+0000","last_undegraded":"2024-09-11T12:53:49.996177+0000","last_fullsized":"2024-09-11T12:53:49.996177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:33.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994907+0000","last_change":"2024-09-11T12:53:48.988457+0000","last_active":"2024-09-11T12:53:49.994907+0000","last_peered":"2024-09-11T12:53:49.994907+0000","last_clean":"2024-09-11T12:53:49.994907+0000","last_became_active":"2024-09-11T12:53:48.988135+0000","last_became_peered":"2024-09-11T12:53:48.988135+0000","last_unstale":"2024-09-11T12:53:49.994907+0000","last_undegraded":"2024-09-11T12:53:49.994907+0000","last_fullsized":"2024-09-11T12:53:49.994907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:53.860743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995729+0000","last_change":"2024-09-11T12:53:49.991780+0000","last_active":"2024-09-11T12:53:49.995729+0000","last_peered":"2024-09-11T12:53:49.995729+0000","last_clean":"2024-09-11T12:53:49.995729+0000","last_became_active":"2024-09-11T12:53:49.991047+0000","last_became_peered":"2024-09-11T12:53:49.991047+0000","last_unstale":"2024-09-11T12:53:49.995729+0000","last_undegraded":"2024-09-11T12:53:49.995729+0000","last_fullsized":"2024-09-11T12:53:49.995729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:25.066295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976127+0000","last_change":"2024-09-11T12:53:48.977234+0000","last_active":"2024-09-11T12:53:49.976127+0000","last_peered":"2024-09-11T12:53:49.976127+0000","last_clean":"2024-09-11T12:53:49.976127+0000","last_became_active":"2024-09-11T12:53:48.976961+0000","last_became_peered":"2024-09-11T12:53:48.976961+0000","last_unstale":"2024-09-11T12:53:49.976127+0000","last_undegraded":"2024-09-11T12:53:49.976127+0000","last_fullsized":"2024-09-11T12:53:49.976127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:57:05.911791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992185+0000","last_change":"2024-09-11T12:53:49.989214+0000","last_active":"2024-09-11T12:53:49.992185+0000","last_peered":"2024-09-11T12:53:49.992185+0000","last_clean":"2024-09-11T12:53:49.992185+0000","last_became_active":"2024-09-11T12:53:49.988905+0000","last_became_peered":"2024-09-11T12:53:49.988905+0000","last_unstale":"2024-09-11T12:53:49.992185+0000","last_undegraded":"2024-09-11T12:53:49.992185+0000","last_fullsized":"2024-09-11T12:53:49.992185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.149519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.966505+0000","last_change":"2024-09-11T12:53:48.979739+0000","last_active":"2024-09-11T12:53:49.966505+0000","last_peered":"2024-09-11T12:53:49.966505+0000","last_clean":"2024-09-11T12:53:49.966505+0000","last_became_active":"2024-09-11T12:53:48.978901+0000","last_became_peered":"2024-09-11T12:53:48.978901+0000","last_unstale":"2024-09-11T12:53:49.966505+0000","last_undegraded":"2024-09-11T12:53:49.966505+0000","last_fullsized":"2024-09-11T12:53:49.966505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:22.054812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.007253+0000","last_change":"2024-09-11T12:53:49.999653+0000","last_active":"2024-09-11T12:53:50.007253+0000","last_peered":"2024-09-11T12:53:50.007253+0000","last_clean":"2024-09-11T12:53:50.007253+0000","last_became_active":"2024-09-11T12:53:49.999348+0000","last_became_peered":"2024-09-11T12:53:49.999348+0000","last_unstale":"2024-09-11T12:53:50.007253+0000","last_undegraded":"2024-09-11T12:53:50.007253+0000","last_fullsized":"2024-09-11T12:53:50.007253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:06:46.408543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.975970+0000","last_change":"2024-09-11T12:53:48.989859+0000","last_active":"2024-09-11T12:53:49.975970+0000","last_peered":"2024-09-11T12:53:49.975970+0000","last_clean":"2024-09-11T12:53:49.975970+0000","last_became_active":"2024-09-11T12:53:48.989571+0000","last_became_peered":"2024-09-11T12:53:48.989571+0000","last_unstale":"2024-09-11T12:53:49.975970+0000","last_undegraded":"2024-09-11T12:53:49.975970+0000","last_fullsized":"2024-09-11T12:53:49.975970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:36.906112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.988885+0000","last_change":"2024-09-11T12:53:49.986709+0000","last_active":"2024-09-11T12:53:49.988885+0000","last_peered":"2024-09-11T12:53:49.988885+0000","last_clean":"2024-09-11T12:53:49.988885+0000","last_became_active":"2024-09-11T12:53:49.986435+0000","last_became_peered":"2024-09-11T12:53:49.986435+0000","last_unstale":"2024-09-11T12:53:49.988885+0000","last_undegraded":"2024-09-11T12:53:49.988885+0000","last_fullsized":"2024-09-11T12:53:49.988885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.977195+0000","last_change":"2024-09-11T12:53:48.983097+0000","last_active":"2024-09-11T12:53:49.977195+0000","last_peered":"2024-09-11T12:53:49.977195+0000","last_clean":"2024-09-11T12:53:49.977195+0000","last_became_active":"2024-09-11T12:53:48.981963+0000","last_became_peered":"2024-09-11T12:53:48.981963+0000","last_unstale":"2024-09-11T12:53:49.977195+0000","last_undegraded":"2024-09-11T12:53:49.977195+0000","last_fullsized":"2024-09-11T12:53:49.977195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:39.963022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.993888+0000","last_change":"2024-09-11T12:53:49.992429+0000","last_active":"2024-09-11T12:53:49.993888+0000","last_peered":"2024-09-11T12:53:49.993888+0000","last_clean":"2024-09-11T12:53:49.993888+0000","last_became_active":"2024-09-11T12:53:49.990492+0000","last_became_peered":"2024-09-11T12:53:49.990492+0000","last_unstale":"2024-09-11T12:53:49.993888+0000","last_undegraded":"2024-09-11T12:53:49.993888+0000","last_fullsized":"2024-09-11T12:53:49.993888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:19.725349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.006252+0000","last_change":"2024-09-11T12:53:48.987581+0000","last_active":"2024-09-11T12:53:50.006252+0000","last_peered":"2024-09-11T12:53:50.006252+0000","last_clean":"2024-09-11T12:53:50.006252+0000","last_became_active":"2024-09-11T12:53:48.987236+0000","last_became_peered":"2024-09-11T12:53:48.987236+0000","last_unstale":"2024-09-11T12:53:50.006252+0000","last_undegraded":"2024-09-11T12:53:50.006252+0000","last_fullsized":"2024-09-11T12:53:50.006252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.164278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997653+0000","last_change":"2024-09-11T12:53:49.995746+0000","last_active":"2024-09-11T12:53:49.997653+0000","last_peered":"2024-09-11T12:53:49.997653+0000","last_clean":"2024-09-11T12:53:49.997653+0000","last_became_active":"2024-09-11T12:53:49.994589+0000","last_became_peered":"2024-09-11T12:53:49.994589+0000","last_unstale":"2024-09-11T12:53:49.997653+0000","last_undegraded":"2024-09-11T12:53:49.997653+0000","last_fullsized":"2024-09-11T12:53:49.997653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009139+0000","last_change":"2024-09-11T12:53:48.983481+0000","last_active":"2024-09-11T12:53:50.009139+0000","last_peered":"2024-09-11T12:53:50.009139+0000","last_clean":"2024-09-11T12:53:50.009139+0000","last_became_active":"2024-09-11T12:53:48.982118+0000","last_became_peered":"2024-09-11T12:53:48.982118+0000","last_unstale":"2024-09-11T12:53:50.009139+0000","last_undegraded":"2024-09-11T12:53:50.009139+0000","last_fullsized":"2024-09-11T12:53:50.009139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:36.906112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992455+0000","last_change":"2024-09-11T12:53:49.985738+0000","last_active":"2024-09-11T12:53:49.992455+0000","last_peered":"2024-09-11T12:53:49.992455+0000","last_clean":"2024-09-11T12:53:49.992455+0000","last_became_active":"2024-09-11T12:53:49.985234+0000","last_became_peered":"2024-09-11T12:53:49.985234+0000","last_unstale":"2024-09-11T12:53:49.992455+0000","last_undegraded":"2024-09-11T12:53:49.992455+0000","last_fullsized":"2024-09-11T12:53:49.992455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:33:53.806460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001675+0000","last_change":"2024-09-11T12:53:48.982676+0000","last_active":"2024-09-11T12:53:50.001675+0000","last_peered":"2024-09-11T12:53:50.001675+0000","last_clean":"2024-09-11T12:53:50.001675+0000","last_became_active":"2024-09-11T12:53:48.982174+0000","last_became_peered":"2024-09-11T12:53:48.982174+0000","last_unstale":"2024-09-11T12:53:50.001675+0000","last_undegraded":"2024-09-11T12:53:50.001675+0000","last_fullsized":"2024-09-11T12:53:50.001675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:38.159456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995711+0000","last_change":"2024-09-11T12:53:49.992391+0000","last_active":"2024-09-11T12:53:49.995711+0000","last_peered":"2024-09-11T12:53:49.995711+0000","last_clean":"2024-09-11T12:53:49.995711+0000","last_became_active":"2024-09-11T12:53:49.990788+0000","last_became_peered":"2024-09-11T12:53:49.990788+0000","last_unstale":"2024-09-11T12:53:49.995711+0000","last_undegraded":"2024-09-11T12:53:49.995711+0000","last_fullsized":"2024-09-11T12:53:49.995711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:07.402433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.979401+0000","last_change":"2024-09-11T12:53:48.984403+0000","last_active":"2024-09-11T12:53:49.979401+0000","last_peered":"2024-09-11T12:53:49.979401+0000","last_clean":"2024-09-11T12:53:49.979401+0000","last_became_active":"2024-09-11T12:53:48.983571+0000","last_became_peered":"2024-09-11T12:53:48.983571+0000","last_unstale":"2024-09-11T12:53:49.979401+0000","last_undegraded":"2024-09-11T12:53:49.979401+0000","last_fullsized":"2024-09-11T12:53:49.979401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:06.399793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992622+0000","last_change":"2024-09-11T12:53:49.987214+0000","last_active":"2024-09-11T12:53:49.992622+0000","last_peered":"2024-09-11T12:53:49.992622+0000","last_clean":"2024-09-11T12:53:49.992622+0000","last_became_active":"2024-09-11T12:53:49.986896+0000","last_became_peered":"2024-09-11T12:53:49.986896+0000","last_unstale":"2024-09-11T12:53:49.992622+0000","last_undegraded":"2024-09-11T12:53:49.992622+0000","last_fullsized":"2024-09-11T12:53:49.992622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:40.280459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.968784+0000","last_change":"2024-09-11T12:53:48.974701+0000","last_active":"2024-09-11T12:53:49.968784+0000","last_peered":"2024-09-11T12:53:49.968784+0000","last_clean":"2024-09-11T12:53:49.968784+0000","last_became_active":"2024-09-11T12:53:48.974401+0000","last_became_peered":"2024-09-11T12:53:48.974401+0000","last_unstale":"2024-09-11T12:53:49.968784+0000","last_undegraded":"2024-09-11T12:53:49.968784+0000","last_fullsized":"2024-09-11T12:53:49.968784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:19.587352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991733+0000","last_change":"2024-09-11T12:53:49.981695+0000","last_active":"2024-09-11T12:53:49.991733+0000","last_peered":"2024-09-11T12:53:49.991733+0000","last_clean":"2024-09-11T12:53:49.991733+0000","last_became_active":"2024-09-11T12:53:49.981491+0000","last_became_peered":"2024-09-11T12:53:49.981491+0000","last_unstale":"2024-09-11T12:53:49.991733+0000","last_undegraded":"2024-09-11T12:53:49.991733+0000","last_fullsized":"2024-09-11T12:53:49.991733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:05.409971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999856+0000","last_change":"2024-09-11T12:53:48.979123+0000","last_active":"2024-09-11T12:53:49.999856+0000","last_peered":"2024-09-11T12:53:49.999856+0000","last_clean":"2024-09-11T12:53:49.999856+0000","last_became_active":"2024-09-11T12:53:48.978885+0000","last_became_peered":"2024-09-11T12:53:48.978885+0000","last_unstale":"2024-09-11T12:53:49.999856+0000","last_undegraded":"2024-09-11T12:53:49.999856+0000","last_fullsized":"2024-09-11T12:53:49.999856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:52.614926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.985319+0000","last_change":"2024-09-11T12:53:49.980347+0000","last_active":"2024-09-11T12:53:49.985319+0000","last_peered":"2024-09-11T12:53:49.985319+0000","last_clean":"2024-09-11T12:53:49.985319+0000","last_became_active":"2024-09-11T12:53:49.980066+0000","last_became_peered":"2024-09-11T12:53:49.980066+0000","last_unstale":"2024-09-11T12:53:49.985319+0000","last_undegraded":"2024-09-11T12:53:49.985319+0000","last_fullsized":"2024-09-11T12:53:49.985319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:59.620107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000275+0000","last_change":"2024-09-11T12:53:48.985225+0000","last_active":"2024-09-11T12:53:50.000275+0000","last_peered":"2024-09-11T12:53:50.000275+0000","last_clean":"2024-09-11T12:53:50.000275+0000","last_became_active":"2024-09-11T12:53:48.984903+0000","last_became_peered":"2024-09-11T12:53:48.984903+0000","last_unstale":"2024-09-11T12:53:50.000275+0000","last_undegraded":"2024-09-11T12:53:50.000275+0000","last_fullsized":"2024-09-11T12:53:50.000275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:39.900007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995155+0000","last_change":"2024-09-11T12:53:49.986150+0000","last_active":"2024-09-11T12:53:49.995155+0000","last_peered":"2024-09-11T12:53:49.995155+0000","last_clean":"2024-09-11T12:53:49.995155+0000","last_became_active":"2024-09-11T12:53:49.985818+0000","last_became_peered":"2024-09-11T12:53:49.985818+0000","last_unstale":"2024-09-11T12:53:49.995155+0000","last_undegraded":"2024-09-11T12:53:49.995155+0000","last_fullsized":"2024-09-11T12:53:49.995155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.002320+0000","last_change":"2024-09-11T12:53:48.982560+0000","last_active":"2024-09-11T12:53:50.002320+0000","last_peered":"2024-09-11T12:53:50.002320+0000","last_clean":"2024-09-11T12:53:50.002320+0000","last_became_active":"2024-09-11T12:53:48.982240+0000","last_became_peered":"2024-09-11T12:53:48.982240+0000","last_unstale":"2024-09-11T12:53:50.002320+0000","last_undegraded":"2024-09-11T12:53:50.002320+0000","last_fullsized":"2024-09-11T12:53:50.002320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:23.496509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995656+0000","last_change":"2024-09-11T12:53:49.984067+0000","last_active":"2024-09-11T12:53:49.995656+0000","last_peered":"2024-09-11T12:53:49.995656+0000","last_clean":"2024-09-11T12:53:49.995656+0000","last_became_active":"2024-09-11T12:53:49.983499+0000","last_became_peered":"2024-09-11T12:53:49.983499+0000","last_unstale":"2024-09-11T12:53:49.995656+0000","last_undegraded":"2024-09-11T12:53:49.995656+0000","last_fullsized":"2024-09-11T12:53:49.995656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:20.589992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.980167+0000","last_change":"2024-09-11T12:53:48.985614+0000","last_active":"2024-09-11T12:53:49.980167+0000","last_peered":"2024-09-11T12:53:49.980167+0000","last_clean":"2024-09-11T12:53:49.980167+0000","last_became_active":"2024-09-11T12:53:48.985328+0000","last_became_peered":"2024-09-11T12:53:48.985328+0000","last_unstale":"2024-09-11T12:53:49.980167+0000","last_undegraded":"2024-09-11T12:53:49.980167+0000","last_fullsized":"2024-09-11T12:53:49.980167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:23.219983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005773+0000","last_change":"2024-09-11T12:53:49.999655+0000","last_active":"2024-09-11T12:53:50.005773+0000","last_peered":"2024-09-11T12:53:50.005773+0000","last_clean":"2024-09-11T12:53:50.005773+0000","last_became_active":"2024-09-11T12:53:49.999064+0000","last_became_peered":"2024-09-11T12:53:49.999064+0000","last_unstale":"2024-09-11T12:53:50.005773+0000","last_undegraded":"2024-09-11T12:53:50.005773+0000","last_fullsized":"2024-09-11T12:53:50.005773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:09.932147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.004538+0000","last_change":"2024-09-11T12:53:48.985368+0000","last_active":"2024-09-11T12:53:50.004538+0000","last_peered":"2024-09-11T12:53:50.004538+0000","last_clean":"2024-09-11T12:53:50.004538+0000","last_became_active":"2024-09-11T12:53:48.984458+0000","last_became_peered":"2024-09-11T12:53:48.984458+0000","last_unstale":"2024-09-11T12:53:50.004538+0000","last_undegraded":"2024-09-11T12:53:50.004538+0000","last_fullsized":"2024-09-11T12:53:50.004538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:11.089175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996568+0000","last_change":"2024-09-11T12:53:49.992181+0000","last_active":"2024-09-11T12:53:49.996568+0000","last_peered":"2024-09-11T12:53:49.996568+0000","last_clean":"2024-09-11T12:53:49.996568+0000","last_became_active":"2024-09-11T12:53:49.991937+0000","last_became_peered":"2024-09-11T12:53:49.991937+0000","last_unstale":"2024-09-11T12:53:49.996568+0000","last_undegraded":"2024-09-11T12:53:49.996568+0000","last_fullsized":"2024-09-11T12:53:49.996568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:41.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.984371+0000","last_change":"2024-09-11T12:53:48.981462+0000","last_active":"2024-09-11T12:53:49.984371+0000","last_peered":"2024-09-11T12:53:49.984371+0000","last_clean":"2024-09-11T12:53:49.984371+0000","last_became_active":"2024-09-11T12:53:48.980739+0000","last_became_peered":"2024-09-11T12:53:48.980739+0000","last_unstale":"2024-09-11T12:53:49.984371+0000","last_undegraded":"2024-09-11T12:53:49.984371+0000","last_fullsized":"2024-09-11T12:53:49.984371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:36:12.794499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997163+0000","last_change":"2024-09-11T12:53:49.985757+0000","last_active":"2024-09-11T12:53:49.997163+0000","last_peered":"2024-09-11T12:53:49.997163+0000","last_clean":"2024-09-11T12:53:49.997163+0000","last_became_active":"2024-09-11T12:53:49.985471+0000","last_became_peered":"2024-09-11T12:53:49.985471+0000","last_unstale":"2024-09-11T12:53:49.997163+0000","last_undegraded":"2024-09-11T12:53:49.997163+0000","last_fullsized":"2024-09-11T12:53:49.997163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:53.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.985183+0000","last_change":"2024-09-11T12:53:48.979205+0000","last_active":"2024-09-11T12:53:49.985183+0000","last_peered":"2024-09-11T12:53:49.985183+0000","last_clean":"2024-09-11T12:53:49.985183+0000","last_became_active":"2024-09-11T12:53:48.978826+0000","last_became_peered":"2024-09-11T12:53:48.978826+0000","last_unstale":"2024-09-11T12:53:49.985183+0000","last_undegraded":"2024-09-11T12:53:49.985183+0000","last_fullsized":"2024-09-11T12:53:49.985183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:19.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.987148+0000","last_change":"2024-09-11T12:53:49.984557+0000","last_active":"2024-09-11T12:53:49.987148+0000","last_peered":"2024-09-11T12:53:49.987148+0000","last_clean":"2024-09-11T12:53:49.987148+0000","last_became_active":"2024-09-11T12:53:49.984245+0000","last_became_peered":"2024-09-11T12:53:49.984245+0000","last_unstale":"2024-09-11T12:53:49.987148+0000","last_undegraded":"2024-09-11T12:53:49.987148+0000","last_fullsized":"2024-09-11T12:53:49.987148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:40.902648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976022+0000","last_change":"2024-09-11T12:53:48.980126+0000","last_active":"2024-09-11T12:53:49.976022+0000","last_peered":"2024-09-11T12:53:49.976022+0000","last_clean":"2024-09-11T12:53:49.976022+0000","last_became_active":"2024-09-11T12:53:48.979811+0000","last_became_peered":"2024-09-11T12:53:48.979811+0000","last_unstale":"2024-09-11T12:53:49.976022+0000","last_undegraded":"2024-09-11T12:53:49.976022+0000","last_fullsized":"2024-09-11T12:53:49.976022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:24.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991184+0000","last_change":"2024-09-11T12:53:49.980257+0000","last_active":"2024-09-11T12:53:49.991184+0000","last_peered":"2024-09-11T12:53:49.991184+0000","last_clean":"2024-09-11T12:53:49.991184+0000","last_became_active":"2024-09-11T12:53:49.979955+0000","last_became_peered":"2024-09-11T12:53:49.979955+0000","last_unstale":"2024-09-11T12:53:49.991184+0000","last_undegraded":"2024-09-11T12:53:49.991184+0000","last_fullsized":"2024-09-11T12:53:49.991184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:17.327331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995523+0000","last_change":"2024-09-11T12:53:48.979643+0000","last_active":"2024-09-11T12:53:49.995523+0000","last_peered":"2024-09-11T12:53:49.995523+0000","last_clean":"2024-09-11T12:53:49.995523+0000","last_became_active":"2024-09-11T12:53:48.979338+0000","last_became_peered":"2024-09-11T12:53:48.979338+0000","last_unstale":"2024-09-11T12:53:49.995523+0000","last_undegraded":"2024-09-11T12:53:49.995523+0000","last_fullsized":"2024-09-11T12:53:49.995523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:31.713435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992632+0000","last_change":"2024-09-11T12:53:49.988900+0000","last_active":"2024-09-11T12:53:49.992632+0000","last_peered":"2024-09-11T12:53:49.992632+0000","last_clean":"2024-09-11T12:53:49.992632+0000","last_became_active":"2024-09-11T12:53:49.988547+0000","last_became_peered":"2024-09-11T12:53:49.988547+0000","last_unstale":"2024-09-11T12:53:49.992632+0000","last_undegraded":"2024-09-11T12:53:49.992632+0000","last_fullsized":"2024-09-11T12:53:49.992632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:09.864178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.968490+0000","last_change":"2024-09-11T12:53:48.982488+0000","last_active":"2024-09-11T12:53:49.968490+0000","last_peered":"2024-09-11T12:53:49.968490+0000","last_clean":"2024-09-11T12:53:49.968490+0000","last_became_active":"2024-09-11T12:53:48.981868+0000","last_became_peered":"2024-09-11T12:53:48.981868+0000","last_unstale":"2024-09-11T12:53:49.968490+0000","last_undegraded":"2024-09-11T12:53:49.968490+0000","last_fullsized":"2024-09-11T12:53:49.968490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:04.148610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995619+0000","last_change":"2024-09-11T12:53:49.987370+0000","last_active":"2024-09-11T12:53:49.995619+0000","last_peered":"2024-09-11T12:53:49.995619+0000","last_clean":"2024-09-11T12:53:49.995619+0000","last_became_active":"2024-09-11T12:53:49.987089+0000","last_became_peered":"2024-09-11T12:53:49.987089+0000","last_unstale":"2024-09-11T12:53:49.995619+0000","last_undegraded":"2024-09-11T12:53:49.995619+0000","last_fullsized":"2024-09-11T12:53:49.995619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.974343+0000","last_change":"2024-09-11T12:53:48.985244+0000","last_active":"2024-09-11T12:53:49.974343+0000","last_peered":"2024-09-11T12:53:49.974343+0000","last_clean":"2024-09-11T12:53:49.974343+0000","last_became_active":"2024-09-11T12:53:48.984860+0000","last_became_peered":"2024-09-11T12:53:48.984860+0000","last_unstale":"2024-09-11T12:53:49.974343+0000","last_undegraded":"2024-09-11T12:53:49.974343+0000","last_fullsized":"2024-09-11T12:53:49.974343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:16.324690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997632+0000","last_change":"2024-09-11T12:53:49.994856+0000","last_active":"2024-09-11T12:53:49.997632+0000","last_peered":"2024-09-11T12:53:49.997632+0000","last_clean":"2024-09-11T12:53:49.997632+0000","last_became_active":"2024-09-11T12:53:49.994186+0000","last_became_peered":"2024-09-11T12:53:49.994186+0000","last_unstale":"2024-09-11T12:53:49.997632+0000","last_undegraded":"2024-09-11T12:53:49.997632+0000","last_fullsized":"2024-09-11T12:53:49.997632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976113+0000","last_change":"2024-09-11T12:53:48.988076+0000","last_active":"2024-09-11T12:53:49.976113+0000","last_peered":"2024-09-11T12:53:49.976113+0000","last_clean":"2024-09-11T12:53:49.976113+0000","last_became_active":"2024-09-11T12:53:48.987776+0000","last_became_peered":"2024-09-11T12:53:48.987776+0000","last_unstale":"2024-09-11T12:53:49.976113+0000","last_undegraded":"2024-09-11T12:53:49.976113+0000","last_fullsized":"2024-09-11T12:53:49.976113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:32.777475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994791+0000","last_change":"2024-09-11T12:53:49.989841+0000","last_active":"2024-09-11T12:53:49.994791+0000","last_peered":"2024-09-11T12:53:49.994791+0000","last_clean":"2024-09-11T12:53:49.994791+0000","last_became_active":"2024-09-11T12:53:49.989580+0000","last_became_peered":"2024-09-11T12:53:49.989580+0000","last_unstale":"2024-09-11T12:53:49.994791+0000","last_undegraded":"2024-09-11T12:53:49.994791+0000","last_fullsized":"2024-09-11T12:53:49.994791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.978317+0000","last_change":"2024-09-11T12:53:48.985710+0000","last_active":"2024-09-11T12:53:49.978317+0000","last_peered":"2024-09-11T12:53:49.978317+0000","last_clean":"2024-09-11T12:53:49.978317+0000","last_became_active":"2024-09-11T12:53:48.983745+0000","last_became_peered":"2024-09-11T12:53:48.983745+0000","last_unstale":"2024-09-11T12:53:49.978317+0000","last_undegraded":"2024-09-11T12:53:49.978317+0000","last_fullsized":"2024-09-11T12:53:49.978317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:45.182724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996429+0000","last_change":"2024-09-11T12:53:49.988191+0000","last_active":"2024-09-11T12:53:49.996429+0000","last_peered":"2024-09-11T12:53:49.996429+0000","last_clean":"2024-09-11T12:53:49.996429+0000","last_became_active":"2024-09-11T12:53:49.987734+0000","last_became_peered":"2024-09-11T12:53:49.987734+0000","last_unstale":"2024-09-11T12:53:49.996429+0000","last_undegraded":"2024-09-11T12:53:49.996429+0000","last_fullsized":"2024-09-11T12:53:49.996429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:55.608657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.969076+0000","last_change":"2024-09-11T12:53:48.980510+0000","last_active":"2024-09-11T12:53:49.969076+0000","last_peered":"2024-09-11T12:53:49.969076+0000","last_clean":"2024-09-11T12:53:49.969076+0000","last_became_active":"2024-09-11T12:53:48.980079+0000","last_became_peered":"2024-09-11T12:53:48.980079+0000","last_unstale":"2024-09-11T12:53:49.969076+0000","last_undegraded":"2024-09-11T12:53:49.969076+0000","last_fullsized":"2024-09-11T12:53:49.969076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:22.268148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999860+0000","last_change":"2024-09-11T12:53:49.998566+0000","last_active":"2024-09-11T12:53:49.999860+0000","last_peered":"2024-09-11T12:53:49.999860+0000","last_clean":"2024-09-11T12:53:49.999860+0000","last_became_active":"2024-09-11T12:53:49.998301+0000","last_became_peered":"2024-09-11T12:53:49.998301+0000","last_unstale":"2024-09-11T12:53:49.999860+0000","last_undegraded":"2024-09-11T12:53:49.999860+0000","last_fullsized":"2024-09-11T12:53:49.999860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:12.091816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.974864+0000","last_change":"2024-09-11T12:53:48.987921+0000","last_active":"2024-09-11T12:53:49.974864+0000","last_peered":"2024-09-11T12:53:49.974864+0000","last_clean":"2024-09-11T12:53:49.974864+0000","last_became_active":"2024-09-11T12:53:48.987618+0000","last_became_peered":"2024-09-11T12:53:48.987618+0000","last_unstale":"2024-09-11T12:53:49.974864+0000","last_undegraded":"2024-09-11T12:53:49.974864+0000","last_fullsized":"2024-09-11T12:53:49.974864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:09.109265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997003+0000","last_change":"2024-09-11T12:53:49.988172+0000","last_active":"2024-09-11T12:53:49.997003+0000","last_peered":"2024-09-11T12:53:49.997003+0000","last_clean":"2024-09-11T12:53:49.997003+0000","last_became_active":"2024-09-11T12:53:49.987185+0000","last_became_peered":"2024-09-11T12:53:49.987185+0000","last_unstale":"2024-09-11T12:53:49.997003+0000","last_undegraded":"2024-09-11T12:53:49.997003+0000","last_fullsized":"2024-09-11T12:53:49.997003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:48.687748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.969096+0000","last_change":"2024-09-11T12:53:48.986576+0000","last_active":"2024-09-11T12:53:49.969096+0000","last_peered":"2024-09-11T12:53:49.969096+0000","last_clean":"2024-09-11T12:53:49.969096+0000","last_became_active":"2024-09-11T12:53:48.985143+0000","last_became_peered":"2024-09-11T12:53:48.985143+0000","last_unstale":"2024-09-11T12:53:49.969096+0000","last_undegraded":"2024-09-11T12:53:49.969096+0000","last_fullsized":"2024-09-11T12:53:49.969096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:38.474205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.983165+0000","last_change":"2024-09-11T12:53:49.973576+0000","last_active":"2024-09-11T12:53:49.983165+0000","last_peered":"2024-09-11T12:53:49.983165+0000","last_clean":"2024-09-11T12:53:49.983165+0000","last_became_active":"2024-09-11T12:53:49.973284+0000","last_became_peered":"2024-09-11T12:53:49.973284+0000","last_unstale":"2024-09-11T12:53:49.983165+0000","last_undegraded":"2024-09-11T12:53:49.983165+0000","last_fullsized":"2024-09-11T12:53:49.983165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:55.780681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001496+0000","last_change":"2024-09-11T12:53:48.984204+0000","last_active":"2024-09-11T12:53:50.001496+0000","last_peered":"2024-09-11T12:53:50.001496+0000","last_clean":"2024-09-11T12:53:50.001496+0000","last_became_active":"2024-09-11T12:53:48.983469+0000","last_became_peered":"2024-09-11T12:53:48.983469+0000","last_unstale":"2024-09-11T12:53:50.001496+0000","last_undegraded":"2024-09-11T12:53:50.001496+0000","last_fullsized":"2024-09-11T12:53:50.001496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:48:31.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":246,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.972386+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:53:49.972386+0000","last_peered":"2024-09-11T12:53:49.972386+0000","last_clean":"2024-09-11T12:53:49.972386+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:53:49.972386+0000","last_undegraded":"2024-09-11T12:53:49.972386+0000","last_fullsized":"2024-09-11T12:53:49.972386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1015551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509495,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1581056,"data_stored":1008706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:53:54.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:53:54.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- 172.21.15.78:0/1391073034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac073c30 msgr2=0x7f46ac15e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- 172.21.15.78:0/1391073034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac073c30 0x7f46ac15e380 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f4698009900 tx=0x7f469801c770 comp rx=0 tx=0).stop 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- 172.21.15.78:0/1391073034 shutdown_connections 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- 172.21.15.78:0/1391073034 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac073c30 0x7f46ac15e380 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- 172.21.15.78:0/1391073034 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 0x7f46ac0b5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- 172.21.15.78:0/1391073034 >> 172.21.15.78:0/1391073034 conn(0x7f46ac06ea90 msgr2=0x7f46ac071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- 172.21.15.78:0/1391073034 shutdown_connections 2024-09-11T12:53:54.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- 172.21.15.78:0/1391073034 wait complete. 2024-09-11T12:53:54.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 Processor -- start 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- start start 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 0x7f46ac070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ac073c30 0x7f46ac151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac15f9b0 con 0x7f46ac15e8c0 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac158600 con 0x7f46ac073c30 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46b2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac15fb30 con 0x7f46ac073850 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44410/0 (socket says 172.21.15.78:44410) 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.268+0000 7f46abe00640 1 -- 172.21.15.78:0/3261513293 learned_addr learned my addr 172.21.15.78:0/3261513293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:54.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46aaa00640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ac073c30 0x7f46ac151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46ab400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 0x7f46ac070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 -- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 msgr2=0x7f46ac070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 0x7f46ac070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 -- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ac073c30 msgr2=0x7f46ac151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ac073c30 0x7f46ac151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 -- 172.21.15.78:0/3261513293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac15ca60 con 0x7f46ac15e8c0 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46abe00640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f46a400faa0 tx=0x7f46a400ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:54.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a401a070 con 0x7f46ac15e8c0 2024-09-11T12:53:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a400d6d0 con 0x7f46ac15e8c0 2024-09-11T12:53:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac15cd40 con 0x7f46ac15e8c0 2024-09-11T12:53:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a400db60 con 0x7f46ac15e8c0 2024-09-11T12:53:54.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac070bc0 con 0x7f46ac15e8c0 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac0b5c00 con 0x7f46ac15e8c0 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f46a401d070 con 0x7f46ac15e8c0 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.272+0000 7f46a3400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 0x7f468c073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.276+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f46a4094810 con 0x7f46ac15e8c0 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.276+0000 7f46ab400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 0x7f468c073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.276+0000 7f46ab400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 0x7f468c073c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f469c005c50 tx=0x7f469c005be0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:54.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.276+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f46a4060560 con 0x7f46ac15e8c0 2024-09-11T12:53:54.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.440+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46ac0d3f50 con 0x7f468c0717d0 2024-09-11T12:53:54.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.444+0000 7f46a3400640 1 -- 172.21.15.78:0/3261513293 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252064 (secure 0 0 0) 0x7f46ac0d3f50 con 0x7f468c0717d0 2024-09-11T12:53:54.450 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:54.452 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 msgr2=0x7f468c073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 0x7f468c073c90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f469c005c50 tx=0x7f469c005be0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 msgr2=0x7f46ac1f5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f46a400faa0 tx=0x7f46a400ff60 comp rx=0 tx=0).stop 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 shutdown_connections 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ac15e8c0 0x7f46ac1f5170 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f468c0717d0 0x7f468c073c90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ac073c30 0x7f46ac151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 --2- 172.21.15.78:0/3261513293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ac073850 0x7f46ac070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 >> 172.21.15.78:0/3261513293 conn(0x7f46ac06ea90 msgr2=0x7f46ac0b44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 shutdown_connections 2024-09-11T12:53:54.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.448+0000 7f46b2400640 1 -- 172.21.15.78:0/3261513293 wait complete. 2024-09-11T12:53:54.469 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-09-11T12:53:52.471967+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":157,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258420,"kb_used_data":8900,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716300,"statfs":{"total":773094113280,"available":772829491200,"internally_reserved":0,"allocated":9113600,"data_stored":4541060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.517731"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000338+0000","last_change":"2024-09-11T12:53:49.994111+0000","last_active":"2024-09-11T12:53:50.000338+0000","last_peered":"2024-09-11T12:53:50.000338+0000","last_clean":"2024-09-11T12:53:50.000338+0000","last_became_active":"2024-09-11T12:53:49.993553+0000","last_became_peered":"2024-09-11T12:53:49.993553+0000","last_unstale":"2024-09-11T12:53:50.000338+0000","last_undegraded":"2024-09-11T12:53:50.000338+0000","last_fullsized":"2024-09-11T12:53:50.000338+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:48:32.688884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999733+0000","last_change":"2024-09-11T12:53:49.994797+0000","last_active":"2024-09-11T12:53:49.999733+0000","last_peered":"2024-09-11T12:53:49.999733+0000","last_clean":"2024-09-11T12:53:49.999733+0000","last_became_active":"2024-09-11T12:53:49.994087+0000","last_became_peered":"2024-09-11T12:53:49.994087+0000","last_unstale":"2024-09-11T12:53:49.999733+0000","last_undegraded":"2024-09-11T12:53:49.999733+0000","last_fullsized":"2024-09-11T12:53:49.999733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:41:37.839887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991897+0000","last_change":"2024-09-11T12:53:49.987897+0000","last_active":"2024-09-11T12:53:49.991897+0000","last_peered":"2024-09-11T12:53:49.991897+0000","last_clean":"2024-09-11T12:53:49.991897+0000","last_became_active":"2024-09-11T12:53:49.987580+0000","last_became_peered":"2024-09-11T12:53:49.987580+0000","last_unstale":"2024-09-11T12:53:49.991897+0000","last_undegraded":"2024-09-11T12:53:49.991897+0000","last_fullsized":"2024-09-11T12:53:49.991897+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:45.722422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005287+0000","last_change":"2024-09-11T12:53:49.997395+0000","last_active":"2024-09-11T12:53:50.005287+0000","last_peered":"2024-09-11T12:53:50.005287+0000","last_clean":"2024-09-11T12:53:50.005287+0000","last_became_active":"2024-09-11T12:53:49.997068+0000","last_became_peered":"2024-09-11T12:53:49.997068+0000","last_unstale":"2024-09-11T12:53:50.005287+0000","last_undegraded":"2024-09-11T12:53:50.005287+0000","last_fullsized":"2024-09-11T12:53:50.005287+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:14.877164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995368+0000","last_change":"2024-09-11T12:53:49.983511+0000","last_active":"2024-09-11T12:53:49.995368+0000","last_peered":"2024-09-11T12:53:49.995368+0000","last_clean":"2024-09-11T12:53:49.995368+0000","last_became_active":"2024-09-11T12:53:49.982983+0000","last_became_peered":"2024-09-11T12:53:49.982983+0000","last_unstale":"2024-09-11T12:53:49.995368+0000","last_undegraded":"2024-09-11T12:53:49.995368+0000","last_fullsized":"2024-09-11T12:53:49.995368+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:10.111906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.988462+0000","last_change":"2024-09-11T12:53:49.985655+0000","last_active":"2024-09-11T12:53:49.988462+0000","last_peered":"2024-09-11T12:53:49.988462+0000","last_clean":"2024-09-11T12:53:49.988462+0000","last_became_active":"2024-09-11T12:53:49.985363+0000","last_became_peered":"2024-09-11T12:53:49.985363+0000","last_unstale":"2024-09-11T12:53:49.988462+0000","last_undegraded":"2024-09-11T12:53:49.988462+0000","last_fullsized":"2024-09-11T12:53:49.988462+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:40.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.998892+0000","last_change":"2024-09-11T12:53:49.990010+0000","last_active":"2024-09-11T12:53:49.998892+0000","last_peered":"2024-09-11T12:53:49.998892+0000","last_clean":"2024-09-11T12:53:49.998892+0000","last_became_active":"2024-09-11T12:53:49.989731+0000","last_became_peered":"2024-09-11T12:53:49.989731+0000","last_unstale":"2024-09-11T12:53:49.998892+0000","last_undegraded":"2024-09-11T12:53:49.998892+0000","last_fullsized":"2024-09-11T12:53:49.998892+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:36.648601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995972+0000","last_change":"2024-09-11T12:53:49.991096+0000","last_active":"2024-09-11T12:53:49.995972+0000","last_peered":"2024-09-11T12:53:49.995972+0000","last_clean":"2024-09-11T12:53:49.995972+0000","last_became_active":"2024-09-11T12:53:49.990845+0000","last_became_peered":"2024-09-11T12:53:49.990845+0000","last_unstale":"2024-09-11T12:53:49.995972+0000","last_undegraded":"2024-09-11T12:53:49.995972+0000","last_fullsized":"2024-09-11T12:53:49.995972+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.986044+0000","last_change":"2024-09-11T12:53:49.984485+0000","last_active":"2024-09-11T12:53:49.986044+0000","last_peered":"2024-09-11T12:53:49.986044+0000","last_clean":"2024-09-11T12:53:49.986044+0000","last_became_active":"2024-09-11T12:53:49.984207+0000","last_became_peered":"2024-09-11T12:53:49.984207+0000","last_unstale":"2024-09-11T12:53:49.986044+0000","last_undegraded":"2024-09-11T12:53:49.986044+0000","last_fullsized":"2024-09-11T12:53:49.986044+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:33.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995992+0000","last_change":"2024-09-11T12:53:49.987810+0000","last_active":"2024-09-11T12:53:49.995992+0000","last_peered":"2024-09-11T12:53:49.995992+0000","last_clean":"2024-09-11T12:53:49.995992+0000","last_became_active":"2024-09-11T12:53:49.987501+0000","last_became_peered":"2024-09-11T12:53:49.987501+0000","last_unstale":"2024-09-11T12:53:49.995992+0000","last_undegraded":"2024-09-11T12:53:49.995992+0000","last_fullsized":"2024-09-11T12:53:49.995992+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991466+0000","last_change":"2024-09-11T12:53:49.980601+0000","last_active":"2024-09-11T12:53:49.991466+0000","last_peered":"2024-09-11T12:53:49.991466+0000","last_clean":"2024-09-11T12:53:49.991466+0000","last_became_active":"2024-09-11T12:53:49.980282+0000","last_became_peered":"2024-09-11T12:53:49.980282+0000","last_unstale":"2024-09-11T12:53:49.991466+0000","last_undegraded":"2024-09-11T12:53:49.991466+0000","last_fullsized":"2024-09-11T12:53:49.991466+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:26.129355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996457+0000","last_change":"2024-09-11T12:53:49.988000+0000","last_active":"2024-09-11T12:53:49.996457+0000","last_peered":"2024-09-11T12:53:49.996457+0000","last_clean":"2024-09-11T12:53:49.996457+0000","last_became_active":"2024-09-11T12:53:49.987724+0000","last_became_peered":"2024-09-11T12:53:49.987724+0000","last_unstale":"2024-09-11T12:53:49.996457+0000","last_undegraded":"2024-09-11T12:53:49.996457+0000","last_fullsized":"2024-09-11T12:53:49.996457+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:41.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995481+0000","last_change":"2024-09-11T12:53:49.991124+0000","last_active":"2024-09-11T12:53:49.995481+0000","last_peered":"2024-09-11T12:53:49.995481+0000","last_clean":"2024-09-11T12:53:49.995481+0000","last_became_active":"2024-09-11T12:53:49.990800+0000","last_became_peered":"2024-09-11T12:53:49.990800+0000","last_unstale":"2024-09-11T12:53:49.995481+0000","last_undegraded":"2024-09-11T12:53:49.995481+0000","last_fullsized":"2024-09-11T12:53:49.995481+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:23.205971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005879+0000","last_change":"2024-09-11T12:53:49.999885+0000","last_active":"2024-09-11T12:53:50.005879+0000","last_peered":"2024-09-11T12:53:50.005879+0000","last_clean":"2024-09-11T12:53:50.005879+0000","last_became_active":"2024-09-11T12:53:49.999408+0000","last_became_peered":"2024-09-11T12:53:49.999408+0000","last_unstale":"2024-09-11T12:53:50.005879+0000","last_undegraded":"2024-09-11T12:53:50.005879+0000","last_fullsized":"2024-09-11T12:53:50.005879+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:21.647957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992210+0000","last_change":"2024-09-11T12:53:49.989420+0000","last_active":"2024-09-11T12:53:49.992210+0000","last_peered":"2024-09-11T12:53:49.992210+0000","last_clean":"2024-09-11T12:53:49.992210+0000","last_became_active":"2024-09-11T12:53:49.989158+0000","last_became_peered":"2024-09-11T12:53:49.989158+0000","last_unstale":"2024-09-11T12:53:49.992210+0000","last_undegraded":"2024-09-11T12:53:49.992210+0000","last_fullsized":"2024-09-11T12:53:49.992210+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:13.311775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.990014+0000","last_change":"2024-09-11T12:53:49.980155+0000","last_active":"2024-09-11T12:53:49.990014+0000","last_peered":"2024-09-11T12:53:49.990014+0000","last_clean":"2024-09-11T12:53:49.990014+0000","last_became_active":"2024-09-11T12:53:49.979606+0000","last_became_peered":"2024-09-11T12:53:49.979606+0000","last_unstale":"2024-09-11T12:53:49.990014+0000","last_undegraded":"2024-09-11T12:53:49.990014+0000","last_fullsized":"2024-09-11T12:53:49.990014+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:34.166919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997806+0000","last_change":"2024-09-11T12:53:49.991908+0000","last_active":"2024-09-11T12:53:49.997806+0000","last_peered":"2024-09-11T12:53:49.997806+0000","last_clean":"2024-09-11T12:53:49.997806+0000","last_became_active":"2024-09-11T12:53:49.991616+0000","last_became_peered":"2024-09-11T12:53:49.991616+0000","last_unstale":"2024-09-11T12:53:49.997806+0000","last_undegraded":"2024-09-11T12:53:49.997806+0000","last_fullsized":"2024-09-11T12:53:49.997806+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:40.902648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.998916+0000","last_change":"2024-09-11T12:53:49.991414+0000","last_active":"2024-09-11T12:53:49.998916+0000","last_peered":"2024-09-11T12:53:49.998916+0000","last_clean":"2024-09-11T12:53:49.998916+0000","last_became_active":"2024-09-11T12:53:49.991141+0000","last_became_peered":"2024-09-11T12:53:49.991141+0000","last_unstale":"2024-09-11T12:53:49.998916+0000","last_undegraded":"2024-09-11T12:53:49.998916+0000","last_fullsized":"2024-09-11T12:53:49.998916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:19.286133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.003600+0000","last_change":"2024-09-11T12:53:49.994210+0000","last_active":"2024-09-11T12:53:50.003600+0000","last_peered":"2024-09-11T12:53:50.003600+0000","last_clean":"2024-09-11T12:53:50.003600+0000","last_became_active":"2024-09-11T12:53:49.993930+0000","last_became_peered":"2024-09-11T12:53:49.993930+0000","last_unstale":"2024-09-11T12:53:50.003600+0000","last_undegraded":"2024-09-11T12:53:50.003600+0000","last_fullsized":"2024-09-11T12:53:50.003600+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:12.949102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005396+0000","last_change":"2024-09-11T12:53:49.998011+0000","last_active":"2024-09-11T12:53:50.005396+0000","last_peered":"2024-09-11T12:53:50.005396+0000","last_clean":"2024-09-11T12:53:50.005396+0000","last_became_active":"2024-09-11T12:53:49.997653+0000","last_became_peered":"2024-09-11T12:53:49.997653+0000","last_unstale":"2024-09-11T12:53:50.005396+0000","last_undegraded":"2024-09-11T12:53:50.005396+0000","last_fullsized":"2024-09-11T12:53:50.005396+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:20:58.612685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009226+0000","last_change":"2024-09-11T12:53:50.005651+0000","last_active":"2024-09-11T12:53:50.009226+0000","last_peered":"2024-09-11T12:53:50.009226+0000","last_clean":"2024-09-11T12:53:50.009226+0000","last_became_active":"2024-09-11T12:53:50.005350+0000","last_became_peered":"2024-09-11T12:53:50.005350+0000","last_unstale":"2024-09-11T12:53:50.009226+0000","last_undegraded":"2024-09-11T12:53:50.009226+0000","last_fullsized":"2024-09-11T12:53:50.009226+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:08:19.610993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.004768+0000","last_change":"2024-09-11T12:53:49.996315+0000","last_active":"2024-09-11T12:53:50.004768+0000","last_peered":"2024-09-11T12:53:50.004768+0000","last_clean":"2024-09-11T12:53:50.004768+0000","last_became_active":"2024-09-11T12:53:49.996007+0000","last_became_peered":"2024-09-11T12:53:49.996007+0000","last_unstale":"2024-09-11T12:53:50.004768+0000","last_undegraded":"2024-09-11T12:53:50.004768+0000","last_fullsized":"2024-09-11T12:53:50.004768+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:28:54.037285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999911+0000","last_change":"2024-09-11T12:53:49.998031+0000","last_active":"2024-09-11T12:53:49.999911+0000","last_peered":"2024-09-11T12:53:49.999911+0000","last_clean":"2024-09-11T12:53:49.999911+0000","last_became_active":"2024-09-11T12:53:49.997689+0000","last_became_peered":"2024-09-11T12:53:49.997689+0000","last_unstale":"2024-09-11T12:53:49.999911+0000","last_undegraded":"2024-09-11T12:53:49.999911+0000","last_fullsized":"2024-09-11T12:53:49.999911+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:59.620107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005611+0000","last_change":"2024-09-11T12:53:49.998056+0000","last_active":"2024-09-11T12:53:50.005611+0000","last_peered":"2024-09-11T12:53:50.005611+0000","last_clean":"2024-09-11T12:53:50.005611+0000","last_became_active":"2024-09-11T12:53:49.997757+0000","last_became_peered":"2024-09-11T12:53:49.997757+0000","last_unstale":"2024-09-11T12:53:50.005611+0000","last_undegraded":"2024-09-11T12:53:50.005611+0000","last_fullsized":"2024-09-11T12:53:50.005611+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:50:05.572626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991639+0000","last_change":"2024-09-11T12:53:49.983507+0000","last_active":"2024-09-11T12:53:49.991639+0000","last_peered":"2024-09-11T12:53:49.991639+0000","last_clean":"2024-09-11T12:53:49.991639+0000","last_became_active":"2024-09-11T12:53:49.983229+0000","last_became_peered":"2024-09-11T12:53:49.983229+0000","last_unstale":"2024-09-11T12:53:49.991639+0000","last_undegraded":"2024-09-11T12:53:49.991639+0000","last_fullsized":"2024-09-11T12:53:49.991639+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:37:26.437611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001447+0000","last_change":"2024-09-11T12:53:49.999888+0000","last_active":"2024-09-11T12:53:50.001447+0000","last_peered":"2024-09-11T12:53:50.001447+0000","last_clean":"2024-09-11T12:53:50.001447+0000","last_became_active":"2024-09-11T12:53:49.999564+0000","last_became_peered":"2024-09-11T12:53:49.999564+0000","last_unstale":"2024-09-11T12:53:50.001447+0000","last_undegraded":"2024-09-11T12:53:50.001447+0000","last_fullsized":"2024-09-11T12:53:50.001447+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.149519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995680+0000","last_change":"2024-09-11T12:53:49.986916+0000","last_active":"2024-09-11T12:53:49.995680+0000","last_peered":"2024-09-11T12:53:49.995680+0000","last_clean":"2024-09-11T12:53:49.995680+0000","last_became_active":"2024-09-11T12:53:49.986485+0000","last_became_peered":"2024-09-11T12:53:49.986485+0000","last_unstale":"2024-09-11T12:53:49.995680+0000","last_undegraded":"2024-09-11T12:53:49.995680+0000","last_fullsized":"2024-09-11T12:53:49.995680+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:37.908752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000145+0000","last_change":"2024-09-11T12:53:49.998436+0000","last_active":"2024-09-11T12:53:50.000145+0000","last_peered":"2024-09-11T12:53:50.000145+0000","last_clean":"2024-09-11T12:53:50.000145+0000","last_became_active":"2024-09-11T12:53:49.998073+0000","last_became_peered":"2024-09-11T12:53:49.998073+0000","last_unstale":"2024-09-11T12:53:50.000145+0000","last_undegraded":"2024-09-11T12:53:50.000145+0000","last_fullsized":"2024-09-11T12:53:50.000145+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:45.722422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009092+0000","last_change":"2024-09-11T12:53:50.001373+0000","last_active":"2024-09-11T12:53:50.009092+0000","last_peered":"2024-09-11T12:53:50.009092+0000","last_clean":"2024-09-11T12:53:50.009092+0000","last_became_active":"2024-09-11T12:53:49.996893+0000","last_became_peered":"2024-09-11T12:53:49.996893+0000","last_unstale":"2024-09-11T12:53:50.009092+0000","last_undegraded":"2024-09-11T12:53:50.009092+0000","last_fullsized":"2024-09-11T12:53:50.009092+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:23.270789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996197+0000","last_change":"2024-09-11T12:53:49.986827+0000","last_active":"2024-09-11T12:53:49.996197+0000","last_peered":"2024-09-11T12:53:49.996197+0000","last_clean":"2024-09-11T12:53:49.996197+0000","last_became_active":"2024-09-11T12:53:49.986399+0000","last_became_peered":"2024-09-11T12:53:49.986399+0000","last_unstale":"2024-09-11T12:53:49.996197+0000","last_undegraded":"2024-09-11T12:53:49.996197+0000","last_fullsized":"2024-09-11T12:53:49.996197+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:16:04.156160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997753+0000","last_change":"2024-09-11T12:53:49.995428+0000","last_active":"2024-09-11T12:53:49.997753+0000","last_peered":"2024-09-11T12:53:49.997753+0000","last_clean":"2024-09-11T12:53:49.997753+0000","last_became_active":"2024-09-11T12:53:49.995145+0000","last_became_peered":"2024-09-11T12:53:49.995145+0000","last_unstale":"2024-09-11T12:53:49.997753+0000","last_undegraded":"2024-09-11T12:53:49.997753+0000","last_fullsized":"2024-09-11T12:53:49.997753+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997283+0000","last_change":"2024-09-11T12:53:49.994112+0000","last_active":"2024-09-11T12:53:49.997283+0000","last_peered":"2024-09-11T12:53:49.997283+0000","last_clean":"2024-09-11T12:53:49.997283+0000","last_became_active":"2024-09-11T12:53:49.993589+0000","last_became_peered":"2024-09-11T12:53:49.993589+0000","last_unstale":"2024-09-11T12:53:49.997283+0000","last_undegraded":"2024-09-11T12:53:49.997283+0000","last_fullsized":"2024-09-11T12:53:49.997283+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:38.895803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996475+0000","last_change":"2024-09-11T12:53:49.992127+0000","last_active":"2024-09-11T12:53:49.996475+0000","last_peered":"2024-09-11T12:53:49.996475+0000","last_clean":"2024-09-11T12:53:49.996475+0000","last_became_active":"2024-09-11T12:53:49.989888+0000","last_became_peered":"2024-09-11T12:53:49.989888+0000","last_unstale":"2024-09-11T12:53:49.996475+0000","last_undegraded":"2024-09-11T12:53:49.996475+0000","last_fullsized":"2024-09-11T12:53:49.996475+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:40.965663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"82'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.007197+0000","last_change":"2024-09-11T12:53:48.989674+0000","last_active":"2024-09-11T12:53:50.007197+0000","last_peered":"2024-09-11T12:53:50.007197+0000","last_clean":"2024-09-11T12:53:50.007197+0000","last_became_active":"2024-09-11T12:53:48.989340+0000","last_became_peered":"2024-09-11T12:53:48.989340+0000","last_unstale":"2024-09-11T12:53:50.007197+0000","last_undegraded":"2024-09-11T12:53:50.007197+0000","last_fullsized":"2024-09-11T12:53:50.007197+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:19.587352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000319+0000","last_change":"2024-09-11T12:53:49.993875+0000","last_active":"2024-09-11T12:53:50.000319+0000","last_peered":"2024-09-11T12:53:50.000319+0000","last_clean":"2024-09-11T12:53:50.000319+0000","last_became_active":"2024-09-11T12:53:49.992356+0000","last_became_peered":"2024-09-11T12:53:49.992356+0000","last_unstale":"2024-09-11T12:53:50.000319+0000","last_undegraded":"2024-09-11T12:53:50.000319+0000","last_fullsized":"2024-09-11T12:53:50.000319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:15.858073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994950+0000","last_change":"2024-09-11T12:53:48.976033+0000","last_active":"2024-09-11T12:53:49.994950+0000","last_peered":"2024-09-11T12:53:49.994950+0000","last_clean":"2024-09-11T12:53:49.994950+0000","last_became_active":"2024-09-11T12:53:48.975625+0000","last_became_peered":"2024-09-11T12:53:48.975625+0000","last_unstale":"2024-09-11T12:53:49.994950+0000","last_undegraded":"2024-09-11T12:53:49.994950+0000","last_fullsized":"2024-09-11T12:53:49.994950+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:10:30.333424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997242+0000","last_change":"2024-09-11T12:53:49.989424+0000","last_active":"2024-09-11T12:53:49.997242+0000","last_peered":"2024-09-11T12:53:49.997242+0000","last_clean":"2024-09-11T12:53:49.997242+0000","last_became_active":"2024-09-11T12:53:49.989145+0000","last_became_peered":"2024-09-11T12:53:49.989145+0000","last_unstale":"2024-09-11T12:53:49.997242+0000","last_undegraded":"2024-09-11T12:53:49.997242+0000","last_fullsized":"2024-09-11T12:53:49.997242+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:16.491656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.003876+0000","last_change":"2024-09-11T12:53:48.988295+0000","last_active":"2024-09-11T12:53:50.003876+0000","last_peered":"2024-09-11T12:53:50.003876+0000","last_clean":"2024-09-11T12:53:50.003876+0000","last_became_active":"2024-09-11T12:53:48.987797+0000","last_became_peered":"2024-09-11T12:53:48.987797+0000","last_unstale":"2024-09-11T12:53:50.003876+0000","last_undegraded":"2024-09-11T12:53:50.003876+0000","last_fullsized":"2024-09-11T12:53:50.003876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:04:24.610208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995006+0000","last_change":"2024-09-11T12:53:49.990699+0000","last_active":"2024-09-11T12:53:49.995006+0000","last_peered":"2024-09-11T12:53:49.995006+0000","last_clean":"2024-09-11T12:53:49.995006+0000","last_became_active":"2024-09-11T12:53:49.990224+0000","last_became_peered":"2024-09-11T12:53:49.990224+0000","last_unstale":"2024-09-11T12:53:49.995006+0000","last_undegraded":"2024-09-11T12:53:49.995006+0000","last_fullsized":"2024-09-11T12:53:49.995006+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:16.076036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.002549+0000","last_change":"2024-09-11T12:53:48.988534+0000","last_active":"2024-09-11T12:53:50.002549+0000","last_peered":"2024-09-11T12:53:50.002549+0000","last_clean":"2024-09-11T12:53:50.002549+0000","last_became_active":"2024-09-11T12:53:48.988129+0000","last_became_peered":"2024-09-11T12:53:48.988129+0000","last_unstale":"2024-09-11T12:53:50.002549+0000","last_undegraded":"2024-09-11T12:53:50.002549+0000","last_fullsized":"2024-09-11T12:53:50.002549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:35:21.266551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.006351+0000","last_change":"2024-09-11T12:53:50.000099+0000","last_active":"2024-09-11T12:53:50.006351+0000","last_peered":"2024-09-11T12:53:50.006351+0000","last_clean":"2024-09-11T12:53:50.006351+0000","last_became_active":"2024-09-11T12:53:49.987102+0000","last_became_peered":"2024-09-11T12:53:49.987102+0000","last_unstale":"2024-09-11T12:53:50.006351+0000","last_undegraded":"2024-09-11T12:53:50.006351+0000","last_fullsized":"2024-09-11T12:53:50.006351+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:54.863383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001874+0000","last_change":"2024-09-11T12:53:48.981414+0000","last_active":"2024-09-11T12:53:50.001874+0000","last_peered":"2024-09-11T12:53:50.001874+0000","last_clean":"2024-09-11T12:53:50.001874+0000","last_became_active":"2024-09-11T12:53:48.981095+0000","last_became_peered":"2024-09-11T12:53:48.981095+0000","last_unstale":"2024-09-11T12:53:50.001874+0000","last_undegraded":"2024-09-11T12:53:50.001874+0000","last_fullsized":"2024-09-11T12:53:50.001874+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:02:45.142820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996177+0000","last_change":"2024-09-11T12:53:49.991481+0000","last_active":"2024-09-11T12:53:49.996177+0000","last_peered":"2024-09-11T12:53:49.996177+0000","last_clean":"2024-09-11T12:53:49.996177+0000","last_became_active":"2024-09-11T12:53:49.991206+0000","last_became_peered":"2024-09-11T12:53:49.991206+0000","last_unstale":"2024-09-11T12:53:49.996177+0000","last_undegraded":"2024-09-11T12:53:49.996177+0000","last_fullsized":"2024-09-11T12:53:49.996177+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:33.780116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994907+0000","last_change":"2024-09-11T12:53:48.988457+0000","last_active":"2024-09-11T12:53:49.994907+0000","last_peered":"2024-09-11T12:53:49.994907+0000","last_clean":"2024-09-11T12:53:49.994907+0000","last_became_active":"2024-09-11T12:53:48.988135+0000","last_became_peered":"2024-09-11T12:53:48.988135+0000","last_unstale":"2024-09-11T12:53:49.994907+0000","last_undegraded":"2024-09-11T12:53:49.994907+0000","last_fullsized":"2024-09-11T12:53:49.994907+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:14:53.860743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995729+0000","last_change":"2024-09-11T12:53:49.991780+0000","last_active":"2024-09-11T12:53:49.995729+0000","last_peered":"2024-09-11T12:53:49.995729+0000","last_clean":"2024-09-11T12:53:49.995729+0000","last_became_active":"2024-09-11T12:53:49.991047+0000","last_became_peered":"2024-09-11T12:53:49.991047+0000","last_unstale":"2024-09-11T12:53:49.995729+0000","last_undegraded":"2024-09-11T12:53:49.995729+0000","last_fullsized":"2024-09-11T12:53:49.995729+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:25.066295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976127+0000","last_change":"2024-09-11T12:53:48.977234+0000","last_active":"2024-09-11T12:53:49.976127+0000","last_peered":"2024-09-11T12:53:49.976127+0000","last_clean":"2024-09-11T12:53:49.976127+0000","last_became_active":"2024-09-11T12:53:48.976961+0000","last_became_peered":"2024-09-11T12:53:48.976961+0000","last_unstale":"2024-09-11T12:53:49.976127+0000","last_undegraded":"2024-09-11T12:53:49.976127+0000","last_fullsized":"2024-09-11T12:53:49.976127+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:57:05.911791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992185+0000","last_change":"2024-09-11T12:53:49.989214+0000","last_active":"2024-09-11T12:53:49.992185+0000","last_peered":"2024-09-11T12:53:49.992185+0000","last_clean":"2024-09-11T12:53:49.992185+0000","last_became_active":"2024-09-11T12:53:49.988905+0000","last_became_peered":"2024-09-11T12:53:49.988905+0000","last_unstale":"2024-09-11T12:53:49.992185+0000","last_undegraded":"2024-09-11T12:53:49.992185+0000","last_fullsized":"2024-09-11T12:53:49.992185+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:52:30.149519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.966505+0000","last_change":"2024-09-11T12:53:48.979739+0000","last_active":"2024-09-11T12:53:49.966505+0000","last_peered":"2024-09-11T12:53:49.966505+0000","last_clean":"2024-09-11T12:53:49.966505+0000","last_became_active":"2024-09-11T12:53:48.978901+0000","last_became_peered":"2024-09-11T12:53:48.978901+0000","last_unstale":"2024-09-11T12:53:49.966505+0000","last_undegraded":"2024-09-11T12:53:49.966505+0000","last_fullsized":"2024-09-11T12:53:49.966505+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:22.054812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.007253+0000","last_change":"2024-09-11T12:53:49.999653+0000","last_active":"2024-09-11T12:53:50.007253+0000","last_peered":"2024-09-11T12:53:50.007253+0000","last_clean":"2024-09-11T12:53:50.007253+0000","last_became_active":"2024-09-11T12:53:49.999348+0000","last_became_peered":"2024-09-11T12:53:49.999348+0000","last_unstale":"2024-09-11T12:53:50.007253+0000","last_undegraded":"2024-09-11T12:53:50.007253+0000","last_fullsized":"2024-09-11T12:53:50.007253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:06:46.408543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.975970+0000","last_change":"2024-09-11T12:53:48.989859+0000","last_active":"2024-09-11T12:53:49.975970+0000","last_peered":"2024-09-11T12:53:49.975970+0000","last_clean":"2024-09-11T12:53:49.975970+0000","last_became_active":"2024-09-11T12:53:48.989571+0000","last_became_peered":"2024-09-11T12:53:48.989571+0000","last_unstale":"2024-09-11T12:53:49.975970+0000","last_undegraded":"2024-09-11T12:53:49.975970+0000","last_fullsized":"2024-09-11T12:53:49.975970+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:36.906112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.988885+0000","last_change":"2024-09-11T12:53:49.986709+0000","last_active":"2024-09-11T12:53:49.988885+0000","last_peered":"2024-09-11T12:53:49.988885+0000","last_clean":"2024-09-11T12:53:49.988885+0000","last_became_active":"2024-09-11T12:53:49.986435+0000","last_became_peered":"2024-09-11T12:53:49.986435+0000","last_unstale":"2024-09-11T12:53:49.988885+0000","last_undegraded":"2024-09-11T12:53:49.988885+0000","last_fullsized":"2024-09-11T12:53:49.988885+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.977195+0000","last_change":"2024-09-11T12:53:48.983097+0000","last_active":"2024-09-11T12:53:49.977195+0000","last_peered":"2024-09-11T12:53:49.977195+0000","last_clean":"2024-09-11T12:53:49.977195+0000","last_became_active":"2024-09-11T12:53:48.981963+0000","last_became_peered":"2024-09-11T12:53:48.981963+0000","last_unstale":"2024-09-11T12:53:49.977195+0000","last_undegraded":"2024-09-11T12:53:49.977195+0000","last_fullsized":"2024-09-11T12:53:49.977195+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:49:39.963022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.993888+0000","last_change":"2024-09-11T12:53:49.992429+0000","last_active":"2024-09-11T12:53:49.993888+0000","last_peered":"2024-09-11T12:53:49.993888+0000","last_clean":"2024-09-11T12:53:49.993888+0000","last_became_active":"2024-09-11T12:53:49.990492+0000","last_became_peered":"2024-09-11T12:53:49.990492+0000","last_unstale":"2024-09-11T12:53:49.993888+0000","last_undegraded":"2024-09-11T12:53:49.993888+0000","last_fullsized":"2024-09-11T12:53:49.993888+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:50:19.725349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"82'3","reported_seq":16,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.006252+0000","last_change":"2024-09-11T12:53:48.987581+0000","last_active":"2024-09-11T12:53:50.006252+0000","last_peered":"2024-09-11T12:53:50.006252+0000","last_clean":"2024-09-11T12:53:50.006252+0000","last_became_active":"2024-09-11T12:53:48.987236+0000","last_became_peered":"2024-09-11T12:53:48.987236+0000","last_unstale":"2024-09-11T12:53:50.006252+0000","last_undegraded":"2024-09-11T12:53:50.006252+0000","last_fullsized":"2024-09-11T12:53:50.006252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:56:33.164278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997653+0000","last_change":"2024-09-11T12:53:49.995746+0000","last_active":"2024-09-11T12:53:49.997653+0000","last_peered":"2024-09-11T12:53:49.997653+0000","last_clean":"2024-09-11T12:53:49.997653+0000","last_became_active":"2024-09-11T12:53:49.994589+0000","last_became_peered":"2024-09-11T12:53:49.994589+0000","last_unstale":"2024-09-11T12:53:49.997653+0000","last_undegraded":"2024-09-11T12:53:49.997653+0000","last_fullsized":"2024-09-11T12:53:49.997653+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:05:26.217569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"82'6","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.009139+0000","last_change":"2024-09-11T12:53:48.983481+0000","last_active":"2024-09-11T12:53:50.009139+0000","last_peered":"2024-09-11T12:53:50.009139+0000","last_clean":"2024-09-11T12:53:50.009139+0000","last_became_active":"2024-09-11T12:53:48.982118+0000","last_became_peered":"2024-09-11T12:53:48.982118+0000","last_unstale":"2024-09-11T12:53:50.009139+0000","last_undegraded":"2024-09-11T12:53:50.009139+0000","last_fullsized":"2024-09-11T12:53:50.009139+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:36.906112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992455+0000","last_change":"2024-09-11T12:53:49.985738+0000","last_active":"2024-09-11T12:53:49.992455+0000","last_peered":"2024-09-11T12:53:49.992455+0000","last_clean":"2024-09-11T12:53:49.992455+0000","last_became_active":"2024-09-11T12:53:49.985234+0000","last_became_peered":"2024-09-11T12:53:49.985234+0000","last_unstale":"2024-09-11T12:53:49.992455+0000","last_undegraded":"2024-09-11T12:53:49.992455+0000","last_fullsized":"2024-09-11T12:53:49.992455+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:33:53.806460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001675+0000","last_change":"2024-09-11T12:53:48.982676+0000","last_active":"2024-09-11T12:53:50.001675+0000","last_peered":"2024-09-11T12:53:50.001675+0000","last_clean":"2024-09-11T12:53:50.001675+0000","last_became_active":"2024-09-11T12:53:48.982174+0000","last_became_peered":"2024-09-11T12:53:48.982174+0000","last_unstale":"2024-09-11T12:53:50.001675+0000","last_undegraded":"2024-09-11T12:53:50.001675+0000","last_fullsized":"2024-09-11T12:53:50.001675+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:26:38.159456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995711+0000","last_change":"2024-09-11T12:53:49.992391+0000","last_active":"2024-09-11T12:53:49.995711+0000","last_peered":"2024-09-11T12:53:49.995711+0000","last_clean":"2024-09-11T12:53:49.995711+0000","last_became_active":"2024-09-11T12:53:49.990788+0000","last_became_peered":"2024-09-11T12:53:49.990788+0000","last_unstale":"2024-09-11T12:53:49.995711+0000","last_undegraded":"2024-09-11T12:53:49.995711+0000","last_fullsized":"2024-09-11T12:53:49.995711+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:07.402433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.979401+0000","last_change":"2024-09-11T12:53:48.984403+0000","last_active":"2024-09-11T12:53:49.979401+0000","last_peered":"2024-09-11T12:53:49.979401+0000","last_clean":"2024-09-11T12:53:49.979401+0000","last_became_active":"2024-09-11T12:53:48.983571+0000","last_became_peered":"2024-09-11T12:53:48.983571+0000","last_unstale":"2024-09-11T12:53:49.979401+0000","last_undegraded":"2024-09-11T12:53:49.979401+0000","last_fullsized":"2024-09-11T12:53:49.979401+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:06.399793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992622+0000","last_change":"2024-09-11T12:53:49.987214+0000","last_active":"2024-09-11T12:53:49.992622+0000","last_peered":"2024-09-11T12:53:49.992622+0000","last_clean":"2024-09-11T12:53:49.992622+0000","last_became_active":"2024-09-11T12:53:49.986896+0000","last_became_peered":"2024-09-11T12:53:49.986896+0000","last_unstale":"2024-09-11T12:53:49.992622+0000","last_undegraded":"2024-09-11T12:53:49.992622+0000","last_fullsized":"2024-09-11T12:53:49.992622+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:40:40.280459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.968784+0000","last_change":"2024-09-11T12:53:48.974701+0000","last_active":"2024-09-11T12:53:49.968784+0000","last_peered":"2024-09-11T12:53:49.968784+0000","last_clean":"2024-09-11T12:53:49.968784+0000","last_became_active":"2024-09-11T12:53:48.974401+0000","last_became_peered":"2024-09-11T12:53:48.974401+0000","last_unstale":"2024-09-11T12:53:49.968784+0000","last_undegraded":"2024-09-11T12:53:49.968784+0000","last_fullsized":"2024-09-11T12:53:49.968784+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:19.587352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991733+0000","last_change":"2024-09-11T12:53:49.981695+0000","last_active":"2024-09-11T12:53:49.991733+0000","last_peered":"2024-09-11T12:53:49.991733+0000","last_clean":"2024-09-11T12:53:49.991733+0000","last_became_active":"2024-09-11T12:53:49.981491+0000","last_became_peered":"2024-09-11T12:53:49.981491+0000","last_unstale":"2024-09-11T12:53:49.991733+0000","last_undegraded":"2024-09-11T12:53:49.991733+0000","last_fullsized":"2024-09-11T12:53:49.991733+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:20:05.409971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"82'8","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999856+0000","last_change":"2024-09-11T12:53:48.979123+0000","last_active":"2024-09-11T12:53:49.999856+0000","last_peered":"2024-09-11T12:53:49.999856+0000","last_clean":"2024-09-11T12:53:49.999856+0000","last_became_active":"2024-09-11T12:53:48.978885+0000","last_became_peered":"2024-09-11T12:53:48.978885+0000","last_unstale":"2024-09-11T12:53:49.999856+0000","last_undegraded":"2024-09-11T12:53:49.999856+0000","last_fullsized":"2024-09-11T12:53:49.999856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:26:52.614926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.985319+0000","last_change":"2024-09-11T12:53:49.980347+0000","last_active":"2024-09-11T12:53:49.985319+0000","last_peered":"2024-09-11T12:53:49.985319+0000","last_clean":"2024-09-11T12:53:49.985319+0000","last_became_active":"2024-09-11T12:53:49.980066+0000","last_became_peered":"2024-09-11T12:53:49.980066+0000","last_unstale":"2024-09-11T12:53:49.985319+0000","last_undegraded":"2024-09-11T12:53:49.985319+0000","last_fullsized":"2024-09-11T12:53:49.985319+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:38:59.620107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.000275+0000","last_change":"2024-09-11T12:53:48.985225+0000","last_active":"2024-09-11T12:53:50.000275+0000","last_peered":"2024-09-11T12:53:50.000275+0000","last_clean":"2024-09-11T12:53:50.000275+0000","last_became_active":"2024-09-11T12:53:48.984903+0000","last_became_peered":"2024-09-11T12:53:48.984903+0000","last_unstale":"2024-09-11T12:53:50.000275+0000","last_undegraded":"2024-09-11T12:53:50.000275+0000","last_fullsized":"2024-09-11T12:53:50.000275+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:39.900007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995155+0000","last_change":"2024-09-11T12:53:49.986150+0000","last_active":"2024-09-11T12:53:49.995155+0000","last_peered":"2024-09-11T12:53:49.995155+0000","last_clean":"2024-09-11T12:53:49.995155+0000","last_became_active":"2024-09-11T12:53:49.985818+0000","last_became_peered":"2024-09-11T12:53:49.985818+0000","last_unstale":"2024-09-11T12:53:49.995155+0000","last_undegraded":"2024-09-11T12:53:49.995155+0000","last_fullsized":"2024-09-11T12:53:49.995155+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.002320+0000","last_change":"2024-09-11T12:53:48.982560+0000","last_active":"2024-09-11T12:53:50.002320+0000","last_peered":"2024-09-11T12:53:50.002320+0000","last_clean":"2024-09-11T12:53:50.002320+0000","last_became_active":"2024-09-11T12:53:48.982240+0000","last_became_peered":"2024-09-11T12:53:48.982240+0000","last_unstale":"2024-09-11T12:53:50.002320+0000","last_undegraded":"2024-09-11T12:53:50.002320+0000","last_fullsized":"2024-09-11T12:53:50.002320+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:23.496509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995656+0000","last_change":"2024-09-11T12:53:49.984067+0000","last_active":"2024-09-11T12:53:49.995656+0000","last_peered":"2024-09-11T12:53:49.995656+0000","last_clean":"2024-09-11T12:53:49.995656+0000","last_became_active":"2024-09-11T12:53:49.983499+0000","last_became_peered":"2024-09-11T12:53:49.983499+0000","last_unstale":"2024-09-11T12:53:49.995656+0000","last_undegraded":"2024-09-11T12:53:49.995656+0000","last_fullsized":"2024-09-11T12:53:49.995656+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:58:20.589992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.980167+0000","last_change":"2024-09-11T12:53:48.985614+0000","last_active":"2024-09-11T12:53:49.980167+0000","last_peered":"2024-09-11T12:53:49.980167+0000","last_clean":"2024-09-11T12:53:49.980167+0000","last_became_active":"2024-09-11T12:53:48.985328+0000","last_became_peered":"2024-09-11T12:53:48.985328+0000","last_unstale":"2024-09-11T12:53:49.980167+0000","last_undegraded":"2024-09-11T12:53:49.980167+0000","last_fullsized":"2024-09-11T12:53:49.980167+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:23.219983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.005773+0000","last_change":"2024-09-11T12:53:49.999655+0000","last_active":"2024-09-11T12:53:50.005773+0000","last_peered":"2024-09-11T12:53:50.005773+0000","last_clean":"2024-09-11T12:53:50.005773+0000","last_became_active":"2024-09-11T12:53:49.999064+0000","last_became_peered":"2024-09-11T12:53:49.999064+0000","last_unstale":"2024-09-11T12:53:50.005773+0000","last_undegraded":"2024-09-11T12:53:50.005773+0000","last_fullsized":"2024-09-11T12:53:50.005773+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:09.932147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.004538+0000","last_change":"2024-09-11T12:53:48.985368+0000","last_active":"2024-09-11T12:53:50.004538+0000","last_peered":"2024-09-11T12:53:50.004538+0000","last_clean":"2024-09-11T12:53:50.004538+0000","last_became_active":"2024-09-11T12:53:48.984458+0000","last_became_peered":"2024-09-11T12:53:48.984458+0000","last_unstale":"2024-09-11T12:53:50.004538+0000","last_undegraded":"2024-09-11T12:53:50.004538+0000","last_fullsized":"2024-09-11T12:53:50.004538+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:11.089175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996568+0000","last_change":"2024-09-11T12:53:49.992181+0000","last_active":"2024-09-11T12:53:49.996568+0000","last_peered":"2024-09-11T12:53:49.996568+0000","last_clean":"2024-09-11T12:53:49.996568+0000","last_became_active":"2024-09-11T12:53:49.991937+0000","last_became_peered":"2024-09-11T12:53:49.991937+0000","last_unstale":"2024-09-11T12:53:49.996568+0000","last_undegraded":"2024-09-11T12:53:49.996568+0000","last_fullsized":"2024-09-11T12:53:49.996568+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:46:41.754682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.984371+0000","last_change":"2024-09-11T12:53:48.981462+0000","last_active":"2024-09-11T12:53:49.984371+0000","last_peered":"2024-09-11T12:53:49.984371+0000","last_clean":"2024-09-11T12:53:49.984371+0000","last_became_active":"2024-09-11T12:53:48.980739+0000","last_became_peered":"2024-09-11T12:53:48.980739+0000","last_unstale":"2024-09-11T12:53:49.984371+0000","last_undegraded":"2024-09-11T12:53:49.984371+0000","last_fullsized":"2024-09-11T12:53:49.984371+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:36:12.794499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997163+0000","last_change":"2024-09-11T12:53:49.985757+0000","last_active":"2024-09-11T12:53:49.997163+0000","last_peered":"2024-09-11T12:53:49.997163+0000","last_clean":"2024-09-11T12:53:49.997163+0000","last_became_active":"2024-09-11T12:53:49.985471+0000","last_became_peered":"2024-09-11T12:53:49.985471+0000","last_unstale":"2024-09-11T12:53:49.997163+0000","last_undegraded":"2024-09-11T12:53:49.997163+0000","last_fullsized":"2024-09-11T12:53:49.997163+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:53.976223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.985183+0000","last_change":"2024-09-11T12:53:48.979205+0000","last_active":"2024-09-11T12:53:49.985183+0000","last_peered":"2024-09-11T12:53:49.985183+0000","last_clean":"2024-09-11T12:53:49.985183+0000","last_became_active":"2024-09-11T12:53:48.978826+0000","last_became_peered":"2024-09-11T12:53:48.978826+0000","last_unstale":"2024-09-11T12:53:49.985183+0000","last_undegraded":"2024-09-11T12:53:49.985183+0000","last_fullsized":"2024-09-11T12:53:49.985183+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:19.918269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.987148+0000","last_change":"2024-09-11T12:53:49.984557+0000","last_active":"2024-09-11T12:53:49.987148+0000","last_peered":"2024-09-11T12:53:49.987148+0000","last_clean":"2024-09-11T12:53:49.987148+0000","last_became_active":"2024-09-11T12:53:49.984245+0000","last_became_peered":"2024-09-11T12:53:49.984245+0000","last_unstale":"2024-09-11T12:53:49.987148+0000","last_undegraded":"2024-09-11T12:53:49.987148+0000","last_fullsized":"2024-09-11T12:53:49.987148+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:25:40.902648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976022+0000","last_change":"2024-09-11T12:53:48.980126+0000","last_active":"2024-09-11T12:53:49.976022+0000","last_peered":"2024-09-11T12:53:49.976022+0000","last_clean":"2024-09-11T12:53:49.976022+0000","last_became_active":"2024-09-11T12:53:48.979811+0000","last_became_peered":"2024-09-11T12:53:48.979811+0000","last_unstale":"2024-09-11T12:53:49.976022+0000","last_undegraded":"2024-09-11T12:53:49.976022+0000","last_fullsized":"2024-09-11T12:53:49.976022+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:24.063655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.991184+0000","last_change":"2024-09-11T12:53:49.980257+0000","last_active":"2024-09-11T12:53:49.991184+0000","last_peered":"2024-09-11T12:53:49.991184+0000","last_clean":"2024-09-11T12:53:49.991184+0000","last_became_active":"2024-09-11T12:53:49.979955+0000","last_became_peered":"2024-09-11T12:53:49.979955+0000","last_unstale":"2024-09-11T12:53:49.991184+0000","last_undegraded":"2024-09-11T12:53:49.991184+0000","last_fullsized":"2024-09-11T12:53:49.991184+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:17.327331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"82'2","reported_seq":15,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995523+0000","last_change":"2024-09-11T12:53:48.979643+0000","last_active":"2024-09-11T12:53:49.995523+0000","last_peered":"2024-09-11T12:53:49.995523+0000","last_clean":"2024-09-11T12:53:49.995523+0000","last_became_active":"2024-09-11T12:53:48.979338+0000","last_became_peered":"2024-09-11T12:53:48.979338+0000","last_unstale":"2024-09-11T12:53:49.995523+0000","last_undegraded":"2024-09-11T12:53:49.995523+0000","last_fullsized":"2024-09-11T12:53:49.995523+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:31.713435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.992632+0000","last_change":"2024-09-11T12:53:49.988900+0000","last_active":"2024-09-11T12:53:49.992632+0000","last_peered":"2024-09-11T12:53:49.992632+0000","last_clean":"2024-09-11T12:53:49.992632+0000","last_became_active":"2024-09-11T12:53:49.988547+0000","last_became_peered":"2024-09-11T12:53:49.988547+0000","last_unstale":"2024-09-11T12:53:49.992632+0000","last_undegraded":"2024-09-11T12:53:49.992632+0000","last_fullsized":"2024-09-11T12:53:49.992632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:09.864178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.968490+0000","last_change":"2024-09-11T12:53:48.982488+0000","last_active":"2024-09-11T12:53:49.968490+0000","last_peered":"2024-09-11T12:53:49.968490+0000","last_clean":"2024-09-11T12:53:49.968490+0000","last_became_active":"2024-09-11T12:53:48.981868+0000","last_became_peered":"2024-09-11T12:53:48.981868+0000","last_unstale":"2024-09-11T12:53:49.968490+0000","last_undegraded":"2024-09-11T12:53:49.968490+0000","last_fullsized":"2024-09-11T12:53:49.968490+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:18:04.148610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.995619+0000","last_change":"2024-09-11T12:53:49.987370+0000","last_active":"2024-09-11T12:53:49.995619+0000","last_peered":"2024-09-11T12:53:49.995619+0000","last_clean":"2024-09-11T12:53:49.995619+0000","last_became_active":"2024-09-11T12:53:49.987089+0000","last_became_peered":"2024-09-11T12:53:49.987089+0000","last_unstale":"2024-09-11T12:53:49.995619+0000","last_undegraded":"2024-09-11T12:53:49.995619+0000","last_fullsized":"2024-09-11T12:53:49.995619+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.974343+0000","last_change":"2024-09-11T12:53:48.985244+0000","last_active":"2024-09-11T12:53:49.974343+0000","last_peered":"2024-09-11T12:53:49.974343+0000","last_clean":"2024-09-11T12:53:49.974343+0000","last_became_active":"2024-09-11T12:53:48.984860+0000","last_became_peered":"2024-09-11T12:53:48.984860+0000","last_unstale":"2024-09-11T12:53:49.974343+0000","last_undegraded":"2024-09-11T12:53:49.974343+0000","last_fullsized":"2024-09-11T12:53:49.974343+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:16.324690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997632+0000","last_change":"2024-09-11T12:53:49.994856+0000","last_active":"2024-09-11T12:53:49.997632+0000","last_peered":"2024-09-11T12:53:49.997632+0000","last_clean":"2024-09-11T12:53:49.997632+0000","last_became_active":"2024-09-11T12:53:49.994186+0000","last_became_peered":"2024-09-11T12:53:49.994186+0000","last_unstale":"2024-09-11T12:53:49.997632+0000","last_undegraded":"2024-09-11T12:53:49.997632+0000","last_fullsized":"2024-09-11T12:53:49.997632+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:43:55.345850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.976113+0000","last_change":"2024-09-11T12:53:48.988076+0000","last_active":"2024-09-11T12:53:49.976113+0000","last_peered":"2024-09-11T12:53:49.976113+0000","last_clean":"2024-09-11T12:53:49.976113+0000","last_became_active":"2024-09-11T12:53:48.987776+0000","last_became_peered":"2024-09-11T12:53:48.987776+0000","last_unstale":"2024-09-11T12:53:49.976113+0000","last_undegraded":"2024-09-11T12:53:49.976113+0000","last_fullsized":"2024-09-11T12:53:49.976113+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:23:32.777475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.994791+0000","last_change":"2024-09-11T12:53:49.989841+0000","last_active":"2024-09-11T12:53:49.994791+0000","last_peered":"2024-09-11T12:53:49.994791+0000","last_clean":"2024-09-11T12:53:49.994791+0000","last_became_active":"2024-09-11T12:53:49.989580+0000","last_became_peered":"2024-09-11T12:53:49.989580+0000","last_unstale":"2024-09-11T12:53:49.994791+0000","last_undegraded":"2024-09-11T12:53:49.994791+0000","last_fullsized":"2024-09-11T12:53:49.994791+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:28.811784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.978317+0000","last_change":"2024-09-11T12:53:48.985710+0000","last_active":"2024-09-11T12:53:49.978317+0000","last_peered":"2024-09-11T12:53:49.978317+0000","last_clean":"2024-09-11T12:53:49.978317+0000","last_became_active":"2024-09-11T12:53:48.983745+0000","last_became_peered":"2024-09-11T12:53:48.983745+0000","last_unstale":"2024-09-11T12:53:49.978317+0000","last_undegraded":"2024-09-11T12:53:49.978317+0000","last_fullsized":"2024-09-11T12:53:49.978317+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:45.182724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.996429+0000","last_change":"2024-09-11T12:53:49.988191+0000","last_active":"2024-09-11T12:53:49.996429+0000","last_peered":"2024-09-11T12:53:49.996429+0000","last_clean":"2024-09-11T12:53:49.996429+0000","last_became_active":"2024-09-11T12:53:49.987734+0000","last_became_peered":"2024-09-11T12:53:49.987734+0000","last_unstale":"2024-09-11T12:53:49.996429+0000","last_undegraded":"2024-09-11T12:53:49.996429+0000","last_fullsized":"2024-09-11T12:53:49.996429+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:55.608657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.969076+0000","last_change":"2024-09-11T12:53:48.980510+0000","last_active":"2024-09-11T12:53:49.969076+0000","last_peered":"2024-09-11T12:53:49.969076+0000","last_clean":"2024-09-11T12:53:49.969076+0000","last_became_active":"2024-09-11T12:53:48.980079+0000","last_became_peered":"2024-09-11T12:53:48.980079+0000","last_unstale":"2024-09-11T12:53:49.969076+0000","last_undegraded":"2024-09-11T12:53:49.969076+0000","last_fullsized":"2024-09-11T12:53:49.969076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:22.268148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.999860+0000","last_change":"2024-09-11T12:53:49.998566+0000","last_active":"2024-09-11T12:53:49.999860+0000","last_peered":"2024-09-11T12:53:49.999860+0000","last_clean":"2024-09-11T12:53:49.999860+0000","last_became_active":"2024-09-11T12:53:49.998301+0000","last_became_peered":"2024-09-11T12:53:49.998301+0000","last_unstale":"2024-09-11T12:53:49.999860+0000","last_undegraded":"2024-09-11T12:53:49.999860+0000","last_fullsized":"2024-09-11T12:53:49.999860+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:12.091816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.974864+0000","last_change":"2024-09-11T12:53:48.987921+0000","last_active":"2024-09-11T12:53:49.974864+0000","last_peered":"2024-09-11T12:53:49.974864+0000","last_clean":"2024-09-11T12:53:49.974864+0000","last_became_active":"2024-09-11T12:53:48.987618+0000","last_became_peered":"2024-09-11T12:53:48.987618+0000","last_unstale":"2024-09-11T12:53:49.974864+0000","last_undegraded":"2024-09-11T12:53:49.974864+0000","last_fullsized":"2024-09-11T12:53:49.974864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:09.109265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.997003+0000","last_change":"2024-09-11T12:53:49.988172+0000","last_active":"2024-09-11T12:53:49.997003+0000","last_peered":"2024-09-11T12:53:49.997003+0000","last_clean":"2024-09-11T12:53:49.997003+0000","last_became_active":"2024-09-11T12:53:49.987185+0000","last_became_peered":"2024-09-11T12:53:49.987185+0000","last_unstale":"2024-09-11T12:53:49.997003+0000","last_undegraded":"2024-09-11T12:53:49.997003+0000","last_fullsized":"2024-09-11T12:53:49.997003+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:48.687748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":13,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.969096+0000","last_change":"2024-09-11T12:53:48.986576+0000","last_active":"2024-09-11T12:53:49.969096+0000","last_peered":"2024-09-11T12:53:49.969096+0000","last_clean":"2024-09-11T12:53:49.969096+0000","last_became_active":"2024-09-11T12:53:48.985143+0000","last_became_peered":"2024-09-11T12:53:48.985143+0000","last_unstale":"2024-09-11T12:53:49.969096+0000","last_undegraded":"2024-09-11T12:53:49.969096+0000","last_fullsized":"2024-09-11T12:53:49.969096+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:38.474205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.983165+0000","last_change":"2024-09-11T12:53:49.973576+0000","last_active":"2024-09-11T12:53:49.983165+0000","last_peered":"2024-09-11T12:53:49.983165+0000","last_clean":"2024-09-11T12:53:49.983165+0000","last_became_active":"2024-09-11T12:53:49.973284+0000","last_became_peered":"2024-09-11T12:53:49.973284+0000","last_unstale":"2024-09-11T12:53:49.983165+0000","last_undegraded":"2024-09-11T12:53:49.983165+0000","last_fullsized":"2024-09-11T12:53:49.983165+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":81,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:48.954042+0000","last_clean_scrub_stamp":"2024-09-11T12:53:48.954042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:55.780681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"82'1","reported_seq":14,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:50.001496+0000","last_change":"2024-09-11T12:53:48.984204+0000","last_active":"2024-09-11T12:53:50.001496+0000","last_peered":"2024-09-11T12:53:50.001496+0000","last_clean":"2024-09-11T12:53:50.001496+0000","last_became_active":"2024-09-11T12:53:48.983469+0000","last_became_peered":"2024-09-11T12:53:48.983469+0000","last_unstale":"2024-09-11T12:53:50.001496+0000","last_undegraded":"2024-09-11T12:53:50.001496+0000","last_fullsized":"2024-09-11T12:53:50.001496+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:53:47.951401+0000","last_clean_scrub_stamp":"2024-09-11T12:53:47.951401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:48:31.686244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":246,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-09-11T12:53:49.972386+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:53:49.972386+0000","last_peered":"2024-09-11T12:53:49.972386+0000","last_clean":"2024-09-11T12:53:49.972386+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:53:49.972386+0000","last_undegraded":"2024-09-11T12:53:49.972386+0000","last_fullsized":"2024-09-11T12:53:49.972386+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509495,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32748,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339092,"statfs":{"total":96636764160,"available":96603230208,"internally_reserved":0,"allocated":1597440,"data_stored":1015551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32152,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339688,"statfs":{"total":96636764160,"available":96603840512,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509495,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509495,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":999424,"data_stored":425113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509495,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509495,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32156,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339684,"statfs":{"total":96636764160,"available":96603836416,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509495,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339692,"statfs":{"total":96636764160,"available":96603844608,"internally_reserved":0,"allocated":987136,"data_stored":418338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32740,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339100,"statfs":{"total":96636764160,"available":96603238400,"internally_reserved":0,"allocated":1581056,"data_stored":1008706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:53:54.473 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:53:54.473 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:53:54.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- 172.21.15.78:0/2579934011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 msgr2=0x7ff1641600b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 --2- 172.21.15.78:0/2579934011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff1641600b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff15c009840 tx=0x7ff15c01c6d0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- 172.21.15.78:0/2579934011 shutdown_connections 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 --2- 172.21.15.78:0/2579934011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff1641600b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- 172.21.15.78:0/2579934011 >> 172.21.15.78:0/2579934011 conn(0x7ff1640ce770 msgr2=0x7ff1640ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- 172.21.15.78:0/2579934011 shutdown_connections 2024-09-11T12:53:54.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- 172.21.15.78:0/2579934011 wait complete. 2024-09-11T12:53:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 Processor -- start 2024-09-11T12:53:54.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- start start 2024-09-11T12:53:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1641528e0 0x7ff164158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff164152cc0 0x7ff1641586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff164169b20 con 0x7ff1640d1290 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641699a0 con 0x7ff164152cc0 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff164169ca0 con 0x7ff1641528e0 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44430/0 (socket says 172.21.15.78:44430) 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 -- 172.21.15.78:0/3186928234 learned_addr learned my addr 172.21.15.78:0/3186928234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff16a200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff164152cc0 0x7ff1641586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff168e00640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1641528e0 0x7ff164158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 -- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1641528e0 msgr2=0x7ff164158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1641528e0 0x7ff164158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 -- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff164152cc0 msgr2=0x7ff1641586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff164152cc0 0x7ff1641586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.664+0000 7ff169800640 1 -- 172.21.15.78:0/3186928234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff164159480 con 0x7ff1640d1290 2024-09-11T12:53:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff169800640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c840 tx=0x7ff15800cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:54.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff158003db0 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff158005070 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1580054a0 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16406cd60 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641591a0 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1641600b0 con 0x7ff1640d1290 2024-09-11T12:53:54.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff158005640 con 0x7ff1640d1290 2024-09-11T12:53:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 0x7ff144073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7ff158091de0 con 0x7ff1640d1290 2024-09-11T12:53:54.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.668+0000 7ff168e00640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 0x7ff144073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:54.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.672+0000 7ff168e00640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 0x7ff144073c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1640b3a40 tx=0x7ff150009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:54.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.672+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff15805db30 con 0x7ff1640d1290 2024-09-11T12:53:54.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.896+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff164158be0 con 0x7ff1640d1290 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.900+0000 7ff161600640 1 -- 172.21.15.78:0/3186928234 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) ==== 190+0+147 (secure 0 0 0) 0x7ff15805d430 con 0x7ff1640d1290 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:53:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:53:54.903 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:53:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 msgr2=0x7ff144073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 0x7ff144073c90 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff1640b3a40 tx=0x7ff150009040 comp rx=0 tx=0).stop 2024-09-11T12:53:54.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 msgr2=0x7ff164156670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c840 tx=0x7ff15800cd00 comp rx=0 tx=0).stop 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 shutdown_connections 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff1440717d0 0x7ff144073c90 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff164152cc0 0x7ff1641586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1641528e0 0x7ff164158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 --2- 172.21.15.78:0/3186928234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1640d1290 0x7ff164156670 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 >> 172.21.15.78:0/3186928234 conn(0x7ff1640ce770 msgr2=0x7ff1640ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 shutdown_connections 2024-09-11T12:53:54.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:54.904+0000 7ff16c200640 1 -- 172.21.15.78:0/3186928234 wait complete. 2024-09-11T12:53:54.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:53:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 -- 172.21.15.78:0/2998233931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340d0bf0 msgr2=0x7fba340d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 --2- 172.21.15.78:0/2998233931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340d0bf0 0x7fba340d10a0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fba340d1780 tx=0x7fba2c01cd80 comp rx=0 tx=0).stop 2024-09-11T12:53:55.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 -- 172.21.15.78:0/2998233931 shutdown_connections 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 --2- 172.21.15.78:0/2998233931 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340d0bf0 0x7fba340d10a0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 --2- 172.21.15.78:0/2998233931 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba3406fdf0 0x7fba340a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 --2- 172.21.15.78:0/2998233931 >> v2:172.21.15.157:3300/0 conn(0x7fba340b14b0 0x7fba3415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 -- 172.21.15.78:0/2998233931 >> 172.21.15.78:0/2998233931 conn(0x7fba34074b90 msgr2=0x7fba340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 -- 172.21.15.78:0/2998233931 shutdown_connections 2024-09-11T12:53:55.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.064+0000 7fba3be00640 1 -- 172.21.15.78:0/2998233931 wait complete. 2024-09-11T12:53:55.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 Processor -- start 2024-09-11T12:53:55.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- start start 2024-09-11T12:53:55.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3406fdf0 0x7fba34146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba340d0bf0 0x7fba341369d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba3406edb0 con 0x7fba340b14b0 2024-09-11T12:53:55.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba3406ec30 con 0x7fba3406fdf0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba3406ef30 con 0x7fba340d0bf0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44448/0 (socket says 172.21.15.78:44448) 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 -- 172.21.15.78:0/2014608921 learned_addr learned my addr 172.21.15.78:0/2014608921 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba39400640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3406fdf0 0x7fba34146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba39e00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba340d0bf0 0x7fba341369d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 -- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba340d0bf0 msgr2=0x7fba341369d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba340d0bf0 0x7fba341369d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 -- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3406fdf0 msgr2=0x7fba34146750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3406fdf0 0x7fba34146750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 -- 172.21.15.78:0/2014608921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba34136f10 con 0x7fba340b14b0 2024-09-11T12:53:55.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba38a00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c970 tx=0x7fba2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20007bb0 con 0x7fba340b14b0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba20007e40 con 0x7fba340b14b0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba20004290 con 0x7fba340b14b0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba34153380 con 0x7fba340b14b0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.068+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba34153940 con 0x7fba340b14b0 2024-09-11T12:53:55.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba3406f3e0 con 0x7fba340b14b0 2024-09-11T12:53:55.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fba20003600 con 0x7fba340b14b0 2024-09-11T12:53:55.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba31600640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 0x7fba14073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fba20092620 con 0x7fba340b14b0 2024-09-11T12:53:55.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba39400640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 0x7fba14073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.072+0000 7fba39400640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 0x7fba14073c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba2800abe0 tx=0x7fba2800a2d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.076+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fba34136f10 con 0x7fba340b14b0 2024-09-11T12:53:55.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.272+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba340b2e00 con 0x7fba340b14b0 2024-09-11T12:53:55.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.272+0000 7fba31600640 1 -- 172.21.15.78:0/2014608921 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+5228 (secure 0 0 0) 0x7fba20014070 con 0x7fba340b14b0 2024-09-11T12:53:55.276 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:53:55.276 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":56,"btime":"2024-09-11T12:53:52:987817+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6537,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/3066636859","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3066636859},{"type":"v1","addr":"172.21.15.78:6833","nonce":3066636859}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:53:55.276 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 56 2024-09-11T12:53:55.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 msgr2=0x7fba14073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 0x7fba14073c60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba2800abe0 tx=0x7fba2800a2d0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 msgr2=0x7fba341faad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fba2000c970 tx=0x7fba2000ce30 comp rx=0 tx=0).stop 2024-09-11T12:53:55.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 shutdown_connections 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba340d0bf0 0x7fba341369d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba340b14b0 0x7fba341faad0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fba140717a0 0x7fba14073c60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 --2- 172.21.15.78:0/2014608921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba3406fdf0 0x7fba34146750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 >> 172.21.15.78:0/2014608921 conn(0x7fba34074b90 msgr2=0x7fba340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 shutdown_connections 2024-09-11T12:53:55.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.276+0000 7fba3be00640 1 -- 172.21.15.78:0/2014608921 wait complete. 2024-09-11T12:53:55.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:53:49.362197+0000', 'modified': '2024-09-11T12:53:52.376088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6546}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6546': {'gid': 6546, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/242442581', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 242442581}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 242442581}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6546, 'qdb_cluster': [6546]} 2024-09-11T12:53:55.701 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:53:55.701 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-11T12:53:55.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- 172.21.15.78:0/4103351631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 msgr2=0x7f046015aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 --2- 172.21.15.78:0/4103351631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f046015aec0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f0458009900 tx=0x7f045801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- 172.21.15.78:0/4103351631 shutdown_connections 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 --2- 172.21.15.78:0/4103351631 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f046015aec0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- 172.21.15.78:0/4103351631 >> 172.21.15.78:0/4103351631 conn(0x7f046006ca80 msgr2=0x7f04600b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- 172.21.15.78:0/4103351631 shutdown_connections 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- 172.21.15.78:0/4103351631 wait complete. 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 Processor -- start 2024-09-11T12:53:55.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- start start 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 0x7f046014a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 0x7f046014a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0460151190 con 0x7f0460070360 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0460151010 con 0x7f0460074260 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.844+0000 7f0466000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0460151310 con 0x7f04600cf760 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44468/0 (socket says 172.21.15.78:44468) 2024-09-11T12:53:55.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 -- 172.21.15.78:0/2526358854 learned_addr learned my addr 172.21.15.78:0/2526358854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045ea00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 0x7f046014a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045fe00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 0x7f046014a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 -- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 msgr2=0x7f046014a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 0x7f046014a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 -- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 msgr2=0x7f046014a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 0x7f046014a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 -- 172.21.15.78:0/2526358854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0460141e30 con 0x7f0460070360 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045ea00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 0x7f046014a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045fe00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 0x7f046014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f045f400640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f045000c790 tx=0x7f045000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0457400640 1 -- 172.21.15.78:0/2526358854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003b80 con 0x7f0460070360 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0457400640 1 -- 172.21.15.78:0/2526358854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0450003d20 con 0x7f0460070360 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0460144630 con 0x7f0460070360 2024-09-11T12:53:55.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0457400640 1 -- 172.21.15.78:0/2526358854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045000aa90 con 0x7f0460070360 2024-09-11T12:53:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0460144bf0 con 0x7f0460070360 2024-09-11T12:53:55.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.848+0000 7f0457400640 1 -- 172.21.15.78:0/2526358854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f045000ac30 con 0x7f0460070360 2024-09-11T12:53:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0457400640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 0x7f0440073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 Processor -- start 2024-09-11T12:53:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0457400640 1 -- 172.21.15.78:0/2526358854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0450091e30 con 0x7f0460070360 2024-09-11T12:53:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 0x7f0440073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 -- start start 2024-09-11T12:53:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 0x7f0440073c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0460149250 tx=0x7f044c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6296e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc624a340 0x55adc624a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55adc60a7270 con 0x55adc62973c0 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adc6096770 con 0x55adc6296aa0 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55adc609a7b0 con 0x55adc624a340 2024-09-11T12:53:55.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44482/0 (socket says 172.21.15.78:44482) 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 -- 172.21.15.78:0/1769845933 learned_addr learned my addr 172.21.15.78:0/1769845933 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045f400640 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6296e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045fe00640 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc624a340 0x55adc624a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 -- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc624a340 msgr2=0x55adc624a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc624a340 0x55adc624a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 -- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 msgr2=0x55adc6296e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6296e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 -- 172.21.15.78:0/1769845933 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc6035f90 con 0x55adc62973c0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f045ea00640 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f044c007e70 tx=0x7f044c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bd0f9b79a32cc61 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f044be00640 1 -- 172.21.15.78:0/1769845933 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c003600 con 0x55adc62973c0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f044be00640 1 -- 172.21.15.78:0/1769845933 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f044c003890 con 0x55adc62973c0 2024-09-11T12:53:55.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.852+0000 7f044be00640 1 -- 172.21.15.78:0/1769845933 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c003ca0 con 0x55adc62973c0 2024-09-11T12:53:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1769845933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 msgr2=0x55adc6249e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f044c007e70 tx=0x7f044c011000 comp rx=0 tx=0).stop 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1769845933 shutdown_connections 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc624a340 0x55adc624a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc6249e00 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- 172.21.15.78:0/1769845933 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6296e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1769845933 >> 172.21.15.78:0/1769845933 conn(0x55adc6277b60 msgr2=0x55adc6279f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1769845933 shutdown_connections 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1769845933 wait complete. 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- start start 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 Processor -- start 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6251620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc62973c0 0x55adc6256d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55adc6166290 con 0x55adc624a340 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adc6042ab0 con 0x55adc6296aa0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55adc6040170 con 0x55adc62973c0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 10 client.0 _ll_register_callbacks cb 0x55adc6237830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44496/0 (socket says 172.21.15.78:44496) 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 -- 172.21.15.78:0/1093355932 learned_addr learned my addr 172.21.15.78:0/1093355932 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045fe00640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc62973c0 0x55adc6256d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045ea00640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6251620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 -- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc62973c0 msgr2=0x55adc6256d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc62973c0 0x55adc6256d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 -- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 msgr2=0x55adc6251620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6251620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 -- 172.21.15.78:0/1093355932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc611b6f0 con 0x55adc624a340 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045ea00640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6251620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f045f400640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0450059a70 tx=0x7f045005d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ab336984cbfe9c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045005db80 con 0x55adc624a340 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0450058df0 con 0x55adc624a340 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55adc602ff10 con 0x55adc624a340 2024-09-11T12:53:55.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f045005d480 con 0x55adc624a340 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 10 client.6678 fetch_fsmap learned FSMap version 56 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55adc60f2c40 con 0x55adc624a340 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55adc602ff10 con 0x55adc624a340 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f0450058ff0 con 0x55adc624a340 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 10 client.6678 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 10 client.6678 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 20 client.6678 populate_metadata read hostname 'smithi078' 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a2a60 0x55adc62a2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 4 client.6678 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.856+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 --> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adc60a91c0 con 0x55adc62a2a60 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f045ea00640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a2a60 0x55adc62a2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f045ea00640 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a2a60 0x55adc62a2e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=215a1851be7d6191 server_cookie=d2cec2a33d61567d in_seq=0 out_seq=0 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0448c00640 10 client.6678 ms_handle_connect on v2:172.21.15.78:6834/242442581 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0448c00640 1 -- 172.21.15.78:0/1093355932 <== mds.0 v2:172.21.15.78:6834/242442581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55adc60a91c0 con 0x55adc62a2a60 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0448c00640 10 client.6678 handle_command_reply: tid=0 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 client.6678 shutdown 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 msgr2=0x55adc62510e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0450059a70 tx=0x7f045005d310 comp rx=0 tx=0).stop 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 shutdown_connections 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc62973c0 0x55adc6256d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a2a60 0x55adc62a2e40 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc6296aa0 0x55adc6251620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 --2- 172.21.15.78:0/1093355932 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc62510e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0448c00640 0 client.6678 ms_handle_reset on v2:172.21.15.78:6834/242442581 2024-09-11T12:53:55.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 >> 172.21.15.78:0/1093355932 conn(0x55adc6277b60 msgr2=0x55adc6249500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 shutdown_connections 2024-09-11T12:53:55.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 1 -- 172.21.15.78:0/1093355932 wait complete. 2024-09-11T12:53:55.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.860+0000 7f0469341000 20 client.6678 trim_cache size 0 max 16384 2024-09-11T12:53:55.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 Processor -- start 2024-09-11T12:53:55.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- start start 2024-09-11T12:53:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc6277080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc62973c0 0x55adc62b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f04600b0c20 con 0x55adc624a340 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0460132100 con 0x55adc62973c0 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f04601f90c0 con 0x55adc6296aa0 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44504/0 (socket says 172.21.15.78:44504) 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 -- 172.21.15.78:0/3734643524 learned_addr learned my addr 172.21.15.78:0/3734643524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045fe00640 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc62973c0 0x55adc62b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045ea00640 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc6277080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 -- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 msgr2=0x55adc6277080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc6277080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 -- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc62973c0 msgr2=0x55adc62b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc62973c0 0x55adc62b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 -- 172.21.15.78:0/3734643524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adc6257070 con 0x55adc624a340 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f045f400640 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f045000c760 tx=0x7f045005d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6086b5e3d605a984 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f044b400640 1 -- 172.21.15.78:0/3734643524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450090040 con 0x55adc624a340 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f044b400640 1 -- 172.21.15.78:0/3734643524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f045005db80 con 0x55adc624a340 2024-09-11T12:53:55.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f044b400640 1 -- 172.21.15.78:0/3734643524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450059050 con 0x55adc624a340 2024-09-11T12:53:55.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- 172.21.15.78:0/3734643524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 msgr2=0x55adc6276b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f045000c760 tx=0x7f045005d380 comp rx=0 tx=0).stop 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- 172.21.15.78:0/3734643524 shutdown_connections 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc62973c0 0x55adc62b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc6277080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 --2- 172.21.15.78:0/3734643524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc624a340 0x55adc6276b40 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- 172.21.15.78:0/3734643524 >> 172.21.15.78:0/3734643524 conn(0x55adc6277b60 msgr2=0x55adc624c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- 172.21.15.78:0/3734643524 shutdown_connections 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.880+0000 7f0469341000 1 -- 172.21.15.78:0/3734643524 wait complete. 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- start start 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 Processor -- start 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc624a340 0x55adc62a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc62b3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55adc609ede0 con 0x55adc62973c0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55adc6099ac0 con 0x55adc624a340 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55adc60a9ea0 con 0x55adc6296aa0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 10 client.0 _ll_register_callbacks cb 0x7f0450064050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44520/0 (socket says 172.21.15.78:44520) 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 -- 172.21.15.78:0/1647131196 learned_addr learned my addr 172.21.15.78:0/1647131196 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045f400640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc624a340 0x55adc62a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045ea00640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc62b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 -- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 msgr2=0x55adc62b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc62b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 -- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc624a340 msgr2=0x55adc62a96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:53:55.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc624a340 0x55adc62a96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 -- 172.21.15.78:0/1647131196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adc604be70 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045fe00640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f04580051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68310b1c36de3d36 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04580b2070 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f0458005510 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045005db80 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) ==== 24+0+0 (secure 0 0 0) 0x7f04580c3050 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 10 client.6684 fetch_fsmap learned FSMap version 56 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) -- 0x55adc6020bf0 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f045005db80 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 56) ==== 3502+0+0 (secure 0 0 0) 0x7f04580c6020 con 0x55adc62973c0 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 10 client.6684 fetch_fsmap finished waiting for FSMap version 56 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 10 client.6684 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:53:55.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 20 client.6684 populate_metadata read hostname 'smithi078' 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a5270 0x55adc62a5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 4 client.6684 mds_command: new command op to 6546 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 --> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] -- command(tid 0: {"prefix": "status"}) -- 0x55adc60a4c60 con 0x55adc62a5270 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045f400640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a5270 0x55adc62a5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f045f400640 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a5270 0x55adc62a5650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0aa9ac03e84b416 server_cookie=d1c949132497402b in_seq=0 out_seq=0 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.884+0000 7f0448c00640 10 client.6684 ms_handle_connect on v2:172.21.15.78:6834/242442581 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0448c00640 1 -- 172.21.15.78:0/1647131196 <== mds.0 v2:172.21.15.78:6834/242442581 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x55adc60a4c60 con 0x55adc62a5270 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0448c00640 10 client.6684 handle_command_reply: tid=0 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 client.6684 shutdown 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 msgr2=0x55adc62b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0458009230 tx=0x7f04580051a0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 shutdown_connections 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55adc62973c0 0x55adc62b3570 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55adc6296aa0 0x55adc62b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.78:6834/242442581,v1:172.21.15.78:6835/242442581] conn(0x55adc62a5270 0x55adc62a5650 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 --2- 172.21.15.78:0/1647131196 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55adc624a340 0x55adc62a96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 >> 172.21.15.78:0/1647131196 conn(0x55adc6277b60 msgr2=0x55adc62a2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0448c00640 0 client.6684 ms_handle_reset on v2:172.21.15.78:6834/242442581 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 shutdown_connections 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 1 -- 172.21.15.78:0/1647131196 wait complete. 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0469341000 20 client.6684 trim_cache size 0 max 16384 2024-09-11T12:53:55.893 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6546, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9254538180000003, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 56, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 83, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 83, 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.441416853 2024-09-11T12:53:55.894 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 msgr2=0x7f0440073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 0x7f0440073c60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f0460149250 tx=0x7f044c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 msgr2=0x7f04601499c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f045000c790 tx=0x7f045000cc50 comp rx=0 tx=0).stop 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 shutdown_connections 2024-09-11T12:53:55.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f04600cf760 0x7f046014a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f04400717a0 0x7f0440073c60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0460074260 0x7f046014a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 --2- 172.21.15.78:0/2526358854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0460070360 0x7f04601499c0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 >> 172.21.15.78:0/2526358854 conn(0x7f046006ca80 msgr2=0x7f046006fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 shutdown_connections 2024-09-11T12:53:55.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:55.888+0000 7f0466000640 1 -- 172.21.15.78:0/2526358854 wait complete. 2024-09-11T12:53:55.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:53:56.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- 172.21.15.78:0/1001658643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa228005240 msgr2=0x7fa22800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 --2- 172.21.15.78:0/1001658643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa228005240 0x7fa22800b9d0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fa23c00c760 tx=0x7fa23c01d180 comp rx=0 tx=0).stop 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- 172.21.15.78:0/1001658643 shutdown_connections 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 --2- 172.21.15.78:0/1001658643 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa228005240 0x7fa22800b9d0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- 172.21.15.78:0/1001658643 >> 172.21.15.78:0/1001658643 conn(0x7fa24406ca80 msgr2=0x7fa2440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- 172.21.15.78:0/1001658643 shutdown_connections 2024-09-11T12:53:56.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- 172.21.15.78:0/1001658643 wait complete. 2024-09-11T12:53:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 Processor -- start 2024-09-11T12:53:56.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- start start 2024-09-11T12:53:56.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 0x7fa2441f0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 0x7fa24406eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2440747a0 con 0x7fa2441f3de0 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa244074620 con 0x7fa228005240 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa249000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa24415e5b0 con 0x7fa244140e20 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44546/0 (socket says 172.21.15.78:44546) 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 -- 172.21.15.78:0/4118550321 learned_addr learned my addr 172.21.15.78:0/4118550321 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa241e00640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 0x7fa2441f0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa241400640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 0x7fa24406eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 -- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 msgr2=0x7fa24406eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 0x7fa24406eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 -- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 msgr2=0x7fa2441f0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:56.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 0x7fa2441f0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa242800640 1 -- 172.21.15.78:0/4118550321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2441efef0 con 0x7fa2441f3de0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa241e00640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 0x7fa2441f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.052+0000 7fa241400640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 0x7fa24406eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa242800640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa23c01cc40 tx=0x7fa23c0012e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0014a0 con 0x7fa2441f3de0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa23c006070 con 0x7fa2441f3de0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0064c0 con 0x7fa2441f3de0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2441f3620 con 0x7fa2441f3de0 2024-09-11T12:53:56.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa244141990 con 0x7fa2441f3de0 2024-09-11T12:53:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa24415eb10 con 0x7fa2441f3de0 2024-09-11T12:53:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa23c0b9030 con 0x7fa2441f3de0 2024-09-11T12:53:56.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.056+0000 7fa23a000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 0x7fa224073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:56.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.060+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fa23c12c110 con 0x7fa2441f3de0 2024-09-11T12:53:56.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.060+0000 7fa241e00640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 0x7fa224073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:56.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.060+0000 7fa241e00640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 0x7fa224073c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa24415c3a0 tx=0x7fa234006b80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:56.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.060+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa2440705a0 con 0x7fa2441f3de0 2024-09-11T12:53:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.268+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa24406f9b0 con 0x7fa2441f3de0 2024-09-11T12:53:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.268+0000 7fa23a000640 1 -- 172.21.15.78:0/4118550321 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7fa23c0f7e60 con 0x7fa2441f3de0 2024-09-11T12:53:56.272 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:53:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 msgr2=0x7fa224073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 0x7fa224073c90 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa24415c3a0 tx=0x7fa234006b80 comp rx=0 tx=0).stop 2024-09-11T12:53:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 msgr2=0x7fa24413edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fa23c01cc40 tx=0x7fa23c0012e0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 shutdown_connections 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2441f3de0 0x7fa24413edd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa244140e20 0x7fa24406eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2240717d0 0x7fa224073c90 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 --2- 172.21.15.78:0/4118550321 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa228005240 0x7fa2441f0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 >> 172.21.15.78:0/4118550321 conn(0x7fa24406ca80 msgr2=0x7fa2440b1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 shutdown_connections 2024-09-11T12:53:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56.272+0000 7fa249000640 1 -- 172.21.15.78:0/4118550321 wait complete. 2024-09-11T12:53:56.292 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:53:56.292 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:53:56.292 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:53:56.293 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:56.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.293 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:53:56.303 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:53:56.303 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:53:56.303 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:53:56.303 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:53:56.304 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42198sec preferred_lft 42198sec 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.305 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:56.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.307 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:53:56.365 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:53:56.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.367 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:53:56.421 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:53:56.421 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:53:56.421 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:53:56.421 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:53:56.421 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:56.422 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:53:56.422 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:53:56.422 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:53:56.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:53:56.422 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:53:56.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:53:56.423 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:53:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:53:56.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:56.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.559 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:53:56.567 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:53:56.567 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:53:56.568 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:56.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.569 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:53:56.635 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:56.635 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:56.683 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:53:56.684 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:56.684 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:56.735 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:53:56.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.735 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:56.823 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:56.823 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:56.836 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:56.836 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:56.836 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:53:56.836 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:53:56.836 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:53:56.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:53:56.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:56.840 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:53:56.906 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:53:56.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:53:57.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:57.013 INFO:teuthology.orchestra.run.smithi078.stdout:client6693 2024-09-11T12:53:57.014 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:53:57.014 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:57.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 -- 172.21.15.78:0/2972712918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 msgr2=0x7fab500b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 --2- 172.21.15.78:0/2972712918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab500b31f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fab500b1610 tx=0x7fab3c01c800 comp rx=0 tx=0).stop 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 -- 172.21.15.78:0/2972712918 shutdown_connections 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 --2- 172.21.15.78:0/2972712918 >> v2:172.21.15.157:3300/0 conn(0x7fab5015dcc0 0x7fab5015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 --2- 172.21.15.78:0/2972712918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab500b31f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 -- 172.21.15.78:0/2972712918 >> 172.21.15.78:0/2972712918 conn(0x7fab500b19a0 msgr2=0x7fab50070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:57.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 -- 172.21.15.78:0/2972712918 shutdown_connections 2024-09-11T12:53:57.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.204+0000 7fab56400640 1 -- 172.21.15.78:0/2972712918 wait complete. 2024-09-11T12:53:57.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 Processor -- start 2024-09-11T12:53:57.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- start start 2024-09-11T12:53:57.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:57.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 0x7fab50149160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 0x7fab50154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fab50168d30 con 0x7fab500b2dc0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fab50168bb0 con 0x7fab5015dcc0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fab50168eb0 con 0x7fab5015c390 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44576/0 (socket says 172.21.15.78:44576) 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 -- 172.21.15.78:0/2323338343 learned_addr learned my addr 172.21.15.78:0/2323338343 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55a00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 0x7fab50154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab4fe00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 0x7fab50149160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 -- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 msgr2=0x7fab50149160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 0x7fab50149160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 -- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 msgr2=0x7fab50154be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 0x7fab50154be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 -- 172.21.15.78:0/2323338343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab50153850 con 0x7fab500b2dc0 2024-09-11T12:53:57.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab4fe00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 0x7fab50149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:57.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55a00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 0x7fab50154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:53:57.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab55000640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c450 tx=0x7fab4000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40012830 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab40017070 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab40012d80 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab50153af0 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.208+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab501540b0 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab500b31f0 con 0x7fab500b2dc0 2024-09-11T12:53:57.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fab4001c460 con 0x7fab500b2dc0 2024-09-11T12:53:57.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab4cc00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 0x7fab30073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:57.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fab40091db0 con 0x7fab500b2dc0 2024-09-11T12:53:57.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab4fe00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 0x7fab30073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:57.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.212+0000 7fab4fe00640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 0x7fab30073c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0af000 tx=0x7fab3c01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:57.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.216+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fab40096070 con 0x7fab500b2dc0 2024-09-11T12:53:57.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.424+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fab50074e00 con 0x7fab500b2dc0 2024-09-11T12:53:57.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.424+0000 7fab4cc00640 1 -- 172.21.15.78:0/2323338343 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) ==== 53+0+73 (secure 0 0 0) 0x7fab4005db00 con 0x7fab500b2dc0 2024-09-11T12:53:57.426 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 msgr2=0x7fab30073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 0x7fab30073c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fab3c0af000 tx=0x7fab3c01b040 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 msgr2=0x7fab50149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fab4000c450 tx=0x7fab4000c910 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 shutdown_connections 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fab300717a0 0x7fab30073c60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fab5015dcc0 0x7fab50154be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fab5015c390 0x7fab50149160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 --2- 172.21.15.78:0/2323338343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fab500b2dc0 0x7fab50149d90 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 >> 172.21.15.78:0/2323338343 conn(0x7fab500b19a0 msgr2=0x7fab500b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 shutdown_connections 2024-09-11T12:53:57.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:57.428+0000 7fab56400640 1 -- 172.21.15.78:0/2323338343 wait complete. 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:53:57.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:57.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.449 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:53:57.459 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42206sec preferred_lft 42206sec 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:53:57.460 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:53:57.461 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.462 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:53:57.462 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:53:57.462 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:53:57.462 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:53:57.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.463 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:53:57.523 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:53:57.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.525 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:53:57.580 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:53:57.580 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:53:57.580 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:53:57.580 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:53:57.581 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:53:57.581 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:53:57.581 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:53:57.581 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:53:57.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:53:57.582 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:53:57.582 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:53:57.680 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:53:57.722 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:57.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.730 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:53:57.739 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:53:57.739 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:53:57.739 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:57.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.739 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:53:57.803 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:57.804 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:57.860 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:53:57.860 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:53:57.860 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:57.915 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:53:57.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:57.915 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:58.007 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:58.007 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:58.021 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:53:58.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:53:58.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:53:58.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:53:58.021 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:53:58.025 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:53:58.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:53:58.025 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:53:58.044 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:53:58.134 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:53:58.137 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:53:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:53:58.143 INFO:teuthology.orchestra.run.smithi157.stdout:client5834 2024-09-11T12:53:58.143 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:53:58.143 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:53:58.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- 172.21.15.78:0/1592384623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159970 msgr2=0x7fed040706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- 172.21.15.78:0/1592384623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159970 0x7fed040706e0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fed040ccc70 tx=0x7fecf401d530 comp rx=0 tx=0).stop 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- 172.21.15.78:0/1592384623 shutdown_connections 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- 172.21.15.78:0/1592384623 >> v2:172.21.15.157:3300/0 conn(0x7fed040b5530 0x7fed040b5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- 172.21.15.78:0/1592384623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159970 0x7fed040706e0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- 172.21.15.78:0/1592384623 >> 172.21.15.78:0/1592384623 conn(0x7fed040b3050 msgr2=0x7fed040b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- 172.21.15.78:0/1592384623 shutdown_connections 2024-09-11T12:53:58.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- 172.21.15.78:0/1592384623 wait complete. 2024-09-11T12:53:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 Processor -- start 2024-09-11T12:53:58.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- start start 2024-09-11T12:53:58.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed040b5530 0x7fed041f8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed04159970 0x7fed041faf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed04166b30 con 0x7fed04159d50 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed041669b0 con 0x7fed04159970 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed09c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed04166cb0 con 0x7fed040b5530 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44586/0 (socket says 172.21.15.78:44586) 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 -- 172.21.15.78:0/3932244996 learned_addr learned my addr 172.21.15.78:0/3932244996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed02000640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed04159970 0x7fed041faf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed02a00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed040b5530 0x7fed041f8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 -- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed040b5530 msgr2=0x7fed041f8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:58.352 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed040b5530 0x7fed041f8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 -- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed04159970 msgr2=0x7fed041faf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed04159970 0x7fed041faf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.348+0000 7fed03400640 1 -- 172.21.15.78:0/3932244996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed040ce1f0 con 0x7fed04159d50 2024-09-11T12:53:58.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fed03400640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00eb20 tx=0x7fecfc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc004b30 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecfc004cf0 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecfc005280 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed041fb680 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed041fbc40 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed040b5960 con 0x7fed04159d50 2024-09-11T12:53:58.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fecfc005420 con 0x7fed04159d50 2024-09-11T12:53:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 0x7fece4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7fecfc092b90 con 0x7fed04159d50 2024-09-11T12:53:58.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.352+0000 7fed02a00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 0x7fece4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:58.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.356+0000 7fed02a00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 0x7fece4073c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ae000 tx=0x7fecf401bd70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:58.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.356+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fecfc05e0f0 con 0x7fed04159d50 2024-09-11T12:53:58.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:58.560+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fed04150130 con 0x7fed04159d50 2024-09-11T12:53:59.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fecfaa00640 1 -- 172.21.15.78:0/3932244996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v57) ==== 136+0+0 (secure 0 0 0) 0x7fecfc062640 con 0x7fed04159d50 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 msgr2=0x7fece4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 0x7fece4073c60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fecf40ae000 tx=0x7fecf401bd70 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 msgr2=0x7fed040cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fecfc00eb20 tx=0x7fecfc00c2d0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 shutdown_connections 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed04159d50 0x7fed040cdcb0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fece40717a0 0x7fece4073c60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed04159970 0x7fed041faf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 --2- 172.21.15.78:0/3932244996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed040b5530 0x7fed041f8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 >> 172.21.15.78:0/3932244996 conn(0x7fed040b3050 msgr2=0x7fed0415c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 shutdown_connections 2024-09-11T12:53:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.016+0000 7fed09c00640 1 -- 172.21.15.78:0/3932244996 wait complete. 2024-09-11T12:53:59.038 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- 172.21.15.78:0/3792229212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740746c0 msgr2=0x7feb7415c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 --2- 172.21.15.78:0/3792229212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740746c0 0x7feb7415c620 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009900 tx=0x7feb6c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- 172.21.15.78:0/3792229212 shutdown_connections 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 --2- 172.21.15.78:0/3792229212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740746c0 0x7feb7415c620 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 --2- 172.21.15.78:0/3792229212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 0x7feb7415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- 172.21.15.78:0/3792229212 >> 172.21.15.78:0/3792229212 conn(0x7feb7406f0c0 msgr2=0x7feb74072660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- 172.21.15.78:0/3792229212 shutdown_connections 2024-09-11T12:53:59.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- 172.21.15.78:0/3792229212 wait complete. 2024-09-11T12:53:59.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 Processor -- start 2024-09-11T12:53:59.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.180+0000 7feb79600640 1 -- start start 2024-09-11T12:53:59.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb740746c0 0x7feb74152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 0x7feb741649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feb74167660 con 0x7feb740b13e0 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feb741674e0 con 0x7feb740746c0 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feb741677e0 con 0x7feb74154d70 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44606/0 (socket says 172.21.15.78:44606) 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 -- 172.21.15.78:0/3276704412 learned_addr learned my addr 172.21.15.78:0/3276704412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73e00640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb740746c0 0x7feb74152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb78c00640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 0x7feb741649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 -- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 msgr2=0x7feb741649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 0x7feb741649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 -- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb740746c0 msgr2=0x7feb74152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb740746c0 0x7feb74152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 -- 172.21.15.78:0/3276704412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb741fb4d0 con 0x7feb740b13e0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb73400640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c400 tx=0x7feb6000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60017070 con 0x7feb740b13e0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb600053e0 con 0x7feb740b13e0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60003860 con 0x7feb740b13e0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741fa730 con 0x7feb740b13e0 2024-09-11T12:53:59.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb741facd0 con 0x7feb740b13e0 2024-09-11T12:53:59.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7415c620 con 0x7feb740b13e0 2024-09-11T12:53:59.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.184+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7feb6001a070 con 0x7feb740b13e0 2024-09-11T12:53:59.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.188+0000 7feb6be00640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 0x7feb54073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.188+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7feb600913c0 con 0x7feb740b13e0 2024-09-11T12:53:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.188+0000 7feb73e00640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 0x7feb54073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.188+0000 7feb73e00640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 0x7feb54073c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb64009ec0 tx=0x7feb64009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.192+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7feb60001130 con 0x7feb740b13e0 2024-09-11T12:53:59.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.380+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7feb74165040 con 0x7feb540717a0 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.380+0000 7feb6be00640 1 -- 172.21.15.78:0/3276704412 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7feb6401b470 con 0x7feb540717a0 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:53:59.382 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:53:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 msgr2=0x7feb54073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 0x7feb54073c60 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7feb64009ec0 tx=0x7feb64009500 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 msgr2=0x7feb74164460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7feb6000c400 tx=0x7feb6000c8c0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 shutdown_connections 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feb74154d70 0x7feb741649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feb740b13e0 0x7feb74164460 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7feb540717a0 0x7feb54073c60 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 --2- 172.21.15.78:0/3276704412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feb740746c0 0x7feb74152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 >> 172.21.15.78:0/3276704412 conn(0x7feb7406f0c0 msgr2=0x7feb7406fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 shutdown_connections 2024-09-11T12:53:59.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.384+0000 7feb79600640 1 -- 172.21.15.78:0/3276704412 wait complete. 2024-09-11T12:53:59.403 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 -- 172.21.15.78:0/3651846713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b3420 msgr2=0x7f185c0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 --2- 172.21.15.78:0/3651846713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b3420 0x7f185c0cf6e0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f184400aa10 tx=0x7f184401dc30 comp rx=0 tx=0).stop 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 -- 172.21.15.78:0/3651846713 shutdown_connections 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 --2- 172.21.15.78:0/3651846713 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0b3420 0x7f185c0cf6e0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 -- 172.21.15.78:0/3651846713 >> 172.21.15.78:0/3651846713 conn(0x7f185c071dd0 msgr2=0x7f185c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 -- 172.21.15.78:0/3651846713 shutdown_connections 2024-09-11T12:53:59.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.544+0000 7f1863800640 1 -- 172.21.15.78:0/3651846713 wait complete. 2024-09-11T12:53:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 Processor -- start 2024-09-11T12:53:59.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- start start 2024-09-11T12:53:59.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c0b3040 0x7f185c1f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f185c0b3420 0x7f185c1644c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f185c0af070 con 0x7f185c0cd770 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f185c0aeef0 con 0x7f185c0b3040 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f185c0af1f0 con 0x7f185c0b3420 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44620/0 (socket says 172.21.15.78:44620) 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 -- 172.21.15.78:0/4115485058 learned_addr learned my addr 172.21.15.78:0/4115485058 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1860e00640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c0b3040 0x7f185c1f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f185be00640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f185c0b3420 0x7f185c1644c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 -- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f185c0b3420 msgr2=0x7f185c1644c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f185c0b3420 0x7f185c1644c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 -- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c0b3040 msgr2=0x7f185c1f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c0b3040 0x7f185c1f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 -- 172.21.15.78:0/4115485058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f185c164a00 con 0x7f185c0cd770 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1861800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f185400c3c0 tx=0x7f185400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854017070 con 0x7f185c0cd770 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18540053c0 con 0x7f185c0cd770 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f185c164ca0 con 0x7f185c0cd770 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1854003860 con 0x7f185c0cd770 2024-09-11T12:53:59.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.548+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f185c165260 con 0x7f185c0cd770 2024-09-11T12:53:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f185c165a30 con 0x7f185c0cd770 2024-09-11T12:53:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f185401a070 con 0x7f185c0cd770 2024-09-11T12:53:59.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1858c00640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 0x7f1838073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f1854091200 con 0x7f185c0cd770 2024-09-11T12:53:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1860e00640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 0x7f1838073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.552+0000 7f1860e00640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 0x7f1838073c60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f184c001ee0 tx=0x7f184c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.556+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f185405c870 con 0x7f185c0cd770 2024-09-11T12:53:59.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f185c0cf6e0 con 0x7f185c0cd770 2024-09-11T12:53:59.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1858c00640 1 -- 172.21.15.78:0/4115485058 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f1854065100 con 0x7f185c0cd770 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 msgr2=0x7f1838073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 0x7f1838073c60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f184c001ee0 tx=0x7f184c009040 comp rx=0 tx=0).stop 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 msgr2=0x7f185c1591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f185400c3c0 tx=0x7f185400c880 comp rx=0 tx=0).stop 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 shutdown_connections 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f185c0cd770 0x7f185c1591c0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f185c0b3420 0x7f185c1644c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f18380717a0 0x7f1838073c60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 --2- 172.21.15.78:0/4115485058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f185c0b3040 0x7f185c1f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 >> 172.21.15.78:0/4115485058 conn(0x7f185c071dd0 msgr2=0x7f185c0afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 shutdown_connections 2024-09-11T12:53:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.728+0000 7f1863800640 1 -- 172.21.15.78:0/4115485058 wait complete. 2024-09-11T12:53:59.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- 172.21.15.78:0/3606158145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415ae90 msgr2=0x7f11d40b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- 172.21.15.78:0/3606158145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415ae90 0x7f11d40b2ec0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f11d4072e00 tx=0x7f11cc01cac0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- 172.21.15.78:0/3606158145 shutdown_connections 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- 172.21.15.78:0/3606158145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415ae90 0x7f11d40b2ec0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- 172.21.15.78:0/3606158145 >> v2:172.21.15.157:3300/0 conn(0x7f11d40b4c00 0x7f11d4154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- 172.21.15.78:0/3606158145 >> 172.21.15.78:0/3606158145 conn(0x7f11d4075a10 msgr2=0x7f11d40734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- 172.21.15.78:0/3606158145 shutdown_connections 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- 172.21.15.78:0/3606158145 wait complete. 2024-09-11T12:53:59.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 Processor -- start 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- start start 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d40b4c00 0x7f11d414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11d415ae90 0x7f11d414a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415dbe0 0x7f11d41faff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f11d4167e80 con 0x7f11d415dbe0 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f11d40b3750 con 0x7f11d40b4c00 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.892+0000 7f11d8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f11d4168000 con 0x7f11d415ae90 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415dbe0 0x7f11d41faff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415dbe0 0x7f11d41faff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44646/0 (socket says 172.21.15.78:44646) 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 -- 172.21.15.78:0/3757518921 learned_addr learned my addr 172.21.15.78:0/3757518921 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d1e00640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d40b4c00 0x7f11d414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d1400640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11d415ae90 0x7f11d414a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 -- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11d415ae90 msgr2=0x7f11d414a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f11d415ae90 0x7f11d414a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 -- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d40b4c00 msgr2=0x7f11d414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11d40b4c00 0x7f11d414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 -- 172.21.15.78:0/3757518921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d414fd40 con 0x7f11d415dbe0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d2800640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11d415dbe0 0x7f11d41faff0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f11cc0986b0 tx=0x7f11cc009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc0b2040 con 0x7f11d415dbe0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11cc0bb070 con 0x7f11d415dbe0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11cc004530 con 0x7f11d415dbe0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d8a00640 1 -- 172.21.15.78:0/3757518921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d4150020 con 0x7f11d415dbe0 2024-09-11T12:53:59.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d8a00640 1 -- 172.21.15.78:0/3757518921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d414fa50 con 0x7f11d415dbe0 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11d8a00640 1 -- 172.21.15.78:0/3757518921 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d4154560 con 0x7f11d415dbe0 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.896+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f11cc0046d0 con 0x7f11d415dbe0 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.900+0000 7f11ca000640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f11b40717a0 0x7f11b4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.900+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f11cc12c910 con 0x7f11d415dbe0 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.900+0000 7f11d1e00640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f11b40717a0 0x7f11b4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:53:59.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.900+0000 7f11d1e00640 1 --2- 172.21.15.78:0/3757518921 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f11b40717a0 0x7f11b4073c60 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11c4004630 tx=0x7f11c400b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:53:59.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:53:59.904+0000 7f11ca000640 1 -- 172.21.15.78:0/3757518921 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f11cc016770 con 0x7f11d415dbe0 2024-09-11T12:54:00.095 INFO:teuthology.orchestra.run.smithi078.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-11T12:54:00.095 INFO:teuthology.orchestra.run.smithi078.stderr:fs volume create [] : Create a CephFS volume 2024-09-11T12:54:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:Error EINVAL: invalid command 2024-09-11T12:54:00.103 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T12:54:00.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-11T12:54:00.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- 172.21.15.78:0/1105484305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce0155e50 msgr2=0x7f4ce006f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- 172.21.15.78:0/1105484305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce0155e50 0x7f4ce006f840 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009900 tx=0x7f4cd001c7c0 comp rx=0 tx=0).stop 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- 172.21.15.78:0/1105484305 shutdown_connections 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- 172.21.15.78:0/1105484305 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce006cf70 0x7f4ce0071dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- 172.21.15.78:0/1105484305 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce0155e50 0x7f4ce006f840 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- 172.21.15.78:0/1105484305 >> 172.21.15.78:0/1105484305 conn(0x7f4ce0074100 msgr2=0x7f4ce0074510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- 172.21.15.78:0/1105484305 shutdown_connections 2024-09-11T12:54:00.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- 172.21.15.78:0/1105484305 wait complete. 2024-09-11T12:54:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 Processor -- start 2024-09-11T12:54:00.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- start start 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 0x7f4ce0158c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0155e50 0x7f4ce01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce0168a70 con 0x7f4ce006cf70 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4ce01688f0 con 0x7f4ce0155e50 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4ce0168bf0 con 0x7f4ce00a9460 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44670/0 (socket says 172.21.15.78:44670) 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 -- 172.21.15.78:0/2420208821 learned_addr learned my addr 172.21.15.78:0/2420208821 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cde800640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0155e50 0x7f4ce01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:00.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdd400640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 0x7f4ce0158c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 -- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 msgr2=0x7f4ce0158c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 0x7f4ce0158c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 -- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0155e50 msgr2=0x7f4ce01591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0155e50 0x7f4ce01591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 -- 172.21.15.78:0/2420208821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ce01f5ae0 con 0x7f4ce006cf70 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdd400640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 0x7f4ce0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cdde00640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4cd001a3f0 tx=0x7f4cd00a64e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd00ac070 con 0x7f4ce006cf70 2024-09-11T12:54:00.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cd0004320 con 0x7f4ce006cf70 2024-09-11T12:54:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.248+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd0005320 con 0x7f4ce006cf70 2024-09-11T12:54:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ce01f5dc0 con 0x7f4ce006cf70 2024-09-11T12:54:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ce01f6380 con 0x7f4ce006cf70 2024-09-11T12:54:00.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ce0071dd0 con 0x7f4ce006cf70 2024-09-11T12:54:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f4cd00af070 con 0x7f4ce006cf70 2024-09-11T12:54:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cd6000640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 0x7f4cc0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5709+0+0 (secure 0 0 0) 0x7f4cd012b850 con 0x7f4ce006cf70 2024-09-11T12:54:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cdd400640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 0x7f4cc0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:00.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.252+0000 7f4cdd400640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 0x7f4cc0073c60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0058d0 tx=0x7f4ccc005860 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:00.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.256+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4cd00bf020 con 0x7f4ce006cf70 2024-09-11T12:54:00.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:00.444+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f4ce00af3c0 con 0x7f4cc00717a0 2024-09-11T12:54:03.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.052+0000 7f4cd6000640 1 -- 172.21.15.78:0/2420208821 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f4ce00af3c0 con 0x7f4cc00717a0 2024-09-11T12:54:03.054 INFO:teuthology.orchestra.run.smithi078.stderr:Volume created successfully (no MDS daemons created) 2024-09-11T12:54:03.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 msgr2=0x7f4cc0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 0x7f4cc0073c60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc0058d0 tx=0x7f4ccc005860 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 msgr2=0x7f4ce00cfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f4cd001a3f0 tx=0x7f4cd00a64e0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 shutdown_connections 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4cc00717a0 0x7f4cc0073c60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4ce0155e50 0x7f4ce01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4ce00a9460 0x7f4ce0158c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 --2- 172.21.15.78:0/2420208821 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ce006cf70 0x7f4ce00cfcb0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 >> 172.21.15.78:0/2420208821 conn(0x7f4ce0074100 msgr2=0x7f4ce00744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 shutdown_connections 2024-09-11T12:54:03.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.056+0000 7f4cdf200640 1 -- 172.21.15.78:0/2420208821 wait complete. 2024-09-11T12:54:03.089 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-11T12:54:03.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- 172.21.15.78:0/3243512282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd748140190 msgr2=0x7fd7480cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- 172.21.15.78:0/3243512282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd748140190 0x7fd7480cc360 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fd74806e6b0 tx=0x7fd73c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- 172.21.15.78:0/3243512282 shutdown_connections 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- 172.21.15.78:0/3243512282 >> v2:172.21.15.157:3300/0 conn(0x7fd74806fea0 0x7fd748073610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- 172.21.15.78:0/3243512282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd748140190 0x7fd7480cc360 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- 172.21.15.78:0/3243512282 >> 172.21.15.78:0/3243512282 conn(0x7fd748075c10 msgr2=0x7fd7480b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- 172.21.15.78:0/3243512282 shutdown_connections 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- 172.21.15.78:0/3243512282 wait complete. 2024-09-11T12:54:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 Processor -- start 2024-09-11T12:54:03.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- start start 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd74806fea0 0x7fd748159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480731a0 0x7fd748139450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd748140190 0x7fd748139990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd74815e5a0 con 0x7fd74806fea0 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd74815e420 con 0x7fd7480731a0 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd74815e720 con 0x7fd748140190 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd74806fea0 0x7fd748159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd74806fea0 0x7fd748159270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44682/0 (socket says 172.21.15.78:44682) 2024-09-11T12:54:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74dc00640 1 -- 172.21.15.78:0/1706116322 learned_addr learned my addr 172.21.15.78:0/1706116322 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74e600640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd748140190 0x7fd748139990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.236+0000 7fd74d200640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480731a0 0x7fd748139450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 -- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd748140190 msgr2=0x7fd748139990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd748140190 0x7fd748139990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 -- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480731a0 msgr2=0x7fd748139450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7480731a0 0x7fd748139450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 -- 172.21.15.78:0/1706116322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7481f8040 con 0x7fd74806fea0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74dc00640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd74806fea0 0x7fd748159270 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fd73c0b0040 tx=0x7fd73c0a6790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c0043c0 con 0x7fd74806fea0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd73c005020 con 0x7fd74806fea0 2024-09-11T12:54:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74f000640 1 -- 172.21.15.78:0/1706116322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7481631e0 con 0x7fd74806fea0 2024-09-11T12:54:03.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd73c005470 con 0x7fd74806fea0 2024-09-11T12:54:03.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74f000640 1 -- 172.21.15.78:0/1706116322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7481637a0 con 0x7fd74806fea0 2024-09-11T12:54:03.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd74f000640 1 -- 172.21.15.78:0/1706116322 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7480735f0 con 0x7fd74806fea0 2024-09-11T12:54:03.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd73c0be060 con 0x7fd74806fea0 2024-09-11T12:54:03.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.240+0000 7fd745600640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7280717a0 0x7fd728073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.244+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7fd73c12c7c0 con 0x7fd74806fea0 2024-09-11T12:54:03.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.244+0000 7fd74d200640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7280717a0 0x7fd728073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.244+0000 7fd74d200640 1 --2- 172.21.15.78:0/1706116322 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7280717a0 0x7fd728073c60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd748140ab0 tx=0x7fd734009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.244+0000 7fd745600640 1 -- 172.21.15.78:0/1706116322 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd73c0164b0 con 0x7fd74806fea0 2024-09-11T12:54:03.436 INFO:teuthology.orchestra.run.smithi078.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-11T12:54:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-11T12:54:03.437 INFO:teuthology.orchestra.run.smithi078.stderr:Error EINVAL: invalid command 2024-09-11T12:54:03.444 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T12:54:03.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- 172.21.15.78:0/1924793441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c4004d50 msgr2=0x7f94c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 --2- 172.21.15.78:0/1924793441 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94c4004d50 0x7f94c40051d0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006fa0 tx=0x7f94d801cee0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94df400640 1 -- 172.21.15.78:0/1924793441 reap_dead start 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- 172.21.15.78:0/1924793441 shutdown_connections 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- 172.21.15.78:0/1924793441 >> 172.21.15.78:0/1924793441 conn(0x7f94e0071dd0 msgr2=0x7f94e00721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- 172.21.15.78:0/1924793441 shutdown_connections 2024-09-11T12:54:03.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- 172.21.15.78:0/1924793441 wait complete. 2024-09-11T12:54:03.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 Processor -- start 2024-09-11T12:54:03.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.588+0000 7f94e5800640 1 -- start start 2024-09-11T12:54:03.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c4004d50 0x7f94e01f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94e013e470 0x7f94e013fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.593 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94e01e5ee0 0x7f94e01510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f94e00b3170 con 0x7f94e01e5ee0 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f94e00b2ff0 con 0x7f94e013e470 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f94e0141f90 con 0x7f94c4004d50 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94e01e5ee0 0x7f94e01510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94e01e5ee0 0x7f94e01510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:44702/0 (socket says 172.21.15.78:44702) 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 -- 172.21.15.78:0/1839221879 learned_addr learned my addr 172.21.15.78:0/1839221879 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94de000640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94e013e470 0x7f94e013fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94dea00640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c4004d50 0x7f94e01f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 -- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c4004d50 msgr2=0x7f94e01f7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c4004d50 0x7f94e01f7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 -- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94e013e470 msgr2=0x7f94e013fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94e013e470 0x7f94e013fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 -- 172.21.15.78:0/1839221879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e01f85f0 con 0x7f94e01e5ee0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94de000640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f94e013e470 0x7f94e013fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94dea00640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f94c4004d50 0x7f94e01f7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94df400640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f94e01e5ee0 0x7f94e01510d0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006fa0 tx=0x7f94d8006bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8000e50 con 0x7f94e01e5ee0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94d8006440 con 0x7f94e01e5ee0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8003840 con 0x7f94e01e5ee0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- 172.21.15.78:0/1839221879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e01f88d0 con 0x7f94e01e5ee0 2024-09-11T12:54:03.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- 172.21.15.78:0/1839221879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e01eacd0 con 0x7f94e01e5ee0 2024-09-11T12:54:03.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94e5800640 1 -- 172.21.15.78:0/1839221879 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e013f000 con 0x7f94e01e5ee0 2024-09-11T12:54:03.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.592+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f94d80039e0 con 0x7f94e01e5ee0 2024-09-11T12:54:03.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.596+0000 7f94d6a00640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94c00717d0 0x7f94c0073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.596+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7f94d812c580 con 0x7f94e01e5ee0 2024-09-11T12:54:03.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.596+0000 7f94dea00640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94c00717d0 0x7f94c0073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.596+0000 7f94dea00640 1 --2- 172.21.15.78:0/1839221879 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94c00717d0 0x7f94c0073c90 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009f30 tx=0x7f94d0009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.600+0000 7f94d6a00640 1 -- 172.21.15.78:0/1839221879 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f94d8016c90 con 0x7f94e01e5ee0 2024-09-11T12:54:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-11T12:54:03.788 INFO:teuthology.orchestra.run.smithi078.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] [] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-11T12:54:03.789 INFO:teuthology.orchestra.run.smithi078.stderr:Error EINVAL: invalid command 2024-09-11T12:54:03.795 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T12:54:03.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-11T12:54:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- 172.21.15.78:0/2753984221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf74003a10 msgr2=0x7fdf74003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 --2- 172.21.15.78:0/2753984221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf74003a10 0x7fdf74003e30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009ee0 tx=0x7fdf8001d600 comp rx=0 tx=0).stop 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf8e800640 1 -- 172.21.15.78:0/2753984221 reap_dead start 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- 172.21.15.78:0/2753984221 shutdown_connections 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- 172.21.15.78:0/2753984221 >> 172.21.15.78:0/2753984221 conn(0x7fdf90071dd0 msgr2=0x7fdf900721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- 172.21.15.78:0/2753984221 shutdown_connections 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- 172.21.15.78:0/2753984221 wait complete. 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 Processor -- start 2024-09-11T12:54:03.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- start start 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf74003a10 0x7fdf90160180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9015e820 0x7fdf90163130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900aef20 con 0x7fdf90163670 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdf900aeda0 con 0x7fdf9015e820 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.940+0000 7fdf95000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdf900af0a0 con 0x7fdf74003a10 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46030/0 (socket says 172.21.15.78:46030) 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 -- 172.21.15.78:0/777231525 learned_addr learned my addr 172.21.15.78:0/777231525 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:03.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8d400640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9015e820 0x7fdf90163130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8de00640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf74003a10 0x7fdf90160180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 -- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf74003a10 msgr2=0x7fdf90160180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf74003a10 0x7fdf90160180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 -- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9015e820 msgr2=0x7fdf90163130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9015e820 0x7fdf90163130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 -- 172.21.15.78:0/777231525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf9015da60 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf8e800640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdf90143e60 tx=0x7fdf8800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf88004240 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf880043e0 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8800daf0 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf9015dd40 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf9015e300 con 0x7fdf90163670 2024-09-11T12:54:03.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf9015e500 con 0x7fdf90163670 2024-09-11T12:54:03.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.944+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fdf8800dc90 con 0x7fdf90163670 2024-09-11T12:54:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.948+0000 7fdf86000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 0x7fdf70073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:03.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.948+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 6543+0+0 (secure 0 0 0) 0x7fdf88092750 con 0x7fdf90163670 2024-09-11T12:54:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.948+0000 7fdf8de00640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 0x7fdf70073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:03.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.948+0000 7fdf8de00640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 0x7fdf70073c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf800af000 tx=0x7fdf8001a3a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:03.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:03.952+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdf9015e500 con 0x7fdf90163670 2024-09-11T12:54:04.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:04.156+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7fdf9015d810 con 0x7fdf700717d0 2024-09-11T12:54:04.160 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:54:04.160+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:54:04.207 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:54:07.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.180+0000 7fdf86000640 1 -- 172.21.15.78:0/777231525 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7fdf9015d810 con 0x7fdf700717d0 2024-09-11T12:54:07.182 INFO:teuthology.orchestra.run.smithi078.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 msgr2=0x7fdf70073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 0x7fdf70073c90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf800af000 tx=0x7fdf8001a3a0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 msgr2=0x7fdf900ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdf90143e60 tx=0x7fdf8800edc0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 shutdown_connections 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdf90163670 0x7fdf900ac4c0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdf700717d0 0x7fdf70073c90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdf9015e820 0x7fdf90163130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 --2- 172.21.15.78:0/777231525 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdf74003a10 0x7fdf90160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 >> 172.21.15.78:0/777231525 conn(0x7fdf90071dd0 msgr2=0x7fdf9006e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 shutdown_connections 2024-09-11T12:54:07.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.184+0000 7fdf95000640 1 -- 172.21.15.78:0/777231525 wait complete. 2024-09-11T12:54:07.209 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:54:07.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:07.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:07.378 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:54:07.379 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:07.379 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:07.379 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:07.388 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:07.388 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:07.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- 172.21.15.78:0/1117354660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 msgr2=0x7fd3480cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- 172.21.15.78:0/1117354660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd3480cc5e0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd340009c80 tx=0x7fd34001cb00 comp rx=0 tx=0).stop 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- 172.21.15.78:0/1117354660 shutdown_connections 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- 172.21.15.78:0/1117354660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd3480cc5e0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- 172.21.15.78:0/1117354660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd34815e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- 172.21.15.78:0/1117354660 >> 172.21.15.78:0/1117354660 conn(0x7fd34806ca80 msgr2=0x7fd3480b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- 172.21.15.78:0/1117354660 shutdown_connections 2024-09-11T12:54:07.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- 172.21.15.78:0/1117354660 wait complete. 2024-09-11T12:54:07.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 Processor -- start 2024-09-11T12:54:07.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- start start 2024-09-11T12:54:07.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd3481512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 0x7fd34814f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd34813ebe0 con 0x7fd34813d250 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd34813ea60 con 0x7fd34815a1c0 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd34813ed60 con 0x7fd3480b4780 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd347400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd347400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46050/0 (socket says 172.21.15.78:46050) 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd347400640 1 -- 172.21.15.78:0/3344435217 learned_addr learned my addr 172.21.15.78:0/3344435217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd347e00640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd3481512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.636+0000 7fd34d200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 0x7fd34814f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 -- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 msgr2=0x7fd3481512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd3481512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 -- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 msgr2=0x7fd34814f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 0x7fd34814f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 -- 172.21.15.78:0/3344435217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd348142e70 con 0x7fd34813d250 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347e00640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd3481512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd34d200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 0x7fd34814f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347400640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd33400b3e0 tx=0x7fd33400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334015020 con 0x7fd34813d250 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd334011920 con 0x7fd34813d250 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd348153960 con 0x7fd34813d250 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334011db0 con 0x7fd34813d250 2024-09-11T12:54:07.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd348153f20 con 0x7fd34813d250 2024-09-11T12:54:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd348154120 con 0x7fd34813d250 2024-09-11T12:54:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd334018020 con 0x7fd34813d250 2024-09-11T12:54:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 0x7fd328073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fd334091a00 con 0x7fd34813d250 2024-09-11T12:54:07.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.640+0000 7fd347e00640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 0x7fd328073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:07.644 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.644+0000 7fd347e00640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 0x7fd328073c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3481fb0d0 tx=0x7fd338005f50 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:07.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.644+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd348154120 con 0x7fd34813d250 2024-09-11T12:54:07.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd34813f250 con 0x7fd34813d250 2024-09-11T12:54:07.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd33fe00640 1 -- 172.21.15.78:0/3344435217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) ==== 80+0+122 (secure 0 0 0) 0x7fd33401f070 con 0x7fd34813d250 2024-09-11T12:54:07.829 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6833/3066636859 2024-09-12T12:54:04.162092+0000 2024-09-11T12:54:07.829 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6832/3066636859 2024-09-12T12:54:04.162092+0000 2024-09-11T12:54:07.829 INFO:teuthology.orchestra.run.smithi078.stderr:listed 2 entries 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 msgr2=0x7fd328073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 0x7fd328073c60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd3481fb0d0 tx=0x7fd338005f50 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 msgr2=0x7fd348152ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd33400b3e0 tx=0x7fd33400b8a0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 shutdown_connections 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd3280717a0 0x7fd328073c60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd34815a1c0 0x7fd34814f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd34813d250 0x7fd348152ef0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 --2- 172.21.15.78:0/3344435217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3480b4780 0x7fd3481512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 >> 172.21.15.78:0/3344435217 conn(0x7fd34806ca80 msgr2=0x7fd34815b300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:07.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 shutdown_connections 2024-09-11T12:54:07.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:07.828+0000 7fd34f200640 1 -- 172.21.15.78:0/3344435217 wait complete. 2024-09-11T12:54:07.850 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:54:07.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:07.850 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:07.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:54:07.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:07.877 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:54:07.928 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:07.928 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:07.929 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:07.938 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:07.938 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:07.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 -- 172.21.15.78:0/4203870341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50074240 msgr2=0x7efc5015ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 --2- 172.21.15.78:0/4203870341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50074240 0x7efc5015ba80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7efc40009840 tx=0x7efc4001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 -- 172.21.15.78:0/4203870341 shutdown_connections 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 --2- 172.21.15.78:0/4203870341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50070360 0x7efc5015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 --2- 172.21.15.78:0/4203870341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50074240 0x7efc5015ba80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 -- 172.21.15.78:0/4203870341 >> 172.21.15.78:0/4203870341 conn(0x7efc5006ca80 msgr2=0x7efc500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:08.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 -- 172.21.15.78:0/4203870341 shutdown_connections 2024-09-11T12:54:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.128+0000 7efc56a00640 1 -- 172.21.15.78:0/4203870341 wait complete. 2024-09-11T12:54:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 Processor -- start 2024-09-11T12:54:08.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 -- start start 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 0x7efc50151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50074240 0x7efc501fb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc500729b0 con 0x7efc5015e5b0 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc50072830 con 0x7efc50070360 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc50167740 con 0x7efc50074240 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46056/0 (socket says 172.21.15.78:46056) 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 -- 172.21.15.78:0/2599815076 learned_addr learned my addr 172.21.15.78:0/2599815076 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc4fe00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 0x7efc50151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc4f400640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50074240 0x7efc501fb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 -- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50074240 msgr2=0x7efc501fb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50074240 0x7efc501fb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 -- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 msgr2=0x7efc50151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 0x7efc50151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 -- 172.21.15.78:0/2599815076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc5013f090 con 0x7efc5015e5b0 2024-09-11T12:54:08.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc4fe00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 0x7efc50151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc54a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efc4800efd0 tx=0x7efc4800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48003b80 con 0x7efc5015e5b0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc48003e10 con 0x7efc5015e5b0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc5013f330 con 0x7efc5015e5b0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.132+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc48004280 con 0x7efc5015e5b0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc5013fbd0 con 0x7efc5015e5b0 2024-09-11T12:54:08.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc5013fdd0 con 0x7efc5015e5b0 2024-09-11T12:54:08.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7efc48004420 con 0x7efc5015e5b0 2024-09-11T12:54:08.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc47e00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 0x7efc30073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7efc48092d90 con 0x7efc5015e5b0 2024-09-11T12:54:08.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc4fe00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 0x7efc30073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.136+0000 7efc4fe00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 0x7efc30073c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc400af000 tx=0x7efc4001b030 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:08.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.140+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efc5013fdd0 con 0x7efc5015e5b0 2024-09-11T12:54:08.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.320+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efc5015ea20 con 0x7efc5015e5b0 2024-09-11T12:54:08.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.320+0000 7efc47e00640 1 -- 172.21.15.78:0/2599815076 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v90) ==== 80+0+122 (secure 0 0 0) 0x7efc4805e320 con 0x7efc5015e5b0 2024-09-11T12:54:08.322 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6833/3066636859 2024-09-12T12:54:04.162092+0000 2024-09-11T12:54:08.322 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.15.78:6832/3066636859 2024-09-12T12:54:04.162092+0000 2024-09-11T12:54:08.323 INFO:teuthology.orchestra.run.smithi078.stderr:listed 2 entries 2024-09-11T12:54:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 msgr2=0x7efc30073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 0x7efc30073c60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7efc400af000 tx=0x7efc4001b030 comp rx=0 tx=0).stop 2024-09-11T12:54:08.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 msgr2=0x7efc501609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7efc4800efd0 tx=0x7efc4800c6a0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 shutdown_connections 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc5015e5b0 0x7efc501609c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc50074240 0x7efc501fb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc300717a0 0x7efc30073c60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 --2- 172.21.15.78:0/2599815076 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc50070360 0x7efc50151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 >> 172.21.15.78:0/2599815076 conn(0x7efc5006ca80 msgr2=0x7efc501509f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 shutdown_connections 2024-09-11T12:54:08.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.324+0000 7efc56a00640 1 -- 172.21.15.78:0/2599815076 wait complete. 2024-09-11T12:54:08.344 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:54:08.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:08.344 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:08.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:54:08.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:08.372 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:54:08.425 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:08.425 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:08.425 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:08.434 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:54:08.434 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:08.434 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:08.434 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:08.477 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:54:08.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:08.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- 172.21.15.78:0/1635310180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 msgr2=0x7f39b80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 --2- 172.21.15.78:0/1635310180 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39b80051d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f39cc001f80 tx=0x7f39cc01cc40 comp rx=0 tx=0).stop 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39d9400640 1 -- 172.21.15.78:0/1635310180 reap_dead start 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- 172.21.15.78:0/1635310180 shutdown_connections 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- 172.21.15.78:0/1635310180 >> 172.21.15.78:0/1635310180 conn(0x7f39d406ca80 msgr2=0x7f39d40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- 172.21.15.78:0/1635310180 shutdown_connections 2024-09-11T12:54:08.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- 172.21.15.78:0/1635310180 wait complete. 2024-09-11T12:54:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 Processor -- start 2024-09-11T12:54:08.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 -- start start 2024-09-11T12:54:08.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.620+0000 7f39db400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d40cc100 0x7f39d41f2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d41594f0 0x7f39d40ae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39d415b250 con 0x7f39b8004d50 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39d415b0d0 con 0x7f39d41594f0 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39d4070100 con 0x7f39d40cc100 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46076/0 (socket says 172.21.15.78:46076) 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 -- 172.21.15.78:0/3585634493 learned_addr learned my addr 172.21.15.78:0/3585634493 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d9400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d41594f0 0x7f39d40ae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d3e00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d40cc100 0x7f39d41f2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 -- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d40cc100 msgr2=0x7f39d41f2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d40cc100 0x7f39d41f2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 -- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d41594f0 msgr2=0x7f39d40ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d41594f0 0x7f39d40ae8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 -- 172.21.15.78:0/3585634493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39d415d6c0 con 0x7f39b8004d50 2024-09-11T12:54:08.626 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d8a00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f39c4012610 tx=0x7f39c4012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4013980 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c4017030 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39d4153de0 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c4013de0 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39d415dc10 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39d4070830 con 0x7f39b8004d50 2024-09-11T12:54:08.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.624+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f39c4010440 con 0x7f39b8004d50 2024-09-11T12:54:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.628+0000 7f39d0c00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 0x7f39b4073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.628+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f39c4095b50 con 0x7f39b8004d50 2024-09-11T12:54:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.628+0000 7f39d3e00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 0x7f39b4073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:08.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.628+0000 7f39d3e00640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 0x7f39b4073c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39d41f51a0 tx=0x7f39c00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:08.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.632+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f39c40611f0 con 0x7f39b8004d50 2024-09-11T12:54:08.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.836+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39d41f1c60 con 0x7f39b8004d50 2024-09-11T12:54:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39d0c00640 1 -- 172.21.15.78:0/3585634493 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5228 (secure 0 0 0) 0x7f39c4065740 con 0x7f39b8004d50 2024-09-11T12:54:08.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:08.840 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":62,"btime":"2024-09-11T12:54:05:164845+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2256690013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2256690013},{"type":"v1","addr":"172.21.15.78:6833","nonce":2256690013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:54:08.840 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 62 2024-09-11T12:54:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 msgr2=0x7f39b4073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 0x7f39b4073c90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f39d41f51a0 tx=0x7f39c00091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 msgr2=0x7f39d41538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f39c4012610 tx=0x7f39c4012a10 comp rx=0 tx=0).stop 2024-09-11T12:54:08.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 shutdown_connections 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39b40717d0 0x7f39b4073c90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39d41594f0 0x7f39d40ae8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39d40cc100 0x7f39d41f2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 --2- 172.21.15.78:0/3585634493 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39b8004d50 0x7f39d41538a0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 >> 172.21.15.78:0/3585634493 conn(0x7f39d406ca80 msgr2=0x7f39d415b490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 shutdown_connections 2024-09-11T12:54:08.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:08.840+0000 7f39db400640 1 -- 172.21.15.78:0/3585634493 wait complete. 2024-09-11T12:54:08.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 -- 172.21.15.78:0/1468669072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 msgr2=0x7fad0815ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 --2- 172.21.15.78:0/1468669072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad0815ed00 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7facf8009900 tx=0x7facf801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 -- 172.21.15.78:0/1468669072 shutdown_connections 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 --2- 172.21.15.78:0/1468669072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad08154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 --2- 172.21.15.78:0/1468669072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad0815ed00 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 -- 172.21.15.78:0/1468669072 >> 172.21.15.78:0/1468669072 conn(0x7fad0806c650 msgr2=0x7fad080d0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 -- 172.21.15.78:0/1468669072 shutdown_connections 2024-09-11T12:54:09.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.000+0000 7fad0ea00640 1 -- 172.21.15.78:0/1468669072 wait complete. 2024-09-11T12:54:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 Processor -- start 2024-09-11T12:54:09.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- start start 2024-09-11T12:54:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 0x7fad08151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad0814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fad0816b470 con 0x7fad0806fd00 2024-09-11T12:54:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fad0816b2f0 con 0x7fad080700e0 2024-09-11T12:54:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fad0816b5f0 con 0x7fad080b4520 2024-09-11T12:54:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46106/0 (socket says 172.21.15.78:46106) 2024-09-11T12:54:09.006 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 -- 172.21.15.78:0/2318475935 learned_addr learned my addr 172.21.15.78:0/2318475935 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07400640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 0x7fad08151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ca00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad0814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 -- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 msgr2=0x7fad0814b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad0814b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 -- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 msgr2=0x7fad08151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 0x7fad08151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 -- 172.21.15.78:0/2318475935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad0814b540 con 0x7fad0806fd00 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ca00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad0814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07400640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 0x7fad08151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad07e00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7facf80b0040 tx=0x7facf8004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf80b2040 con 0x7fad0806fd00 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facf8005120 con 0x7fad0806fd00 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad0814ed20 con 0x7fad0806fd00 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facf8005550 con 0x7fad0806fd00 2024-09-11T12:54:09.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad081fb840 con 0x7fad0806fd00 2024-09-11T12:54:09.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.004+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad0815ed00 con 0x7fad0806fd00 2024-09-11T12:54:09.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.008+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7facf80be060 con 0x7fad0806fd00 2024-09-11T12:54:09.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.008+0000 7facffe00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 0x7face8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.008+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7facf812b810 con 0x7fad0806fd00 2024-09-11T12:54:09.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.008+0000 7fad07400640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 0x7face8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.008+0000 7fad07400640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 0x7face8073c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf4005d00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.012+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7facf80164d0 con 0x7fad0806fd00 2024-09-11T12:54:09.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.212+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fad080d1af0 con 0x7fad0806fd00 2024-09-11T12:54:09.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.212+0000 7facffe00640 1 -- 172.21.15.78:0/2318475935 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5228 (secure 0 0 0) 0x7facf80f7470 con 0x7fad0806fd00 2024-09-11T12:54:09.214 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:09.214 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":62,"btime":"2024-09-11T12:54:05:164845+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2256690013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2256690013},{"type":"v1","addr":"172.21.15.78:6833","nonce":2256690013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:54:09.214 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 62 2024-09-11T12:54:09.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 msgr2=0x7face8073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 0x7face8073c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7facf4005fd0 tx=0x7facf4005d00 comp rx=0 tx=0).stop 2024-09-11T12:54:09.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 msgr2=0x7fad081faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7facf80b0040 tx=0x7facf8004220 comp rx=0 tx=0).stop 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 shutdown_connections 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fad080b4520 0x7fad0814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7face80717a0 0x7face8073c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fad080700e0 0x7fad08151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 --2- 172.21.15.78:0/2318475935 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fad0806fd00 0x7fad081faf10 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 >> 172.21.15.78:0/2318475935 conn(0x7fad0806c650 msgr2=0x7fad08153eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 shutdown_connections 2024-09-11T12:54:09.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.216+0000 7fad0ea00640 1 -- 172.21.15.78:0/2318475935 wait complete. 2024-09-11T12:54:09.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- 172.21.15.78:0/2794298752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 msgr2=0x7ff824070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- 172.21.15.78:0/2794298752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824070a00 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b2c60 tx=0x7ff81401dbe0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- 172.21.15.78:0/2794298752 shutdown_connections 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- 172.21.15.78:0/2794298752 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240b4db0 0x7ff8240b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- 172.21.15.78:0/2794298752 >> v2:172.21.15.157:3300/0 conn(0x7ff8240736d0 0x7ff82415af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- 172.21.15.78:0/2794298752 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824070a00 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- 172.21.15.78:0/2794298752 >> 172.21.15.78:0/2794298752 conn(0x7ff82406ca80 msgr2=0x7ff8240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- 172.21.15.78:0/2794298752 shutdown_connections 2024-09-11T12:54:09.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- 172.21.15.78:0/2794298752 wait complete. 2024-09-11T12:54:09.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 Processor -- start 2024-09-11T12:54:09.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- start start 2024-09-11T12:54:09.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240736d0 0x7ff8241550c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8240b4db0 0x7ff824155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff82406f0c0 con 0x7ff824157ae0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff82406ef40 con 0x7ff8240b4db0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff829000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff82406f240 con 0x7ff8240736d0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff822800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff821e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240736d0 0x7ff8241550c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff822800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46120/0 (socket says 172.21.15.78:46120) 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff822800640 1 -- 172.21.15.78:0/3226552747 learned_addr learned my addr 172.21.15.78:0/3226552747 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.376+0000 7ff821400640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8240b4db0 0x7ff824155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 -- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240736d0 msgr2=0x7ff8241550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240736d0 0x7ff8241550c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 -- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8240b4db0 msgr2=0x7ff824155600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8240b4db0 0x7ff824155600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 -- 172.21.15.78:0/3226552747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8240cfbc0 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff822800640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d750 tx=0x7ff81c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c017070 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff81c009d20 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8241ed6b0 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00de60 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82415c0d0 con 0x7ff824157ae0 2024-09-11T12:54:09.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8240b5220 con 0x7ff824157ae0 2024-09-11T12:54:09.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff81c005280 con 0x7ff824157ae0 2024-09-11T12:54:09.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 0x7ff804073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.380+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7ff81c0920b0 con 0x7ff824157ae0 2024-09-11T12:54:09.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.384+0000 7ff821e00640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 0x7ff804073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.384+0000 7ff821e00640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 0x7ff804073c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8140b0000 tx=0x7ff81401c420 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.384+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff81c05d5c0 con 0x7ff824157ae0 2024-09-11T12:54:09.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.556+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff824071e00 con 0x7ff824157ae0 2024-09-11T12:54:09.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.556+0000 7ff81a000640 1 -- 172.21.15.78:0/3226552747 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16849 (secure 0 0 0) 0x7ff81c061b10 con 0x7ff824157ae0 2024-09-11T12:54:09.559 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:09.559 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:07.172356+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:47.045831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:48.326392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6833/3066636859":"2024-09-12T12:54:04.162092+0000","172.21.15.78:6832/3066636859":"2024-09-12T12:54:04.162092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 msgr2=0x7ff804073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 0x7ff804073c60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff8140b0000 tx=0x7ff81401c420 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 msgr2=0x7ff824146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00d750 tx=0x7ff81c00dc10 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 shutdown_connections 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff824157ae0 0x7ff824146f70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff8040717a0 0x7ff804073c60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff8240b4db0 0x7ff824155600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 --2- 172.21.15.78:0/3226552747 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff8240736d0 0x7ff8241550c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 >> 172.21.15.78:0/3226552747 conn(0x7ff82406ca80 msgr2=0x7ff824072600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 shutdown_connections 2024-09-11T12:54:09.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.560+0000 7ff829000640 1 -- 172.21.15.78:0/3226552747 wait complete. 2024-09-11T12:54:09.580 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:54:09.580 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 -- 172.21.15.78:0/3823013602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe640d0bf0 msgr2=0x7fbe640d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 --2- 172.21.15.78:0/3823013602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe640d0bf0 0x7fbe640d1080 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009bc0 tx=0x7fbe5c01cac0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 -- 172.21.15.78:0/3823013602 shutdown_connections 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 --2- 172.21.15.78:0/3823013602 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe640d0bf0 0x7fbe640d1080 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 -- 172.21.15.78:0/3823013602 >> 172.21.15.78:0/3823013602 conn(0x7fbe64074b90 msgr2=0x7fbe640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 -- 172.21.15.78:0/3823013602 shutdown_connections 2024-09-11T12:54:09.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.720+0000 7fbe6c000640 1 -- 172.21.15.78:0/3823013602 wait complete. 2024-09-11T12:54:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 Processor -- start 2024-09-11T12:54:09.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 -- start start 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6406fdf0 0x7fbe64142f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe640d0bf0 0x7fbe64148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe6416a0a0 con 0x7fbe640b14b0 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe64169f20 con 0x7fbe6406fdf0 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe6416a220 con 0x7fbe640d0bf0 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46124/0 (socket says 172.21.15.78:46124) 2024-09-11T12:54:09.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 -- 172.21.15.78:0/870846228 learned_addr learned my addr 172.21.15.78:0/870846228 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe6a000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe640d0bf0 0x7fbe64148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe69600640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6406fdf0 0x7fbe64142f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 -- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe640d0bf0 msgr2=0x7fbe64148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe640d0bf0 0x7fbe64148330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 -- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6406fdf0 msgr2=0x7fbe64142f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6406fdf0 0x7fbe64142f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 -- 172.21.15.78:0/870846228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe64143490 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe68c00640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe50001230 tx=0x7fbe5000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50016020 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5000a930 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.724+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000ad80 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe641f96a0 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe641f9c60 con 0x7fbe640b14b0 2024-09-11T12:54:09.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe640d1080 con 0x7fbe640b14b0 2024-09-11T12:54:09.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fbe50019020 con 0x7fbe640b14b0 2024-09-11T12:54:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe61600640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 0x7fbe44073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fbe50091cf0 con 0x7fbe640b14b0 2024-09-11T12:54:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.728+0000 7fbe69600640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 0x7fbe44073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:09.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.732+0000 7fbe69600640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 0x7fbe44073c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe58001820 tx=0x7fbe58009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:09.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.732+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbe50096070 con 0x7fbe640b14b0 2024-09-11T12:54:09.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.932+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe640ce400 con 0x7fbe640b14b0 2024-09-11T12:54:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.932+0000 7fbe61600640 1 -- 172.21.15.78:0/870846228 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5228 (secure 0 0 0) 0x7fbe50020030 con 0x7fbe640b14b0 2024-09-11T12:54:09.936 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:09.936 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":62,"btime":"2024-09-11T12:54:05:164845+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2256690013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2256690013},{"type":"v1","addr":"172.21.15.78:6833","nonce":2256690013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:54:09.936 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 62 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 msgr2=0x7fbe44073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 0x7fbe44073c60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe58001820 tx=0x7fbe58009040 comp rx=0 tx=0).stop 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 msgr2=0x7fbe641f8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fbe50001230 tx=0x7fbe5000cb90 comp rx=0 tx=0).stop 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 shutdown_connections 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe640d0bf0 0x7fbe64148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe640b14b0 0x7fbe641f8e30 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe440717a0 0x7fbe44073c60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 --2- 172.21.15.78:0/870846228 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe6406fdf0 0x7fbe64142f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 >> 172.21.15.78:0/870846228 conn(0x7fbe64074b90 msgr2=0x7fbe6415e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 shutdown_connections 2024-09-11T12:54:09.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:09.936+0000 7fbe6c000640 1 -- 172.21.15.78:0/870846228 wait complete. 2024-09-11T12:54:09.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.096+0000 7fd4d5400640 1 -- 172.21.15.78:0/2693057545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d00b3070 msgr2=0x7fd4d00b3520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.096+0000 7fd4d5400640 1 --2- 172.21.15.78:0/2693057545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d00b3070 0x7fd4d00b3520 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009c80 tx=0x7fd4c801ccd0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- 172.21.15.78:0/2693057545 shutdown_connections 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 --2- 172.21.15.78:0/2693057545 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d00b3070 0x7fd4d00b3520 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 --2- 172.21.15.78:0/2693057545 >> v2:172.21.15.157:3300/0 conn(0x7fd4d01527f0 0x7fd4d015ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- 172.21.15.78:0/2693057545 >> 172.21.15.78:0/2693057545 conn(0x7fd4d00b1c50 msgr2=0x7fd4d0070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:10.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- 172.21.15.78:0/2693057545 shutdown_connections 2024-09-11T12:54:10.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- 172.21.15.78:0/2693057545 wait complete. 2024-09-11T12:54:10.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 Processor -- start 2024-09-11T12:54:10.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- start start 2024-09-11T12:54:10.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4d00b3070 0x7fd4d006e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4d0152bd0 0x7fd4d014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d0168a70 con 0x7fd4d01527f0 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd4d01688f0 con 0x7fd4d00b3070 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4d5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd4d0168bf0 con 0x7fd4d0152bd0 2024-09-11T12:54:10.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46144/0 (socket says 172.21.15.78:46144) 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 -- 172.21.15.78:0/3677939907 learned_addr learned my addr 172.21.15.78:0/3677939907 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4ce800640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4d00b3070 0x7fd4d006e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cf200640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4d0152bd0 0x7fd4d014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 -- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4d0152bd0 msgr2=0x7fd4d014f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4d0152bd0 0x7fd4d014f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 -- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4d00b3070 msgr2=0x7fd4d006e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4d00b3070 0x7fd4d006e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 -- 172.21.15.78:0/3677939907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4d013a240 con 0x7fd4d01527f0 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4cde00640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c500 tx=0x7fd4bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.100+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc017070 con 0x7fd4d01527f0 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4bc0053c0 con 0x7fd4d01527f0 2024-09-11T12:54:10.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4d013a520 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc012890 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4d013aae0 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4d00b3520 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd4bc01a070 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4c6a00640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 0x7fd4b0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fd4bc0912c0 con 0x7fd4d01527f0 2024-09-11T12:54:10.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.104+0000 7fd4ce800640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 0x7fd4b0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.108+0000 7fd4ce800640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 0x7fd4b0073c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009be0 tx=0x7fd4c00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.108+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd4bc05c8e0 con 0x7fd4d01527f0 2024-09-11T12:54:10.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd4d00695e0 con 0x7fd4d01527f0 2024-09-11T12:54:10.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4c6a00640 1 -- 172.21.15.78:0/3677939907 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) ==== 94+0+5228 (secure 0 0 0) 0x7fd4bc065170 con 0x7fd4d01527f0 2024-09-11T12:54:10.313 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:10.313 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":62,"btime":"2024-09-11T12:54:05:164845+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2256690013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2256690013},{"type":"v1","addr":"172.21.15.78:6833","nonce":2256690013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:53:49.362197+0000","modified":"2024-09-11T12:53:52.376088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6546},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6546":{"gid":6546,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/242442581","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":242442581},{"type":"v1","addr":"172.21.15.78:6835","nonce":242442581}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6546,"qdb_cluster":[6546]},"id":6}]} 2024-09-11T12:54:10.313 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 62 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 msgr2=0x7fd4b0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 0x7fd4b0073c60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009be0 tx=0x7fd4c00091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 msgr2=0x7fd4d0139960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00c500 tx=0x7fd4bc00c9c0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 shutdown_connections 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd4d0152bd0 0x7fd4d014f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd4d01527f0 0x7fd4d0139960 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd4b00717a0 0x7fd4b0073c60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 --2- 172.21.15.78:0/3677939907 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd4d00b3070 0x7fd4d006e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 >> 172.21.15.78:0/3677939907 conn(0x7fd4d00b1c50 msgr2=0x7fd4d0070690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:10.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 shutdown_connections 2024-09-11T12:54:10.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.312+0000 7fd4d5400640 1 -- 172.21.15.78:0/3677939907 wait complete. 2024-09-11T12:54:10.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:10.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 -- 172.21.15.78:0/2593939095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3006ff80 msgr2=0x7f6a30070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2593939095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3006ff80 0x7f6a30070410 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009c80 tx=0x7f6a2801cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 -- 172.21.15.78:0/2593939095 shutdown_connections 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2593939095 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a3006ff80 0x7f6a30070410 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2593939095 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a30142bf0 0x7f6a300cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 -- 172.21.15.78:0/2593939095 >> 172.21.15.78:0/2593939095 conn(0x7f6a300cfeb0 msgr2=0x7f6a300d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 -- 172.21.15.78:0/2593939095 shutdown_connections 2024-09-11T12:54:10.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.476+0000 7f6a2f200640 1 -- 172.21.15.78:0/2593939095 wait complete. 2024-09-11T12:54:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 Processor -- start 2024-09-11T12:54:10.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- start start 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3006ff80 0x7f6a301f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a300ce3c0 0x7f6a30159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a30073940 con 0x7f6a30142bf0 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a300737c0 con 0x7f6a300ce3c0 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a30073ac0 con 0x7f6a3006ff80 2024-09-11T12:54:10.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46162/0 (socket says 172.21.15.78:46162) 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 -- 172.21.15.78:0/2931211973 learned_addr learned my addr 172.21.15.78:0/2931211973 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2de00640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3006ff80 0x7f6a301f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2d400640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a300ce3c0 0x7f6a30159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 -- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3006ff80 msgr2=0x7f6a301f6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3006ff80 0x7f6a301f6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 -- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a300ce3c0 msgr2=0x7f6a30159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a300ce3c0 0x7f6a30159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 -- 172.21.15.78:0/2931211973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3013cd10 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2e800640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009bd0 tx=0x7f6a280a5990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280bb070 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a28004d40 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a280b2040 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a30159930 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.480+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a3013af30 con 0x7f6a30142bf0 2024-09-11T12:54:10.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a30070410 con 0x7f6a30142bf0 2024-09-11T12:54:10.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f6a280be070 con 0x7f6a30142bf0 2024-09-11T12:54:10.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a26000640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 0x7f6a10073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7f6a2812be20 con 0x7f6a30142bf0 2024-09-11T12:54:10.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a2de00640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 0x7f6a10073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.484+0000 7f6a2de00640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 0x7f6a10073c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005fd0 tx=0x7f6a20005d10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.490 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.488+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6a28016990 con 0x7f6a30142bf0 2024-09-11T12:54:10.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.656+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a30139580 con 0x7f6a30142bf0 2024-09-11T12:54:10.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a26000640 1 -- 172.21.15.78:0/2931211973 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+16849 (secure 0 0 0) 0x7f6a280f7a80 con 0x7f6a30142bf0 2024-09-11T12:54:10.660 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:10.660 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":90,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:07.172356+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:53:47.045831+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-11T12:53:48.326392+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"83","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6833/3066636859":"2024-09-12T12:54:04.162092+0000","172.21.15.78:6832/3066636859":"2024-09-12T12:54:04.162092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 msgr2=0x7f6a10073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 0x7f6a10073c60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6a20005fd0 tx=0x7f6a20005d10 comp rx=0 tx=0).stop 2024-09-11T12:54:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 msgr2=0x7f6a301f7f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6a28009bd0 tx=0x7f6a280a5990 comp rx=0 tx=0).stop 2024-09-11T12:54:10.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 shutdown_connections 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a30142bf0 0x7f6a301f7f10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a100717a0 0x7f6a10073c60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a300ce3c0 0x7f6a30159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 --2- 172.21.15.78:0/2931211973 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a3006ff80 0x7f6a301f6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 >> 172.21.15.78:0/2931211973 conn(0x7f6a300cfeb0 msgr2=0x7f6a300b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 shutdown_connections 2024-09-11T12:54:10.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.660+0000 7f6a2f200640 1 -- 172.21.15.78:0/2931211973 wait complete. 2024-09-11T12:54:10.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:54:10.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- 172.21.15.78:0/283984738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd240b4c30 msgr2=0x7fcd2415cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 --2- 172.21.15.78:0/283984738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd240b4c30 0x7fcd2415cf60 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009900 tx=0x7fcd1c01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- 172.21.15.78:0/283984738 shutdown_connections 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 --2- 172.21.15.78:0/283984738 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd240b4c30 0x7fcd2415cf60 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- 172.21.15.78:0/283984738 >> 172.21.15.78:0/283984738 conn(0x7fcd2406ca80 msgr2=0x7fcd240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- 172.21.15.78:0/283984738 shutdown_connections 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- 172.21.15.78:0/283984738 wait complete. 2024-09-11T12:54:10.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 Processor -- start 2024-09-11T12:54:10.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- start start 2024-09-11T12:54:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd240aed20 0x7fcd2414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd240b4c30 0x7fcd2414a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240cd460 con 0x7fcd2415cad0 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd240cd2e0 con 0x7fcd240b4c30 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd2a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd240cd5e0 con 0x7fcd240aed20 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46180/0 (socket says 172.21.15.78:46180) 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 -- 172.21.15.78:0/3553042191 learned_addr learned my addr 172.21.15.78:0/3553042191 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd22a00640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd240b4c30 0x7fcd2414a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23400640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd240aed20 0x7fcd2414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 -- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd240aed20 msgr2=0x7fcd2414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd240aed20 0x7fcd2414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 -- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd240b4c30 msgr2=0x7fcd2414a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd240b4c30 0x7fcd2414a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:10.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 -- 172.21.15.78:0/3553042191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2414ecf0 con 0x7fcd2415cad0 2024-09-11T12:54:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.824+0000 7fcd23e00640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009230 tx=0x7fcd1c0a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c0b2040 con 0x7fcd2415cad0 2024-09-11T12:54:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd1c0042b0 con 0x7fcd2415cad0 2024-09-11T12:54:10.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1c003860 con 0x7fcd2415cad0 2024-09-11T12:54:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2414ab70 con 0x7fcd2415cad0 2024-09-11T12:54:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2414f240 con 0x7fcd2415cad0 2024-09-11T12:54:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd240cdb90 con 0x7fcd2415cad0 2024-09-11T12:54:10.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.828+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fcd1c0bf020 con 0x7fcd2415cad0 2024-09-11T12:54:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.832+0000 7fcd1b400640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 0x7fcd04073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.832+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5795+0+0 (secure 0 0 0) 0x7fcd1c12c3c0 con 0x7fcd2415cad0 2024-09-11T12:54:10.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.832+0000 7fcd23400640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 0x7fcd04073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:10.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.832+0000 7fcd23400640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 0x7fcd04073c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd14007fd0 tx=0x7fcd14009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:10.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:10.836+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcd1c0c6020 con 0x7fcd2415cad0 2024-09-11T12:54:11.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.036+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcd240695e0 con 0x7fcd2415cad0 2024-09-11T12:54:11.037 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:54:11.036+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:54:11.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.036+0000 7fcd1b400640 1 -- 172.21.15.78:0/3553042191 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) ==== 161+0+0 (secure 0 0 0) 0x7fcd1c0164d0 con 0x7fcd2415cad0 2024-09-11T12:54:11.039 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:54:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 msgr2=0x7fcd04073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 0x7fcd04073c60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcd14007fd0 tx=0x7fcd14009040 comp rx=0 tx=0).stop 2024-09-11T12:54:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 msgr2=0x7fcd240b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:11.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c009230 tx=0x7fcd1c0a57f0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 shutdown_connections 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd2415cad0 0x7fcd240b0a70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd040717a0 0x7fcd04073c60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd240b4c30 0x7fcd2414a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 --2- 172.21.15.78:0/3553042191 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd240aed20 0x7fcd2414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 >> 172.21.15.78:0/3553042191 conn(0x7fcd2406ca80 msgr2=0x7fcd2406fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 shutdown_connections 2024-09-11T12:54:11.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.040+0000 7fcd2a200640 1 -- 172.21.15.78:0/3553042191 wait complete. 2024-09-11T12:54:11.048 INFO:tasks.ceph.mds.a.smithi078.stderr: -2762> 2024-09-11T12:53:51.948+0000 7fd31a800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:54:11.063 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:54:11.096 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:54:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.212+0000 7f97e0c00640 1 -- 172.21.15.78:0/4093064233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc06d060 msgr2=0x7f97dc150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:11.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.212+0000 7f97e0c00640 1 --2- 172.21.15.78:0/4093064233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc06d060 0x7f97dc150a70 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0cd290 tx=0x7f97cc01ccc0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- 172.21.15.78:0/4093064233 shutdown_connections 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- 172.21.15.78:0/4093064233 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc071d90 0x7f97dc072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- 172.21.15.78:0/4093064233 >> v2:172.21.15.157:3300/0 conn(0x7f97dc15a530 0x7f97dc15a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- 172.21.15.78:0/4093064233 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc06d060 0x7f97dc150a70 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- 172.21.15.78:0/4093064233 >> 172.21.15.78:0/4093064233 conn(0x7f97dc075a10 msgr2=0x7f97dc0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- 172.21.15.78:0/4093064233 shutdown_connections 2024-09-11T12:54:11.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- 172.21.15.78:0/4093064233 wait complete. 2024-09-11T12:54:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 Processor -- start 2024-09-11T12:54:11.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- start start 2024-09-11T12:54:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc06d060 0x7f97dc151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 0x7f97dc14c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0b49f0 con 0x7f97dc071d90 2024-09-11T12:54:11.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0b4870 con 0x7f97dc06d060 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97e0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0b4b70 con 0x7f97dc15a530 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46214/0 (socket says 172.21.15.78:46214) 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 -- 172.21.15.78:0/2591694334 learned_addr learned my addr 172.21.15.78:0/2591694334 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9e00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc06d060 0x7f97dc151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97da800640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 0x7f97dc14c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 -- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 msgr2=0x7f97dc14c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 0x7f97dc14c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 -- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc06d060 msgr2=0x7f97dc151240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc06d060 0x7f97dc151240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 -- 172.21.15.78:0/2591694334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc164c70 con 0x7f97dc071d90 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97da800640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 0x7f97dc14c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:11.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d9400640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f97c800d750 tx=0x7f97c800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:11.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.216+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8017070 con 0x7f97dc071d90 2024-09-11T12:54:11.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97c8009d20 con 0x7f97dc071d90 2024-09-11T12:54:11.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c800de60 con 0x7f97dc071d90 2024-09-11T12:54:11.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc164f50 con 0x7f97dc071d90 2024-09-11T12:54:11.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc165510 con 0x7f97dc071d90 2024-09-11T12:54:11.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc072200 con 0x7f97dc071d90 2024-09-11T12:54:11.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f97c8004280 con 0x7f97dc071d90 2024-09-11T12:54:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d2000640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 0x7f97bc073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f97c80923b0 con 0x7f97dc071d90 2024-09-11T12:54:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d9e00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 0x7f97bc073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:11.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.220+0000 7f97d9e00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 0x7f97bc073c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0af000 tx=0x7f97cc01b500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:11.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.224+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f97c8020020 con 0x7f97dc071d90 2024-09-11T12:54:11.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:11.424+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f97dc15bff0 con 0x7f97dc071d90 2024-09-11T12:54:12.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97d2000640 1 -- 172.21.15.78:0/2591694334 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) ==== 104+0+0 (secure 0 0 0) 0x7f97c805df40 con 0x7f97dc071d90 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 msgr2=0x7f97bc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 0x7f97bc073c60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f97cc0af000 tx=0x7f97cc01b500 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 msgr2=0x7f97dc148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f97c800d750 tx=0x7f97c800dc10 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 shutdown_connections 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f97dc15a530 0x7f97dc14c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f97dc071d90 0x7f97dc148960 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f97bc0717a0 0x7f97bc073c60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 --2- 172.21.15.78:0/2591694334 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f97dc06d060 0x7f97dc151240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 >> 172.21.15.78:0/2591694334 conn(0x7f97dc075a10 msgr2=0x7f97dc14dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 shutdown_connections 2024-09-11T12:54:12.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.039+0000 7f97e0c00640 1 -- 172.21.15.78:0/2591694334 wait complete. 2024-09-11T12:54:12.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:54:12.162 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:54:12.166 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:54:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 -- 172.21.15.78:0/136375757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5790 msgr2=0x7f10c80723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 --2- 172.21.15.78:0/136375757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5790 0x7f10c80723f0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f10c80749e0 tx=0x7f10b801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 -- 172.21.15.78:0/136375757 shutdown_connections 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 --2- 172.21.15.78:0/136375757 >> v2:172.21.15.157:3300/0 conn(0x7f10c80b5b70 0x7f10c8072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 --2- 172.21.15.78:0/136375757 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5790 0x7f10c80723f0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 -- 172.21.15.78:0/136375757 >> 172.21.15.78:0/136375757 conn(0x7f10c80b3050 msgr2=0x7f10c80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 -- 172.21.15.78:0/136375757 shutdown_connections 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.255+0000 7f10ce600640 1 -- 172.21.15.78:0/136375757 wait complete. 2024-09-11T12:54:12.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 Processor -- start 2024-09-11T12:54:12.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- start start 2024-09-11T12:54:12.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c80b5790 0x7f10c8149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:12.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c8151340 0x7f10c806ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10c8167b60 con 0x7f10c80b5b70 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f10c815bdf0 con 0x7f10c80b5790 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f10c8167ce0 con 0x7f10c8151340 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46234/0 (socket says 172.21.15.78:46234) 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 -- 172.21.15.78:0/236824232 learned_addr learned my addr 172.21.15.78:0/236824232 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10c7400640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c80b5790 0x7f10c8149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10c7e00640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c8151340 0x7f10c806ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 -- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c8151340 msgr2=0x7f10c806ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c8151340 0x7f10c806ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 -- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c80b5790 msgr2=0x7f10c8149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c80b5790 0x7f10c8149450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 -- 172.21.15.78:0/236824232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c8145e40 con 0x7f10c80b5b70 2024-09-11T12:54:12.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bfe00640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c970 tx=0x7f10b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b0007c70 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10b000f450 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c815db90 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10b000f8a0 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.259+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c81453d0 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.263+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c814b130 con 0x7f10c80b5b70 2024-09-11T12:54:12.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.263+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f10b000fa40 con 0x7f10c80b5b70 2024-09-11T12:54:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.263+0000 7f10bf400640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 0x7f10a4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.263+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5881+0+0 (secure 0 0 0) 0x7f10b00927a0 con 0x7f10c80b5b70 2024-09-11T12:54:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.263+0000 7f10c7400640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 0x7f10a4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.267+0000 7f10c7400640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 0x7f10a4073c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10b80af000 tx=0x7f10b801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.267+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f10c814b130 con 0x7f10c80b5b70 2024-09-11T12:54:12.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:12.435+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f10c81517f0 con 0x7f10c80b5b70 2024-09-11T12:54:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.203+0000 7f10bf400640 1 -- 172.21.15.78:0/236824232 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f10c8145e40 con 0x7f10c80b5b70 2024-09-11T12:54:13.207 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 msgr2=0x7f10a4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 0x7f10a4073c60 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f10b80af000 tx=0x7f10b801b100 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 msgr2=0x7f10c8144e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f10b000c970 tx=0x7f10b000ce30 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 shutdown_connections 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f10c8151340 0x7f10c806ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f10c80b5b70 0x7f10c8144e90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10a40717a0 0x7f10a4073c60 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 --2- 172.21.15.78:0/236824232 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f10c80b5790 0x7f10c8149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 >> 172.21.15.78:0/236824232 conn(0x7f10c80b3050 msgr2=0x7f10c80b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:13.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 shutdown_connections 2024-09-11T12:54:13.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.207+0000 7f10ce600640 1 -- 172.21.15.78:0/236824232 wait complete. 2024-09-11T12:54:13.232 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:54:13.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- 172.21.15.78:0/4257591710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 msgr2=0x7face8073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 --2- 172.21.15.78:0/4257591710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 0x7face8073950 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7facd8009900 tx=0x7facd801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- 172.21.15.78:0/4257591710 shutdown_connections 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 --2- 172.21.15.78:0/4257591710 >> v2:172.21.15.157:3300/0 conn(0x7face8072520 0x7face806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 --2- 172.21.15.78:0/4257591710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 0x7face8073950 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- 172.21.15.78:0/4257591710 >> 172.21.15.78:0/4257591710 conn(0x7face80b3050 msgr2=0x7face80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- 172.21.15.78:0/4257591710 shutdown_connections 2024-09-11T12:54:13.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- 172.21.15.78:0/4257591710 wait complete. 2024-09-11T12:54:13.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 Processor -- start 2024-09-11T12:54:13.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.379+0000 7facecc00640 1 -- start start 2024-09-11T12:54:13.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7face8073e90 0x7face80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 0x7face814f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7face8070340 con 0x7face8072520 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7face80701c0 con 0x7face8073e90 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7face80704c0 con 0x7face8151050 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46250/0 (socket says 172.21.15.78:46250) 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 -- 172.21.15.78:0/1690136600 learned_addr learned my addr 172.21.15.78:0/1690136600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 -- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 msgr2=0x7face814f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 0x7face814f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 -- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7face8073e90 msgr2=0x7face80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:13.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7face8073e90 0x7face80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:13.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 -- 172.21.15.78:0/1690136600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face80ce410 con 0x7face8072520 2024-09-11T12:54:13.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7face5e00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7facd8009850 tx=0x7facd8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:13.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd80b2040 con 0x7face8072520 2024-09-11T12:54:13.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facd8004130 con 0x7face8072520 2024-09-11T12:54:13.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8003600 con 0x7face8072520 2024-09-11T12:54:13.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81ec990 con 0x7face8072520 2024-09-11T12:54:13.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81ecf50 con 0x7face8072520 2024-09-11T12:54:13.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7face806f5d0 con 0x7face8072520 2024-09-11T12:54:13.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.383+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7facd80bf020 con 0x7face8072520 2024-09-11T12:54:13.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.387+0000 7facde000640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 0x7facc8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:13.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.387+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5443+0+0 (secure 0 0 0) 0x7facd812c110 con 0x7face8072520 2024-09-11T12:54:13.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.387+0000 7face5400640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 0x7facc8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:13.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.387+0000 7face5400640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 0x7facc8073c60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facd4007fd0 tx=0x7facd40078e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:13.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.391+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7facd80c6020 con 0x7face8072520 2024-09-11T12:54:13.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:13.559+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7face80cfab0 con 0x7face8072520 2024-09-11T12:54:14.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facde000640 1 -- 172.21.15.78:0/1690136600 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7facd80164d0 con 0x7face8072520 2024-09-11T12:54:14.209 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 msgr2=0x7facc8073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 0x7facc8073c60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7facd4007fd0 tx=0x7facd40078e0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 msgr2=0x7face814cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7facd8009850 tx=0x7facd8005e70 comp rx=0 tx=0).stop 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 shutdown_connections 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7face8151050 0x7face814f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7facc80717a0 0x7facc8073c60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7face8073e90 0x7face80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 --2- 172.21.15.78:0/1690136600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7face8072520 0x7face814cce0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 >> 172.21.15.78:0/1690136600 conn(0x7face80b3050 msgr2=0x7face815a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:14.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 shutdown_connections 2024-09-11T12:54:14.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.207+0000 7facecc00640 1 -- 172.21.15.78:0/1690136600 wait complete. 2024-09-11T12:54:14.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- 172.21.15.78:0/3886578447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 msgr2=0x7f3ff0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 --2- 172.21.15.78:0/3886578447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f3ff0004c90 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009b10 tx=0x7f3ff401d230 comp rx=0 tx=0).stop 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- 172.21.15.78:0/3886578447 shutdown_connections 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 --2- 172.21.15.78:0/3886578447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f3ff0004c90 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- 172.21.15.78:0/3886578447 >> 172.21.15.78:0/3886578447 conn(0x7f400c06ca80 msgr2=0x7f400c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- 172.21.15.78:0/3886578447 shutdown_connections 2024-09-11T12:54:14.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- 172.21.15.78:0/3886578447 wait complete. 2024-09-11T12:54:14.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 Processor -- start 2024-09-11T12:54:14.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.375+0000 7f4011e00640 1 -- start start 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c15daa0 0x7f400c1f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c0afa60 0x7f400c0b1110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f400c15b040 con 0x7f3ff0004810 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f400c15aec0 con 0x7f400c0afa60 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f400c15b1c0 con 0x7f400c15daa0 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52744/0 (socket says 172.21.15.78:52744) 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 -- 172.21.15.78:0/1367655671 learned_addr learned my addr 172.21.15.78:0/1367655671 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4003e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c15daa0 0x7f400c1f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400b400640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c0afa60 0x7f400c0b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 -- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c15daa0 msgr2=0x7f400c1f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c15daa0 0x7f400c1f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 -- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c0afa60 msgr2=0x7f400c0b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c0afa60 0x7f400c0b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 -- 172.21.15.78:0/1367655671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c156eb0 con 0x7f3ff0004810 2024-09-11T12:54:14.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f400aa00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc012610 tx=0x7f3ffc012ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc010990 con 0x7f3ff0004810 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ffc017030 con 0x7f3ff0004810 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc010dd0 con 0x7f3ff0004810 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c1f4c30 con 0x7f3ff0004810 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c157120 con 0x7f3ff0004810 2024-09-11T12:54:14.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.379+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400c157320 con 0x7f3ff0004810 2024-09-11T12:54:14.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.383+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f3ffc013450 con 0x7f3ff0004810 2024-09-11T12:54:14.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.383+0000 7f4003400640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 0x7f3fe8073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.383+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f3ffc095870 con 0x7f3ff0004810 2024-09-11T12:54:14.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.383+0000 7f4003e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 0x7f3fe8073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.383+0000 7f4003e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 0x7f3fe8073c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40ad000 tx=0x7f3ff4019fd0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:14.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.387+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f400c157320 con 0x7f3ff0004810 2024-09-11T12:54:14.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.551+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f400c0703b0 con 0x7f3ff0004810 2024-09-11T12:54:14.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.555+0000 7f4003400640 1 -- 172.21.15.78:0/1367655671 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f3ffc061860 con 0x7f3ff0004810 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 msgr2=0x7f3fe8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 0x7f3fe8073c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40ad000 tx=0x7f3ff4019fd0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 msgr2=0x7f400c0b09b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc012610 tx=0x7f3ffc012ad0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 shutdown_connections 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe80717d0 0x7f3fe8073c90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c0afa60 0x7f400c0b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c15daa0 0x7f400c1f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 --2- 172.21.15.78:0/1367655671 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3ff0004810 0x7f400c0b09b0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 >> 172.21.15.78:0/1367655671 conn(0x7f400c06ca80 msgr2=0x7f400c15b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:14.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 shutdown_connections 2024-09-11T12:54:14.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.559+0000 7f4011e00640 1 -- 172.21.15.78:0/1367655671 wait complete. 2024-09-11T12:54:14.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-11T12:54:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 -- 172.21.15.78:0/2284093621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c070690 msgr2=0x7fc87c15d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 --2- 172.21.15.78:0/2284093621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c070690 0x7fc87c15d850 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc874009c80 tx=0x7fc87401cc50 comp rx=0 tx=0).stop 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 -- 172.21.15.78:0/2284093621 shutdown_connections 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 --2- 172.21.15.78:0/2284093621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c070690 0x7fc87c15d850 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 --2- 172.21.15.78:0/2284093621 >> v2:172.21.15.157:3300/0 conn(0x7fc87c1527c0 0x7fc87c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 -- 172.21.15.78:0/2284093621 >> 172.21.15.78:0/2284093621 conn(0x7fc87c072c30 msgr2=0x7fc87c06e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 -- 172.21.15.78:0/2284093621 shutdown_connections 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 -- 172.21.15.78:0/2284093621 wait complete. 2024-09-11T12:54:14.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.719+0000 7fc87b200640 1 Processor -- start 2024-09-11T12:54:14.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- start start 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc87c070da0 0x7fc87c14cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 0x7fc87c145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c168780 con 0x7fc87c155bf0 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc87c15eab0 con 0x7fc87c070da0 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc87c168900 con 0x7fc87c1527c0 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52758/0 (socket says 172.21.15.78:52758) 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 -- 172.21.15.78:0/2474994558 learned_addr learned my addr 172.21.15.78:0/2474994558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc879e00640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc87c070da0 0x7fc87c14cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc873e00640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 0x7fc87c145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 -- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 msgr2=0x7fc87c145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 0x7fc87c145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 -- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc87c070da0 msgr2=0x7fc87c14cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc87c070da0 0x7fc87c14cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 -- 172.21.15.78:0/2474994558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc87c146090 con 0x7fc87c155bf0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc873e00640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 0x7fc87c145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87a800640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc87401c6c0 tx=0x7fc874005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8740b2040 con 0x7fc87c155bf0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8740a5b20 con 0x7fc87c155bf0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc874004240 con 0x7fc87c155bf0 2024-09-11T12:54:14.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc87c146370 con 0x7fc87c155bf0 2024-09-11T12:54:14.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc87c146930 con 0x7fc87c155bf0 2024-09-11T12:54:14.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.723+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc87c15d850 con 0x7fc87c155bf0 2024-09-11T12:54:14.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.727+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fc8740bd030 con 0x7fc87c155bf0 2024-09-11T12:54:14.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.727+0000 7fc872000640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 0x7fc858073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:14.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.727+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7fc87412bcb0 con 0x7fc87c155bf0 2024-09-11T12:54:14.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.727+0000 7fc879e00640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 0x7fc858073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:14.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.727+0000 7fc879e00640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 0x7fc858073c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009b40 tx=0x7fc86c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:14.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.731+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc8740c4020 con 0x7fc87c155bf0 2024-09-11T12:54:14.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:14.955+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fc87c070b80 con 0x7fc87c155bf0 2024-09-11T12:54:15.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.219+0000 7fc872000640 1 -- 172.21.15.78:0/2474994558 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v257) ==== 146+0+0 (secure 0 0 0) 0x7fc874016990 con 0x7fc87c155bf0 2024-09-11T12:54:15.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 msgr2=0x7fc858073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 0x7fc858073c90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009b40 tx=0x7fc86c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 msgr2=0x7fc87c147790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fc87401c6c0 tx=0x7fc874005c40 comp rx=0 tx=0).stop 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 shutdown_connections 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc87c155bf0 0x7fc87c147790 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc87c1527c0 0x7fc87c145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8580717d0 0x7fc858073c90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 --2- 172.21.15.78:0/2474994558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc87c070da0 0x7fc87c14cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 >> 172.21.15.78:0/2474994558 conn(0x7fc87c070690 msgr2=0x7fc87c0b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:15.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 shutdown_connections 2024-09-11T12:54:15.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.223+0000 7fc87b200640 1 -- 172.21.15.78:0/2474994558 wait complete. 2024-09-11T12:54:15.250 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-11T12:54:15.250 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-11T12:54:15.250 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:54:15.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:15.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- 172.21.15.78:0/1184009101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395006f250 msgr2=0x7f3950157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 --2- 172.21.15.78:0/1184009101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395006f250 0x7f3950157ae0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f394801c850 comp rx=0 tx=0).stop 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- 172.21.15.78:0/1184009101 shutdown_connections 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 --2- 172.21.15.78:0/1184009101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f395006f250 0x7f3950157ae0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- 172.21.15.78:0/1184009101 >> 172.21.15.78:0/1184009101 conn(0x7f395006ca80 msgr2=0x7f39500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- 172.21.15.78:0/1184009101 shutdown_connections 2024-09-11T12:54:15.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- 172.21.15.78:0/1184009101 wait complete. 2024-09-11T12:54:15.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 Processor -- start 2024-09-11T12:54:15.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.395+0000 7f3958200640 1 -- start start 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f395006f250 0x7f395015e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39500736d0 0x7f3950147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3950168a00 con 0x7f3950070a00 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3950168880 con 0x7f395006f250 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3950168b80 con 0x7f39500736d0 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52774/0 (socket says 172.21.15.78:52774) 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 -- 172.21.15.78:0/1037156589 learned_addr learned my addr 172.21.15.78:0/1037156589 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3955800640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f395006f250 0x7f395015e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3956200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39500736d0 0x7f3950147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 -- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39500736d0 msgr2=0x7f3950147490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39500736d0 0x7f3950147490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 -- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f395006f250 msgr2=0x7f395015e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f395006f250 0x7f395015e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 -- 172.21.15.78:0/1037156589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3950153930 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3954e00640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f393c00b440 tx=0x7f393c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c015020 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f393c011980 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3950153650 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c011dd0 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39501f8e50 con 0x7f3950070a00 2024-09-11T12:54:15.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3950157ae0 con 0x7f3950070a00 2024-09-11T12:54:15.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.399+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f393c018020 con 0x7f3950070a00 2024-09-11T12:54:15.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.403+0000 7f394d600640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 0x7f3930073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.403+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f393c091ab0 con 0x7f3950070a00 2024-09-11T12:54:15.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.403+0000 7f3955800640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 0x7f3930073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.403+0000 7f3955800640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 0x7f3930073c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3944001ee0 tx=0x7f3944009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:15.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.407+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f393c01f030 con 0x7f3950070a00 2024-09-11T12:54:15.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.571+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3950072020 con 0x7f3950070a00 2024-09-11T12:54:15.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f394d600640 1 -- 172.21.15.78:0/1037156589 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+13161 (secure 0 0 0) 0x7f393c00d040 con 0x7f3950070a00 2024-09-11T12:54:15.577 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:15.577 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":93,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:14.208689+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/242442581":"2024-09-12T12:54:11.038445+0000","172.21.15.78:6834/242442581":"2024-09-12T12:54:11.038445+0000","172.21.15.78:6833/3066636859":"2024-09-12T12:54:04.162092+0000","172.21.15.78:6832/3066636859":"2024-09-12T12:54:04.162092+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 msgr2=0x7f3930073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 0x7f3930073c60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f3944001ee0 tx=0x7f3944009040 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 msgr2=0x7f3950152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f393c00b440 tx=0x7f393c00b900 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 shutdown_connections 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39500736d0 0x7f3950147490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3950070a00 0x7f3950152e20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39300717a0 0x7f3930073c60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 --2- 172.21.15.78:0/1037156589 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f395006f250 0x7f395015e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 >> 172.21.15.78:0/1037156589 conn(0x7f395006ca80 msgr2=0x7f3950072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 shutdown_connections 2024-09-11T12:54:15.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.575+0000 7f3958200640 1 -- 172.21.15.78:0/1037156589 wait complete. 2024-09-11T12:54:15.599 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:54:15.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:54:15.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- 172.21.15.78:0/830250096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc1518b0 msgr2=0x7f9ebc0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/830250096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc1518b0 0x7f9ebc0b1a10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9eac009c80 tx=0x7f9eac01cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- 172.21.15.78:0/830250096 shutdown_connections 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/830250096 >> v2:172.21.15.157:3300/0 conn(0x7f9ebc0b1f50 0x7f9ebc0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/830250096 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc1518b0 0x7f9ebc0b1a10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- 172.21.15.78:0/830250096 >> 172.21.15.78:0/830250096 conn(0x7f9ebc071dd0 msgr2=0x7f9ebc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- 172.21.15.78:0/830250096 shutdown_connections 2024-09-11T12:54:15.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- 172.21.15.78:0/830250096 wait complete. 2024-09-11T12:54:15.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 Processor -- start 2024-09-11T12:54:15.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- start start 2024-09-11T12:54:15.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ebc0b1f50 0x7f9ebc143330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0ce150 0x7f9ebc1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc1677e0 con 0x7f9ebc1518b0 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9ebc151e80 con 0x7f9ebc0b1f50 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ec1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9ebc167960 con 0x7f9ebc0ce150 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ebb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ebb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52776/0 (socket says 172.21.15.78:52776) 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ebb400640 1 -- 172.21.15.78:0/1673591189 learned_addr learned my addr 172.21.15.78:0/1673591189 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9ebaa00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ebc0b1f50 0x7f9ebc143330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.743+0000 7f9eba000640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0ce150 0x7f9ebc1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 -- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0ce150 msgr2=0x7f9ebc1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0ce150 0x7f9ebc1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 -- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ebc0b1f50 msgr2=0x7f9ebc143330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ebc0b1f50 0x7f9ebc143330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 -- 172.21.15.78:0/1673591189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ebc15a110 con 0x7f9ebc1518b0 2024-09-11T12:54:15.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ebb400640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c3a0 tx=0x7f9eb400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4017070 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ebc15a3f0 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9eb4003600 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4003a50 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ebc15a9b0 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ebc0cdc10 con 0x7f9ebc1518b0 2024-09-11T12:54:15.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f9eb401a070 con 0x7f9ebc1518b0 2024-09-11T12:54:15.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 0x7f9e9c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:15.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.747+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f9eb40913a0 con 0x7f9ebc1518b0 2024-09-11T12:54:15.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.751+0000 7f9ebaa00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 0x7f9e9c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:15.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.751+0000 7f9ebaa00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 0x7f9e9c073c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0af000 tx=0x7f9eac01b5e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:15.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.751+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9eb4021020 con 0x7f9ebc1518b0 2024-09-11T12:54:15.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9ebc069500 con 0x7f9ebc1518b0 2024-09-11T12:54:15.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9eb2a00640 1 -- 172.21.15.78:0/1673591189 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f9eb405ccb0 con 0x7f9ebc1518b0 2024-09-11T12:54:15.921 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:54:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 msgr2=0x7f9e9c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 0x7f9e9c073c60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0af000 tx=0x7f9eac01b5e0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 msgr2=0x7f9ebc1467c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9eb400c3a0 tx=0x7f9eb400c860 comp rx=0 tx=0).stop 2024-09-11T12:54:15.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 shutdown_connections 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9ebc1518b0 0x7f9ebc1467c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9ebc0ce150 0x7f9ebc1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9e9c0717a0 0x7f9e9c073c60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 --2- 172.21.15.78:0/1673591189 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9ebc0b1f50 0x7f9ebc143330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 >> 172.21.15.78:0/1673591189 conn(0x7f9ebc071dd0 msgr2=0x7f9ebc15c940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 shutdown_connections 2024-09-11T12:54:15.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:15.919+0000 7f9ec1c00640 1 -- 172.21.15.78:0/1673591189 wait complete. 2024-09-11T12:54:15.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-11T12:54:16.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- 172.21.15.78:0/3916013242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 msgr2=0x7fa0f415ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- 172.21.15.78:0/3916013242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f415ba90 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009900 tx=0x7fa0ec01c8a0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- 172.21.15.78:0/3916013242 shutdown_connections 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- 172.21.15.78:0/3916013242 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f415ba90 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- 172.21.15.78:0/3916013242 >> v2:172.21.15.157:3300/0 conn(0x7fa0f40ccb20 0x7fa0f415b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- 172.21.15.78:0/3916013242 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40b4780 0x7fa0f4073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- 172.21.15.78:0/3916013242 >> 172.21.15.78:0/3916013242 conn(0x7fa0f406ca80 msgr2=0x7fa0f40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- 172.21.15.78:0/3916013242 shutdown_connections 2024-09-11T12:54:16.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- 172.21.15.78:0/3916013242 wait complete. 2024-09-11T12:54:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 Processor -- start 2024-09-11T12:54:16.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- start start 2024-09-11T12:54:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f40b4780 0x7fa0f414d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:16.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40ccb20 0x7fa0f4150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f406f190 con 0x7fa0f40b4b60 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f406f010 con 0x7fa0f40b4780 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.083+0000 7fa0fa400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f406f310 con 0x7fa0f40ccb20 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f3400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f40b4780 0x7fa0f414d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:16.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52798/0 (socket says 172.21.15.78:52798) 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 -- 172.21.15.78:0/2950992058 learned_addr learned my addr 172.21.15.78:0/2950992058 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f3e00640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40ccb20 0x7fa0f4150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 -- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40ccb20 msgr2=0x7fa0f4150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40ccb20 0x7fa0f4150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 -- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f40b4780 msgr2=0x7fa0f414d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f40b4780 0x7fa0f414d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 -- 172.21.15.78:0/2950992058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f41578f0 con 0x7fa0f40b4b60 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0f2a00640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00e9f0 tx=0x7fa0dc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc00cd80 con 0x7fa0f40b4b60 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa0dc003a70 con 0x7fa0f40b4b60 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f41f7600 con 0x7fa0f40b4b60 2024-09-11T12:54:16.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0dc004020 con 0x7fa0f40b4b60 2024-09-11T12:54:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f41f72d0 con 0x7fa0f40b4b60 2024-09-11T12:54:16.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f4073f70 con 0x7fa0f40b4b60 2024-09-11T12:54:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.087+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa0dc0041c0 con 0x7fa0f40b4b60 2024-09-11T12:54:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.091+0000 7fa0eb400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 0x7fa0d0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.091+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7fa0dc092590 con 0x7fa0f40b4b60 2024-09-11T12:54:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.091+0000 7fa0f3400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 0x7fa0d0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:16.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.091+0000 7fa0f3400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 0x7fa0d0073c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4004630 tx=0x7fa0e400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:16.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.091+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa0dc016030 con 0x7fa0f40b4b60 2024-09-11T12:54:16.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:16.319+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fa0f40720e0 con 0x7fa0f40b4b60 2024-09-11T12:54:17.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.227+0000 7fa0eb400640 1 -- 172.21.15.78:0/2950992058 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v259) ==== 152+0+0 (secure 0 0 0) 0x7fa0dc05de30 con 0x7fa0f40b4b60 2024-09-11T12:54:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 msgr2=0x7fa0d0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 0x7fa0d0073c60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa0e4004630 tx=0x7fa0e400b2e0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 msgr2=0x7fa0f41504c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc00e9f0 tx=0x7fa0dc00eeb0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 shutdown_connections 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0f40ccb20 0x7fa0f4150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0f40b4b60 0x7fa0f41504c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa0d00717a0 0x7fa0d0073c60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 --2- 172.21.15.78:0/2950992058 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0f40b4780 0x7fa0f414d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 >> 172.21.15.78:0/2950992058 conn(0x7fa0f406ca80 msgr2=0x7fa0f415b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 shutdown_connections 2024-09-11T12:54:17.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.231+0000 7fa0fa400640 1 -- 172.21.15.78:0/2950992058 wait complete. 2024-09-11T12:54:17.256 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:54:17.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- 172.21.15.78:0/3729922458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 msgr2=0x7f87f0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 --2- 172.21.15.78:0/3729922458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f87f0004c90 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009b10 tx=0x7f87f401d210 comp rx=0 tx=0).stop 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f880b400640 1 -- 172.21.15.78:0/3729922458 reap_dead start 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- 172.21.15.78:0/3729922458 shutdown_connections 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- 172.21.15.78:0/3729922458 >> 172.21.15.78:0/3729922458 conn(0x7f880c06ca80 msgr2=0x7f880c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- 172.21.15.78:0/3729922458 shutdown_connections 2024-09-11T12:54:17.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- 172.21.15.78:0/3729922458 wait complete. 2024-09-11T12:54:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 Processor -- start 2024-09-11T12:54:17.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.399+0000 7f8811c00640 1 -- start start 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 0x7f880c151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880c0ae8c0 0x7f880c0aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f880c0b50c0 con 0x7f87f0004810 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f880c0b4f40 con 0x7f880c0ae8c0 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f880c0b5240 con 0x7f880c0cf770 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52818/0 (socket says 172.21.15.78:52818) 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 -- 172.21.15.78:0/1277189523 learned_addr learned my addr 172.21.15.78:0/1277189523 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8802000640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 0x7f880c151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880b400640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880c0ae8c0 0x7f880c0aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 -- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 msgr2=0x7f880c151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 0x7f880c151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 -- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880c0ae8c0 msgr2=0x7f880c0aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880c0ae8c0 0x7f880c0aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 -- 172.21.15.78:0/1277189523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c15b720 con 0x7f87f0004810 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8802000640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 0x7f880c151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f880aa00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f87fc011a80 tx=0x7f87fc011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00fe10 con 0x7f87f0004810 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f87fc016070 con 0x7f87f0004810 2024-09-11T12:54:17.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0127a0 con 0x7f87f0004810 2024-09-11T12:54:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1f7930 con 0x7f87f0004810 2024-09-11T12:54:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1f7ef0 con 0x7f87f0004810 2024-09-11T12:54:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0736d0 con 0x7f87f0004810 2024-09-11T12:54:17.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.403+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f87fc018030 con 0x7f87f0004810 2024-09-11T12:54:17.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.407+0000 7f8803400640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 0x7f87e8073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.407+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f87fc0925d0 con 0x7f87f0004810 2024-09-11T12:54:17.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.407+0000 7f8802000640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 0x7f87e8073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.407+0000 7f8802000640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 0x7f87e8073c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87f40ad000 tx=0x7f87f4019fb0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:17.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.411+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f87fc05e5c0 con 0x7f87f0004810 2024-09-11T12:54:17.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f880c151da0 con 0x7f87f0004810 2024-09-11T12:54:17.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8803400640 1 -- 172.21.15.78:0/1277189523 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+1998 (secure 0 0 0) 0x7f87fc05dec0 con 0x7f87f0004810 2024-09-11T12:54:17.585 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:54:17.586 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:54:17.587 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:54:17.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 msgr2=0x7f87e8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 0x7f87e8073c90 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87f40ad000 tx=0x7f87f4019fb0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 msgr2=0x7f880c161700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f87fc011a80 tx=0x7f87fc011f40 comp rx=0 tx=0).stop 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 shutdown_connections 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87e80717d0 0x7f87e8073c90 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f880c0ae8c0 0x7f880c0aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f880c0cf770 0x7f880c151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 --2- 172.21.15.78:0/1277189523 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87f0004810 0x7f880c161700 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 >> 172.21.15.78:0/1277189523 conn(0x7f880c06ca80 msgr2=0x7f880c072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 shutdown_connections 2024-09-11T12:54:17.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.583+0000 7f8811c00640 1 -- 172.21.15.78:0/1277189523 wait complete. 2024-09-11T12:54:17.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:54:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- 172.21.15.78:0/1567747687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 msgr2=0x7f5be00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 --2- 172.21.15.78:0/1567747687 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5be00051d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4009b60 tx=0x7f5bf401d2b0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bfa800640 1 -- 172.21.15.78:0/1567747687 reap_dead start 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- 172.21.15.78:0/1567747687 shutdown_connections 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- 172.21.15.78:0/1567747687 >> 172.21.15.78:0/1567747687 conn(0x7f5bfc071dd0 msgr2=0x7f5bfc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- 172.21.15.78:0/1567747687 shutdown_connections 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- 172.21.15.78:0/1567747687 wait complete. 2024-09-11T12:54:17.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 Processor -- start 2024-09-11T12:54:17.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- start start 2024-09-11T12:54:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bfc137200 0x7f5bfc138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bfc13a1d0 0x7f5bfc13b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc141fb0 con 0x7f5be0004d50 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc141e30 con 0x7f5bfc137200 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5c01000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bfc0b2de0 con 0x7f5bfc13a1d0 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52832/0 (socket says 172.21.15.78:52832) 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 -- 172.21.15.78:0/860169824 learned_addr learned my addr 172.21.15.78:0/860169824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bfa800640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bfc13a1d0 0x7f5bfc13b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9400640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bfc137200 0x7f5bfc138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 -- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bfc13a1d0 msgr2=0x7f5bfc13b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bfc13a1d0 0x7f5bfc13b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 -- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bfc137200 msgr2=0x7f5bfc138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bfc137200 0x7f5bfc138fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.751+0000 7f5bf9e00640 1 -- 172.21.15.78:0/860169824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc139520 con 0x7f5be0004d50 2024-09-11T12:54:17.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf9e00640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5bec010c50 tx=0x7f5bec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec017070 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bec003ba0 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bec004030 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc1f5910 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc1f5ed0 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc1416e0 con 0x7f5be0004d50 2024-09-11T12:54:17.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f5bec0041d0 con 0x7f5be0004d50 2024-09-11T12:54:17.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 0x7f5bdc073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:17.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f5bec0928c0 con 0x7f5be0004d50 2024-09-11T12:54:17.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.755+0000 7f5bf9400640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 0x7f5bdc073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:17.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.759+0000 7f5bf9400640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 0x7f5bdc073c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5be8001250 tx=0x7f5be80011a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:17.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.759+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5bec05e270 con 0x7f5be0004d50 2024-09-11T12:54:17.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.927+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5bfc15c020 con 0x7f5be0004d50 2024-09-11T12:54:17.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5bf2000640 1 -- 172.21.15.78:0/860169824 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f5bec0627c0 con 0x7f5be0004d50 2024-09-11T12:54:17.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 msgr2=0x7f5bdc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 0x7f5bdc073c90 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5be8001250 tx=0x7f5be80011a0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 msgr2=0x7f5bfc1f52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:17.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f5bec010c50 tx=0x7f5bec00e720 comp rx=0 tx=0).stop 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 shutdown_connections 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bfc13a1d0 0x7f5bfc13b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bdc0717d0 0x7f5bdc073c90 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bfc137200 0x7f5bfc138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 --2- 172.21.15.78:0/860169824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5be0004d50 0x7f5bfc1f52b0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 >> 172.21.15.78:0/860169824 conn(0x7f5bfc071dd0 msgr2=0x7f5bfc0cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 shutdown_connections 2024-09-11T12:54:17.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:17.931+0000 7f5c01000640 1 -- 172.21.15.78:0/860169824 wait complete. 2024-09-11T12:54:17.955 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:54:17.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:18.050 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:18.154 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:54:18.155 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:18.155 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:18.163 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:54:18.163 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:18.163 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:18.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:54:18.173 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:18.173 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:18.211 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:54:18.211 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:18.211 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:18.224 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:54:18.224 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:54:18.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.387+0000 7f05f7a00640 1 -- 172.21.15.78:0/332823133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f00cf760 msgr2=0x7f05f0074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.387+0000 7f05f7a00640 1 --2- 172.21.15.78:0/332823133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f00cf760 0x7f05f0074260 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009bc0 tx=0x7f05e001cb70 comp rx=0 tx=0).stop 2024-09-11T12:54:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- 172.21.15.78:0/332823133 shutdown_connections 2024-09-11T12:54:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 --2- 172.21.15.78:0/332823133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f00cf760 0x7f05f0074260 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- 172.21.15.78:0/332823133 >> 172.21.15.78:0/332823133 conn(0x7f05f006ca80 msgr2=0x7f05f00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- 172.21.15.78:0/332823133 shutdown_connections 2024-09-11T12:54:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- 172.21.15.78:0/332823133 wait complete. 2024-09-11T12:54:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 Processor -- start 2024-09-11T12:54:18.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- start start 2024-09-11T12:54:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 0x7f05f00b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 0x7f05f01f9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f05f01519a0 con 0x7f05f006fc50 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f05f0151820 con 0x7f05f00cf760 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f05f0151b20 con 0x7f05f015aec0 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52844/0 (socket says 172.21.15.78:52844) 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 -- 172.21.15.78:0/1908726574 learned_addr learned my addr 172.21.15.78:0/1908726574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05efe00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 0x7f05f00b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 -- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 msgr2=0x7f05f01f9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 0x7f05f01f9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 0x7f05f01f9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 -- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 msgr2=0x7f05f00b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 0x7f05f00b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 -- 172.21.15.78:0/1908726574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05f0141e40 con 0x7f05f006fc50 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05efe00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 0x7f05f00b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 0x7f05f01f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f5000640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009b10 tx=0x7f05e00a5bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00bb070 con 0x7f05f006fc50 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05e0004e00 con 0x7f05f006fc50 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e00b2040 con 0x7f05f006fc50 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.391+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05f0142120 con 0x7f05f006fc50 2024-09-11T12:54:18.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05f01426e0 con 0x7f05f006fc50 2024-09-11T12:54:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05f0074240 con 0x7f05f006fc50 2024-09-11T12:54:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f05e0004420 con 0x7f05f006fc50 2024-09-11T12:54:18.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05ecc00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 0x7f05d0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f05e012bd20 con 0x7f05f006fc50 2024-09-11T12:54:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05efe00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 0x7f05d0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.395+0000 7f05efe00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 0x7f05d0073c60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05f0148920 tx=0x7f05dc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:18.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.399+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f05e0016790 con 0x7f05f006fc50 2024-09-11T12:54:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.599+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05f014e3b0 con 0x7f05f006fc50 2024-09-11T12:54:18.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.599+0000 7f05ecc00640 1 -- 172.21.15.78:0/1908726574 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) ==== 94+0+3917 (secure 0 0 0) 0x7f05e00f7d10 con 0x7f05f006fc50 2024-09-11T12:54:18.603 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:18.603 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":64,"btime":"2024-09-11T12:54:12:040634+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6262,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3685111978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3685111978},{"type":"v1","addr":"172.21.15.157:6801","nonce":3685111978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6265,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3712922497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":3712922497},{"type":"v1","addr":"172.21.15.157:6835","nonce":3712922497}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6487,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3137515312","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":3137515312},{"type":"v1","addr":"172.21.15.78:6835","nonce":3137515312}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":64},{"gid":6759,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/2256690013","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":2256690013},{"type":"v1","addr":"172.21.15.78:6833","nonce":2256690013}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":62}],"filesystems":[]} 2024-09-11T12:54:18.603 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 64 2024-09-11T12:54:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 msgr2=0x7f05d0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 0x7f05d0073c60 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f05f0148920 tx=0x7f05dc009040 comp rx=0 tx=0).stop 2024-09-11T12:54:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 msgr2=0x7f05f00b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f05e0009b10 tx=0x7f05e00a5bc0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 shutdown_connections 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f05f015aec0 0x7f05f01f9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f05d00717a0 0x7f05d0073c60 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f05f00cf760 0x7f05f00b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 --2- 172.21.15.78:0/1908726574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f05f006fc50 0x7f05f00b2ec0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 >> 172.21.15.78:0/1908726574 conn(0x7f05f006ca80 msgr2=0x7f05f015bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 shutdown_connections 2024-09-11T12:54:18.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.603+0000 7f05f7a00640 1 -- 172.21.15.78:0/1908726574 wait complete. 2024-09-11T12:54:18.625 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:54:18.625 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:54:18.625 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:54:18.625 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:54:18.626 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:54:18.626 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:54:18.626 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:54:18.626 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:54:18.626 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:54:18.626 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:54:18.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:54:18.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:54:18.627 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:54:18.627 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:54:18.627 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:54:18.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:54:18.637 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:54:18.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:54:18.639 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:54:18.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:54:18.643 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:54:18.643 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:54:18.644 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:54:18.644 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:54:18.685 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:54:18.686 DEBUG:teuthology.parallel:result is None 2024-09-11T12:54:18.687 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:54:18.687 DEBUG:teuthology.parallel:result is None 2024-09-11T12:54:18.687 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:54:18.688 DEBUG:teuthology.parallel:result is None 2024-09-11T12:54:18.688 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:54:18.688 DEBUG:teuthology.parallel:result is None 2024-09-11T12:54:18.689 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:54:18.772 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:54:18.772 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:54:18.780 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:54:18.781 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 -- 172.21.15.78:0/2220421553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 msgr2=0x7f704c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 --2- 172.21.15.78:0/2220421553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c073f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f703c009870 tx=0x7f703c01c820 comp rx=0 tx=0).stop 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 -- 172.21.15.78:0/2220421553 shutdown_connections 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 --2- 172.21.15.78:0/2220421553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c073f70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 -- 172.21.15.78:0/2220421553 >> 172.21.15.78:0/2220421553 conn(0x7f704c06ca80 msgr2=0x7f704c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 -- 172.21.15.78:0/2220421553 shutdown_connections 2024-09-11T12:54:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.835+0000 7f7053a00640 1 -- 172.21.15.78:0/2220421553 wait complete. 2024-09-11T12:54:18.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 Processor -- start 2024-09-11T12:54:18.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- start start 2024-09-11T12:54:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f704c15bc60 0x7f704c1f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f704c169db0 con 0x7f704c0ccb20 2024-09-11T12:54:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f704c169c30 con 0x7f704c0b4780 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f704c169f30 con 0x7f704c15bc60 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52878/0 (socket says 172.21.15.78:52878) 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 -- 172.21.15.78:0/970985598 learned_addr learned my addr 172.21.15.78:0/970985598 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7051000640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7051a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f704c15bc60 0x7f704c1f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 -- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f704c15bc60 msgr2=0x7f704c1f8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f704c15bc60 0x7f704c1f8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 -- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 msgr2=0x7f704c144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 -- 172.21.15.78:0/970985598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c147950 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f704be00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f703800c4b0 tx=0x7f703800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380052d0 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c150ab0 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7038005470 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038012870 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c151070 con 0x7f704c0ccb20 2024-09-11T12:54:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.839+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c073f70 con 0x7f704c0ccb20 2024-09-11T12:54:18.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.843+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f7038012a10 con 0x7f704c0ccb20 2024-09-11T12:54:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.843+0000 7f7048c00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 0x7f702c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.843+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5029+0+0 (secure 0 0 0) 0x7f70380919f0 con 0x7f704c0ccb20 2024-09-11T12:54:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.843+0000 7f7051000640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 0x7f702c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:18.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.843+0000 7f7051000640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 0x7f702c073c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f703c0af000 tx=0x7f703c01b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:18.847+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f703805d9e0 con 0x7f704c0ccb20 2024-09-11T12:54:19.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.023+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f704c15cf60 con 0x7f704c0ccb20 2024-09-11T12:54:19.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.247+0000 7f7048c00640 1 -- 172.21.15.78:0/970985598 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v95) ==== 97+0+0 (secure 0 0 0) 0x7f703805d2e0 con 0x7f704c0ccb20 2024-09-11T12:54:19.249 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:54:19.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.247+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 msgr2=0x7f702c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.247+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 0x7f702c073c60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f703c0af000 tx=0x7f703c01b060 comp rx=0 tx=0).stop 2024-09-11T12:54:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.247+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 msgr2=0x7f704c1f84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.247+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f703800c4b0 tx=0x7f703800c970 comp rx=0 tx=0).stop 2024-09-11T12:54:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 shutdown_connections 2024-09-11T12:54:19.252 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f704c15bc60 0x7f704c1f8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f704c0ccb20 0x7f704c1f84d0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f702c0717a0 0x7f702c073c60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 --2- 172.21.15.78:0/970985598 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f704c0b4780 0x7f704c144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 >> 172.21.15.78:0/970985598 conn(0x7f704c06ca80 msgr2=0x7f704c15c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 shutdown_connections 2024-09-11T12:54:19.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.251+0000 7f7053a00640 1 -- 172.21.15.78:0/970985598 wait complete. 2024-09-11T12:54:19.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:54:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- 172.21.15.78:0/939269656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e0004090 msgr2=0x7f83e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 --2- 172.21.15.78:0/939269656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83e0004090 0x7f83e0004810 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f83e800bb10 tx=0x7f83e801cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f83fb400640 1 -- 172.21.15.78:0/939269656 reap_dead start 2024-09-11T12:54:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- 172.21.15.78:0/939269656 shutdown_connections 2024-09-11T12:54:19.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- 172.21.15.78:0/939269656 >> 172.21.15.78:0/939269656 conn(0x7f83fc06ca80 msgr2=0x7f83fc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- 172.21.15.78:0/939269656 shutdown_connections 2024-09-11T12:54:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- 172.21.15.78:0/939269656 wait complete. 2024-09-11T12:54:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 Processor -- start 2024-09-11T12:54:19.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.415+0000 7f8401e00640 1 -- start start 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e0004090 0x7f83fc1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83fc15e2d0 0x7f83fc0af3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc15b350 con 0x7f83fc1595b0 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83fc15b1d0 con 0x7f83fc15e2d0 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83fc15b4d0 con 0x7f83e0004090 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52892/0 (socket says 172.21.15.78:52892) 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 -- 172.21.15.78:0/1012374293 learned_addr learned my addr 172.21.15.78:0/1012374293 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83faa00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e0004090 0x7f83fc1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fb400640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83fc15e2d0 0x7f83fc0af3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 -- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e0004090 msgr2=0x7f83fc1f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e0004090 0x7f83fc1f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 -- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83fc15e2d0 msgr2=0x7f83fc0af3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83fc15e2d0 0x7f83fc0af3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 -- 172.21.15.78:0/1012374293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83fc06f080 con 0x7f83fc1595b0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83fa000640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f83e801cbd0 tx=0x7f83e80058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80be070 con 0x7f83fc1595b0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83e8005130 con 0x7f83fc1595b0 2024-09-11T12:54:19.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83fc0769e0 con 0x7f83fc1595b0 2024-09-11T12:54:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e80ab050 con 0x7f83fc1595b0 2024-09-11T12:54:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83fc158510 con 0x7f83fc1595b0 2024-09-11T12:54:19.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.419+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83fc070a00 con 0x7f83fc1595b0 2024-09-11T12:54:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.423+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f83e80c1070 con 0x7f83fc1595b0 2024-09-11T12:54:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.423+0000 7f83f3400640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 0x7f83dc073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.423+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f83e812bb90 con 0x7f83fc1595b0 2024-09-11T12:54:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.423+0000 7f83faa00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 0x7f83dc073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.423+0000 7f83faa00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 0x7f83dc073c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83fc15a440 tx=0x7f83ec00d490 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:19.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.427+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f83e8016ab0 con 0x7f83fc1595b0 2024-09-11T12:54:19.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f83fc0aeb80 con 0x7f83fc1595b0 2024-09-11T12:54:19.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f83f3400640 1 -- 172.21.15.78:0/1012374293 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7f83e80f74c0 con 0x7f83fc1595b0 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:54:19.657 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.658 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.659 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:54:19.661 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.662 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:54:19.663 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:54:19.665 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:54:19.667 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:54:19.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:19.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:54:19.670 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 msgr2=0x7f83dc073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 0x7f83dc073c90 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f83fc15a440 tx=0x7f83ec00d490 comp rx=0 tx=0).stop 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 msgr2=0x7f83fc15dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f83e801cbd0 tx=0x7f83e80058a0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 shutdown_connections 2024-09-11T12:54:19.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83dc0717d0 0x7f83dc073c90 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83fc15e2d0 0x7f83fc0af3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83fc1595b0 0x7f83fc15dd90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 --2- 172.21.15.78:0/1012374293 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83e0004090 0x7f83fc1f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 >> 172.21.15.78:0/1012374293 conn(0x7f83fc06ca80 msgr2=0x7f83fc162f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 shutdown_connections 2024-09-11T12:54:19.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.655+0000 7f8401e00640 1 -- 172.21.15.78:0/1012374293 wait complete. 2024-09-11T12:54:19.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:54:19.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- 172.21.15.78:0/1542154277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f0003a10 msgr2=0x7fe8f0003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 --2- 172.21.15.78:0/1542154277 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe8f0003a10 0x7fe8f0003e30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00bb10 tx=0x7fe8fc01cde0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90be00640 1 -- 172.21.15.78:0/1542154277 reap_dead start 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- 172.21.15.78:0/1542154277 shutdown_connections 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- 172.21.15.78:0/1542154277 >> 172.21.15.78:0/1542154277 conn(0x7fe90c06ca80 msgr2=0x7fe90c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- 172.21.15.78:0/1542154277 shutdown_connections 2024-09-11T12:54:19.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- 172.21.15.78:0/1542154277 wait complete. 2024-09-11T12:54:19.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 Processor -- start 2024-09-11T12:54:19.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- start start 2024-09-11T12:54:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f0003a10 0x7fe90c140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90c156030 0x7fe90c1511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c15b320 con 0x7fe90c1f5f10 2024-09-11T12:54:19.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe90c15b1a0 con 0x7fe90c156030 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe912800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe90c15b4a0 con 0x7fe8f0003a10 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52900/0 (socket says 172.21.15.78:52900) 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 -- 172.21.15.78:0/3846100912 learned_addr learned my addr 172.21.15.78:0/3846100912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90b400640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f0003a10 0x7fe90c140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90be00640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90c156030 0x7fe90c1511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 -- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f0003a10 msgr2=0x7fe90c140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f0003a10 0x7fe90c140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 -- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90c156030 msgr2=0x7fe90c1511a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90c156030 0x7fe90c1511a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.819+0000 7fe90aa00640 1 -- 172.21.15.78:0/3846100912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90c154a50 con 0x7fe90c1f5f10 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe90aa00640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800fa30 tx=0x7fe8f800fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:19.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800bcb0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8f8004020 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8004450 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90c154d30 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90c1552f0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe90c1541a0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fe8f8019030 con 0x7fe90c1f5f10 2024-09-11T12:54:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 0x7fe8ec073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7fe8f80922e0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.823+0000 7fe90b400640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 0x7fe8ec073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:19.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.827+0000 7fe90b400640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 0x7fe8ec073c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0b0000 tx=0x7fe8fc0a7040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:19.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.827+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe8f805db90 con 0x7fe90c1f5f10 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.995+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe90c0709b0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.995+0000 7fe903400640 1 -- 172.21.15.78:0/3846100912 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) ==== 84+0+65 (secure 0 0 0) 0x7fe8f80620e0 con 0x7fe90c1f5f10 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:19.998 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:54:20.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 msgr2=0x7fe8ec073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:20.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 0x7fe8ec073c90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0b0000 tx=0x7fe8fc0a7040 comp rx=0 tx=0).stop 2024-09-11T12:54:20.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 msgr2=0x7fe90c155af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:20.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fe8f800fa30 tx=0x7fe8f800fe30 comp rx=0 tx=0).stop 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 shutdown_connections 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8ec0717d0 0x7fe8ec073c90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90c156030 0x7fe90c1511a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe90c1f5f10 0x7fe90c155af0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 --2- 172.21.15.78:0/3846100912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe8f0003a10 0x7fe90c140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 >> 172.21.15.78:0/3846100912 conn(0x7fe90c06ca80 msgr2=0x7fe90c153680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 shutdown_connections 2024-09-11T12:54:20.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:19.999+0000 7fe912800640 1 -- 172.21.15.78:0/3846100912 wait complete. 2024-09-11T12:54:20.020 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:54:20.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- 172.21.15.78:0/1447638843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 msgr2=0x7f239c151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 --2- 172.21.15.78:0/1447638843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c151a50 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2394009900 tx=0x7f239401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- 172.21.15.78:0/1447638843 shutdown_connections 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 --2- 172.21.15.78:0/1447638843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c151a50 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- 172.21.15.78:0/1447638843 >> 172.21.15.78:0/1447638843 conn(0x7f239c0b3050 msgr2=0x7f239c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- 172.21.15.78:0/1447638843 shutdown_connections 2024-09-11T12:54:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- 172.21.15.78:0/1447638843 wait complete. 2024-09-11T12:54:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 Processor -- start 2024-09-11T12:54:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.163+0000 7f23a3a00640 1 -- start start 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c0b5790 0x7f239c0afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f239c0b5b70 0x7f239c0b00a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f239c169050 con 0x7f239c072b00 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f239c168ed0 con 0x7f239c0b5790 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f239c1691d0 con 0x7f239c0b5b70 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52920/0 (socket says 172.21.15.78:52920) 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 -- 172.21.15.78:0/2261257957 learned_addr learned my addr 172.21.15.78:0/2261257957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f239c0b5b70 0x7f239c0b00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f239be00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c0b5790 0x7f239c0afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 -- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f239c0b5b70 msgr2=0x7f239c0b00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f239c0b5b70 0x7f239c0b00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 -- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c0b5790 msgr2=0x7f239c0afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c0b5790 0x7f239c0afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 -- 172.21.15.78:0/2261257957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f239c14e100 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a1000640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c870 tx=0x7f238c00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c003b80 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f238c003d20 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c00aab0 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f239c14e3e0 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f239c152330 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.167+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f239c0723f0 con 0x7f239c072b00 2024-09-11T12:54:20.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.171+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f238c00ac50 con 0x7f239c072b00 2024-09-11T12:54:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.171+0000 7f2398c00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 0x7f237c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.171+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f238c091b30 con 0x7f239c072b00 2024-09-11T12:54:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.171+0000 7f239be00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 0x7f237c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:20.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.171+0000 7f239be00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 0x7f237c073c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f239c14ee30 tx=0x7f2388009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:20.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.175+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f238c05dbd0 con 0x7f239c072b00 2024-09-11T12:54:20.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:20.347+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f239c15e290 con 0x7f239c072b00 2024-09-11T12:54:21.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.255+0000 7f2398c00640 1 -- 172.21.15.78:0/2261257957 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) ==== 138+0+0 (secure 0 0 0) 0x7f238c05d4d0 con 0x7f239c072b00 2024-09-11T12:54:21.257 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 msgr2=0x7f237c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 0x7f237c073c60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f239c14ee30 tx=0x7f2388009040 comp rx=0 tx=0).stop 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 msgr2=0x7f239c13c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f238c00c870 tx=0x7f238c00cd30 comp rx=0 tx=0).stop 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 shutdown_connections 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f239c0b5b70 0x7f239c0b00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f237c0717a0 0x7f237c073c60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f239c0b5790 0x7f239c0afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 --2- 172.21.15.78:0/2261257957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f239c072b00 0x7f239c13c700 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 >> 172.21.15.78:0/2261257957 conn(0x7f239c0b3050 msgr2=0x7f239c06ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 shutdown_connections 2024-09-11T12:54:21.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.259+0000 7f23a3a00640 1 -- 172.21.15.78:0/2261257957 wait complete. 2024-09-11T12:54:21.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- 172.21.15.78:0/2144702548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 msgr2=0x7f54f015c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- 172.21.15.78:0/2144702548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f015c980 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f54f015c050 tx=0x7f54ec01ccf0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- 172.21.15.78:0/2144702548 shutdown_connections 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- 172.21.15.78:0/2144702548 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f015c980 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- 172.21.15.78:0/2144702548 >> v2:172.21.15.157:3300/0 conn(0x7f54f00701b0 0x7f54f015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- 172.21.15.78:0/2144702548 >> 172.21.15.78:0/2144702548 conn(0x7f54f00b2ec0 msgr2=0x7f54f00b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- 172.21.15.78:0/2144702548 shutdown_connections 2024-09-11T12:54:21.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- 172.21.15.78:0/2144702548 wait complete. 2024-09-11T12:54:21.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 Processor -- start 2024-09-11T12:54:21.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- start start 2024-09-11T12:54:21.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f00701b0 0x7f54f0159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f015dfc0 0x7f54f014eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01680d0 con 0x7f54f0072e70 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0168250 con 0x7f54f015dfc0 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f8800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f54f01683d0 con 0x7f54f00701b0 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52940/0 (socket says 172.21.15.78:52940) 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f5400640 1 -- 172.21.15.78:0/559240783 learned_addr learned my addr 172.21.15.78:0/559240783 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f6800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f015dfc0 0x7f54f014eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:21.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.439+0000 7f54f5e00640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f00701b0 0x7f54f0159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 -- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f00701b0 msgr2=0x7f54f0159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f00701b0 0x7f54f0159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 -- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f015dfc0 msgr2=0x7f54f014eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f015dfc0 0x7f54f014eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 -- 172.21.15.78:0/559240783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f01fbf20 con 0x7f54f0072e70 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f5400640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c930 tx=0x7f54e000cdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e0007c70 con 0x7f54f0072e70 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e000f450 con 0x7f54f0072e70 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e000f8a0 con 0x7f54f0072e70 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f0149420 con 0x7f54f0072e70 2024-09-11T12:54:21.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01499e0 con 0x7f54f0072e70 2024-09-11T12:54:21.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54f006faa0 con 0x7f54f0072e70 2024-09-11T12:54:21.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f54e000fa40 con 0x7f54f0072e70 2024-09-11T12:54:21.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.443+0000 7f54ea000640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 0x7f54d4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:21.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.447+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5227+0+0 (secure 0 0 0) 0x7f54e0092b40 con 0x7f54f0072e70 2024-09-11T12:54:21.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.447+0000 7f54f5e00640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 0x7f54d4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:21.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.447+0000 7f54f5e00640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 0x7f54d4073c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f54e400b810 tx=0x7f54e4008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:21.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.447+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f54e0027080 con 0x7f54f0072e70 2024-09-11T12:54:21.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:21.615+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f54f00b16e0 con 0x7f54f0072e70 2024-09-11T12:54:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54ea000640 1 -- 172.21.15.78:0/559240783 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) ==== 170+0+0 (secure 0 0 0) 0x7f54e0014030 con 0x7f54f0072e70 2024-09-11T12:54:22.260 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 msgr2=0x7f54d4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 0x7f54d4073c60 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f54e400b810 tx=0x7f54e4008040 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 msgr2=0x7f54f01fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f54e000c930 tx=0x7f54e000cdf0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 shutdown_connections 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f54d40717a0 0x7f54d4073c60 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f54f015dfc0 0x7f54f014eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f54f0072e70 0x7f54f01fb9e0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 --2- 172.21.15.78:0/559240783 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f54f00701b0 0x7f54f0159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 >> 172.21.15.78:0/559240783 conn(0x7f54f00b2ec0 msgr2=0x7f54f015c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 shutdown_connections 2024-09-11T12:54:22.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.259+0000 7f54f8800640 1 -- 172.21.15.78:0/559240783 wait complete. 2024-09-11T12:54:22.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- 172.21.15.78:0/2686409101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 msgr2=0x7fbf040b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- 172.21.15.78:0/2686409101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf040b3040 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7fbf0406d1a0 tx=0x7fbef001c890 comp rx=0 tx=0).stop 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- 172.21.15.78:0/2686409101 shutdown_connections 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- 172.21.15.78:0/2686409101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf040b3040 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- 172.21.15.78:0/2686409101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040b1f10 0x7fbf040cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- 172.21.15.78:0/2686409101 >> v2:172.21.15.157:3300/0 conn(0x7fbf040cf6e0 0x7fbf0415a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- 172.21.15.78:0/2686409101 >> 172.21.15.78:0/2686409101 conn(0x7fbf04071dd0 msgr2=0x7fbf040721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- 172.21.15.78:0/2686409101 shutdown_connections 2024-09-11T12:54:22.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- 172.21.15.78:0/2686409101 wait complete. 2024-09-11T12:54:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 Processor -- start 2024-09-11T12:54:22.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- start start 2024-09-11T12:54:22.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 0x7fbf041fa840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 0x7fbf041fad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040af050 con 0x7fbf040b1a80 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040aeed0 con 0x7fbf040b1f10 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040af1d0 con 0x7fbf040cf6e0 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf09c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf09c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52960/0 (socket says 172.21.15.78:52960) 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf09c00640 1 -- 172.21.15.78:0/1866979335 learned_addr learned my addr 172.21.15.78:0/1866979335 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf0a600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 0x7fbf041fad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.443+0000 7fbf09200640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 0x7fbf041fa840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 -- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 msgr2=0x7fbf041fad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 0x7fbf041fad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 -- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 msgr2=0x7fbf041fa840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 0x7fbf041fa840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 -- 172.21.15.78:0/1866979335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0415eb00 con 0x7fbf040b1a80 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf0a600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 0x7fbf041fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09200640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 0x7fbf041fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf09c00640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbef800ea40 tx=0x7fbef800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef800ce40 con 0x7fbf040b1a80 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef8004020 con 0x7fbf040b1a80 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8004470 con 0x7fbf040b1a80 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0415eda0 con 0x7fbf040b1a80 2024-09-11T12:54:22.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0415f360 con 0x7fbf040b1a80 2024-09-11T12:54:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0415f9b0 con 0x7fbf040b1a80 2024-09-11T12:54:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fbef8003600 con 0x7fbf040b1a80 2024-09-11T12:54:22.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.447+0000 7fbf01600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 0x7fbee4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:22.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.451+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5609+0+0 (secure 0 0 0) 0x7fbef8092980 con 0x7fbf040b1a80 2024-09-11T12:54:22.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.451+0000 7fbf09200640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 0x7fbee4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:22.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.451+0000 7fbf09200640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 0x7fbee4073c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef001b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:22.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.455+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbef805e620 con 0x7fbf040b1a80 2024-09-11T12:54:22.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:22.655+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbf0415f7a0 con 0x7fbf040b1a80 2024-09-11T12:54:23.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.283+0000 7fbf01600640 1 -- 172.21.15.78:0/1866979335 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v67) ==== 174+0+0 (secure 0 0 0) 0x7fbef805df20 con 0x7fbf040b1a80 2024-09-11T12:54:23.291 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 16 and data pool 17 2024-09-11T12:54:23.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.291+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 msgr2=0x7fbee4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.291+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 0x7fbee4073c60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbef00af000 tx=0x7fbef001b0d0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.291+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 msgr2=0x7fbf04140ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.291+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7fbef800ea40 tx=0x7fbef800ef00 comp rx=0 tx=0).stop 2024-09-11T12:54:23.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 shutdown_connections 2024-09-11T12:54:23.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbf040cf6e0 0x7fbf041fad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbee40717a0 0x7fbee4073c60 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbf040b1f10 0x7fbf041fa840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 --2- 172.21.15.78:0/1866979335 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbf040b1a80 0x7fbf04140ec0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 >> 172.21.15.78:0/1866979335 conn(0x7fbf04071dd0 msgr2=0x7fbf04143590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:23.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 shutdown_connections 2024-09-11T12:54:23.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.295+0000 7fbf0c600640 1 -- 172.21.15.78:0/1866979335 wait complete. 2024-09-11T12:54:23.335 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:23.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 -- 172.21.15.78:0/1173672266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c015fc70 msgr2=0x7fa3c0072a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/1173672266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c015fc70 0x7fa3c0072a40 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009900 tx=0x7fa3ac01c850 comp rx=0 tx=0).stop 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 -- 172.21.15.78:0/1173672266 shutdown_connections 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/1173672266 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c015fc70 0x7fa3c0072a40 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/1173672266 >> v2:172.21.15.157:3300/0 conn(0x7fa3c0072660 0x7fa3c0070600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 -- 172.21.15.78:0/1173672266 >> 172.21.15.78:0/1173672266 conn(0x7fa3c0073770 msgr2=0x7fa3c0073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 -- 172.21.15.78:0/1173672266 shutdown_connections 2024-09-11T12:54:23.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.475+0000 7fa3c6c00640 1 -- 172.21.15.78:0/1173672266 wait complete. 2024-09-11T12:54:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 Processor -- start 2024-09-11T12:54:23.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- start start 2024-09-11T12:54:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c0072660 0x7fa3c01555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c015fc70 0x7fa3c0148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0168d80 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0168c00 con 0x7fa3c015fc70 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c0168f00 con 0x7fa3c0072660 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52974/0 (socket says 172.21.15.78:52974) 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 -- 172.21.15.78:0/362222112 learned_addr learned my addr 172.21.15.78:0/362222112 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c5800640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c0072660 0x7fa3c01555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6200640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c015fc70 0x7fa3c0148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 -- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c0072660 msgr2=0x7fa3c01555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c0072660 0x7fa3c01555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 -- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c015fc70 msgr2=0x7fa3c0148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c015fc70 0x7fa3c0148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 -- 172.21.15.78:0/362222112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c0159180 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c4e00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009850 tx=0x7fa3ac0a5620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac0b2040 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3ac0042c0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac005030 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c01564f0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c0149030 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.479+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c0070600 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.483+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa3ac0051f0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.483+0000 7fa3bd600640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 0x7fa3a0073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.483+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa3ac12b7a0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.483+0000 7fa3c5800640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 0x7fa3a0073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.483+0000 7fa3c5800640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 0x7fa3a0073c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009be0 tx=0x7fa3b40091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:23.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.487+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa3ac0164d0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.651+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3c006a860 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3bd600640 1 -- 172.21.15.78:0/362222112 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16692 (secure 0 0 0) 0x7fa3ac0f6ea0 con 0x7fa3c00cf6e0 2024-09-11T12:54:23.657 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:23.657 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":100,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:22.659638+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:54:20.362554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-11T12:54:21.629928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:23.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 msgr2=0x7fa3a0073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 0x7fa3a0073c90 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009be0 tx=0x7fa3b40091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 msgr2=0x7fa3c0152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac009850 tx=0x7fa3ac0a5620 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 shutdown_connections 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa3a00717d0 0x7fa3a0073c90 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa3c015fc70 0x7fa3c0148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa3c00cf6e0 0x7fa3c0152d00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 --2- 172.21.15.78:0/362222112 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa3c0072660 0x7fa3c01555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 >> 172.21.15.78:0/362222112 conn(0x7fa3c0073770 msgr2=0x7fa3c0159800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:23.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 shutdown_connections 2024-09-11T12:54:23.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.655+0000 7fa3c6c00640 1 -- 172.21.15.78:0/362222112 wait complete. 2024-09-11T12:54:23.678 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- 172.21.15.78:0/1594010147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15e5b0 msgr2=0x7fa4dc15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- 172.21.15.78:0/1594010147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15e9f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8009900 tx=0x7fa4c801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- 172.21.15.78:0/1594010147 shutdown_connections 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- 172.21.15.78:0/1594010147 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15e9f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- 172.21.15.78:0/1594010147 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 0x7fa4dc15ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- 172.21.15.78:0/1594010147 >> 172.21.15.78:0/1594010147 conn(0x7fa4dc06ca80 msgr2=0x7fa4dc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- 172.21.15.78:0/1594010147 shutdown_connections 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- 172.21.15.78:0/1594010147 wait complete. 2024-09-11T12:54:23.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 Processor -- start 2024-09-11T12:54:23.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- start start 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc06fc50 0x7fa4dc158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 0x7fa4dc142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1680d0 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc167f50 con 0x7fa4dc06fc50 2024-09-11T12:54:23.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4e1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa4dc168250 con 0x7fa4dc074240 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:52990/0 (socket says 172.21.15.78:52990) 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 -- 172.21.15.78:0/929102789 learned_addr learned my addr 172.21.15.78:0/929102789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4d9400640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 0x7fa4dc142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4d9e00640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc06fc50 0x7fa4dc158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 -- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 msgr2=0x7fa4dc142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 0x7fa4dc142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 -- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc06fc50 msgr2=0x7fa4dc158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc06fc50 0x7fa4dc158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:23.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 -- 172.21.15.78:0/929102789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc1f9120 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.819+0000 7fa4da800640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c710 tx=0x7fa4d400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4003b80 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d4003d20 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc1f9400 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d400aab0 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1f99c0 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc15ba80 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa4d400ac50 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 0x7fa4bc073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa4d4091ce0 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.823+0000 7fa4d9e00640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 0x7fa4bc073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:23.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.827+0000 7fa4d9e00640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 0x7fa4bc073c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009be0 tx=0x7fa4cc0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:23.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.827+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa4d400ce20 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.995+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4dc15c350 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.995+0000 7fa4d2000640 1 -- 172.21.15.78:0/929102789 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16692 (secure 0 0 0) 0x7fa4d405d370 con 0x7fa4dc15e5b0 2024-09-11T12:54:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:23.999 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":100,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:22.659638+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:54:20.362554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-11T12:54:21.629928+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 msgr2=0x7fa4bc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 0x7fa4bc073c60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc009be0 tx=0x7fa4cc0091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 msgr2=0x7fa4dc15c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c710 tx=0x7fa4d400cbd0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 shutdown_connections 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa4dc15e5b0 0x7fa4dc15c870 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4dc074240 0x7fa4dc142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa4bc0717a0 0x7fa4bc073c60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 --2- 172.21.15.78:0/929102789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4dc06fc50 0x7fa4dc158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 >> 172.21.15.78:0/929102789 conn(0x7fa4dc06ca80 msgr2=0x7fa4dc15da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 shutdown_connections 2024-09-11T12:54:24.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:23.999+0000 7fa4e1000640 1 -- 172.21.15.78:0/929102789 wait complete. 2024-09-11T12:54:24.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 -- 172.21.15.78:0/2548644167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa13c004220 msgr2=0x7fa13c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 --2- 172.21.15.78:0/2548644167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa13c004220 0x7fa13c004810 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa140009b60 tx=0x7fa14001cfb0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15a000640 1 -- 172.21.15.78:0/2548644167 reap_dead start 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 -- 172.21.15.78:0/2548644167 shutdown_connections 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 -- 172.21.15.78:0/2548644167 >> 172.21.15.78:0/2548644167 conn(0x7fa154071dd0 msgr2=0x7fa1540721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 -- 172.21.15.78:0/2548644167 shutdown_connections 2024-09-11T12:54:24.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.163+0000 7fa15c000640 1 -- 172.21.15.78:0/2548644167 wait complete. 2024-09-11T12:54:24.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 Processor -- start 2024-09-11T12:54:24.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- start start 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa13c004220 0x7fa1541f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1541f4870 0x7fa15414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540cdfa0 con 0x7fa15414cd00 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1540cde20 con 0x7fa1541f4870 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa154070290 con 0x7fa13c004220 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57418/0 (socket says 172.21.15.78:57418) 2024-09-11T12:54:24.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 -- 172.21.15.78:0/3077118883 learned_addr learned my addr 172.21.15.78:0/3077118883 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa159600640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa13c004220 0x7fa1541f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa158c00640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1541f4870 0x7fa15414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 -- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa13c004220 msgr2=0x7fa1541f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa13c004220 0x7fa1541f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 -- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1541f4870 msgr2=0x7fa15414c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1541f4870 0x7fa15414c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 -- 172.21.15.78:0/3077118883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1541f8e30 con 0x7fa15414cd00 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15a000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001230 tx=0x7fa14c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c015070 con 0x7fa15414cd00 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa14c011860 con 0x7fa15414cd00 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c011cb0 con 0x7fa15414cd00 2024-09-11T12:54:24.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1541f4db0 con 0x7fa15414cd00 2024-09-11T12:54:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1541f5370 con 0x7fa15414cd00 2024-09-11T12:54:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.167+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1540707f0 con 0x7fa15414cd00 2024-09-11T12:54:24.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.171+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fa14c018070 con 0x7fa15414cd00 2024-09-11T12:54:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.171+0000 7fa151600640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 0x7fa134073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.171+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fa14c092a40 con 0x7fa15414cd00 2024-09-11T12:54:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.171+0000 7fa159600640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 0x7fa134073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:24.174 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.171+0000 7fa159600640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 0x7fa134073c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1540aee10 tx=0x7fa148006ea0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:24.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.175+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa1541f8e30 con 0x7fa15414cd00 2024-09-11T12:54:24.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:24.379+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa1540b1690 con 0x7fa15414cd00 2024-09-11T12:54:25.258 INFO:tasks.ceph.mds.c.smithi078.stderr:2024-09-11T12:54:25.255+0000 7f19c5a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:54:25.301 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa151600640 1 -- 172.21.15.78:0/3077118883 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) ==== 118+0+0 (secure 0 0 0) 0x7fa14c05e5f0 con 0x7fa15414cd00 2024-09-11T12:54:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 msgr2=0x7fa134073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:25.303 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 0x7fa134073c90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa1540aee10 tx=0x7fa148006ea0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 msgr2=0x7fa1541ee450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa14c001230 tx=0x7fa14c0039d0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 shutdown_connections 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa15414cd00 0x7fa1541ee450 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1340717d0 0x7fa134073c90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1541f4870 0x7fa15414c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 --2- 172.21.15.78:0/3077118883 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa13c004220 0x7fa1541f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 >> 172.21.15.78:0/3077118883 conn(0x7fa154071dd0 msgr2=0x7fa1540b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 shutdown_connections 2024-09-11T12:54:25.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.299+0000 7fa15c000640 1 -- 172.21.15.78:0/3077118883 wait complete. 2024-09-11T12:54:25.325 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:54:25.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- 172.21.15.78:0/1116942946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff30406f290 msgr2=0x7ff3040a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 --2- 172.21.15.78:0/1116942946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff30406f290 0x7ff3040a99b0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009c80 tx=0x7ff2fc01cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- 172.21.15.78:0/1116942946 shutdown_connections 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 --2- 172.21.15.78:0/1116942946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff30406f290 0x7ff3040a99b0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 --2- 172.21.15.78:0/1116942946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3041528e0 0x7ff30415fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- 172.21.15.78:0/1116942946 >> 172.21.15.78:0/1116942946 conn(0x7ff3040745a0 msgr2=0x7ff3040749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:25.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- 172.21.15.78:0/1116942946 shutdown_connections 2024-09-11T12:54:25.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- 172.21.15.78:0/1116942946 wait complete. 2024-09-11T12:54:25.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 Processor -- start 2024-09-11T12:54:25.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.471+0000 7ff30b600640 1 -- start start 2024-09-11T12:54:25.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 0x7ff304141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30415bee0 0x7ff3040b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3041689f0 con 0x7ff3041528e0 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff304168870 con 0x7ff30406f290 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff304168b70 con 0x7ff30415bee0 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57432/0 (socket says 172.21.15.78:57432) 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 -- 172.21.15.78:0/1335509843 learned_addr learned my addr 172.21.15.78:0/1335509843 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff308c00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 0x7ff304141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff309600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30415bee0 0x7ff3040b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 -- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30415bee0 msgr2=0x7ff3040b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30415bee0 0x7ff3040b0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 -- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 msgr2=0x7ff304141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 0x7ff304141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 -- 172.21.15.78:0/1335509843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30414eb80 con 0x7ff3041528e0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff308c00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 0x7ff304141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff303e00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3f0 tx=0x7ff2f000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0017070 con 0x7ff3041528e0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f00053c0 con 0x7ff3041528e0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30414ee60 con 0x7ff3041528e0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0003860 con 0x7ff3041528e0 2024-09-11T12:54:25.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30414f420 con 0x7ff3041528e0 2024-09-11T12:54:25.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3040a99b0 con 0x7ff3041528e0 2024-09-11T12:54:25.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.475+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff2f001a070 con 0x7ff3041528e0 2024-09-11T12:54:25.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.479+0000 7ff300c00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 0x7ff2e4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:25.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.479+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff2f0091290 con 0x7ff3041528e0 2024-09-11T12:54:25.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.479+0000 7ff308c00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 0x7ff2e4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:25.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.479+0000 7ff308c00640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 0x7ff2e4073c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff30414e030 tx=0x7ff2f800b2c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:25.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.483+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff2f005cfe0 con 0x7ff3041528e0 2024-09-11T12:54:25.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:25.683+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff3040a9cb0 con 0x7ff3041528e0 2024-09-11T12:54:26.302 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.299+0000 7ff300c00640 1 -- 172.21.15.78:0/1335509843 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) ==== 122+0+0 (secure 0 0 0) 0x7ff2f0065220 con 0x7ff3041528e0 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 msgr2=0x7ff2e4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 0x7ff2e4073c60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff30414e030 tx=0x7ff2f800b2c0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 msgr2=0x7ff30414e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7ff2f000c3f0 tx=0x7ff2f000c8b0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 shutdown_connections 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff30415bee0 0x7ff3040b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3041528e0 0x7ff30414e0c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff2e40717a0 0x7ff2e4073c60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 --2- 172.21.15.78:0/1335509843 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff30406f290 0x7ff304141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 >> 172.21.15.78:0/1335509843 conn(0x7ff3040745a0 msgr2=0x7ff30415d8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 shutdown_connections 2024-09-11T12:54:26.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.303+0000 7ff30b600640 1 -- 172.21.15.78:0/1335509843 wait complete. 2024-09-11T12:54:26.327 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:54:26.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.467+0000 7fc8e6600640 1 -- 172.21.15.78:0/1997465060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8c4004030 msgr2=0x7fc8c4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.467+0000 7fc8e6600640 1 --2- 172.21.15.78:0/1997465060 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8c4004030 0x7fc8c4004810 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc009b60 tx=0x7fc8cc01d270 comp rx=0 tx=0).stop 2024-09-11T12:54:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.467+0000 7fc8dfe00640 1 -- 172.21.15.78:0/1997465060 reap_dead start 2024-09-11T12:54:26.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- 172.21.15.78:0/1997465060 shutdown_connections 2024-09-11T12:54:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- 172.21.15.78:0/1997465060 >> 172.21.15.78:0/1997465060 conn(0x7fc8e006ca80 msgr2=0x7fc8e00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- 172.21.15.78:0/1997465060 shutdown_connections 2024-09-11T12:54:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- 172.21.15.78:0/1997465060 wait complete. 2024-09-11T12:54:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 Processor -- start 2024-09-11T12:54:26.473 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- start start 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 0x7fc8e014de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 0x7fc8e0150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e015cce0 con 0x7fc8e01457a0 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00b4ed0 con 0x7fc8c4004030 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8e6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e015ce60 con 0x7fc8e0145c20 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8df400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 0x7fc8e014de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8df400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 0x7fc8e014de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:37388/0 (socket says 172.21.15.78:37388) 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8df400640 1 -- 172.21.15.78:0/2030196947 learned_addr learned my addr 172.21.15.78:0/2030196947 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:26.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57436/0 (socket says 172.21.15.78:57436) 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dfe00640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 0x7fc8e0150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 -- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 msgr2=0x7fc8e0150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 0x7fc8e0150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 -- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 msgr2=0x7fc8e014de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 0x7fc8e014de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 -- 172.21.15.78:0/2030196947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e014cb70 con 0x7fc8e01457a0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dfe00640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 0x7fc8e0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.471+0000 7fc8dea00640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0ac000 tx=0x7fc8cc005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc0b1040 con 0x7fc8e01457a0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8cc004310 con 0x7fc8e01457a0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e014ce50 con 0x7fc8e01457a0 2024-09-11T12:54:26.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc009730 con 0x7fc8e01457a0 2024-09-11T12:54:26.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e014d410 con 0x7fc8e01457a0 2024-09-11T12:54:26.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e00cf280 con 0x7fc8e01457a0 2024-09-11T12:54:26.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fc8cc0be060 con 0x7fc8e01457a0 2024-09-11T12:54:26.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.475+0000 7fc8d7400640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 0x7fc8c0073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.479+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fc8cc12bfe0 con 0x7fc8e01457a0 2024-09-11T12:54:26.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.479+0000 7fc8df400640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 0x7fc8c0073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.479+0000 7fc8df400640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 0x7fc8c0073c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00058d0 tx=0x7fc8d00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:26.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.483+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc8cc016f80 con 0x7fc8e01457a0 2024-09-11T12:54:26.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.683+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8e014fe30 con 0x7fc8e01457a0 2024-09-11T12:54:26.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.683+0000 7fc8d7400640 1 -- 172.21.15.78:0/2030196947 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5228 (secure 0 0 0) 0x7fc8cc0f7540 con 0x7fc8e01457a0 2024-09-11T12:54:26.686 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:26.686 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"btime":"2024-09-11T12:54:26:302435+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:26.687 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2024-09-11T12:54:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 msgr2=0x7fc8c0073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 0x7fc8c0073c60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00058d0 tx=0x7fc8d00091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 msgr2=0x7fc8e01523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc0ac000 tx=0x7fc8cc005560 comp rx=0 tx=0).stop 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 shutdown_connections 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8e0145c20 0x7fc8e0150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8e01457a0 0x7fc8e01523b0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8c00717a0 0x7fc8c0073c60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 --2- 172.21.15.78:0/2030196947 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8c4004030 0x7fc8e014de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 >> 172.21.15.78:0/2030196947 conn(0x7fc8e006ca80 msgr2=0x7fc8e00af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 shutdown_connections 2024-09-11T12:54:26.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.687+0000 7fc8e6600640 1 -- 172.21.15.78:0/2030196947 wait complete. 2024-09-11T12:54:26.709 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:26.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 -- 172.21.15.78:0/736847938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 msgr2=0x7fd8981520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 --2- 172.21.15.78:0/736847938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd8981520b0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fd890009c80 tx=0x7fd89001cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 -- 172.21.15.78:0/736847938 shutdown_connections 2024-09-11T12:54:26.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 --2- 172.21.15.78:0/736847938 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd8981520b0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 --2- 172.21.15.78:0/736847938 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd89815f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 -- 172.21.15.78:0/736847938 >> 172.21.15.78:0/736847938 conn(0x7fd8980725a0 msgr2=0x7fd8980729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 -- 172.21.15.78:0/736847938 shutdown_connections 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.847+0000 7fd8a0000640 1 -- 172.21.15.78:0/736847938 wait complete. 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 Processor -- start 2024-09-11T12:54:26.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- start start 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 0x7fd898146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd8980ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd8980b5310 con 0x7fd89815fb10 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd8980b5190 con 0x7fd8980d18a0 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd8980b5490 con 0x7fd898160250 2024-09-11T12:54:26.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57464/0 (socket says 172.21.15.78:57464) 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 -- 172.21.15.78:0/1815977778 learned_addr learned my addr 172.21.15.78:0/1815977778 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89e000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd8980ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89d600640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 0x7fd898146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 -- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 msgr2=0x7fd8980ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd8980ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 -- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 msgr2=0x7fd898146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 0x7fd898146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 -- 172.21.15.78:0/1815977778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8980ced10 con 0x7fd89815fb10 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89e000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd8980ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89d600640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 0x7fd898146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd89cc00640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c9a0 tx=0x7fd88400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:26.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd884007bb0 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd884007e40 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89814cfc0 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88400f6b0 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.851+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89814d580 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8980b59b0 con 0x7fd89815fb10 2024-09-11T12:54:26.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd88400f850 con 0x7fd89815fb10 2024-09-11T12:54:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd895600640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 0x7fd878073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd884092770 con 0x7fd89815fb10 2024-09-11T12:54:26.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd89d600640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 0x7fd878073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:26.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.855+0000 7fd89d600640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 0x7fd878073c60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd88c003e40 tx=0x7fd88c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:26.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:26.859+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd884014030 con 0x7fd89815fb10 2024-09-11T12:54:27.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.027+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd8980b1830 con 0x7fd89815fb10 2024-09-11T12:54:27.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.027+0000 7fd895600640 1 -- 172.21.15.78:0/1815977778 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16679 (secure 0 0 0) 0x7fd88405e3b0 con 0x7fd89815fb10 2024-09-11T12:54:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:27.032 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":101,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:24.284782+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:54:20.362554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-11T12:54:21.629928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 msgr2=0x7fd878073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 0x7fd878073c60 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd88c003e40 tx=0x7fd88c009040 comp rx=0 tx=0).stop 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 msgr2=0x7fd89814c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fd88400c9a0 tx=0x7fd88400ce60 comp rx=0 tx=0).stop 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 shutdown_connections 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd898160250 0x7fd8980ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd89815fb10 0x7fd89814c830 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd8780717a0 0x7fd878073c60 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 --2- 172.21.15.78:0/1815977778 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd8980d18a0 0x7fd898146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 >> 172.21.15.78:0/1815977778 conn(0x7fd8980725a0 msgr2=0x7fd8980b72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 shutdown_connections 2024-09-11T12:54:27.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.031+0000 7fd8a0000640 1 -- 172.21.15.78:0/1815977778 wait complete. 2024-09-11T12:54:27.053 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:54:27.054 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:54:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- 172.21.15.78:0/1757487215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f815e1d0 msgr2=0x7ff6f80ce710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/1757487215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f815e1d0 0x7ff6f80ce710 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ff6e4009900 tx=0x7ff6e401c8b0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- 172.21.15.78:0/1757487215 shutdown_connections 2024-09-11T12:54:27.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/1757487215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f815e1d0 0x7ff6f80ce710 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- 172.21.15.78:0/1757487215 >> 172.21.15.78:0/1757487215 conn(0x7ff6f80cfeb0 msgr2=0x7ff6f80d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- 172.21.15.78:0/1757487215 shutdown_connections 2024-09-11T12:54:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- 172.21.15.78:0/1757487215 wait complete. 2024-09-11T12:54:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 Processor -- start 2024-09-11T12:54:27.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.191+0000 7ff6f7e00640 1 -- start start 2024-09-11T12:54:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 0x7ff6f81515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f815e1d0 0x7ff6f813b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f80b3820 con 0x7ff6f8143290 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff6f80b36a0 con 0x7ff6f815e1d0 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6f80b39a0 con 0x7ff6f815ddf0 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 0x7ff6f81515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57484/0 (socket says 172.21.15.78:57484) 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6a00640 1 -- 172.21.15.78:0/68865711 learned_addr learned my addr 172.21.15.78:0/68865711 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:27.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7400640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f815e1d0 0x7ff6f813b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 -- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f815e1d0 msgr2=0x7ff6f813b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f815e1d0 0x7ff6f813b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 -- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 msgr2=0x7ff6f8150880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 -- 172.21.15.78:0/68865711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6f80af770 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6a00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f6000640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 0x7ff6f81515f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ac000 tx=0x7ff6e4005790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40b2040 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6e4003910 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6f8138ce0 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6e4003d60 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6f81392a0 con 0x7ff6f815ddf0 2024-09-11T12:54:27.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.195+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6f80ce710 con 0x7ff6f815ddf0 2024-09-11T12:54:27.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.199+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff6e40ad440 con 0x7ff6f815ddf0 2024-09-11T12:54:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.199+0000 7ff6eea00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 0x7ff6d8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.199+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff6e412c5d0 con 0x7ff6f815ddf0 2024-09-11T12:54:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.199+0000 7ff6f6a00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 0x7ff6d8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.199+0000 7ff6f6a00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 0x7ff6d8073c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8004470 tx=0x7ff6e8003700 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.203+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff6e40164d0 con 0x7ff6f815ddf0 2024-09-11T12:54:27.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.367+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff6f80afa50 con 0x7ff6d80717a0 2024-09-11T12:54:27.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.371+0000 7ff6eea00640 1 -- 172.21.15.78:0/68865711 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252252 (secure 0 0 0) 0x7ff6e800fce0 con 0x7ff6d80717a0 2024-09-11T12:54:27.376 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:27.378 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 msgr2=0x7ff6d8073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 0x7ff6d8073c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8004470 tx=0x7ff6e8003700 comp rx=0 tx=0).stop 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 msgr2=0x7ff6f81515f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 0x7ff6f81515f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff6e40ac000 tx=0x7ff6e4005790 comp rx=0 tx=0).stop 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 shutdown_connections 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6d80717a0 0x7ff6d8073c60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6f815e1d0 0x7ff6f813b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6f815ddf0 0x7ff6f81515f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 --2- 172.21.15.78:0/68865711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6f8143290 0x7ff6f8150880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 >> 172.21.15.78:0/68865711 conn(0x7ff6f80cfeb0 msgr2=0x7ff6f80b1ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 shutdown_connections 2024-09-11T12:54:27.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.375+0000 7ff6f7e00640 1 -- 172.21.15.78:0/68865711 wait complete. 2024-09-11T12:54:27.397 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-11T12:54:26.479584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260532,"kb_used_data":10984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714188,"statfs":{"total":773094113280,"available":772827328512,"internally_reserved":0,"allocated":11247616,"data_stored":5537388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220044"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586649+0000","last_change":"2024-09-11T12:54:23.314807+0000","last_active":"2024-09-11T12:54:24.586649+0000","last_peered":"2024-09-11T12:54:24.586649+0000","last_clean":"2024-09-11T12:54:24.586649+0000","last_became_active":"2024-09-11T12:54:23.314584+0000","last_became_peered":"2024-09-11T12:54:23.314584+0000","last_unstale":"2024-09-11T12:54:24.586649+0000","last_undegraded":"2024-09-11T12:54:24.586649+0000","last_fullsized":"2024-09-11T12:54:24.586649+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:58:17.027168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296124+0000","last_change":"2024-09-11T12:54:23.293096+0000","last_active":"2024-09-11T12:54:24.296124+0000","last_peered":"2024-09-11T12:54:24.296124+0000","last_clean":"2024-09-11T12:54:24.296124+0000","last_became_active":"2024-09-11T12:54:23.292804+0000","last_became_peered":"2024-09-11T12:54:23.292804+0000","last_unstale":"2024-09-11T12:54:24.296124+0000","last_undegraded":"2024-09-11T12:54:24.296124+0000","last_fullsized":"2024-09-11T12:54:24.296124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:24.159603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586247+0000","last_change":"2024-09-11T12:54:23.303442+0000","last_active":"2024-09-11T12:54:24.586247+0000","last_peered":"2024-09-11T12:54:24.586247+0000","last_clean":"2024-09-11T12:54:24.586247+0000","last_became_active":"2024-09-11T12:54:23.303128+0000","last_became_peered":"2024-09-11T12:54:23.303128+0000","last_unstale":"2024-09-11T12:54:24.586247+0000","last_undegraded":"2024-09-11T12:54:24.586247+0000","last_fullsized":"2024-09-11T12:54:24.586247+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:19.274192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586204+0000","last_change":"2024-09-11T12:54:23.304579+0000","last_active":"2024-09-11T12:54:24.586204+0000","last_peered":"2024-09-11T12:54:24.586204+0000","last_clean":"2024-09-11T12:54:24.586204+0000","last_became_active":"2024-09-11T12:54:23.304323+0000","last_became_peered":"2024-09-11T12:54:23.304323+0000","last_unstale":"2024-09-11T12:54:24.586204+0000","last_undegraded":"2024-09-11T12:54:24.586204+0000","last_fullsized":"2024-09-11T12:54:24.586204+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:49.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296049+0000","last_change":"2024-09-11T12:54:23.304151+0000","last_active":"2024-09-11T12:54:24.296049+0000","last_peered":"2024-09-11T12:54:24.296049+0000","last_clean":"2024-09-11T12:54:24.296049+0000","last_became_active":"2024-09-11T12:54:23.303822+0000","last_became_peered":"2024-09-11T12:54:23.303822+0000","last_unstale":"2024-09-11T12:54:24.296049+0000","last_undegraded":"2024-09-11T12:54:24.296049+0000","last_fullsized":"2024-09-11T12:54:24.296049+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:56.510751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586135+0000","last_change":"2024-09-11T12:54:23.307753+0000","last_active":"2024-09-11T12:54:24.586135+0000","last_peered":"2024-09-11T12:54:24.586135+0000","last_clean":"2024-09-11T12:54:24.586135+0000","last_became_active":"2024-09-11T12:54:23.307433+0000","last_became_peered":"2024-09-11T12:54:23.307433+0000","last_unstale":"2024-09-11T12:54:24.586135+0000","last_undegraded":"2024-09-11T12:54:24.586135+0000","last_fullsized":"2024-09-11T12:54:24.586135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:26.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.294971+0000","last_change":"2024-09-11T12:54:23.295519+0000","last_active":"2024-09-11T12:54:24.294971+0000","last_peered":"2024-09-11T12:54:24.294971+0000","last_clean":"2024-09-11T12:54:24.294971+0000","last_became_active":"2024-09-11T12:54:23.295230+0000","last_became_peered":"2024-09-11T12:54:23.295230+0000","last_unstale":"2024-09-11T12:54:24.294971+0000","last_undegraded":"2024-09-11T12:54:24.294971+0000","last_fullsized":"2024-09-11T12:54:24.294971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:23.920249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586219+0000","last_change":"2024-09-11T12:54:23.307624+0000","last_active":"2024-09-11T12:54:24.586219+0000","last_peered":"2024-09-11T12:54:24.586219+0000","last_clean":"2024-09-11T12:54:24.586219+0000","last_became_active":"2024-09-11T12:54:23.307332+0000","last_became_peered":"2024-09-11T12:54:23.307332+0000","last_unstale":"2024-09-11T12:54:24.586219+0000","last_undegraded":"2024-09-11T12:54:24.586219+0000","last_fullsized":"2024-09-11T12:54:24.586219+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:28.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586822+0000","last_change":"2024-09-11T12:54:23.308884+0000","last_active":"2024-09-11T12:54:24.586822+0000","last_peered":"2024-09-11T12:54:24.586822+0000","last_clean":"2024-09-11T12:54:24.586822+0000","last_became_active":"2024-09-11T12:54:23.308365+0000","last_became_peered":"2024-09-11T12:54:23.308365+0000","last_unstale":"2024-09-11T12:54:24.586822+0000","last_undegraded":"2024-09-11T12:54:24.586822+0000","last_fullsized":"2024-09-11T12:54:24.586822+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:37:35.357853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586795+0000","last_change":"2024-09-11T12:54:23.306835+0000","last_active":"2024-09-11T12:54:24.586795+0000","last_peered":"2024-09-11T12:54:24.586795+0000","last_clean":"2024-09-11T12:54:24.586795+0000","last_became_active":"2024-09-11T12:54:23.306375+0000","last_became_peered":"2024-09-11T12:54:23.306375+0000","last_unstale":"2024-09-11T12:54:24.586795+0000","last_undegraded":"2024-09-11T12:54:24.586795+0000","last_fullsized":"2024-09-11T12:54:24.586795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:46.188989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296044+0000","last_change":"2024-09-11T12:54:23.304969+0000","last_active":"2024-09-11T12:54:24.296044+0000","last_peered":"2024-09-11T12:54:24.296044+0000","last_clean":"2024-09-11T12:54:24.296044+0000","last_became_active":"2024-09-11T12:54:23.303641+0000","last_became_peered":"2024-09-11T12:54:23.303641+0000","last_unstale":"2024-09-11T12:54:24.296044+0000","last_undegraded":"2024-09-11T12:54:24.296044+0000","last_fullsized":"2024-09-11T12:54:24.296044+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:49:58.007081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586398+0000","last_change":"2024-09-11T12:54:23.304023+0000","last_active":"2024-09-11T12:54:24.586398+0000","last_peered":"2024-09-11T12:54:24.586398+0000","last_clean":"2024-09-11T12:54:24.586398+0000","last_became_active":"2024-09-11T12:54:23.303678+0000","last_became_peered":"2024-09-11T12:54:23.303678+0000","last_unstale":"2024-09-11T12:54:24.586398+0000","last_undegraded":"2024-09-11T12:54:24.586398+0000","last_fullsized":"2024-09-11T12:54:24.586398+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:33.333132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586191+0000","last_change":"2024-09-11T12:54:23.307978+0000","last_active":"2024-09-11T12:54:24.586191+0000","last_peered":"2024-09-11T12:54:24.586191+0000","last_clean":"2024-09-11T12:54:24.586191+0000","last_became_active":"2024-09-11T12:54:23.306930+0000","last_became_peered":"2024-09-11T12:54:23.306930+0000","last_unstale":"2024-09-11T12:54:24.586191+0000","last_undegraded":"2024-09-11T12:54:24.586191+0000","last_fullsized":"2024-09-11T12:54:24.586191+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585917+0000","last_change":"2024-09-11T12:54:23.303383+0000","last_active":"2024-09-11T12:54:24.585917+0000","last_peered":"2024-09-11T12:54:24.585917+0000","last_clean":"2024-09-11T12:54:24.585917+0000","last_became_active":"2024-09-11T12:54:23.303058+0000","last_became_peered":"2024-09-11T12:54:23.303058+0000","last_unstale":"2024-09-11T12:54:24.585917+0000","last_undegraded":"2024-09-11T12:54:24.585917+0000","last_fullsized":"2024-09-11T12:54:24.585917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586065+0000","last_change":"2024-09-11T12:54:23.303864+0000","last_active":"2024-09-11T12:54:24.586065+0000","last_peered":"2024-09-11T12:54:24.586065+0000","last_clean":"2024-09-11T12:54:24.586065+0000","last_became_active":"2024-09-11T12:54:23.299676+0000","last_became_peered":"2024-09-11T12:54:23.299676+0000","last_unstale":"2024-09-11T12:54:24.586065+0000","last_undegraded":"2024-09-11T12:54:24.586065+0000","last_fullsized":"2024-09-11T12:54:24.586065+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:30.412135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586051+0000","last_change":"2024-09-11T12:54:23.310421+0000","last_active":"2024-09-11T12:54:24.586051+0000","last_peered":"2024-09-11T12:54:24.586051+0000","last_clean":"2024-09-11T12:54:24.586051+0000","last_became_active":"2024-09-11T12:54:23.310138+0000","last_became_peered":"2024-09-11T12:54:23.310138+0000","last_unstale":"2024-09-11T12:54:24.586051+0000","last_undegraded":"2024-09-11T12:54:24.586051+0000","last_fullsized":"2024-09-11T12:54:24.586051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:45.456278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296274+0000","last_change":"2024-09-11T12:54:23.300135+0000","last_active":"2024-09-11T12:54:24.296274+0000","last_peered":"2024-09-11T12:54:24.296274+0000","last_clean":"2024-09-11T12:54:24.296274+0000","last_became_active":"2024-09-11T12:54:23.299857+0000","last_became_peered":"2024-09-11T12:54:23.299857+0000","last_unstale":"2024-09-11T12:54:24.296274+0000","last_undegraded":"2024-09-11T12:54:24.296274+0000","last_fullsized":"2024-09-11T12:54:24.296274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302808+0000","last_change":"2024-09-11T12:54:23.300798+0000","last_active":"2024-09-11T12:54:24.302808+0000","last_peered":"2024-09-11T12:54:24.302808+0000","last_clean":"2024-09-11T12:54:24.302808+0000","last_became_active":"2024-09-11T12:54:23.300520+0000","last_became_peered":"2024-09-11T12:54:23.300520+0000","last_unstale":"2024-09-11T12:54:24.302808+0000","last_undegraded":"2024-09-11T12:54:24.302808+0000","last_fullsized":"2024-09-11T12:54:24.302808+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586276+0000","last_change":"2024-09-11T12:54:23.306652+0000","last_active":"2024-09-11T12:54:24.586276+0000","last_peered":"2024-09-11T12:54:24.586276+0000","last_clean":"2024-09-11T12:54:24.586276+0000","last_became_active":"2024-09-11T12:54:23.306036+0000","last_became_peered":"2024-09-11T12:54:23.306036+0000","last_unstale":"2024-09-11T12:54:24.586276+0000","last_undegraded":"2024-09-11T12:54:24.586276+0000","last_fullsized":"2024-09-11T12:54:24.586276+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:57:02.296525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295904+0000","last_change":"2024-09-11T12:54:23.296956+0000","last_active":"2024-09-11T12:54:24.295904+0000","last_peered":"2024-09-11T12:54:24.295904+0000","last_clean":"2024-09-11T12:54:24.295904+0000","last_became_active":"2024-09-11T12:54:23.296616+0000","last_became_peered":"2024-09-11T12:54:23.296616+0000","last_unstale":"2024-09-11T12:54:24.295904+0000","last_undegraded":"2024-09-11T12:54:24.295904+0000","last_fullsized":"2024-09-11T12:54:24.295904+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:16.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296171+0000","last_change":"2024-09-11T12:54:23.291243+0000","last_active":"2024-09-11T12:54:24.296171+0000","last_peered":"2024-09-11T12:54:24.296171+0000","last_clean":"2024-09-11T12:54:24.296171+0000","last_became_active":"2024-09-11T12:54:23.290931+0000","last_became_peered":"2024-09-11T12:54:23.290931+0000","last_unstale":"2024-09-11T12:54:24.296171+0000","last_undegraded":"2024-09-11T12:54:24.296171+0000","last_fullsized":"2024-09-11T12:54:24.296171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586888+0000","last_change":"2024-09-11T12:54:23.306150+0000","last_active":"2024-09-11T12:54:24.586888+0000","last_peered":"2024-09-11T12:54:24.586888+0000","last_clean":"2024-09-11T12:54:24.586888+0000","last_became_active":"2024-09-11T12:54:23.298357+0000","last_became_peered":"2024-09-11T12:54:23.298357+0000","last_unstale":"2024-09-11T12:54:24.586888+0000","last_undegraded":"2024-09-11T12:54:24.586888+0000","last_fullsized":"2024-09-11T12:54:24.586888+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:49:58.007081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295062+0000","last_change":"2024-09-11T12:54:23.301764+0000","last_active":"2024-09-11T12:54:24.295062+0000","last_peered":"2024-09-11T12:54:24.295062+0000","last_clean":"2024-09-11T12:54:24.295062+0000","last_became_active":"2024-09-11T12:54:23.298424+0000","last_became_peered":"2024-09-11T12:54:23.298424+0000","last_unstale":"2024-09-11T12:54:24.295062+0000","last_undegraded":"2024-09-11T12:54:24.295062+0000","last_fullsized":"2024-09-11T12:54:24.295062+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:57.117951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585976+0000","last_change":"2024-09-11T12:54:23.297177+0000","last_active":"2024-09-11T12:54:24.585976+0000","last_peered":"2024-09-11T12:54:24.585976+0000","last_clean":"2024-09-11T12:54:24.585976+0000","last_became_active":"2024-09-11T12:54:23.296877+0000","last_became_peered":"2024-09-11T12:54:23.296877+0000","last_unstale":"2024-09-11T12:54:24.585976+0000","last_undegraded":"2024-09-11T12:54:24.585976+0000","last_fullsized":"2024-09-11T12:54:24.585976+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:37.633645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296327+0000","last_change":"2024-09-11T12:54:23.299467+0000","last_active":"2024-09-11T12:54:24.296327+0000","last_peered":"2024-09-11T12:54:24.296327+0000","last_clean":"2024-09-11T12:54:24.296327+0000","last_became_active":"2024-09-11T12:54:23.299138+0000","last_became_peered":"2024-09-11T12:54:23.299138+0000","last_unstale":"2024-09-11T12:54:24.296327+0000","last_undegraded":"2024-09-11T12:54:24.296327+0000","last_fullsized":"2024-09-11T12:54:24.296327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586943+0000","last_change":"2024-09-11T12:54:23.306615+0000","last_active":"2024-09-11T12:54:24.586943+0000","last_peered":"2024-09-11T12:54:24.586943+0000","last_clean":"2024-09-11T12:54:24.586943+0000","last_became_active":"2024-09-11T12:54:23.306061+0000","last_became_peered":"2024-09-11T12:54:23.306061+0000","last_unstale":"2024-09-11T12:54:24.586943+0000","last_undegraded":"2024-09-11T12:54:24.586943+0000","last_fullsized":"2024-09-11T12:54:24.586943+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:30:31.142765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586891+0000","last_change":"2024-09-11T12:54:23.318248+0000","last_active":"2024-09-11T12:54:24.586891+0000","last_peered":"2024-09-11T12:54:24.586891+0000","last_clean":"2024-09-11T12:54:24.586891+0000","last_became_active":"2024-09-11T12:54:23.317965+0000","last_became_peered":"2024-09-11T12:54:23.317965+0000","last_unstale":"2024-09-11T12:54:24.586891+0000","last_undegraded":"2024-09-11T12:54:24.586891+0000","last_fullsized":"2024-09-11T12:54:24.586891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:54.769868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587196+0000","last_change":"2024-09-11T12:54:23.316393+0000","last_active":"2024-09-11T12:54:24.587196+0000","last_peered":"2024-09-11T12:54:24.587196+0000","last_clean":"2024-09-11T12:54:24.587196+0000","last_became_active":"2024-09-11T12:54:23.316149+0000","last_became_peered":"2024-09-11T12:54:23.316149+0000","last_unstale":"2024-09-11T12:54:24.587196+0000","last_undegraded":"2024-09-11T12:54:24.587196+0000","last_fullsized":"2024-09-11T12:54:24.587196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295862+0000","last_change":"2024-09-11T12:54:23.297197+0000","last_active":"2024-09-11T12:54:24.295862+0000","last_peered":"2024-09-11T12:54:24.295862+0000","last_clean":"2024-09-11T12:54:24.295862+0000","last_became_active":"2024-09-11T12:54:23.295221+0000","last_became_peered":"2024-09-11T12:54:23.295221+0000","last_unstale":"2024-09-11T12:54:24.295862+0000","last_undegraded":"2024-09-11T12:54:24.295862+0000","last_fullsized":"2024-09-11T12:54:24.295862+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:37.633645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295310+0000","last_change":"2024-09-11T12:54:23.286993+0000","last_active":"2024-09-11T12:54:24.295310+0000","last_peered":"2024-09-11T12:54:24.295310+0000","last_clean":"2024-09-11T12:54:24.295310+0000","last_became_active":"2024-09-11T12:54:23.286618+0000","last_became_peered":"2024-09-11T12:54:23.286618+0000","last_unstale":"2024-09-11T12:54:24.295310+0000","last_undegraded":"2024-09-11T12:54:24.295310+0000","last_fullsized":"2024-09-11T12:54:24.295310+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:51.619705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586148+0000","last_change":"2024-09-11T12:54:23.312290+0000","last_active":"2024-09-11T12:54:24.586148+0000","last_peered":"2024-09-11T12:54:24.586148+0000","last_clean":"2024-09-11T12:54:24.586148+0000","last_became_active":"2024-09-11T12:54:23.311794+0000","last_became_peered":"2024-09-11T12:54:23.311794+0000","last_unstale":"2024-09-11T12:54:24.586148+0000","last_undegraded":"2024-09-11T12:54:24.586148+0000","last_fullsized":"2024-09-11T12:54:24.586148+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:44.081258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585836+0000","last_change":"2024-09-11T12:54:23.297447+0000","last_active":"2024-09-11T12:54:24.585836+0000","last_peered":"2024-09-11T12:54:24.585836+0000","last_clean":"2024-09-11T12:54:24.585836+0000","last_became_active":"2024-09-11T12:54:23.297153+0000","last_became_peered":"2024-09-11T12:54:23.297153+0000","last_unstale":"2024-09-11T12:54:24.585836+0000","last_undegraded":"2024-09-11T12:54:24.585836+0000","last_fullsized":"2024-09-11T12:54:24.585836+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:09:38.705928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586933+0000","last_change":"2024-09-11T12:54:23.301455+0000","last_active":"2024-09-11T12:54:24.586933+0000","last_peered":"2024-09-11T12:54:24.586933+0000","last_clean":"2024-09-11T12:54:24.586933+0000","last_became_active":"2024-09-11T12:54:23.301075+0000","last_became_peered":"2024-09-11T12:54:23.301075+0000","last_unstale":"2024-09-11T12:54:24.586933+0000","last_undegraded":"2024-09-11T12:54:24.586933+0000","last_fullsized":"2024-09-11T12:54:24.586933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:23.920249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295741+0000","last_change":"2024-09-11T12:54:22.283172+0000","last_active":"2024-09-11T12:54:24.295741+0000","last_peered":"2024-09-11T12:54:24.295741+0000","last_clean":"2024-09-11T12:54:24.295741+0000","last_became_active":"2024-09-11T12:54:22.282891+0000","last_became_peered":"2024-09-11T12:54:22.282891+0000","last_unstale":"2024-09-11T12:54:24.295741+0000","last_undegraded":"2024-09-11T12:54:24.295741+0000","last_fullsized":"2024-09-11T12:54:24.295741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:53:43.834988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302931+0000","last_change":"2024-09-11T12:54:23.299031+0000","last_active":"2024-09-11T12:54:24.302931+0000","last_peered":"2024-09-11T12:54:24.302931+0000","last_clean":"2024-09-11T12:54:24.302931+0000","last_became_active":"2024-09-11T12:54:23.297625+0000","last_became_peered":"2024-09-11T12:54:23.297625+0000","last_unstale":"2024-09-11T12:54:24.302931+0000","last_undegraded":"2024-09-11T12:54:24.302931+0000","last_fullsized":"2024-09-11T12:54:24.302931+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586406+0000","last_change":"2024-09-11T12:54:22.287565+0000","last_active":"2024-09-11T12:54:24.586406+0000","last_peered":"2024-09-11T12:54:24.586406+0000","last_clean":"2024-09-11T12:54:24.586406+0000","last_became_active":"2024-09-11T12:54:22.287182+0000","last_became_peered":"2024-09-11T12:54:22.287182+0000","last_unstale":"2024-09-11T12:54:24.586406+0000","last_undegraded":"2024-09-11T12:54:24.586406+0000","last_fullsized":"2024-09-11T12:54:24.586406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:02.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586588+0000","last_change":"2024-09-11T12:54:23.297506+0000","last_active":"2024-09-11T12:54:24.586588+0000","last_peered":"2024-09-11T12:54:24.586588+0000","last_clean":"2024-09-11T12:54:24.586588+0000","last_became_active":"2024-09-11T12:54:23.297208+0000","last_became_peered":"2024-09-11T12:54:23.297208+0000","last_unstale":"2024-09-11T12:54:24.586588+0000","last_undegraded":"2024-09-11T12:54:24.586588+0000","last_fullsized":"2024-09-11T12:54:24.586588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:19.901425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296179+0000","last_change":"2024-09-11T12:54:22.291984+0000","last_active":"2024-09-11T12:54:24.296179+0000","last_peered":"2024-09-11T12:54:24.296179+0000","last_clean":"2024-09-11T12:54:24.296179+0000","last_became_active":"2024-09-11T12:54:22.291728+0000","last_became_peered":"2024-09-11T12:54:22.291728+0000","last_unstale":"2024-09-11T12:54:24.296179+0000","last_undegraded":"2024-09-11T12:54:24.296179+0000","last_fullsized":"2024-09-11T12:54:24.296179+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:31.918087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586486+0000","last_change":"2024-09-11T12:54:23.307078+0000","last_active":"2024-09-11T12:54:24.586486+0000","last_peered":"2024-09-11T12:54:24.586486+0000","last_clean":"2024-09-11T12:54:24.586486+0000","last_became_active":"2024-09-11T12:54:23.306414+0000","last_became_peered":"2024-09-11T12:54:23.306414+0000","last_unstale":"2024-09-11T12:54:24.586486+0000","last_undegraded":"2024-09-11T12:54:24.586486+0000","last_fullsized":"2024-09-11T12:54:24.586486+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:52.001445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295961+0000","last_change":"2024-09-11T12:54:22.291256+0000","last_active":"2024-09-11T12:54:24.295961+0000","last_peered":"2024-09-11T12:54:24.295961+0000","last_clean":"2024-09-11T12:54:24.295961+0000","last_became_active":"2024-09-11T12:54:22.290938+0000","last_became_peered":"2024-09-11T12:54:22.290938+0000","last_unstale":"2024-09-11T12:54:24.295961+0000","last_undegraded":"2024-09-11T12:54:24.295961+0000","last_fullsized":"2024-09-11T12:54:24.295961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:48.156054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586339+0000","last_change":"2024-09-11T12:54:23.304781+0000","last_active":"2024-09-11T12:54:24.586339+0000","last_peered":"2024-09-11T12:54:24.586339+0000","last_clean":"2024-09-11T12:54:24.586339+0000","last_became_active":"2024-09-11T12:54:23.303758+0000","last_became_peered":"2024-09-11T12:54:23.303758+0000","last_unstale":"2024-09-11T12:54:24.586339+0000","last_undegraded":"2024-09-11T12:54:24.586339+0000","last_fullsized":"2024-09-11T12:54:24.586339+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:53.563835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295282+0000","last_change":"2024-09-11T12:54:22.284410+0000","last_active":"2024-09-11T12:54:24.295282+0000","last_peered":"2024-09-11T12:54:24.295282+0000","last_clean":"2024-09-11T12:54:24.295282+0000","last_became_active":"2024-09-11T12:54:22.283653+0000","last_became_peered":"2024-09-11T12:54:22.283653+0000","last_unstale":"2024-09-11T12:54:24.295282+0000","last_undegraded":"2024-09-11T12:54:24.295282+0000","last_fullsized":"2024-09-11T12:54:24.295282+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:28.069265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295733+0000","last_change":"2024-09-11T12:54:23.295479+0000","last_active":"2024-09-11T12:54:24.295733+0000","last_peered":"2024-09-11T12:54:24.295733+0000","last_clean":"2024-09-11T12:54:24.295733+0000","last_became_active":"2024-09-11T12:54:23.295162+0000","last_became_peered":"2024-09-11T12:54:23.295162+0000","last_unstale":"2024-09-11T12:54:24.295733+0000","last_undegraded":"2024-09-11T12:54:24.295733+0000","last_fullsized":"2024-09-11T12:54:24.295733+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:45.118571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587137+0000","last_change":"2024-09-11T12:54:22.287556+0000","last_active":"2024-09-11T12:54:24.587137+0000","last_peered":"2024-09-11T12:54:24.587137+0000","last_clean":"2024-09-11T12:54:24.587137+0000","last_became_active":"2024-09-11T12:54:22.286075+0000","last_became_peered":"2024-09-11T12:54:22.286075+0000","last_unstale":"2024-09-11T12:54:24.587137+0000","last_undegraded":"2024-09-11T12:54:24.587137+0000","last_fullsized":"2024-09-11T12:54:24.587137+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:18.894624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586206+0000","last_change":"2024-09-11T12:54:23.300572+0000","last_active":"2024-09-11T12:54:24.586206+0000","last_peered":"2024-09-11T12:54:24.586206+0000","last_clean":"2024-09-11T12:54:24.586206+0000","last_became_active":"2024-09-11T12:54:23.299448+0000","last_became_peered":"2024-09-11T12:54:23.299448+0000","last_unstale":"2024-09-11T12:54:24.586206+0000","last_undegraded":"2024-09-11T12:54:24.586206+0000","last_fullsized":"2024-09-11T12:54:24.586206+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:34.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295169+0000","last_change":"2024-09-11T12:54:22.276340+0000","last_active":"2024-09-11T12:54:24.295169+0000","last_peered":"2024-09-11T12:54:24.295169+0000","last_clean":"2024-09-11T12:54:24.295169+0000","last_became_active":"2024-09-11T12:54:22.276012+0000","last_became_peered":"2024-09-11T12:54:22.276012+0000","last_unstale":"2024-09-11T12:54:24.295169+0000","last_undegraded":"2024-09-11T12:54:24.295169+0000","last_fullsized":"2024-09-11T12:54:24.295169+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:02.321194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.294889+0000","last_change":"2024-09-11T12:54:23.301748+0000","last_active":"2024-09-11T12:54:24.294889+0000","last_peered":"2024-09-11T12:54:24.294889+0000","last_clean":"2024-09-11T12:54:24.294889+0000","last_became_active":"2024-09-11T12:54:23.301460+0000","last_became_peered":"2024-09-11T12:54:23.301460+0000","last_unstale":"2024-09-11T12:54:24.294889+0000","last_undegraded":"2024-09-11T12:54:24.294889+0000","last_fullsized":"2024-09-11T12:54:24.294889+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:54.769868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587034+0000","last_change":"2024-09-11T12:54:22.300759+0000","last_active":"2024-09-11T12:54:24.587034+0000","last_peered":"2024-09-11T12:54:24.587034+0000","last_clean":"2024-09-11T12:54:24.587034+0000","last_became_active":"2024-09-11T12:54:22.300235+0000","last_became_peered":"2024-09-11T12:54:22.300235+0000","last_unstale":"2024-09-11T12:54:24.587034+0000","last_undegraded":"2024-09-11T12:54:24.587034+0000","last_fullsized":"2024-09-11T12:54:24.587034+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:57:42.820534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295759+0000","last_change":"2024-09-11T12:54:23.295938+0000","last_active":"2024-09-11T12:54:24.295759+0000","last_peered":"2024-09-11T12:54:24.295759+0000","last_clean":"2024-09-11T12:54:24.295759+0000","last_became_active":"2024-09-11T12:54:23.295595+0000","last_became_peered":"2024-09-11T12:54:23.295595+0000","last_unstale":"2024-09-11T12:54:24.295759+0000","last_undegraded":"2024-09-11T12:54:24.295759+0000","last_fullsized":"2024-09-11T12:54:24.295759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:03.756772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295896+0000","last_change":"2024-09-11T12:54:22.284963+0000","last_active":"2024-09-11T12:54:24.295896+0000","last_peered":"2024-09-11T12:54:24.295896+0000","last_clean":"2024-09-11T12:54:24.295896+0000","last_became_active":"2024-09-11T12:54:22.284479+0000","last_became_peered":"2024-09-11T12:54:22.284479+0000","last_unstale":"2024-09-11T12:54:24.295896+0000","last_undegraded":"2024-09-11T12:54:24.295896+0000","last_fullsized":"2024-09-11T12:54:24.295896+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:04.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.303006+0000","last_change":"2024-09-11T12:54:23.316264+0000","last_active":"2024-09-11T12:54:24.303006+0000","last_peered":"2024-09-11T12:54:24.303006+0000","last_clean":"2024-09-11T12:54:24.303006+0000","last_became_active":"2024-09-11T12:54:23.315795+0000","last_became_peered":"2024-09-11T12:54:23.315795+0000","last_unstale":"2024-09-11T12:54:24.303006+0000","last_undegraded":"2024-09-11T12:54:24.303006+0000","last_fullsized":"2024-09-11T12:54:24.303006+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:28.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587385+0000","last_change":"2024-09-11T12:54:22.285893+0000","last_active":"2024-09-11T12:54:24.587385+0000","last_peered":"2024-09-11T12:54:24.587385+0000","last_clean":"2024-09-11T12:54:24.587385+0000","last_became_active":"2024-09-11T12:54:22.285588+0000","last_became_peered":"2024-09-11T12:54:22.285588+0000","last_unstale":"2024-09-11T12:54:24.587385+0000","last_undegraded":"2024-09-11T12:54:24.587385+0000","last_fullsized":"2024-09-11T12:54:24.587385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:25.458947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587328+0000","last_change":"2024-09-11T12:54:23.284864+0000","last_active":"2024-09-11T12:54:24.587328+0000","last_peered":"2024-09-11T12:54:24.587328+0000","last_clean":"2024-09-11T12:54:24.587328+0000","last_became_active":"2024-09-11T12:54:23.284585+0000","last_became_peered":"2024-09-11T12:54:23.284585+0000","last_unstale":"2024-09-11T12:54:24.587328+0000","last_undegraded":"2024-09-11T12:54:24.587328+0000","last_fullsized":"2024-09-11T12:54:24.587328+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586511+0000","last_change":"2024-09-11T12:54:22.289982+0000","last_active":"2024-09-11T12:54:24.586511+0000","last_peered":"2024-09-11T12:54:24.586511+0000","last_clean":"2024-09-11T12:54:24.586511+0000","last_became_active":"2024-09-11T12:54:22.289679+0000","last_became_peered":"2024-09-11T12:54:22.289679+0000","last_unstale":"2024-09-11T12:54:24.586511+0000","last_undegraded":"2024-09-11T12:54:24.586511+0000","last_fullsized":"2024-09-11T12:54:24.586511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:34.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295556+0000","last_change":"2024-09-11T12:54:23.298774+0000","last_active":"2024-09-11T12:54:24.295556+0000","last_peered":"2024-09-11T12:54:24.295556+0000","last_clean":"2024-09-11T12:54:24.295556+0000","last_became_active":"2024-09-11T12:54:23.296168+0000","last_became_peered":"2024-09-11T12:54:23.296168+0000","last_unstale":"2024-09-11T12:54:24.295556+0000","last_undegraded":"2024-09-11T12:54:24.295556+0000","last_fullsized":"2024-09-11T12:54:24.295556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:16.318999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586648+0000","last_change":"2024-09-11T12:54:22.300532+0000","last_active":"2024-09-11T12:54:24.586648+0000","last_peered":"2024-09-11T12:54:24.586648+0000","last_clean":"2024-09-11T12:54:24.586648+0000","last_became_active":"2024-09-11T12:54:22.300229+0000","last_became_peered":"2024-09-11T12:54:22.300229+0000","last_unstale":"2024-09-11T12:54:24.586648+0000","last_undegraded":"2024-09-11T12:54:24.586648+0000","last_fullsized":"2024-09-11T12:54:24.586648+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:34.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587037+0000","last_change":"2024-09-11T12:54:23.285089+0000","last_active":"2024-09-11T12:54:24.587037+0000","last_peered":"2024-09-11T12:54:24.587037+0000","last_clean":"2024-09-11T12:54:24.587037+0000","last_became_active":"2024-09-11T12:54:23.284855+0000","last_became_peered":"2024-09-11T12:54:23.284855+0000","last_unstale":"2024-09-11T12:54:24.587037+0000","last_undegraded":"2024-09-11T12:54:24.587037+0000","last_fullsized":"2024-09-11T12:54:24.587037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:24.159603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587166+0000","last_change":"2024-09-11T12:54:22.286903+0000","last_active":"2024-09-11T12:54:24.587166+0000","last_peered":"2024-09-11T12:54:24.587166+0000","last_clean":"2024-09-11T12:54:24.587166+0000","last_became_active":"2024-09-11T12:54:22.286642+0000","last_became_peered":"2024-09-11T12:54:22.286642+0000","last_unstale":"2024-09-11T12:54:24.587166+0000","last_undegraded":"2024-09-11T12:54:24.587166+0000","last_fullsized":"2024-09-11T12:54:24.587166+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:49:56.941915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587837+0000","last_change":"2024-09-11T12:54:23.295636+0000","last_active":"2024-09-11T12:54:24.587837+0000","last_peered":"2024-09-11T12:54:24.587837+0000","last_clean":"2024-09-11T12:54:24.587837+0000","last_became_active":"2024-09-11T12:54:23.295322+0000","last_became_peered":"2024-09-11T12:54:23.295322+0000","last_unstale":"2024-09-11T12:54:24.587837+0000","last_undegraded":"2024-09-11T12:54:24.587837+0000","last_fullsized":"2024-09-11T12:54:24.587837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302655+0000","last_change":"2024-09-11T12:54:22.282177+0000","last_active":"2024-09-11T12:54:24.302655+0000","last_peered":"2024-09-11T12:54:24.302655+0000","last_clean":"2024-09-11T12:54:24.302655+0000","last_became_active":"2024-09-11T12:54:22.281878+0000","last_became_peered":"2024-09-11T12:54:22.281878+0000","last_unstale":"2024-09-11T12:54:24.302655+0000","last_undegraded":"2024-09-11T12:54:24.302655+0000","last_fullsized":"2024-09-11T12:54:24.302655+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:56.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586344+0000","last_change":"2024-09-11T12:54:23.286786+0000","last_active":"2024-09-11T12:54:24.586344+0000","last_peered":"2024-09-11T12:54:24.586344+0000","last_clean":"2024-09-11T12:54:24.586344+0000","last_became_active":"2024-09-11T12:54:23.286511+0000","last_became_peered":"2024-09-11T12:54:23.286511+0000","last_unstale":"2024-09-11T12:54:24.586344+0000","last_undegraded":"2024-09-11T12:54:24.586344+0000","last_fullsized":"2024-09-11T12:54:24.586344+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:42.783808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295835+0000","last_change":"2024-09-11T12:54:22.279237+0000","last_active":"2024-09-11T12:54:24.295835+0000","last_peered":"2024-09-11T12:54:24.295835+0000","last_clean":"2024-09-11T12:54:24.295835+0000","last_became_active":"2024-09-11T12:54:22.278861+0000","last_became_peered":"2024-09-11T12:54:22.278861+0000","last_unstale":"2024-09-11T12:54:24.295835+0000","last_undegraded":"2024-09-11T12:54:24.295835+0000","last_fullsized":"2024-09-11T12:54:24.295835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:20.985728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296352+0000","last_change":"2024-09-11T12:54:23.317193+0000","last_active":"2024-09-11T12:54:24.296352+0000","last_peered":"2024-09-11T12:54:24.296352+0000","last_clean":"2024-09-11T12:54:24.296352+0000","last_became_active":"2024-09-11T12:54:23.316928+0000","last_became_peered":"2024-09-11T12:54:23.316928+0000","last_unstale":"2024-09-11T12:54:24.296352+0000","last_undegraded":"2024-09-11T12:54:24.296352+0000","last_fullsized":"2024-09-11T12:54:24.296352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:58.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587029+0000","last_change":"2024-09-11T12:54:22.286749+0000","last_active":"2024-09-11T12:54:24.587029+0000","last_peered":"2024-09-11T12:54:24.587029+0000","last_clean":"2024-09-11T12:54:24.587029+0000","last_became_active":"2024-09-11T12:54:22.286459+0000","last_became_peered":"2024-09-11T12:54:22.286459+0000","last_unstale":"2024-09-11T12:54:24.587029+0000","last_undegraded":"2024-09-11T12:54:24.587029+0000","last_fullsized":"2024-09-11T12:54:24.587029+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:12.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587593+0000","last_change":"2024-09-11T12:54:23.317522+0000","last_active":"2024-09-11T12:54:24.587593+0000","last_peered":"2024-09-11T12:54:24.587593+0000","last_clean":"2024-09-11T12:54:24.587593+0000","last_became_active":"2024-09-11T12:54:23.317222+0000","last_became_peered":"2024-09-11T12:54:23.317222+0000","last_unstale":"2024-09-11T12:54:24.587593+0000","last_undegraded":"2024-09-11T12:54:24.587593+0000","last_fullsized":"2024-09-11T12:54:24.587593+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:51.619705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296353+0000","last_change":"2024-09-11T12:54:22.287379+0000","last_active":"2024-09-11T12:54:24.296353+0000","last_peered":"2024-09-11T12:54:24.296353+0000","last_clean":"2024-09-11T12:54:24.296353+0000","last_became_active":"2024-09-11T12:54:22.283856+0000","last_became_peered":"2024-09-11T12:54:22.283856+0000","last_unstale":"2024-09-11T12:54:24.296353+0000","last_undegraded":"2024-09-11T12:54:24.296353+0000","last_fullsized":"2024-09-11T12:54:24.296353+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:11.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586283+0000","last_change":"2024-09-11T12:54:23.316024+0000","last_active":"2024-09-11T12:54:24.586283+0000","last_peered":"2024-09-11T12:54:24.586283+0000","last_clean":"2024-09-11T12:54:24.586283+0000","last_became_active":"2024-09-11T12:54:23.315634+0000","last_became_peered":"2024-09-11T12:54:23.315634+0000","last_unstale":"2024-09-11T12:54:24.586283+0000","last_undegraded":"2024-09-11T12:54:24.586283+0000","last_fullsized":"2024-09-11T12:54:24.586283+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:34.980309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302753+0000","last_change":"2024-09-11T12:54:22.300516+0000","last_active":"2024-09-11T12:54:24.302753+0000","last_peered":"2024-09-11T12:54:24.302753+0000","last_clean":"2024-09-11T12:54:24.302753+0000","last_became_active":"2024-09-11T12:54:22.300214+0000","last_became_peered":"2024-09-11T12:54:22.300214+0000","last_unstale":"2024-09-11T12:54:24.302753+0000","last_undegraded":"2024-09-11T12:54:24.302753+0000","last_fullsized":"2024-09-11T12:54:24.302753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:44.111771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586483+0000","last_change":"2024-09-11T12:54:23.307256+0000","last_active":"2024-09-11T12:54:24.586483+0000","last_peered":"2024-09-11T12:54:24.586483+0000","last_clean":"2024-09-11T12:54:24.586483+0000","last_became_active":"2024-09-11T12:54:23.306965+0000","last_became_peered":"2024-09-11T12:54:23.306965+0000","last_unstale":"2024-09-11T12:54:24.586483+0000","last_undegraded":"2024-09-11T12:54:24.586483+0000","last_fullsized":"2024-09-11T12:54:24.586483+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:16.049460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295454+0000","last_change":"2024-09-11T12:54:22.281928+0000","last_active":"2024-09-11T12:54:24.295454+0000","last_peered":"2024-09-11T12:54:24.295454+0000","last_clean":"2024-09-11T12:54:24.295454+0000","last_became_active":"2024-09-11T12:54:22.281530+0000","last_became_peered":"2024-09-11T12:54:22.281530+0000","last_unstale":"2024-09-11T12:54:24.295454+0000","last_undegraded":"2024-09-11T12:54:24.295454+0000","last_fullsized":"2024-09-11T12:54:24.295454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:40.676738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585966+0000","last_change":"2024-09-11T12:54:23.304321+0000","last_active":"2024-09-11T12:54:24.585966+0000","last_peered":"2024-09-11T12:54:24.585966+0000","last_clean":"2024-09-11T12:54:24.585966+0000","last_became_active":"2024-09-11T12:54:23.304015+0000","last_became_peered":"2024-09-11T12:54:23.304015+0000","last_unstale":"2024-09-11T12:54:24.585966+0000","last_undegraded":"2024-09-11T12:54:24.585966+0000","last_fullsized":"2024-09-11T12:54:24.585966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587085+0000","last_change":"2024-09-11T12:54:22.292840+0000","last_active":"2024-09-11T12:54:24.587085+0000","last_peered":"2024-09-11T12:54:24.587085+0000","last_clean":"2024-09-11T12:54:24.587085+0000","last_became_active":"2024-09-11T12:54:22.292572+0000","last_became_peered":"2024-09-11T12:54:22.292572+0000","last_unstale":"2024-09-11T12:54:24.587085+0000","last_undegraded":"2024-09-11T12:54:24.587085+0000","last_fullsized":"2024-09-11T12:54:24.587085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:08:33.168923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585929+0000","last_change":"2024-09-11T12:54:23.295550+0000","last_active":"2024-09-11T12:54:24.585929+0000","last_peered":"2024-09-11T12:54:24.585929+0000","last_clean":"2024-09-11T12:54:24.585929+0000","last_became_active":"2024-09-11T12:54:23.295301+0000","last_became_peered":"2024-09-11T12:54:23.295301+0000","last_unstale":"2024-09-11T12:54:24.585929+0000","last_undegraded":"2024-09-11T12:54:24.585929+0000","last_fullsized":"2024-09-11T12:54:24.585929+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:30.735031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295772+0000","last_change":"2024-09-11T12:54:22.284786+0000","last_active":"2024-09-11T12:54:24.295772+0000","last_peered":"2024-09-11T12:54:24.295772+0000","last_clean":"2024-09-11T12:54:24.295772+0000","last_became_active":"2024-09-11T12:54:22.284485+0000","last_became_peered":"2024-09-11T12:54:22.284485+0000","last_unstale":"2024-09-11T12:54:24.295772+0000","last_undegraded":"2024-09-11T12:54:24.295772+0000","last_fullsized":"2024-09-11T12:54:24.295772+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:14.052663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586052+0000","last_change":"2024-09-11T12:54:23.306341+0000","last_active":"2024-09-11T12:54:24.586052+0000","last_peered":"2024-09-11T12:54:24.586052+0000","last_clean":"2024-09-11T12:54:24.586052+0000","last_became_active":"2024-09-11T12:54:23.306085+0000","last_became_peered":"2024-09-11T12:54:23.306085+0000","last_unstale":"2024-09-11T12:54:24.586052+0000","last_undegraded":"2024-09-11T12:54:24.586052+0000","last_fullsized":"2024-09-11T12:54:24.586052+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586162+0000","last_change":"2024-09-11T12:54:22.292981+0000","last_active":"2024-09-11T12:54:24.586162+0000","last_peered":"2024-09-11T12:54:24.586162+0000","last_clean":"2024-09-11T12:54:24.586162+0000","last_became_active":"2024-09-11T12:54:22.292743+0000","last_became_peered":"2024-09-11T12:54:22.292743+0000","last_unstale":"2024-09-11T12:54:24.586162+0000","last_undegraded":"2024-09-11T12:54:24.586162+0000","last_fullsized":"2024-09-11T12:54:24.586162+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:32:41.359164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587305+0000","last_change":"2024-09-11T12:54:23.315721+0000","last_active":"2024-09-11T12:54:24.587305+0000","last_peered":"2024-09-11T12:54:24.587305+0000","last_clean":"2024-09-11T12:54:24.587305+0000","last_became_active":"2024-09-11T12:54:23.315391+0000","last_became_peered":"2024-09-11T12:54:23.315391+0000","last_unstale":"2024-09-11T12:54:24.587305+0000","last_undegraded":"2024-09-11T12:54:24.587305+0000","last_fullsized":"2024-09-11T12:54:24.587305+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586566+0000","last_change":"2024-09-11T12:54:22.281607+0000","last_active":"2024-09-11T12:54:24.586566+0000","last_peered":"2024-09-11T12:54:24.586566+0000","last_clean":"2024-09-11T12:54:24.586566+0000","last_became_active":"2024-09-11T12:54:22.281332+0000","last_became_peered":"2024-09-11T12:54:22.281332+0000","last_unstale":"2024-09-11T12:54:24.586566+0000","last_undegraded":"2024-09-11T12:54:24.586566+0000","last_fullsized":"2024-09-11T12:54:24.586566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:35.367639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302809+0000","last_change":"2024-09-11T12:54:23.316537+0000","last_active":"2024-09-11T12:54:24.302809+0000","last_peered":"2024-09-11T12:54:24.302809+0000","last_clean":"2024-09-11T12:54:24.302809+0000","last_became_active":"2024-09-11T12:54:23.316078+0000","last_became_peered":"2024-09-11T12:54:23.316078+0000","last_unstale":"2024-09-11T12:54:24.302809+0000","last_undegraded":"2024-09-11T12:54:24.302809+0000","last_fullsized":"2024-09-11T12:54:24.302809+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587406+0000","last_change":"2024-09-11T12:54:22.285756+0000","last_active":"2024-09-11T12:54:24.587406+0000","last_peered":"2024-09-11T12:54:24.587406+0000","last_clean":"2024-09-11T12:54:24.587406+0000","last_became_active":"2024-09-11T12:54:22.285461+0000","last_became_peered":"2024-09-11T12:54:22.285461+0000","last_unstale":"2024-09-11T12:54:24.587406+0000","last_undegraded":"2024-09-11T12:54:24.587406+0000","last_fullsized":"2024-09-11T12:54:24.587406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:22.913448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587495+0000","last_change":"2024-09-11T12:54:23.308626+0000","last_active":"2024-09-11T12:54:24.587495+0000","last_peered":"2024-09-11T12:54:24.587495+0000","last_clean":"2024-09-11T12:54:24.587495+0000","last_became_active":"2024-09-11T12:54:23.308104+0000","last_became_peered":"2024-09-11T12:54:23.308104+0000","last_unstale":"2024-09-11T12:54:24.587495+0000","last_undegraded":"2024-09-11T12:54:24.587495+0000","last_fullsized":"2024-09-11T12:54:24.587495+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:37.268981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"101'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.304281+0000","last_change":"2024-09-11T12:54:22.284167+0000","last_active":"2024-09-11T12:54:24.304281+0000","last_peered":"2024-09-11T12:54:24.304281+0000","last_clean":"2024-09-11T12:54:24.304281+0000","last_became_active":"2024-09-11T12:54:22.283819+0000","last_became_peered":"2024-09-11T12:54:22.283819+0000","last_unstale":"2024-09-11T12:54:24.304281+0000","last_undegraded":"2024-09-11T12:54:24.304281+0000","last_fullsized":"2024-09-11T12:54:24.304281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:49.564759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586549+0000","last_change":"2024-09-11T12:54:23.299220+0000","last_active":"2024-09-11T12:54:24.586549+0000","last_peered":"2024-09-11T12:54:24.586549+0000","last_clean":"2024-09-11T12:54:24.586549+0000","last_became_active":"2024-09-11T12:54:23.298934+0000","last_became_peered":"2024-09-11T12:54:23.298934+0000","last_unstale":"2024-09-11T12:54:24.586549+0000","last_undegraded":"2024-09-11T12:54:24.586549+0000","last_fullsized":"2024-09-11T12:54:24.586549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:13.789321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586814+0000","last_change":"2024-09-11T12:54:22.284407+0000","last_active":"2024-09-11T12:54:24.586814+0000","last_peered":"2024-09-11T12:54:24.586814+0000","last_clean":"2024-09-11T12:54:24.586814+0000","last_became_active":"2024-09-11T12:54:22.284135+0000","last_became_peered":"2024-09-11T12:54:22.284135+0000","last_unstale":"2024-09-11T12:54:24.586814+0000","last_undegraded":"2024-09-11T12:54:24.586814+0000","last_fullsized":"2024-09-11T12:54:24.586814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:29.812151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302840+0000","last_change":"2024-09-11T12:54:23.315053+0000","last_active":"2024-09-11T12:54:24.302840+0000","last_peered":"2024-09-11T12:54:24.302840+0000","last_clean":"2024-09-11T12:54:24.302840+0000","last_became_active":"2024-09-11T12:54:23.314676+0000","last_became_peered":"2024-09-11T12:54:23.314676+0000","last_unstale":"2024-09-11T12:54:24.302840+0000","last_undegraded":"2024-09-11T12:54:24.302840+0000","last_fullsized":"2024-09-11T12:54:24.302840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587255+0000","last_change":"2024-09-11T12:54:22.289564+0000","last_active":"2024-09-11T12:54:24.587255+0000","last_peered":"2024-09-11T12:54:24.587255+0000","last_clean":"2024-09-11T12:54:24.587255+0000","last_became_active":"2024-09-11T12:54:22.289307+0000","last_became_peered":"2024-09-11T12:54:22.289307+0000","last_unstale":"2024-09-11T12:54:24.587255+0000","last_undegraded":"2024-09-11T12:54:24.587255+0000","last_fullsized":"2024-09-11T12:54:24.587255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:53.763067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295742+0000","last_change":"2024-09-11T12:54:23.284960+0000","last_active":"2024-09-11T12:54:24.295742+0000","last_peered":"2024-09-11T12:54:24.295742+0000","last_clean":"2024-09-11T12:54:24.295742+0000","last_became_active":"2024-09-11T12:54:23.284632+0000","last_became_peered":"2024-09-11T12:54:23.284632+0000","last_unstale":"2024-09-11T12:54:24.295742+0000","last_undegraded":"2024-09-11T12:54:24.295742+0000","last_fullsized":"2024-09-11T12:54:24.295742+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:50.571559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295311+0000","last_change":"2024-09-11T12:54:22.277045+0000","last_active":"2024-09-11T12:54:24.295311+0000","last_peered":"2024-09-11T12:54:24.295311+0000","last_clean":"2024-09-11T12:54:24.295311+0000","last_became_active":"2024-09-11T12:54:22.276685+0000","last_became_peered":"2024-09-11T12:54:22.276685+0000","last_unstale":"2024-09-11T12:54:24.295311+0000","last_undegraded":"2024-09-11T12:54:24.295311+0000","last_fullsized":"2024-09-11T12:54:24.295311+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:10.956963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295980+0000","last_change":"2024-09-11T12:54:23.305703+0000","last_active":"2024-09-11T12:54:24.295980+0000","last_peered":"2024-09-11T12:54:24.295980+0000","last_clean":"2024-09-11T12:54:24.295980+0000","last_became_active":"2024-09-11T12:54:23.305337+0000","last_became_peered":"2024-09-11T12:54:23.305337+0000","last_unstale":"2024-09-11T12:54:24.295980+0000","last_undegraded":"2024-09-11T12:54:24.295980+0000","last_fullsized":"2024-09-11T12:54:24.295980+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:42.783808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295560+0000","last_change":"2024-09-11T12:54:22.289227+0000","last_active":"2024-09-11T12:54:24.295560+0000","last_peered":"2024-09-11T12:54:24.295560+0000","last_clean":"2024-09-11T12:54:24.295560+0000","last_became_active":"2024-09-11T12:54:22.288770+0000","last_became_peered":"2024-09-11T12:54:22.288770+0000","last_unstale":"2024-09-11T12:54:24.295560+0000","last_undegraded":"2024-09-11T12:54:24.295560+0000","last_fullsized":"2024-09-11T12:54:24.295560+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:09:37.699127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586295+0000","last_change":"2024-09-11T12:54:23.300195+0000","last_active":"2024-09-11T12:54:24.586295+0000","last_peered":"2024-09-11T12:54:24.586295+0000","last_clean":"2024-09-11T12:54:24.586295+0000","last_became_active":"2024-09-11T12:54:23.299862+0000","last_became_peered":"2024-09-11T12:54:23.299862+0000","last_unstale":"2024-09-11T12:54:24.586295+0000","last_undegraded":"2024-09-11T12:54:24.586295+0000","last_fullsized":"2024-09-11T12:54:24.586295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:59.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295766+0000","last_change":"2024-09-11T12:54:22.292101+0000","last_active":"2024-09-11T12:54:24.295766+0000","last_peered":"2024-09-11T12:54:24.295766+0000","last_clean":"2024-09-11T12:54:24.295766+0000","last_became_active":"2024-09-11T12:54:22.291824+0000","last_became_peered":"2024-09-11T12:54:22.291824+0000","last_unstale":"2024-09-11T12:54:24.295766+0000","last_undegraded":"2024-09-11T12:54:24.295766+0000","last_fullsized":"2024-09-11T12:54:24.295766+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:48.789636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586319+0000","last_change":"2024-09-11T12:54:23.312068+0000","last_active":"2024-09-11T12:54:24.586319+0000","last_peered":"2024-09-11T12:54:24.586319+0000","last_clean":"2024-09-11T12:54:24.586319+0000","last_became_active":"2024-09-11T12:54:23.311424+0000","last_became_peered":"2024-09-11T12:54:23.311424+0000","last_unstale":"2024-09-11T12:54:24.586319+0000","last_undegraded":"2024-09-11T12:54:24.586319+0000","last_fullsized":"2024-09-11T12:54:24.586319+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:25:18.915354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586655+0000","last_change":"2024-09-11T12:54:22.280306+0000","last_active":"2024-09-11T12:54:24.586655+0000","last_peered":"2024-09-11T12:54:24.586655+0000","last_clean":"2024-09-11T12:54:24.586655+0000","last_became_active":"2024-09-11T12:54:22.279691+0000","last_became_peered":"2024-09-11T12:54:22.279691+0000","last_unstale":"2024-09-11T12:54:24.586655+0000","last_undegraded":"2024-09-11T12:54:24.586655+0000","last_fullsized":"2024-09-11T12:54:24.586655+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:39.525780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586132+0000","last_change":"2024-09-11T12:54:23.297043+0000","last_active":"2024-09-11T12:54:24.586132+0000","last_peered":"2024-09-11T12:54:24.586132+0000","last_clean":"2024-09-11T12:54:24.586132+0000","last_became_active":"2024-09-11T12:54:23.296787+0000","last_became_peered":"2024-09-11T12:54:23.296787+0000","last_unstale":"2024-09-11T12:54:24.586132+0000","last_undegraded":"2024-09-11T12:54:24.586132+0000","last_fullsized":"2024-09-11T12:54:24.586132+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:16.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295022+0000","last_change":"2024-09-11T12:54:22.280451+0000","last_active":"2024-09-11T12:54:24.295022+0000","last_peered":"2024-09-11T12:54:24.295022+0000","last_clean":"2024-09-11T12:54:24.295022+0000","last_became_active":"2024-09-11T12:54:22.280063+0000","last_became_peered":"2024-09-11T12:54:22.280063+0000","last_unstale":"2024-09-11T12:54:24.295022+0000","last_undegraded":"2024-09-11T12:54:24.295022+0000","last_fullsized":"2024-09-11T12:54:24.295022+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:46:01.586894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":281,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586352+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:54:24.586352+0000","last_peered":"2024-09-11T12:54:24.586352+0000","last_clean":"2024-09-11T12:54:24.586352+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:54:24.586352+0000","last_undegraded":"2024-09-11T12:54:24.586352+0000","last_fullsized":"2024-09-11T12:54:24.586352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1851392,"data_stored":1134963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1253376,"data_stored":544049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1257472,"data_stored":544583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":543979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1265664,"data_stored":545661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":543979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1257472,"data_stored":544103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1863680,"data_stored":1136071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:54:27.401 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:54:27.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- 172.21.15.78:0/3371734716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 msgr2=0x7f0dc80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 --2- 172.21.15.78:0/3371734716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0dc80051d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00c640 tx=0x7f0ddc01fa60 comp rx=0 tx=0).stop 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de8c00640 1 -- 172.21.15.78:0/3371734716 reap_dead start 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- 172.21.15.78:0/3371734716 shutdown_connections 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- 172.21.15.78:0/3371734716 >> 172.21.15.78:0/3371734716 conn(0x7f0de40b1c50 msgr2=0x7f0de4070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- 172.21.15.78:0/3371734716 shutdown_connections 2024-09-11T12:54:27.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- 172.21.15.78:0/3371734716 wait complete. 2024-09-11T12:54:27.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 Processor -- start 2024-09-11T12:54:27.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- start start 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0de414c5d0 0x7f0de406f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de41415c0 0x7f0de415e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0de415cf00 con 0x7f0dc8004d50 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0de415cd80 con 0x7f0de41415c0 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0deac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0de40b3500 con 0x7f0de414c5d0 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57508/0 (socket says 172.21.15.78:57508) 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 -- 172.21.15.78:0/847217264 learned_addr learned my addr 172.21.15.78:0/847217264 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3400640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0de414c5d0 0x7f0de406f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de8c00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de41415c0 0x7f0de415e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 -- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0de414c5d0 msgr2=0x7f0de406f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0de414c5d0 0x7f0de406f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 -- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de41415c0 msgr2=0x7f0de415e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de41415c0 0x7f0de415e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.583+0000 7f0de3e00640 1 -- 172.21.15.78:0/847217264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de4141b80 con 0x7f0dc8004d50 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0de3e00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4011c20 tx=0x7f0dd400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd401e070 con 0x7f0dc8004d50 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dd400e7c0 con 0x7f0dc8004d50 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400ebf0 con 0x7f0dc8004d50 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de4141e60 con 0x7f0dc8004d50 2024-09-11T12:54:27.589 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de414cd50 con 0x7f0dc8004d50 2024-09-11T12:54:27.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de415f9b0 con 0x7f0dc8004d50 2024-09-11T12:54:27.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f0dd4021070 con 0x7f0dc8004d50 2024-09-11T12:54:27.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 0x7f0dc4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.587+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f0dd4092fe0 con 0x7f0dc8004d50 2024-09-11T12:54:27.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.591+0000 7f0de3400640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 0x7f0dc4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.592 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.591+0000 7f0de3400640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 0x7f0dc4073c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0de414e460 tx=0x7f0dd00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.591+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0dd405e5a0 con 0x7f0dc8004d50 2024-09-11T12:54:27.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.755+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0de415afa0 con 0x7f0dc40717a0 2024-09-11T12:54:27.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.759+0000 7f0ddbe00640 1 -- 172.21.15.78:0/847217264 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252252 (secure 0 0 0) 0x7f0de415afa0 con 0x7f0dc40717a0 2024-09-11T12:54:27.764 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:27.765 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:54:27.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 msgr2=0x7f0dc4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 0x7f0dc4073c60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0de414e460 tx=0x7f0dd00091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 msgr2=0x7f0de413f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4011c20 tx=0x7f0dd400f790 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 shutdown_connections 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0dc40717a0 0x7f0dc4073c60 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0de41415c0 0x7f0de415e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0de414c5d0 0x7f0de406f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 --2- 172.21.15.78:0/847217264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0dc8004d50 0x7f0de413f530 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 >> 172.21.15.78:0/847217264 conn(0x7f0de40b1c50 msgr2=0x7f0de415b7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 shutdown_connections 2024-09-11T12:54:27.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.763+0000 7f0deac00640 1 -- 172.21.15.78:0/847217264 wait complete. 2024-09-11T12:54:27.784 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-09-11T12:54:26.479584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":163,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260532,"kb_used_data":10984,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754714188,"statfs":{"total":773094113280,"available":772827328512,"internally_reserved":0,"allocated":11247616,"data_stored":5537388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.220044"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586649+0000","last_change":"2024-09-11T12:54:23.314807+0000","last_active":"2024-09-11T12:54:24.586649+0000","last_peered":"2024-09-11T12:54:24.586649+0000","last_clean":"2024-09-11T12:54:24.586649+0000","last_became_active":"2024-09-11T12:54:23.314584+0000","last_became_peered":"2024-09-11T12:54:23.314584+0000","last_unstale":"2024-09-11T12:54:24.586649+0000","last_undegraded":"2024-09-11T12:54:24.586649+0000","last_fullsized":"2024-09-11T12:54:24.586649+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:58:17.027168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296124+0000","last_change":"2024-09-11T12:54:23.293096+0000","last_active":"2024-09-11T12:54:24.296124+0000","last_peered":"2024-09-11T12:54:24.296124+0000","last_clean":"2024-09-11T12:54:24.296124+0000","last_became_active":"2024-09-11T12:54:23.292804+0000","last_became_peered":"2024-09-11T12:54:23.292804+0000","last_unstale":"2024-09-11T12:54:24.296124+0000","last_undegraded":"2024-09-11T12:54:24.296124+0000","last_fullsized":"2024-09-11T12:54:24.296124+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:24.159603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586247+0000","last_change":"2024-09-11T12:54:23.303442+0000","last_active":"2024-09-11T12:54:24.586247+0000","last_peered":"2024-09-11T12:54:24.586247+0000","last_clean":"2024-09-11T12:54:24.586247+0000","last_became_active":"2024-09-11T12:54:23.303128+0000","last_became_peered":"2024-09-11T12:54:23.303128+0000","last_unstale":"2024-09-11T12:54:24.586247+0000","last_undegraded":"2024-09-11T12:54:24.586247+0000","last_fullsized":"2024-09-11T12:54:24.586247+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:27:19.274192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586204+0000","last_change":"2024-09-11T12:54:23.304579+0000","last_active":"2024-09-11T12:54:24.586204+0000","last_peered":"2024-09-11T12:54:24.586204+0000","last_clean":"2024-09-11T12:54:24.586204+0000","last_became_active":"2024-09-11T12:54:23.304323+0000","last_became_peered":"2024-09-11T12:54:23.304323+0000","last_unstale":"2024-09-11T12:54:24.586204+0000","last_undegraded":"2024-09-11T12:54:24.586204+0000","last_fullsized":"2024-09-11T12:54:24.586204+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:50:49.981499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296049+0000","last_change":"2024-09-11T12:54:23.304151+0000","last_active":"2024-09-11T12:54:24.296049+0000","last_peered":"2024-09-11T12:54:24.296049+0000","last_clean":"2024-09-11T12:54:24.296049+0000","last_became_active":"2024-09-11T12:54:23.303822+0000","last_became_peered":"2024-09-11T12:54:23.303822+0000","last_unstale":"2024-09-11T12:54:24.296049+0000","last_undegraded":"2024-09-11T12:54:24.296049+0000","last_fullsized":"2024-09-11T12:54:24.296049+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:51:56.510751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586135+0000","last_change":"2024-09-11T12:54:23.307753+0000","last_active":"2024-09-11T12:54:24.586135+0000","last_peered":"2024-09-11T12:54:24.586135+0000","last_clean":"2024-09-11T12:54:24.586135+0000","last_became_active":"2024-09-11T12:54:23.307433+0000","last_became_peered":"2024-09-11T12:54:23.307433+0000","last_unstale":"2024-09-11T12:54:24.586135+0000","last_undegraded":"2024-09-11T12:54:24.586135+0000","last_fullsized":"2024-09-11T12:54:24.586135+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:26.465748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.294971+0000","last_change":"2024-09-11T12:54:23.295519+0000","last_active":"2024-09-11T12:54:24.294971+0000","last_peered":"2024-09-11T12:54:24.294971+0000","last_clean":"2024-09-11T12:54:24.294971+0000","last_became_active":"2024-09-11T12:54:23.295230+0000","last_became_peered":"2024-09-11T12:54:23.295230+0000","last_unstale":"2024-09-11T12:54:24.294971+0000","last_undegraded":"2024-09-11T12:54:24.294971+0000","last_fullsized":"2024-09-11T12:54:24.294971+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:23.920249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586219+0000","last_change":"2024-09-11T12:54:23.307624+0000","last_active":"2024-09-11T12:54:24.586219+0000","last_peered":"2024-09-11T12:54:24.586219+0000","last_clean":"2024-09-11T12:54:24.586219+0000","last_became_active":"2024-09-11T12:54:23.307332+0000","last_became_peered":"2024-09-11T12:54:23.307332+0000","last_unstale":"2024-09-11T12:54:24.586219+0000","last_undegraded":"2024-09-11T12:54:24.586219+0000","last_fullsized":"2024-09-11T12:54:24.586219+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:28.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586822+0000","last_change":"2024-09-11T12:54:23.308884+0000","last_active":"2024-09-11T12:54:24.586822+0000","last_peered":"2024-09-11T12:54:24.586822+0000","last_clean":"2024-09-11T12:54:24.586822+0000","last_became_active":"2024-09-11T12:54:23.308365+0000","last_became_peered":"2024-09-11T12:54:23.308365+0000","last_unstale":"2024-09-11T12:54:24.586822+0000","last_undegraded":"2024-09-11T12:54:24.586822+0000","last_fullsized":"2024-09-11T12:54:24.586822+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:37:35.357853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586795+0000","last_change":"2024-09-11T12:54:23.306835+0000","last_active":"2024-09-11T12:54:24.586795+0000","last_peered":"2024-09-11T12:54:24.586795+0000","last_clean":"2024-09-11T12:54:24.586795+0000","last_became_active":"2024-09-11T12:54:23.306375+0000","last_became_peered":"2024-09-11T12:54:23.306375+0000","last_unstale":"2024-09-11T12:54:24.586795+0000","last_undegraded":"2024-09-11T12:54:24.586795+0000","last_fullsized":"2024-09-11T12:54:24.586795+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:46.188989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296044+0000","last_change":"2024-09-11T12:54:23.304969+0000","last_active":"2024-09-11T12:54:24.296044+0000","last_peered":"2024-09-11T12:54:24.296044+0000","last_clean":"2024-09-11T12:54:24.296044+0000","last_became_active":"2024-09-11T12:54:23.303641+0000","last_became_peered":"2024-09-11T12:54:23.303641+0000","last_unstale":"2024-09-11T12:54:24.296044+0000","last_undegraded":"2024-09-11T12:54:24.296044+0000","last_fullsized":"2024-09-11T12:54:24.296044+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:49:58.007081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586398+0000","last_change":"2024-09-11T12:54:23.304023+0000","last_active":"2024-09-11T12:54:24.586398+0000","last_peered":"2024-09-11T12:54:24.586398+0000","last_clean":"2024-09-11T12:54:24.586398+0000","last_became_active":"2024-09-11T12:54:23.303678+0000","last_became_peered":"2024-09-11T12:54:23.303678+0000","last_unstale":"2024-09-11T12:54:24.586398+0000","last_undegraded":"2024-09-11T12:54:24.586398+0000","last_fullsized":"2024-09-11T12:54:24.586398+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:33.333132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586191+0000","last_change":"2024-09-11T12:54:23.307978+0000","last_active":"2024-09-11T12:54:24.586191+0000","last_peered":"2024-09-11T12:54:24.586191+0000","last_clean":"2024-09-11T12:54:24.586191+0000","last_became_active":"2024-09-11T12:54:23.306930+0000","last_became_peered":"2024-09-11T12:54:23.306930+0000","last_unstale":"2024-09-11T12:54:24.586191+0000","last_undegraded":"2024-09-11T12:54:24.586191+0000","last_fullsized":"2024-09-11T12:54:24.586191+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585917+0000","last_change":"2024-09-11T12:54:23.303383+0000","last_active":"2024-09-11T12:54:24.585917+0000","last_peered":"2024-09-11T12:54:24.585917+0000","last_clean":"2024-09-11T12:54:24.585917+0000","last_became_active":"2024-09-11T12:54:23.303058+0000","last_became_peered":"2024-09-11T12:54:23.303058+0000","last_unstale":"2024-09-11T12:54:24.585917+0000","last_undegraded":"2024-09-11T12:54:24.585917+0000","last_fullsized":"2024-09-11T12:54:24.585917+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586065+0000","last_change":"2024-09-11T12:54:23.303864+0000","last_active":"2024-09-11T12:54:24.586065+0000","last_peered":"2024-09-11T12:54:24.586065+0000","last_clean":"2024-09-11T12:54:24.586065+0000","last_became_active":"2024-09-11T12:54:23.299676+0000","last_became_peered":"2024-09-11T12:54:23.299676+0000","last_unstale":"2024-09-11T12:54:24.586065+0000","last_undegraded":"2024-09-11T12:54:24.586065+0000","last_fullsized":"2024-09-11T12:54:24.586065+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:30.412135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586051+0000","last_change":"2024-09-11T12:54:23.310421+0000","last_active":"2024-09-11T12:54:24.586051+0000","last_peered":"2024-09-11T12:54:24.586051+0000","last_clean":"2024-09-11T12:54:24.586051+0000","last_became_active":"2024-09-11T12:54:23.310138+0000","last_became_peered":"2024-09-11T12:54:23.310138+0000","last_unstale":"2024-09-11T12:54:24.586051+0000","last_undegraded":"2024-09-11T12:54:24.586051+0000","last_fullsized":"2024-09-11T12:54:24.586051+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:45.456278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296274+0000","last_change":"2024-09-11T12:54:23.300135+0000","last_active":"2024-09-11T12:54:24.296274+0000","last_peered":"2024-09-11T12:54:24.296274+0000","last_clean":"2024-09-11T12:54:24.296274+0000","last_became_active":"2024-09-11T12:54:23.299857+0000","last_became_peered":"2024-09-11T12:54:23.299857+0000","last_unstale":"2024-09-11T12:54:24.296274+0000","last_undegraded":"2024-09-11T12:54:24.296274+0000","last_fullsized":"2024-09-11T12:54:24.296274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302808+0000","last_change":"2024-09-11T12:54:23.300798+0000","last_active":"2024-09-11T12:54:24.302808+0000","last_peered":"2024-09-11T12:54:24.302808+0000","last_clean":"2024-09-11T12:54:24.302808+0000","last_became_active":"2024-09-11T12:54:23.300520+0000","last_became_peered":"2024-09-11T12:54:23.300520+0000","last_unstale":"2024-09-11T12:54:24.302808+0000","last_undegraded":"2024-09-11T12:54:24.302808+0000","last_fullsized":"2024-09-11T12:54:24.302808+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586276+0000","last_change":"2024-09-11T12:54:23.306652+0000","last_active":"2024-09-11T12:54:24.586276+0000","last_peered":"2024-09-11T12:54:24.586276+0000","last_clean":"2024-09-11T12:54:24.586276+0000","last_became_active":"2024-09-11T12:54:23.306036+0000","last_became_peered":"2024-09-11T12:54:23.306036+0000","last_unstale":"2024-09-11T12:54:24.586276+0000","last_undegraded":"2024-09-11T12:54:24.586276+0000","last_fullsized":"2024-09-11T12:54:24.586276+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:57:02.296525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295904+0000","last_change":"2024-09-11T12:54:23.296956+0000","last_active":"2024-09-11T12:54:24.295904+0000","last_peered":"2024-09-11T12:54:24.295904+0000","last_clean":"2024-09-11T12:54:24.295904+0000","last_became_active":"2024-09-11T12:54:23.296616+0000","last_became_peered":"2024-09-11T12:54:23.296616+0000","last_unstale":"2024-09-11T12:54:24.295904+0000","last_undegraded":"2024-09-11T12:54:24.295904+0000","last_fullsized":"2024-09-11T12:54:24.295904+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:16.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296171+0000","last_change":"2024-09-11T12:54:23.291243+0000","last_active":"2024-09-11T12:54:24.296171+0000","last_peered":"2024-09-11T12:54:24.296171+0000","last_clean":"2024-09-11T12:54:24.296171+0000","last_became_active":"2024-09-11T12:54:23.290931+0000","last_became_peered":"2024-09-11T12:54:23.290931+0000","last_unstale":"2024-09-11T12:54:24.296171+0000","last_undegraded":"2024-09-11T12:54:24.296171+0000","last_fullsized":"2024-09-11T12:54:24.296171+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586888+0000","last_change":"2024-09-11T12:54:23.306150+0000","last_active":"2024-09-11T12:54:24.586888+0000","last_peered":"2024-09-11T12:54:24.586888+0000","last_clean":"2024-09-11T12:54:24.586888+0000","last_became_active":"2024-09-11T12:54:23.298357+0000","last_became_peered":"2024-09-11T12:54:23.298357+0000","last_unstale":"2024-09-11T12:54:24.586888+0000","last_undegraded":"2024-09-11T12:54:24.586888+0000","last_fullsized":"2024-09-11T12:54:24.586888+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:49:58.007081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295062+0000","last_change":"2024-09-11T12:54:23.301764+0000","last_active":"2024-09-11T12:54:24.295062+0000","last_peered":"2024-09-11T12:54:24.295062+0000","last_clean":"2024-09-11T12:54:24.295062+0000","last_became_active":"2024-09-11T12:54:23.298424+0000","last_became_peered":"2024-09-11T12:54:23.298424+0000","last_unstale":"2024-09-11T12:54:24.295062+0000","last_undegraded":"2024-09-11T12:54:24.295062+0000","last_fullsized":"2024-09-11T12:54:24.295062+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:57.117951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585976+0000","last_change":"2024-09-11T12:54:23.297177+0000","last_active":"2024-09-11T12:54:24.585976+0000","last_peered":"2024-09-11T12:54:24.585976+0000","last_clean":"2024-09-11T12:54:24.585976+0000","last_became_active":"2024-09-11T12:54:23.296877+0000","last_became_peered":"2024-09-11T12:54:23.296877+0000","last_unstale":"2024-09-11T12:54:24.585976+0000","last_undegraded":"2024-09-11T12:54:24.585976+0000","last_fullsized":"2024-09-11T12:54:24.585976+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:37.633645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296327+0000","last_change":"2024-09-11T12:54:23.299467+0000","last_active":"2024-09-11T12:54:24.296327+0000","last_peered":"2024-09-11T12:54:24.296327+0000","last_clean":"2024-09-11T12:54:24.296327+0000","last_became_active":"2024-09-11T12:54:23.299138+0000","last_became_peered":"2024-09-11T12:54:23.299138+0000","last_unstale":"2024-09-11T12:54:24.296327+0000","last_undegraded":"2024-09-11T12:54:24.296327+0000","last_fullsized":"2024-09-11T12:54:24.296327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586943+0000","last_change":"2024-09-11T12:54:23.306615+0000","last_active":"2024-09-11T12:54:24.586943+0000","last_peered":"2024-09-11T12:54:24.586943+0000","last_clean":"2024-09-11T12:54:24.586943+0000","last_became_active":"2024-09-11T12:54:23.306061+0000","last_became_peered":"2024-09-11T12:54:23.306061+0000","last_unstale":"2024-09-11T12:54:24.586943+0000","last_undegraded":"2024-09-11T12:54:24.586943+0000","last_fullsized":"2024-09-11T12:54:24.586943+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:30:31.142765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586891+0000","last_change":"2024-09-11T12:54:23.318248+0000","last_active":"2024-09-11T12:54:24.586891+0000","last_peered":"2024-09-11T12:54:24.586891+0000","last_clean":"2024-09-11T12:54:24.586891+0000","last_became_active":"2024-09-11T12:54:23.317965+0000","last_became_peered":"2024-09-11T12:54:23.317965+0000","last_unstale":"2024-09-11T12:54:24.586891+0000","last_undegraded":"2024-09-11T12:54:24.586891+0000","last_fullsized":"2024-09-11T12:54:24.586891+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:54.769868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587196+0000","last_change":"2024-09-11T12:54:23.316393+0000","last_active":"2024-09-11T12:54:24.587196+0000","last_peered":"2024-09-11T12:54:24.587196+0000","last_clean":"2024-09-11T12:54:24.587196+0000","last_became_active":"2024-09-11T12:54:23.316149+0000","last_became_peered":"2024-09-11T12:54:23.316149+0000","last_unstale":"2024-09-11T12:54:24.587196+0000","last_undegraded":"2024-09-11T12:54:24.587196+0000","last_fullsized":"2024-09-11T12:54:24.587196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295862+0000","last_change":"2024-09-11T12:54:23.297197+0000","last_active":"2024-09-11T12:54:24.295862+0000","last_peered":"2024-09-11T12:54:24.295862+0000","last_clean":"2024-09-11T12:54:24.295862+0000","last_became_active":"2024-09-11T12:54:23.295221+0000","last_became_peered":"2024-09-11T12:54:23.295221+0000","last_unstale":"2024-09-11T12:54:24.295862+0000","last_undegraded":"2024-09-11T12:54:24.295862+0000","last_fullsized":"2024-09-11T12:54:24.295862+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:23:37.633645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295310+0000","last_change":"2024-09-11T12:54:23.286993+0000","last_active":"2024-09-11T12:54:24.295310+0000","last_peered":"2024-09-11T12:54:24.295310+0000","last_clean":"2024-09-11T12:54:24.295310+0000","last_became_active":"2024-09-11T12:54:23.286618+0000","last_became_peered":"2024-09-11T12:54:23.286618+0000","last_unstale":"2024-09-11T12:54:24.295310+0000","last_undegraded":"2024-09-11T12:54:24.295310+0000","last_fullsized":"2024-09-11T12:54:24.295310+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:51.619705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586148+0000","last_change":"2024-09-11T12:54:23.312290+0000","last_active":"2024-09-11T12:54:24.586148+0000","last_peered":"2024-09-11T12:54:24.586148+0000","last_clean":"2024-09-11T12:54:24.586148+0000","last_became_active":"2024-09-11T12:54:23.311794+0000","last_became_peered":"2024-09-11T12:54:23.311794+0000","last_unstale":"2024-09-11T12:54:24.586148+0000","last_undegraded":"2024-09-11T12:54:24.586148+0000","last_fullsized":"2024-09-11T12:54:24.586148+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:44.081258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585836+0000","last_change":"2024-09-11T12:54:23.297447+0000","last_active":"2024-09-11T12:54:24.585836+0000","last_peered":"2024-09-11T12:54:24.585836+0000","last_clean":"2024-09-11T12:54:24.585836+0000","last_became_active":"2024-09-11T12:54:23.297153+0000","last_became_peered":"2024-09-11T12:54:23.297153+0000","last_unstale":"2024-09-11T12:54:24.585836+0000","last_undegraded":"2024-09-11T12:54:24.585836+0000","last_fullsized":"2024-09-11T12:54:24.585836+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:09:38.705928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586933+0000","last_change":"2024-09-11T12:54:23.301455+0000","last_active":"2024-09-11T12:54:24.586933+0000","last_peered":"2024-09-11T12:54:24.586933+0000","last_clean":"2024-09-11T12:54:24.586933+0000","last_became_active":"2024-09-11T12:54:23.301075+0000","last_became_peered":"2024-09-11T12:54:23.301075+0000","last_unstale":"2024-09-11T12:54:24.586933+0000","last_undegraded":"2024-09-11T12:54:24.586933+0000","last_fullsized":"2024-09-11T12:54:24.586933+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:23.920249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"100'4","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295741+0000","last_change":"2024-09-11T12:54:22.283172+0000","last_active":"2024-09-11T12:54:24.295741+0000","last_peered":"2024-09-11T12:54:24.295741+0000","last_clean":"2024-09-11T12:54:24.295741+0000","last_became_active":"2024-09-11T12:54:22.282891+0000","last_became_peered":"2024-09-11T12:54:22.282891+0000","last_unstale":"2024-09-11T12:54:24.295741+0000","last_undegraded":"2024-09-11T12:54:24.295741+0000","last_fullsized":"2024-09-11T12:54:24.295741+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:53:43.834988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302931+0000","last_change":"2024-09-11T12:54:23.299031+0000","last_active":"2024-09-11T12:54:24.302931+0000","last_peered":"2024-09-11T12:54:24.302931+0000","last_clean":"2024-09-11T12:54:24.302931+0000","last_became_active":"2024-09-11T12:54:23.297625+0000","last_became_peered":"2024-09-11T12:54:23.297625+0000","last_unstale":"2024-09-11T12:54:24.302931+0000","last_undegraded":"2024-09-11T12:54:24.302931+0000","last_fullsized":"2024-09-11T12:54:24.302931+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586406+0000","last_change":"2024-09-11T12:54:22.287565+0000","last_active":"2024-09-11T12:54:24.586406+0000","last_peered":"2024-09-11T12:54:24.586406+0000","last_clean":"2024-09-11T12:54:24.586406+0000","last_became_active":"2024-09-11T12:54:22.287182+0000","last_became_peered":"2024-09-11T12:54:22.287182+0000","last_unstale":"2024-09-11T12:54:24.586406+0000","last_undegraded":"2024-09-11T12:54:24.586406+0000","last_fullsized":"2024-09-11T12:54:24.586406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:02.749971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586588+0000","last_change":"2024-09-11T12:54:23.297506+0000","last_active":"2024-09-11T12:54:24.586588+0000","last_peered":"2024-09-11T12:54:24.586588+0000","last_clean":"2024-09-11T12:54:24.586588+0000","last_became_active":"2024-09-11T12:54:23.297208+0000","last_became_peered":"2024-09-11T12:54:23.297208+0000","last_unstale":"2024-09-11T12:54:24.586588+0000","last_undegraded":"2024-09-11T12:54:24.586588+0000","last_fullsized":"2024-09-11T12:54:24.586588+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:19.901425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296179+0000","last_change":"2024-09-11T12:54:22.291984+0000","last_active":"2024-09-11T12:54:24.296179+0000","last_peered":"2024-09-11T12:54:24.296179+0000","last_clean":"2024-09-11T12:54:24.296179+0000","last_became_active":"2024-09-11T12:54:22.291728+0000","last_became_peered":"2024-09-11T12:54:22.291728+0000","last_unstale":"2024-09-11T12:54:24.296179+0000","last_undegraded":"2024-09-11T12:54:24.296179+0000","last_fullsized":"2024-09-11T12:54:24.296179+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:31.918087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586486+0000","last_change":"2024-09-11T12:54:23.307078+0000","last_active":"2024-09-11T12:54:24.586486+0000","last_peered":"2024-09-11T12:54:24.586486+0000","last_clean":"2024-09-11T12:54:24.586486+0000","last_became_active":"2024-09-11T12:54:23.306414+0000","last_became_peered":"2024-09-11T12:54:23.306414+0000","last_unstale":"2024-09-11T12:54:24.586486+0000","last_undegraded":"2024-09-11T12:54:24.586486+0000","last_fullsized":"2024-09-11T12:54:24.586486+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:07:52.001445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295961+0000","last_change":"2024-09-11T12:54:22.291256+0000","last_active":"2024-09-11T12:54:24.295961+0000","last_peered":"2024-09-11T12:54:24.295961+0000","last_clean":"2024-09-11T12:54:24.295961+0000","last_became_active":"2024-09-11T12:54:22.290938+0000","last_became_peered":"2024-09-11T12:54:22.290938+0000","last_unstale":"2024-09-11T12:54:24.295961+0000","last_undegraded":"2024-09-11T12:54:24.295961+0000","last_fullsized":"2024-09-11T12:54:24.295961+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:29:48.156054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586339+0000","last_change":"2024-09-11T12:54:23.304781+0000","last_active":"2024-09-11T12:54:24.586339+0000","last_peered":"2024-09-11T12:54:24.586339+0000","last_clean":"2024-09-11T12:54:24.586339+0000","last_became_active":"2024-09-11T12:54:23.303758+0000","last_became_peered":"2024-09-11T12:54:23.303758+0000","last_unstale":"2024-09-11T12:54:24.586339+0000","last_undegraded":"2024-09-11T12:54:24.586339+0000","last_fullsized":"2024-09-11T12:54:24.586339+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:53.563835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295282+0000","last_change":"2024-09-11T12:54:22.284410+0000","last_active":"2024-09-11T12:54:24.295282+0000","last_peered":"2024-09-11T12:54:24.295282+0000","last_clean":"2024-09-11T12:54:24.295282+0000","last_became_active":"2024-09-11T12:54:22.283653+0000","last_became_peered":"2024-09-11T12:54:22.283653+0000","last_unstale":"2024-09-11T12:54:24.295282+0000","last_undegraded":"2024-09-11T12:54:24.295282+0000","last_fullsized":"2024-09-11T12:54:24.295282+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:28.069265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295733+0000","last_change":"2024-09-11T12:54:23.295479+0000","last_active":"2024-09-11T12:54:24.295733+0000","last_peered":"2024-09-11T12:54:24.295733+0000","last_clean":"2024-09-11T12:54:24.295733+0000","last_became_active":"2024-09-11T12:54:23.295162+0000","last_became_peered":"2024-09-11T12:54:23.295162+0000","last_unstale":"2024-09-11T12:54:24.295733+0000","last_undegraded":"2024-09-11T12:54:24.295733+0000","last_fullsized":"2024-09-11T12:54:24.295733+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:45.118571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587137+0000","last_change":"2024-09-11T12:54:22.287556+0000","last_active":"2024-09-11T12:54:24.587137+0000","last_peered":"2024-09-11T12:54:24.587137+0000","last_clean":"2024-09-11T12:54:24.587137+0000","last_became_active":"2024-09-11T12:54:22.286075+0000","last_became_peered":"2024-09-11T12:54:22.286075+0000","last_unstale":"2024-09-11T12:54:24.587137+0000","last_undegraded":"2024-09-11T12:54:24.587137+0000","last_fullsized":"2024-09-11T12:54:24.587137+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:08:18.894624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586206+0000","last_change":"2024-09-11T12:54:23.300572+0000","last_active":"2024-09-11T12:54:24.586206+0000","last_peered":"2024-09-11T12:54:24.586206+0000","last_clean":"2024-09-11T12:54:24.586206+0000","last_became_active":"2024-09-11T12:54:23.299448+0000","last_became_peered":"2024-09-11T12:54:23.299448+0000","last_unstale":"2024-09-11T12:54:24.586206+0000","last_undegraded":"2024-09-11T12:54:24.586206+0000","last_fullsized":"2024-09-11T12:54:24.586206+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:49:34.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295169+0000","last_change":"2024-09-11T12:54:22.276340+0000","last_active":"2024-09-11T12:54:24.295169+0000","last_peered":"2024-09-11T12:54:24.295169+0000","last_clean":"2024-09-11T12:54:24.295169+0000","last_became_active":"2024-09-11T12:54:22.276012+0000","last_became_peered":"2024-09-11T12:54:22.276012+0000","last_unstale":"2024-09-11T12:54:24.295169+0000","last_undegraded":"2024-09-11T12:54:24.295169+0000","last_fullsized":"2024-09-11T12:54:24.295169+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:02.321194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.294889+0000","last_change":"2024-09-11T12:54:23.301748+0000","last_active":"2024-09-11T12:54:24.294889+0000","last_peered":"2024-09-11T12:54:24.294889+0000","last_clean":"2024-09-11T12:54:24.294889+0000","last_became_active":"2024-09-11T12:54:23.301460+0000","last_became_peered":"2024-09-11T12:54:23.301460+0000","last_unstale":"2024-09-11T12:54:24.294889+0000","last_undegraded":"2024-09-11T12:54:24.294889+0000","last_fullsized":"2024-09-11T12:54:24.294889+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:54.769868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587034+0000","last_change":"2024-09-11T12:54:22.300759+0000","last_active":"2024-09-11T12:54:24.587034+0000","last_peered":"2024-09-11T12:54:24.587034+0000","last_clean":"2024-09-11T12:54:24.587034+0000","last_became_active":"2024-09-11T12:54:22.300235+0000","last_became_peered":"2024-09-11T12:54:22.300235+0000","last_unstale":"2024-09-11T12:54:24.587034+0000","last_undegraded":"2024-09-11T12:54:24.587034+0000","last_fullsized":"2024-09-11T12:54:24.587034+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:57:42.820534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295759+0000","last_change":"2024-09-11T12:54:23.295938+0000","last_active":"2024-09-11T12:54:24.295759+0000","last_peered":"2024-09-11T12:54:24.295759+0000","last_clean":"2024-09-11T12:54:24.295759+0000","last_became_active":"2024-09-11T12:54:23.295595+0000","last_became_peered":"2024-09-11T12:54:23.295595+0000","last_unstale":"2024-09-11T12:54:24.295759+0000","last_undegraded":"2024-09-11T12:54:24.295759+0000","last_fullsized":"2024-09-11T12:54:24.295759+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:03.756772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295896+0000","last_change":"2024-09-11T12:54:22.284963+0000","last_active":"2024-09-11T12:54:24.295896+0000","last_peered":"2024-09-11T12:54:24.295896+0000","last_clean":"2024-09-11T12:54:24.295896+0000","last_became_active":"2024-09-11T12:54:22.284479+0000","last_became_peered":"2024-09-11T12:54:22.284479+0000","last_unstale":"2024-09-11T12:54:24.295896+0000","last_undegraded":"2024-09-11T12:54:24.295896+0000","last_fullsized":"2024-09-11T12:54:24.295896+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:49:04.986865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.303006+0000","last_change":"2024-09-11T12:54:23.316264+0000","last_active":"2024-09-11T12:54:24.303006+0000","last_peered":"2024-09-11T12:54:24.303006+0000","last_clean":"2024-09-11T12:54:24.303006+0000","last_became_active":"2024-09-11T12:54:23.315795+0000","last_became_peered":"2024-09-11T12:54:23.315795+0000","last_unstale":"2024-09-11T12:54:24.303006+0000","last_undegraded":"2024-09-11T12:54:24.303006+0000","last_fullsized":"2024-09-11T12:54:24.303006+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:51:28.163086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587385+0000","last_change":"2024-09-11T12:54:22.285893+0000","last_active":"2024-09-11T12:54:24.587385+0000","last_peered":"2024-09-11T12:54:24.587385+0000","last_clean":"2024-09-11T12:54:24.587385+0000","last_became_active":"2024-09-11T12:54:22.285588+0000","last_became_peered":"2024-09-11T12:54:22.285588+0000","last_unstale":"2024-09-11T12:54:24.587385+0000","last_undegraded":"2024-09-11T12:54:24.587385+0000","last_fullsized":"2024-09-11T12:54:24.587385+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:25.458947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587328+0000","last_change":"2024-09-11T12:54:23.284864+0000","last_active":"2024-09-11T12:54:24.587328+0000","last_peered":"2024-09-11T12:54:24.587328+0000","last_clean":"2024-09-11T12:54:24.587328+0000","last_became_active":"2024-09-11T12:54:23.284585+0000","last_became_peered":"2024-09-11T12:54:23.284585+0000","last_unstale":"2024-09-11T12:54:24.587328+0000","last_undegraded":"2024-09-11T12:54:24.587328+0000","last_fullsized":"2024-09-11T12:54:24.587328+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"100'3","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586511+0000","last_change":"2024-09-11T12:54:22.289982+0000","last_active":"2024-09-11T12:54:24.586511+0000","last_peered":"2024-09-11T12:54:24.586511+0000","last_clean":"2024-09-11T12:54:24.586511+0000","last_became_active":"2024-09-11T12:54:22.289679+0000","last_became_peered":"2024-09-11T12:54:22.289679+0000","last_unstale":"2024-09-11T12:54:24.586511+0000","last_undegraded":"2024-09-11T12:54:24.586511+0000","last_fullsized":"2024-09-11T12:54:24.586511+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:21:34.107880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295556+0000","last_change":"2024-09-11T12:54:23.298774+0000","last_active":"2024-09-11T12:54:24.295556+0000","last_peered":"2024-09-11T12:54:24.295556+0000","last_clean":"2024-09-11T12:54:24.295556+0000","last_became_active":"2024-09-11T12:54:23.296168+0000","last_became_peered":"2024-09-11T12:54:23.296168+0000","last_unstale":"2024-09-11T12:54:24.295556+0000","last_undegraded":"2024-09-11T12:54:24.295556+0000","last_fullsized":"2024-09-11T12:54:24.295556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:16.318999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"100'6","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586648+0000","last_change":"2024-09-11T12:54:22.300532+0000","last_active":"2024-09-11T12:54:24.586648+0000","last_peered":"2024-09-11T12:54:24.586648+0000","last_clean":"2024-09-11T12:54:24.586648+0000","last_became_active":"2024-09-11T12:54:22.300229+0000","last_became_peered":"2024-09-11T12:54:22.300229+0000","last_unstale":"2024-09-11T12:54:24.586648+0000","last_undegraded":"2024-09-11T12:54:24.586648+0000","last_fullsized":"2024-09-11T12:54:24.586648+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:50:34.294702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587037+0000","last_change":"2024-09-11T12:54:23.285089+0000","last_active":"2024-09-11T12:54:24.587037+0000","last_peered":"2024-09-11T12:54:24.587037+0000","last_clean":"2024-09-11T12:54:24.587037+0000","last_became_active":"2024-09-11T12:54:23.284855+0000","last_became_peered":"2024-09-11T12:54:23.284855+0000","last_unstale":"2024-09-11T12:54:24.587037+0000","last_undegraded":"2024-09-11T12:54:24.587037+0000","last_fullsized":"2024-09-11T12:54:24.587037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:24.159603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587166+0000","last_change":"2024-09-11T12:54:22.286903+0000","last_active":"2024-09-11T12:54:24.587166+0000","last_peered":"2024-09-11T12:54:24.587166+0000","last_clean":"2024-09-11T12:54:24.587166+0000","last_became_active":"2024-09-11T12:54:22.286642+0000","last_became_peered":"2024-09-11T12:54:22.286642+0000","last_unstale":"2024-09-11T12:54:24.587166+0000","last_undegraded":"2024-09-11T12:54:24.587166+0000","last_fullsized":"2024-09-11T12:54:24.587166+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:49:56.941915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587837+0000","last_change":"2024-09-11T12:54:23.295636+0000","last_active":"2024-09-11T12:54:24.587837+0000","last_peered":"2024-09-11T12:54:24.587837+0000","last_clean":"2024-09-11T12:54:24.587837+0000","last_became_active":"2024-09-11T12:54:23.295322+0000","last_became_peered":"2024-09-11T12:54:23.295322+0000","last_unstale":"2024-09-11T12:54:24.587837+0000","last_undegraded":"2024-09-11T12:54:24.587837+0000","last_fullsized":"2024-09-11T12:54:24.587837+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:39.713579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302655+0000","last_change":"2024-09-11T12:54:22.282177+0000","last_active":"2024-09-11T12:54:24.302655+0000","last_peered":"2024-09-11T12:54:24.302655+0000","last_clean":"2024-09-11T12:54:24.302655+0000","last_became_active":"2024-09-11T12:54:22.281878+0000","last_became_peered":"2024-09-11T12:54:22.281878+0000","last_unstale":"2024-09-11T12:54:24.302655+0000","last_undegraded":"2024-09-11T12:54:24.302655+0000","last_fullsized":"2024-09-11T12:54:24.302655+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:37:56.111150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586344+0000","last_change":"2024-09-11T12:54:23.286786+0000","last_active":"2024-09-11T12:54:24.586344+0000","last_peered":"2024-09-11T12:54:24.586344+0000","last_clean":"2024-09-11T12:54:24.586344+0000","last_became_active":"2024-09-11T12:54:23.286511+0000","last_became_peered":"2024-09-11T12:54:23.286511+0000","last_unstale":"2024-09-11T12:54:24.586344+0000","last_undegraded":"2024-09-11T12:54:24.586344+0000","last_fullsized":"2024-09-11T12:54:24.586344+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:42.783808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"100'8","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295835+0000","last_change":"2024-09-11T12:54:22.279237+0000","last_active":"2024-09-11T12:54:24.295835+0000","last_peered":"2024-09-11T12:54:24.295835+0000","last_clean":"2024-09-11T12:54:24.295835+0000","last_became_active":"2024-09-11T12:54:22.278861+0000","last_became_peered":"2024-09-11T12:54:22.278861+0000","last_unstale":"2024-09-11T12:54:24.295835+0000","last_undegraded":"2024-09-11T12:54:24.295835+0000","last_fullsized":"2024-09-11T12:54:24.295835+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:20.985728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296352+0000","last_change":"2024-09-11T12:54:23.317193+0000","last_active":"2024-09-11T12:54:24.296352+0000","last_peered":"2024-09-11T12:54:24.296352+0000","last_clean":"2024-09-11T12:54:24.296352+0000","last_became_active":"2024-09-11T12:54:23.316928+0000","last_became_peered":"2024-09-11T12:54:23.316928+0000","last_unstale":"2024-09-11T12:54:24.296352+0000","last_undegraded":"2024-09-11T12:54:24.296352+0000","last_fullsized":"2024-09-11T12:54:24.296352+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:17:58.371076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587029+0000","last_change":"2024-09-11T12:54:22.286749+0000","last_active":"2024-09-11T12:54:24.587029+0000","last_peered":"2024-09-11T12:54:24.587029+0000","last_clean":"2024-09-11T12:54:24.587029+0000","last_became_active":"2024-09-11T12:54:22.286459+0000","last_became_peered":"2024-09-11T12:54:22.286459+0000","last_unstale":"2024-09-11T12:54:24.587029+0000","last_undegraded":"2024-09-11T12:54:24.587029+0000","last_fullsized":"2024-09-11T12:54:24.587029+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:12.332034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587593+0000","last_change":"2024-09-11T12:54:23.317522+0000","last_active":"2024-09-11T12:54:24.587593+0000","last_peered":"2024-09-11T12:54:24.587593+0000","last_clean":"2024-09-11T12:54:24.587593+0000","last_became_active":"2024-09-11T12:54:23.317222+0000","last_became_peered":"2024-09-11T12:54:23.317222+0000","last_unstale":"2024-09-11T12:54:24.587593+0000","last_undegraded":"2024-09-11T12:54:24.587593+0000","last_fullsized":"2024-09-11T12:54:24.587593+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:51.619705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.296353+0000","last_change":"2024-09-11T12:54:22.287379+0000","last_active":"2024-09-11T12:54:24.296353+0000","last_peered":"2024-09-11T12:54:24.296353+0000","last_clean":"2024-09-11T12:54:24.296353+0000","last_became_active":"2024-09-11T12:54:22.283856+0000","last_became_peered":"2024-09-11T12:54:22.283856+0000","last_unstale":"2024-09-11T12:54:24.296353+0000","last_undegraded":"2024-09-11T12:54:24.296353+0000","last_fullsized":"2024-09-11T12:54:24.296353+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:11.193783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586283+0000","last_change":"2024-09-11T12:54:23.316024+0000","last_active":"2024-09-11T12:54:24.586283+0000","last_peered":"2024-09-11T12:54:24.586283+0000","last_clean":"2024-09-11T12:54:24.586283+0000","last_became_active":"2024-09-11T12:54:23.315634+0000","last_became_peered":"2024-09-11T12:54:23.315634+0000","last_unstale":"2024-09-11T12:54:24.586283+0000","last_undegraded":"2024-09-11T12:54:24.586283+0000","last_fullsized":"2024-09-11T12:54:24.586283+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:34.980309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302753+0000","last_change":"2024-09-11T12:54:22.300516+0000","last_active":"2024-09-11T12:54:24.302753+0000","last_peered":"2024-09-11T12:54:24.302753+0000","last_clean":"2024-09-11T12:54:24.302753+0000","last_became_active":"2024-09-11T12:54:22.300214+0000","last_became_peered":"2024-09-11T12:54:22.300214+0000","last_unstale":"2024-09-11T12:54:24.302753+0000","last_undegraded":"2024-09-11T12:54:24.302753+0000","last_fullsized":"2024-09-11T12:54:24.302753+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:02:44.111771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586483+0000","last_change":"2024-09-11T12:54:23.307256+0000","last_active":"2024-09-11T12:54:24.586483+0000","last_peered":"2024-09-11T12:54:24.586483+0000","last_clean":"2024-09-11T12:54:24.586483+0000","last_became_active":"2024-09-11T12:54:23.306965+0000","last_became_peered":"2024-09-11T12:54:23.306965+0000","last_unstale":"2024-09-11T12:54:24.586483+0000","last_undegraded":"2024-09-11T12:54:24.586483+0000","last_fullsized":"2024-09-11T12:54:24.586483+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:16.049460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295454+0000","last_change":"2024-09-11T12:54:22.281928+0000","last_active":"2024-09-11T12:54:24.295454+0000","last_peered":"2024-09-11T12:54:24.295454+0000","last_clean":"2024-09-11T12:54:24.295454+0000","last_became_active":"2024-09-11T12:54:22.281530+0000","last_became_peered":"2024-09-11T12:54:22.281530+0000","last_unstale":"2024-09-11T12:54:24.295454+0000","last_undegraded":"2024-09-11T12:54:24.295454+0000","last_fullsized":"2024-09-11T12:54:24.295454+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:40.676738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585966+0000","last_change":"2024-09-11T12:54:23.304321+0000","last_active":"2024-09-11T12:54:24.585966+0000","last_peered":"2024-09-11T12:54:24.585966+0000","last_clean":"2024-09-11T12:54:24.585966+0000","last_became_active":"2024-09-11T12:54:23.304015+0000","last_became_peered":"2024-09-11T12:54:23.304015+0000","last_unstale":"2024-09-11T12:54:24.585966+0000","last_undegraded":"2024-09-11T12:54:24.585966+0000","last_fullsized":"2024-09-11T12:54:24.585966+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587085+0000","last_change":"2024-09-11T12:54:22.292840+0000","last_active":"2024-09-11T12:54:24.587085+0000","last_peered":"2024-09-11T12:54:24.587085+0000","last_clean":"2024-09-11T12:54:24.587085+0000","last_became_active":"2024-09-11T12:54:22.292572+0000","last_became_peered":"2024-09-11T12:54:22.292572+0000","last_unstale":"2024-09-11T12:54:24.587085+0000","last_undegraded":"2024-09-11T12:54:24.587085+0000","last_fullsized":"2024-09-11T12:54:24.587085+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:08:33.168923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.585929+0000","last_change":"2024-09-11T12:54:23.295550+0000","last_active":"2024-09-11T12:54:24.585929+0000","last_peered":"2024-09-11T12:54:24.585929+0000","last_clean":"2024-09-11T12:54:24.585929+0000","last_became_active":"2024-09-11T12:54:23.295301+0000","last_became_peered":"2024-09-11T12:54:23.295301+0000","last_unstale":"2024-09-11T12:54:24.585929+0000","last_undegraded":"2024-09-11T12:54:24.585929+0000","last_fullsized":"2024-09-11T12:54:24.585929+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:43:30.735031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295772+0000","last_change":"2024-09-11T12:54:22.284786+0000","last_active":"2024-09-11T12:54:24.295772+0000","last_peered":"2024-09-11T12:54:24.295772+0000","last_clean":"2024-09-11T12:54:24.295772+0000","last_became_active":"2024-09-11T12:54:22.284485+0000","last_became_peered":"2024-09-11T12:54:22.284485+0000","last_unstale":"2024-09-11T12:54:24.295772+0000","last_undegraded":"2024-09-11T12:54:24.295772+0000","last_fullsized":"2024-09-11T12:54:24.295772+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:14.052663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586052+0000","last_change":"2024-09-11T12:54:23.306341+0000","last_active":"2024-09-11T12:54:24.586052+0000","last_peered":"2024-09-11T12:54:24.586052+0000","last_clean":"2024-09-11T12:54:24.586052+0000","last_became_active":"2024-09-11T12:54:23.306085+0000","last_became_peered":"2024-09-11T12:54:23.306085+0000","last_unstale":"2024-09-11T12:54:24.586052+0000","last_undegraded":"2024-09-11T12:54:24.586052+0000","last_fullsized":"2024-09-11T12:54:24.586052+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586162+0000","last_change":"2024-09-11T12:54:22.292981+0000","last_active":"2024-09-11T12:54:24.586162+0000","last_peered":"2024-09-11T12:54:24.586162+0000","last_clean":"2024-09-11T12:54:24.586162+0000","last_became_active":"2024-09-11T12:54:22.292743+0000","last_became_peered":"2024-09-11T12:54:22.292743+0000","last_unstale":"2024-09-11T12:54:24.586162+0000","last_undegraded":"2024-09-11T12:54:24.586162+0000","last_fullsized":"2024-09-11T12:54:24.586162+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:32:41.359164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587305+0000","last_change":"2024-09-11T12:54:23.315721+0000","last_active":"2024-09-11T12:54:24.587305+0000","last_peered":"2024-09-11T12:54:24.587305+0000","last_clean":"2024-09-11T12:54:24.587305+0000","last_became_active":"2024-09-11T12:54:23.315391+0000","last_became_peered":"2024-09-11T12:54:23.315391+0000","last_unstale":"2024-09-11T12:54:24.587305+0000","last_undegraded":"2024-09-11T12:54:24.587305+0000","last_fullsized":"2024-09-11T12:54:24.587305+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:28:18.114203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586566+0000","last_change":"2024-09-11T12:54:22.281607+0000","last_active":"2024-09-11T12:54:24.586566+0000","last_peered":"2024-09-11T12:54:24.586566+0000","last_clean":"2024-09-11T12:54:24.586566+0000","last_became_active":"2024-09-11T12:54:22.281332+0000","last_became_peered":"2024-09-11T12:54:22.281332+0000","last_unstale":"2024-09-11T12:54:24.586566+0000","last_undegraded":"2024-09-11T12:54:24.586566+0000","last_fullsized":"2024-09-11T12:54:24.586566+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:26:35.367639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302809+0000","last_change":"2024-09-11T12:54:23.316537+0000","last_active":"2024-09-11T12:54:24.302809+0000","last_peered":"2024-09-11T12:54:24.302809+0000","last_clean":"2024-09-11T12:54:24.302809+0000","last_became_active":"2024-09-11T12:54:23.316078+0000","last_became_peered":"2024-09-11T12:54:23.316078+0000","last_unstale":"2024-09-11T12:54:24.302809+0000","last_undegraded":"2024-09-11T12:54:24.302809+0000","last_fullsized":"2024-09-11T12:54:24.302809+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:43:27.796678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587406+0000","last_change":"2024-09-11T12:54:22.285756+0000","last_active":"2024-09-11T12:54:24.587406+0000","last_peered":"2024-09-11T12:54:24.587406+0000","last_clean":"2024-09-11T12:54:24.587406+0000","last_became_active":"2024-09-11T12:54:22.285461+0000","last_became_peered":"2024-09-11T12:54:22.285461+0000","last_unstale":"2024-09-11T12:54:24.587406+0000","last_undegraded":"2024-09-11T12:54:24.587406+0000","last_fullsized":"2024-09-11T12:54:24.587406+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:27:22.913448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587495+0000","last_change":"2024-09-11T12:54:23.308626+0000","last_active":"2024-09-11T12:54:24.587495+0000","last_peered":"2024-09-11T12:54:24.587495+0000","last_clean":"2024-09-11T12:54:24.587495+0000","last_became_active":"2024-09-11T12:54:23.308104+0000","last_became_peered":"2024-09-11T12:54:23.308104+0000","last_unstale":"2024-09-11T12:54:24.587495+0000","last_undegraded":"2024-09-11T12:54:24.587495+0000","last_fullsized":"2024-09-11T12:54:24.587495+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:37.268981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"101'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.304281+0000","last_change":"2024-09-11T12:54:22.284167+0000","last_active":"2024-09-11T12:54:24.304281+0000","last_peered":"2024-09-11T12:54:24.304281+0000","last_clean":"2024-09-11T12:54:24.304281+0000","last_became_active":"2024-09-11T12:54:22.283819+0000","last_became_peered":"2024-09-11T12:54:22.283819+0000","last_unstale":"2024-09-11T12:54:24.304281+0000","last_undegraded":"2024-09-11T12:54:24.304281+0000","last_fullsized":"2024-09-11T12:54:24.304281+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:49.564759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586549+0000","last_change":"2024-09-11T12:54:23.299220+0000","last_active":"2024-09-11T12:54:24.586549+0000","last_peered":"2024-09-11T12:54:24.586549+0000","last_clean":"2024-09-11T12:54:24.586549+0000","last_became_active":"2024-09-11T12:54:23.298934+0000","last_became_peered":"2024-09-11T12:54:23.298934+0000","last_unstale":"2024-09-11T12:54:24.586549+0000","last_undegraded":"2024-09-11T12:54:24.586549+0000","last_fullsized":"2024-09-11T12:54:24.586549+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:13.789321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586814+0000","last_change":"2024-09-11T12:54:22.284407+0000","last_active":"2024-09-11T12:54:24.586814+0000","last_peered":"2024-09-11T12:54:24.586814+0000","last_clean":"2024-09-11T12:54:24.586814+0000","last_became_active":"2024-09-11T12:54:22.284135+0000","last_became_peered":"2024-09-11T12:54:22.284135+0000","last_unstale":"2024-09-11T12:54:24.586814+0000","last_undegraded":"2024-09-11T12:54:24.586814+0000","last_fullsized":"2024-09-11T12:54:24.586814+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:03:29.812151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.302840+0000","last_change":"2024-09-11T12:54:23.315053+0000","last_active":"2024-09-11T12:54:24.302840+0000","last_peered":"2024-09-11T12:54:24.302840+0000","last_clean":"2024-09-11T12:54:24.302840+0000","last_became_active":"2024-09-11T12:54:23.314676+0000","last_became_peered":"2024-09-11T12:54:23.314676+0000","last_unstale":"2024-09-11T12:54:24.302840+0000","last_undegraded":"2024-09-11T12:54:24.302840+0000","last_fullsized":"2024-09-11T12:54:24.302840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:59.849496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.587255+0000","last_change":"2024-09-11T12:54:22.289564+0000","last_active":"2024-09-11T12:54:24.587255+0000","last_peered":"2024-09-11T12:54:24.587255+0000","last_clean":"2024-09-11T12:54:24.587255+0000","last_became_active":"2024-09-11T12:54:22.289307+0000","last_became_peered":"2024-09-11T12:54:22.289307+0000","last_unstale":"2024-09-11T12:54:24.587255+0000","last_undegraded":"2024-09-11T12:54:24.587255+0000","last_fullsized":"2024-09-11T12:54:24.587255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:22:53.763067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295742+0000","last_change":"2024-09-11T12:54:23.284960+0000","last_active":"2024-09-11T12:54:24.295742+0000","last_peered":"2024-09-11T12:54:24.295742+0000","last_clean":"2024-09-11T12:54:24.295742+0000","last_became_active":"2024-09-11T12:54:23.284632+0000","last_became_peered":"2024-09-11T12:54:23.284632+0000","last_unstale":"2024-09-11T12:54:24.295742+0000","last_undegraded":"2024-09-11T12:54:24.295742+0000","last_fullsized":"2024-09-11T12:54:24.295742+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:50.571559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295311+0000","last_change":"2024-09-11T12:54:22.277045+0000","last_active":"2024-09-11T12:54:24.295311+0000","last_peered":"2024-09-11T12:54:24.295311+0000","last_clean":"2024-09-11T12:54:24.295311+0000","last_became_active":"2024-09-11T12:54:22.276685+0000","last_became_peered":"2024-09-11T12:54:22.276685+0000","last_unstale":"2024-09-11T12:54:24.295311+0000","last_undegraded":"2024-09-11T12:54:24.295311+0000","last_fullsized":"2024-09-11T12:54:24.295311+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:05:10.956963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295980+0000","last_change":"2024-09-11T12:54:23.305703+0000","last_active":"2024-09-11T12:54:24.295980+0000","last_peered":"2024-09-11T12:54:24.295980+0000","last_clean":"2024-09-11T12:54:24.295980+0000","last_became_active":"2024-09-11T12:54:23.305337+0000","last_became_peered":"2024-09-11T12:54:23.305337+0000","last_unstale":"2024-09-11T12:54:24.295980+0000","last_undegraded":"2024-09-11T12:54:24.295980+0000","last_fullsized":"2024-09-11T12:54:24.295980+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:42.783808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295560+0000","last_change":"2024-09-11T12:54:22.289227+0000","last_active":"2024-09-11T12:54:24.295560+0000","last_peered":"2024-09-11T12:54:24.295560+0000","last_clean":"2024-09-11T12:54:24.295560+0000","last_became_active":"2024-09-11T12:54:22.288770+0000","last_became_peered":"2024-09-11T12:54:22.288770+0000","last_unstale":"2024-09-11T12:54:24.295560+0000","last_undegraded":"2024-09-11T12:54:24.295560+0000","last_fullsized":"2024-09-11T12:54:24.295560+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:09:37.699127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586295+0000","last_change":"2024-09-11T12:54:23.300195+0000","last_active":"2024-09-11T12:54:24.586295+0000","last_peered":"2024-09-11T12:54:24.586295+0000","last_clean":"2024-09-11T12:54:24.586295+0000","last_became_active":"2024-09-11T12:54:23.299862+0000","last_became_peered":"2024-09-11T12:54:23.299862+0000","last_unstale":"2024-09-11T12:54:24.586295+0000","last_undegraded":"2024-09-11T12:54:24.586295+0000","last_fullsized":"2024-09-11T12:54:24.586295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:59.609063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295766+0000","last_change":"2024-09-11T12:54:22.292101+0000","last_active":"2024-09-11T12:54:24.295766+0000","last_peered":"2024-09-11T12:54:24.295766+0000","last_clean":"2024-09-11T12:54:24.295766+0000","last_became_active":"2024-09-11T12:54:22.291824+0000","last_became_peered":"2024-09-11T12:54:22.291824+0000","last_unstale":"2024-09-11T12:54:24.295766+0000","last_undegraded":"2024-09-11T12:54:24.295766+0000","last_fullsized":"2024-09-11T12:54:24.295766+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:48.789636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586319+0000","last_change":"2024-09-11T12:54:23.312068+0000","last_active":"2024-09-11T12:54:24.586319+0000","last_peered":"2024-09-11T12:54:24.586319+0000","last_clean":"2024-09-11T12:54:24.586319+0000","last_became_active":"2024-09-11T12:54:23.311424+0000","last_became_peered":"2024-09-11T12:54:23.311424+0000","last_unstale":"2024-09-11T12:54:24.586319+0000","last_undegraded":"2024-09-11T12:54:24.586319+0000","last_fullsized":"2024-09-11T12:54:24.586319+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:25:18.915354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586655+0000","last_change":"2024-09-11T12:54:22.280306+0000","last_active":"2024-09-11T12:54:24.586655+0000","last_peered":"2024-09-11T12:54:24.586655+0000","last_clean":"2024-09-11T12:54:24.586655+0000","last_became_active":"2024-09-11T12:54:22.279691+0000","last_became_peered":"2024-09-11T12:54:22.279691+0000","last_unstale":"2024-09-11T12:54:24.586655+0000","last_undegraded":"2024-09-11T12:54:24.586655+0000","last_fullsized":"2024-09-11T12:54:24.586655+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:39.525780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586132+0000","last_change":"2024-09-11T12:54:23.297043+0000","last_active":"2024-09-11T12:54:24.586132+0000","last_peered":"2024-09-11T12:54:24.586132+0000","last_clean":"2024-09-11T12:54:24.586132+0000","last_became_active":"2024-09-11T12:54:23.296787+0000","last_became_peered":"2024-09-11T12:54:23.296787+0000","last_unstale":"2024-09-11T12:54:24.586132+0000","last_undegraded":"2024-09-11T12:54:24.586132+0000","last_fullsized":"2024-09-11T12:54:24.586132+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:22.258823+0000","last_clean_scrub_stamp":"2024-09-11T12:54:22.258823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:16:16.007323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.295022+0000","last_change":"2024-09-11T12:54:22.280451+0000","last_active":"2024-09-11T12:54:24.295022+0000","last_peered":"2024-09-11T12:54:24.295022+0000","last_clean":"2024-09-11T12:54:24.295022+0000","last_became_active":"2024-09-11T12:54:22.280063+0000","last_became_peered":"2024-09-11T12:54:22.280063+0000","last_unstale":"2024-09-11T12:54:24.295022+0000","last_undegraded":"2024-09-11T12:54:24.295022+0000","last_fullsized":"2024-09-11T12:54:24.295022+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:54:21.252022+0000","last_clean_scrub_stamp":"2024-09-11T12:54:21.252022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:46:01.586894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":281,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-11T12:54:24.586352+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:54:24.586352+0000","last_peered":"2024-09-11T12:54:24.586352+0000","last_clean":"2024-09-11T12:54:24.586352+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:54:24.586352+0000","last_undegraded":"2024-09-11T12:54:24.586352+0000","last_fullsized":"2024-09-11T12:54:24.586352+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33000,"kb_used_data":1808,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338840,"statfs":{"total":96636764160,"available":96602972160,"internally_reserved":0,"allocated":1851392,"data_stored":1134963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1253376,"data_stored":544049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509502,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1257472,"data_stored":544583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":543979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509502,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32428,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339412,"statfs":{"total":96636764160,"available":96603557888,"internally_reserved":0,"allocated":1265664,"data_stored":545661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509502,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32412,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339428,"statfs":{"total":96636764160,"available":96603574272,"internally_reserved":0,"allocated":1249280,"data_stored":543979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509502,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32424,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339416,"statfs":{"total":96636764160,"available":96603561984,"internally_reserved":0,"allocated":1257472,"data_stored":544103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542206,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1863680,"data_stored":1136071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:54:27.788 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:54:27.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- 172.21.15.78:0/2147055327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c1524f0 msgr2=0x7ff25c152960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- 172.21.15.78:0/2147055327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c1524f0 0x7ff25c152960 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff254009bc0 tx=0x7ff25401cba0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- 172.21.15.78:0/2147055327 shutdown_connections 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- 172.21.15.78:0/2147055327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c1524f0 0x7ff25c152960 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- 172.21.15.78:0/2147055327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cebc0 0x7ff25c0704b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- 172.21.15.78:0/2147055327 >> 172.21.15.78:0/2147055327 conn(0x7ff25c0b2ec0 msgr2=0x7ff25c0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- 172.21.15.78:0/2147055327 shutdown_connections 2024-09-11T12:54:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- 172.21.15.78:0/2147055327 wait complete. 2024-09-11T12:54:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 Processor -- start 2024-09-11T12:54:27.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- start start 2024-09-11T12:54:27.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c0cebc0 0x7ff25c149f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cefa0 0x7ff25c14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c168920 con 0x7ff25c1524f0 2024-09-11T12:54:27.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c1687a0 con 0x7ff25c0cebc0 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff263c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c168aa0 con 0x7ff25c0cefa0 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff261c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff261c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57518/0 (socket says 172.21.15.78:57518) 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff261c00640 1 -- 172.21.15.78:0/3949566397 learned_addr learned my addr 172.21.15.78:0/3949566397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff25be00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cefa0 0x7ff25c14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.979+0000 7ff261200640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c0cebc0 0x7ff25c149f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 -- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cefa0 msgr2=0x7ff25c14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cefa0 0x7ff25c14a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 -- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c0cebc0 msgr2=0x7ff25c149f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c0cebc0 0x7ff25c149f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:27.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 -- 172.21.15.78:0/3949566397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c0ac270 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff261c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff25401a7d0 tx=0x7ff254009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540bb070 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff254004d40 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c0ac550 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2540b2040 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c0acb10 con 0x7ff25c1524f0 2024-09-11T12:54:27.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c0704b0 con 0x7ff25c1524f0 2024-09-11T12:54:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ff2540be070 con 0x7ff25c1524f0 2024-09-11T12:54:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 0x7ff23c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.983+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ff25412baf0 con 0x7ff25c1524f0 2024-09-11T12:54:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.987+0000 7ff261200640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 0x7ff23c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:27.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.987+0000 7ff261200640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 0x7ff23c073c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009be0 tx=0x7ff24c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:27.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:27.991+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff254016790 con 0x7ff25c1524f0 2024-09-11T12:54:28.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.215+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff25c072b80 con 0x7ff25c1524f0 2024-09-11T12:54:28.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.219+0000 7ff258c00640 1 -- 172.21.15.78:0/3949566397 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7ff2540f7160 con 0x7ff25c1524f0 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:54:28.222 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:54:28.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 msgr2=0x7ff23c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 0x7ff23c073c60 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff24c009be0 tx=0x7ff24c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 msgr2=0x7ff25c150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff25401a7d0 tx=0x7ff254009260 comp rx=0 tx=0).stop 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 shutdown_connections 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff25c1524f0 0x7ff25c150210 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff25c0cefa0 0x7ff25c14a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff23c0717a0 0x7ff23c073c60 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 --2- 172.21.15.78:0/3949566397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff25c0cebc0 0x7ff25c149f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 >> 172.21.15.78:0/3949566397 conn(0x7ff25c0b2ec0 msgr2=0x7ff25c15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 shutdown_connections 2024-09-11T12:54:28.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.223+0000 7ff263c00640 1 -- 172.21.15.78:0/3949566397 wait complete. 2024-09-11T12:54:28.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- 172.21.15.78:0/2961154208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc0cf760 msgr2=0x7fe7fc074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- 172.21.15.78:0/2961154208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc074260 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009900 tx=0x7fe7ec01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- 172.21.15.78:0/2961154208 shutdown_connections 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- 172.21.15.78:0/2961154208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc06fc50 0x7fe7fc0700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- 172.21.15.78:0/2961154208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc074260 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- 172.21.15.78:0/2961154208 >> 172.21.15.78:0/2961154208 conn(0x7fe7fc06ca80 msgr2=0x7fe7fc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:28.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- 172.21.15.78:0/2961154208 shutdown_connections 2024-09-11T12:54:28.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- 172.21.15.78:0/2961154208 wait complete. 2024-09-11T12:54:28.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 Processor -- start 2024-09-11T12:54:28.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- start start 2024-09-11T12:54:28.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc15aec0 0x7fe7fc1563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc151010 con 0x7fe7fc06fc50 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe7fc15f2d0 con 0x7fe7fc0cf760 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe803c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe7fc151190 con 0x7fe7fc15aec0 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe801200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe801200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57524/0 (socket says 172.21.15.78:57524) 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe801200640 1 -- 172.21.15.78:0/2347972506 learned_addr learned my addr 172.21.15.78:0/2347972506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.383+0000 7fe7fbe00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc15aec0 0x7fe7fc1563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:28.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 -- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc15aec0 msgr2=0x7fe7fc1563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc15aec0 0x7fe7fc1563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 -- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7fc0cf760 msgr2=0x7fe7fc155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 -- 172.21.15.78:0/2347972506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7fc14a1d0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe801200640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009850 tx=0x7fe7ec004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec0b2040 con 0x7fe7fc06fc50 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ec003600 con 0x7fe7fc06fc50 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7fc155690 con 0x7fe7fc06fc50 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ec003a10 con 0x7fe7fc06fc50 2024-09-11T12:54:28.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7fc14f3a0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7fc0700e0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fe7ec003bb0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 0x7fe7dc073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:28.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fe7ec12beb0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.387+0000 7fe7fbe00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 0x7fe7dc073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:28.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.391+0000 7fe7fbe00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 0x7fe7dc073c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc150fe0 tx=0x7fe7e8008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:28.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.391+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe7ec0be060 con 0x7fe7fc06fc50 2024-09-11T12:54:28.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.591+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7fc074580 con 0x7fe7fc06fc50 2024-09-11T12:54:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.591+0000 7fe7f8c00640 1 -- 172.21.15.78:0/2347972506 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5228 (secure 0 0 0) 0x7fe7ec0164d0 con 0x7fe7fc06fc50 2024-09-11T12:54:28.595 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:28.595 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":71,"btime":"2024-09-11T12:54:26:302435+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:28.595 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 71 2024-09-11T12:54:28.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 msgr2=0x7fe7dc073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 0x7fe7dc073c60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc150fe0 tx=0x7fe7e8008040 comp rx=0 tx=0).stop 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 msgr2=0x7fe7fc149c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec009850 tx=0x7fe7ec004040 comp rx=0 tx=0).stop 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 shutdown_connections 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7fc15aec0 0x7fe7fc1563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe7dc0717a0 0x7fe7dc073c60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe7fc0cf760 0x7fe7fc155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 --2- 172.21.15.78:0/2347972506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7fc06fc50 0x7fe7fc149c90 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 >> 172.21.15.78:0/2347972506 conn(0x7fe7fc06ca80 msgr2=0x7fe7fc15b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 shutdown_connections 2024-09-11T12:54:28.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:28.595+0000 7fe803c00640 1 -- 172.21.15.78:0/2347972506 wait complete. 2024-09-11T12:54:29.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:54:22.659605+0000', 'modified': '2024-09-11T12:54:25.688540+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6894}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6894': {'gid': 6894, 'name': 'c', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/785402870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6833', 'nonce': 785402870}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 785402870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6894, 'qdb_cluster': [6894]} 2024-09-11T12:54:29.020 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:54:29.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- 172.21.15.78:0/460011050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 msgr2=0x7f6f2000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 --2- 172.21.15.78:0/460011050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f2000b9d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009b40 tx=0x7f6f3001d290 comp rx=0 tx=0).stop 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- 172.21.15.78:0/460011050 shutdown_connections 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 --2- 172.21.15.78:0/460011050 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f2000b9d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- 172.21.15.78:0/460011050 >> 172.21.15.78:0/460011050 conn(0x7f6f3806ca80 msgr2=0x7f6f380b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- 172.21.15.78:0/460011050 shutdown_connections 2024-09-11T12:54:29.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- 172.21.15.78:0/460011050 wait complete. 2024-09-11T12:54:29.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 Processor -- start 2024-09-11T12:54:29.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- start start 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 0x7f6f381550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 0x7f6f380721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38074300 con 0x7f6f200051d0 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f38074180 con 0x7f6f3815cf30 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f40600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f38074480 con 0x7f6f38071d60 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f3dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f3dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57550/0 (socket says 172.21.15.78:57550) 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.163+0000 7f6f3dc00640 1 -- 172.21.15.78:0/1339460841 learned_addr learned my addr 172.21.15.78:0/1339460841 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3e600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 0x7f6f380721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 0x7f6f381550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 -- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 msgr2=0x7f6f380721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 0x7f6f380721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 -- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 msgr2=0x7f6f381550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 0x7f6f381550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 -- 172.21.15.78:0/1339460841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f38155610 con 0x7f6f200051d0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3e600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 0x7f6f380721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 0x7f6f381550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0016f0 tx=0x7f6f2c001bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f35600640 1 -- 172.21.15.78:0/1339460841 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c015070 con 0x7f6f200051d0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f35600640 1 -- 172.21.15.78:0/1339460841 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f2c00e9e0 con 0x7f6f200051d0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f381f7a60 con 0x7f6f200051d0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f35600640 1 -- 172.21.15.78:0/1339460841 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c00edd0 con 0x7f6f200051d0 2024-09-11T12:54:29.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381509f0 con 0x7f6f200051d0 2024-09-11T12:54:29.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.167+0000 7f6f35600640 1 -- 172.21.15.78:0/1339460841 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f6f2c018070 con 0x7f6f200051d0 2024-09-11T12:54:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f35600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 0x7f6f18073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 Processor -- start 2024-09-11T12:54:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f35600640 1 -- 172.21.15.78:0/1339460841 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c092910 con 0x7f6f200051d0 2024-09-11T12:54:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- start start 2024-09-11T12:54:29.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 0x7f6f18073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 0x7f6f18073c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001890 tx=0x7f6f240011c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f18bde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1871720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f16ce270 con 0x55f9f18be3c0 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9f16bd770 con 0x55f9f18bdaa0 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f9f16c17b0 con 0x55f9f1871340 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57554/0 (socket says 172.21.15.78:57554) 2024-09-11T12:54:29.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 -- 172.21.15.78:0/1059912217 learned_addr learned my addr 172.21.15.78:0/1059912217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f18bde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3e600640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1871720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 -- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 msgr2=0x55f9f1871720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1871720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 -- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 msgr2=0x55f9f18bde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f18bde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 -- 172.21.15.78:0/1059912217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f165cf90 con 0x55f9f18be3c0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f18bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f3d200640 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6f380cd5e0 tx=0x7f6f2400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69b0f4d69a723e29 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f2a000640 1 -- 172.21.15.78:0/1059912217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24012d40 con 0x55f9f18be3c0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f2a000640 1 -- 172.21.15.78:0/1059912217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f24023070 con 0x55f9f18be3c0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f2a000640 1 -- 172.21.15.78:0/1059912217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2401e690 con 0x55f9f18be3c0 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- 172.21.15.78:0/1059912217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 msgr2=0x55f9f1870e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f6f380cd5e0 tx=0x7f6f2400c000 comp rx=0 tx=0).stop 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- 172.21.15.78:0/1059912217 shutdown_connections 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1871720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18be3c0 0x55f9f1870e00 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 --2- 172.21.15.78:0/1059912217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f18bde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- 172.21.15.78:0/1059912217 >> 172.21.15.78:0/1059912217 conn(0x55f9f189eb60 msgr2=0x55f9f18a0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- 172.21.15.78:0/1059912217 shutdown_connections 2024-09-11T12:54:29.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.171+0000 7f6f439ae000 1 -- 172.21.15.78:0/1059912217 wait complete. 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- start start 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 Processor -- start 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f1871340 0x55f9f1873cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1874750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f178d290 con 0x55f9f18bdaa0 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9f1669ab0 con 0x55f9f1871340 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f9f1667170 con 0x55f9f18be3c0 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 10 client.0 _ll_register_callbacks cb 0x55f9f185e830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f1871340 0x55f9f1873cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57568/0 (socket says 172.21.15.78:57568) 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 -- 172.21.15.78:0/3631620428 learned_addr learned my addr 172.21.15.78:0/3631620428 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3e600640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1874750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 -- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 msgr2=0x55f9f1874750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1874750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 -- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f1871340 msgr2=0x55f9f1873cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f1871340 0x55f9f1873cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 -- 172.21.15.78:0/3631620428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f17426f0 con 0x55f9f18bdaa0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3d200640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001270 tx=0x7f6f24014e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fa720524bfe6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24014270 con 0x55f9f18bdaa0 2024-09-11T12:54:29.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f9f1656f10 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f6f24014410 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6f24038050 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 10 client.7005 fetch_fsmap learned FSMap version 71 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55f9f1719c40 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f9f1656f10 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f6f2403b020 con 0x55f9f18bdaa0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 10 client.7005 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 10 client.7005 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 20 client.7005 populate_metadata read hostname 'smithi078' 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f1875160 0x55f9f18cfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 4 client.7005 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 --> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9f16d01c0 con 0x55f9f1875160 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f1875160 0x55f9f18cfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f1ea00640 10 client.7005 ms_handle_connect on v2:172.21.15.78:6833/785402870 2024-09-11T12:54:29.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.175+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f1875160 0x55f9f18cfd60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9ae8fc200089fbb server_cookie=44c29ca8766ccf56 in_seq=0 out_seq=0 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f1ea00640 1 -- 172.21.15.78:0/3631620428 <== mds.0 v2:172.21.15.78:6833/785402870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f9f16d01c0 con 0x55f9f1875160 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f1ea00640 10 client.7005 handle_command_reply: tid=0 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 client.7005 shutdown 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 msgr2=0x55f9f1874210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001270 tx=0x7f6f24014e80 comp rx=0 tx=0).stop 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 shutdown_connections 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f1875160 0x55f9f18cfd60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1874750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1874210 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 --2- 172.21.15.78:0/3631620428 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f1871340 0x55f9f1873cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f1ea00640 0 client.7005 ms_handle_reset on v2:172.21.15.78:6833/785402870 2024-09-11T12:54:29.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 >> 172.21.15.78:0/3631620428 conn(0x55f9f189eb60 msgr2=0x55f9f1870500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 shutdown_connections 2024-09-11T12:54:29.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 1 -- 172.21.15.78:0/3631620428 wait complete. 2024-09-11T12:54:29.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.179+0000 7f6f439ae000 20 client.7005 trim_cache size 0 max 16384 2024-09-11T12:54:29.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.195+0000 7f6f439ae000 1 Processor -- start 2024-09-11T12:54:29.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.195+0000 7f6f439ae000 1 -- start start 2024-09-11T12:54:29.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1870500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18be3c0 0x55f9f18d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6f382001c0 con 0x55f9f18bdaa0 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6f380ae1d0 con 0x55f9f18be3c0 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6f381ffc50 con 0x55f9f1871340 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57578/0 (socket says 172.21.15.78:57578) 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 -- 172.21.15.78:0/4171778039 learned_addr learned my addr 172.21.15.78:0/4171778039 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1870500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3e600640 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18be3c0 0x55f9f18d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 -- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 msgr2=0x55f9f1870500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1870500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 -- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18be3c0 msgr2=0x55f9f18d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18be3c0 0x55f9f18d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 -- 172.21.15.78:0/4171778039 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9f1672e70 con 0x55f9f18bdaa0 2024-09-11T12:54:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f3d200640 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001e90 tx=0x7f6f24000f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5325fb88f8b32c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f29600640 1 -- 172.21.15.78:0/4171778039 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24041070 con 0x55f9f18bdaa0 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f29600640 1 -- 172.21.15.78:0/4171778039 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f2402c030 con 0x55f9f18bdaa0 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f29600640 1 -- 172.21.15.78:0/4171778039 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f24035030 con 0x55f9f18bdaa0 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- 172.21.15.78:0/4171778039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 msgr2=0x55f9f1870a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001e90 tx=0x7f6f24000f00 comp rx=0 tx=0).stop 2024-09-11T12:54:29.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- 172.21.15.78:0/4171778039 shutdown_connections 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18be3c0 0x55f9f18d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f18bdaa0 0x55f9f1870a40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- 172.21.15.78:0/4171778039 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f1871340 0x55f9f1870500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- 172.21.15.78:0/4171778039 >> 172.21.15.78:0/4171778039 conn(0x55f9f189eb60 msgr2=0x55f9f186fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- 172.21.15.78:0/4171778039 shutdown_connections 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- 172.21.15.78:0/4171778039 wait complete. 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- start start 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 Processor -- start 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f1931560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1931aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f9f16c5de0 con 0x55f9f1871340 2024-09-11T12:54:29.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f9f16c0ac0 con 0x55f9f18bdaa0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f9f16d0ea0 con 0x55f9f18be3c0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.199+0000 7f6f439ae000 10 client.0 _ll_register_callbacks cb 0x7f6f24010b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57584/0 (socket says 172.21.15.78:57584) 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 -- 172.21.15.78:0/563365722 learned_addr learned my addr 172.21.15.78:0/563365722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3e600640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1931aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3d200640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f1931560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 -- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 msgr2=0x55f9f1931aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1931aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 -- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 msgr2=0x55f9f1931560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f1931560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 -- 172.21.15.78:0/563365722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9f1647bf0 con 0x55f9f1871340 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3dc00640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c05b0c0 tx=0x7f6f2c05a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661b0faedcda84eb server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:54:29.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c01f040 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f6f2c05d0b0 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f2402c030 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) ==== 24+0+0 (secure 0 0 0) 0x7f6f2c062450 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 10 client.7011 fetch_fsmap learned FSMap version 71 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) -- 0x55f9f1663c60 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f2402c030 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 71) ==== 3502+0+0 (secure 0 0 0) 0x7f6f2c0597c0 con 0x55f9f1871340 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 10 client.7011 fetch_fsmap finished waiting for FSMap version 71 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 10 client.7011 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 20 client.7011 populate_metadata read hostname 'smithi078' 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f18ed760 0x55f9f18ec770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 4 client.7011 mds_command: new command op to 6894 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:54:29.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 --> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] -- command(tid 0: {"prefix": "status"}) -- 0x55f9f16cbc60 con 0x55f9f18ed760 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3d200640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f18ed760 0x55f9f18ec770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f3d200640 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f18ed760 0x55f9f18ec770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b957a57f6d08e0a9 server_cookie=f416085c54a98cbf in_seq=0 out_seq=0 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 10 client.7011 ms_handle_connect on v2:172.21.15.78:6833/785402870 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 1 -- 172.21.15.78:0/563365722 <== mds.0 v2:172.21.15.78:6833/785402870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f9f16cbc60 con 0x55f9f18ed760 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 10 client.7011 handle_command_reply: tid=0 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 client.7011 shutdown 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 msgr2=0x55f9f1931020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c05b0c0 tx=0x7f6f2c05a3b0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 shutdown_connections 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:6833/785402870,v1:172.21.15.78:6835/785402870] conn(0x55f9f18ed760 0x55f9f18ec770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f9f18be3c0 0x55f9f1931aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f9f18bdaa0 0x55f9f1931560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 --2- 172.21.15.78:0/563365722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f9f1871340 0x55f9f1931020 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f1ea00640 0 client.7011 ms_handle_reset on v2:172.21.15.78:6833/785402870 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 >> 172.21.15.78:0/563365722 conn(0x55f9f189eb60 msgr2=0x55f9f1870500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 shutdown_connections 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 1 -- 172.21.15.78:0/563365722 wait complete. 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f439ae000 20 client.7011 trim_cache size 0 max 16384 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 6894, 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:54:29.211 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9182660279999997, 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 71, 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 101, 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 101, 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.431160208 2024-09-11T12:54:29.212 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 msgr2=0x7f6f18073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 0x7f6f18073c60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f6f24001890 tx=0x7f6f240011c0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 msgr2=0x7f6f380aefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c0016f0 tx=0x7f6f2c001bb0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 shutdown_connections 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6f38071d60 0x7f6f380721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f180717a0 0x7f6f18073c60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6f3815cf30 0x7f6f381550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 --2- 172.21.15.78:0/1339460841 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6f200051d0 0x7f6f380aefe0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 >> 172.21.15.78:0/1339460841 conn(0x7f6f3806ca80 msgr2=0x7f6f380b19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 shutdown_connections 2024-09-11T12:54:29.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.203+0000 7f6f40600640 1 -- 172.21.15.78:0/1339460841 wait complete. 2024-09-11T12:54:29.226 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:54:29.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- 172.21.15.78:0/4273328500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4070a00 msgr2=0x7f0ad40736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 --2- 172.21.15.78:0/4273328500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4070a00 0x7f0ad40736d0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f0ad415dad0 tx=0x7f0ac401c830 comp rx=0 tx=0).stop 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- 172.21.15.78:0/4273328500 shutdown_connections 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 --2- 172.21.15.78:0/4273328500 >> v2:172.21.15.157:3300/0 conn(0x7f0ad406eb40 0x7f0ad406efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 --2- 172.21.15.78:0/4273328500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4070a00 0x7f0ad40736d0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- 172.21.15.78:0/4273328500 >> 172.21.15.78:0/4273328500 conn(0x7f0ad406ca80 msgr2=0x7f0ad40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- 172.21.15.78:0/4273328500 shutdown_connections 2024-09-11T12:54:29.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- 172.21.15.78:0/4273328500 wait complete. 2024-09-11T12:54:29.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 Processor -- start 2024-09-11T12:54:29.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.367+0000 7f0adbc00640 1 -- start start 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad406eb40 0x7f0ad4159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 0x7f0ad415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad40b51a0 con 0x7f0ad4157ae0 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad40b5020 con 0x7f0ad4070a00 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad40b5320 con 0x7f0ad406eb40 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57600/0 (socket says 172.21.15.78:57600) 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 -- 172.21.15.78:0/43925022 learned_addr learned my addr 172.21.15.78:0/43925022 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9200640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad406eb40 0x7f0ad4159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad3e00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 0x7f0ad415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 -- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad406eb40 msgr2=0x7f0ad4159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad406eb40 0x7f0ad4159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 -- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 msgr2=0x7f0ad415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 0x7f0ad415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 -- 172.21.15.78:0/43925022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad415bc20 con 0x7f0ad4157ae0 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad3e00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 0x7f0ad415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:29.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad9c00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00cce0 tx=0x7f0acc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc00f450 con 0x7f0ad4157ae0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0acc00f700 con 0x7f0ad4157ae0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0acc00fb50 con 0x7f0ad4157ae0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad415bf00 con 0x7f0ad4157ae0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad415c4c0 con 0x7f0ad4157ae0 2024-09-11T12:54:29.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad406efb0 con 0x7f0ad4157ae0 2024-09-11T12:54:29.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.371+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f0acc00fcf0 con 0x7f0ad4157ae0 2024-09-11T12:54:29.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.375+0000 7f0ad0c00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 0x7f0ab4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:29.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.375+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f0acc092b20 con 0x7f0ad4157ae0 2024-09-11T12:54:29.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.375+0000 7f0ad9200640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 0x7f0ab4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:29.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.375+0000 7f0ad9200640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 0x7f0ab4073c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40af000 tx=0x7f0ac401b070 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:29.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.379+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0acc014030 con 0x7f0ad4157ae0 2024-09-11T12:54:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.579+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0ad41584b0 con 0x7f0ad4157ae0 2024-09-11T12:54:29.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.579+0000 7f0ad0c00640 1 -- 172.21.15.78:0/43925022 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f0acc05e760 con 0x7f0ad4157ae0 2024-09-11T12:54:29.582 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 msgr2=0x7f0ab4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 0x7f0ab4073c60 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40af000 tx=0x7f0ac401b070 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 msgr2=0x7f0ad415d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0acc00cce0 tx=0x7f0acc00a620 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 shutdown_connections 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ad4157ae0 0x7f0ad415d2d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ab40717a0 0x7f0ab4073c60 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0ad4070a00 0x7f0ad415cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 --2- 172.21.15.78:0/43925022 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0ad406eb40 0x7f0ad4159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 >> 172.21.15.78:0/43925022 conn(0x7f0ad406ca80 msgr2=0x7f0ad40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 shutdown_connections 2024-09-11T12:54:29.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29.583+0000 7f0adbc00640 1 -- 172.21.15.78:0/43925022 wait complete. 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:54:29.604 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:54:29.605 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:54:29.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:29.605 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:54:29.614 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:29.615 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42165sec preferred_lft 42165sec 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.616 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:54:29.617 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:54:29.618 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:29.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:29.618 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:54:29.672 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:54:29.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:29.675 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:54:29.730 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:54:29.730 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:54:29.730 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:54:29.730 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:29.731 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:54:29.731 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:54:29.731 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:54:29.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:54:29.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:54:29.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:54:29.732 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:54:29.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:54:29.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:54:29.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:29.864 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:54:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:54:29.873 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:54:29.873 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:54:29.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:29.874 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:54:29.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:54:29.935 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:29.983 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:54:29.983 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:54:29.984 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:30.039 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:54:30.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:30.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:30.131 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:54:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:54:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:54:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:54:30.147 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:54:30.151 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:54:30.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:30.152 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:30.171 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:54:30.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:54:30.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:54:30.285 INFO:teuthology.orchestra.run.smithi078.stdout:client6071 2024-09-11T12:54:30.285 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:30.285 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:30.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- 172.21.15.78:0/101950814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a1815a1c0 msgr2=0x7f4a180cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 --2- 172.21.15.78:0/101950814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a1815a1c0 0x7f4a180cbed0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009900 tx=0x7f4a0001c820 comp rx=0 tx=0).stop 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- 172.21.15.78:0/101950814 shutdown_connections 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 --2- 172.21.15.78:0/101950814 >> v2:172.21.15.157:3300/0 conn(0x7f4a1813d250 0x7f4a180cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 --2- 172.21.15.78:0/101950814 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a1815a1c0 0x7f4a180cbed0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- 172.21.15.78:0/101950814 >> 172.21.15.78:0/101950814 conn(0x7f4a1806ca80 msgr2=0x7f4a180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- 172.21.15.78:0/101950814 shutdown_connections 2024-09-11T12:54:30.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- 172.21.15.78:0/101950814 wait complete. 2024-09-11T12:54:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 Processor -- start 2024-09-11T12:54:30.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.475+0000 7f4a1e600640 1 -- start start 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4780 0x7f4a18142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815a1c0 0x7f4a180b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1813ead0 con 0x7f4a1813d250 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1813e950 con 0x7f4a1815a1c0 2024-09-11T12:54:30.482 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4a1813ec50 con 0x7f4a180b4780 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57602/0 (socket says 172.21.15.78:57602) 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 -- 172.21.15.78:0/224331916 learned_addr learned my addr 172.21.15.78:0/224331916 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a17400640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4780 0x7f4a18142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a17e00640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815a1c0 0x7f4a180b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 -- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4780 msgr2=0x7f4a18142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4780 0x7f4a18142e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 -- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815a1c0 msgr2=0x7f4a180b0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815a1c0 0x7f4a180b0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 -- 172.21.15.78:0/224331916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a180b1770 con 0x7f4a1813d250 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0fe00640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009850 tx=0x7f4a000a5740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a000b2040 con 0x7f4a1813d250 2024-09-11T12:54:30.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a000043b0 con 0x7f4a1813d250 2024-09-11T12:54:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a180b1a50 con 0x7f4a1813d250 2024-09-11T12:54:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00005280 con 0x7f4a1813d250 2024-09-11T12:54:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a18140e10 con 0x7f4a1813d250 2024-09-11T12:54:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.479+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a1813f100 con 0x7f4a1813d250 2024-09-11T12:54:30.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.483+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f4a00005420 con 0x7f4a1813d250 2024-09-11T12:54:30.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.483+0000 7f4a0f400640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 0x7f49f4073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:30.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.483+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f4a0012be40 con 0x7f4a1813d250 2024-09-11T12:54:30.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.483+0000 7f4a17400640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 0x7f49f4073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:30.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.483+0000 7f4a17400640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 0x7f49f4073c60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a080016f0 tx=0x7f4a08009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:30.489 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.487+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4a000164d0 con 0x7f4a1813d250 2024-09-11T12:54:30.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.691+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4a180cfce0 con 0x7f4a1813d250 2024-09-11T12:54:30.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.691+0000 7f4a0f400640 1 -- 172.21.15.78:0/224331916 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) ==== 53+0+73 (secure 0 0 0) 0x7f4a000f7b10 con 0x7f4a1813d250 2024-09-11T12:54:30.695 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 msgr2=0x7f49f4073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 0x7f49f4073c60 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4a080016f0 tx=0x7f4a08009040 comp rx=0 tx=0).stop 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 msgr2=0x7f4a180a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f4a00009850 tx=0x7f4a000a5740 comp rx=0 tx=0).stop 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 shutdown_connections 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f49f40717a0 0x7f49f4073c60 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4a1815a1c0 0x7f4a180b0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4a1813d250 0x7f4a180a9460 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 --2- 172.21.15.78:0/224331916 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4a180b4780 0x7f4a18142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:30.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 >> 172.21.15.78:0/224331916 conn(0x7f4a1806ca80 msgr2=0x7f4a1813c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:30.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 shutdown_connections 2024-09-11T12:54:30.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:30.695+0000 7f4a1e600640 1 -- 172.21.15.78:0/224331916 wait complete. 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:54:30.716 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:54:30.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:30.717 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:54:30.727 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42173sec preferred_lft 42173sec 2024-09-11T12:54:30.728 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:54:30.729 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:54:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.730 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:54:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:54:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:54:30.730 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:54:30.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:30.731 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:54:30.795 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:54:30.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:30.798 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:54:30.855 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:54:30.856 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:30.856 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:30.856 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:54:30.856 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:54:30.856 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:54:30.951 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:54:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:54:30.996 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:54:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:54:31.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:31.002 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:54:31.011 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:54:31.012 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:54:31.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:54:31.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:31.012 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:54:31.075 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:54:31.075 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:31.127 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:54:31.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:54:31.127 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:31.182 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:54:31.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:31.183 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:31.271 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:31.272 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:54:31.285 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:54:31.286 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:54:31.289 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:54:31.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:31.290 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:31.311 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:54:31.402 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:54:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:54:31.405 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:54:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:54:31.411 INFO:teuthology.orchestra.run.smithi157.stdout:client7026 2024-09-11T12:54:31.412 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:31.412 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:31.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- 172.21.15.78:0/1311271043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51500051d0 msgr2=0x7f515000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 --2- 172.21.15.78:0/1311271043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51500051d0 0x7f515000b9d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f516c0af690 tx=0x7f516401f980 comp rx=0 tx=0).stop 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516be00640 1 -- 172.21.15.78:0/1311271043 reap_dead start 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- 172.21.15.78:0/1311271043 shutdown_connections 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- 172.21.15.78:0/1311271043 >> 172.21.15.78:0/1311271043 conn(0x7f516c0b3050 msgr2=0x7f516c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- 172.21.15.78:0/1311271043 shutdown_connections 2024-09-11T12:54:31.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- 172.21.15.78:0/1311271043 wait complete. 2024-09-11T12:54:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 Processor -- start 2024-09-11T12:54:31.570 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- start start 2024-09-11T12:54:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51500051d0 0x7f516c1f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:31.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f516c1577a0 0x7f516c1f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0cdd20 con 0x7f516c1f7f10 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f516c0cdba0 con 0x7f51500051d0 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f5172200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f516c0cdea0 con 0x7f516c1577a0 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57612/0 (socket says 172.21.15.78:57612) 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516aa00640 1 -- 172.21.15.78:0/2952745030 learned_addr learned my addr 172.21.15.78:0/2952745030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516b400640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51500051d0 0x7f516c1f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.567+0000 7f516be00640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f516c1577a0 0x7f516c1f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 -- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f516c1577a0 msgr2=0x7f516c1f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f516c1577a0 0x7f516c1f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 -- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51500051d0 msgr2=0x7f516c1f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51500051d0 0x7f516c1f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 -- 172.21.15.78:0/2952745030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516c157fa0 con 0x7f516c1f7f10 2024-09-11T12:54:31.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f516aa00640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5158003f00 tx=0x7f5158001810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158015070 con 0x7f516c1f7f10 2024-09-11T12:54:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5158011920 con 0x7f516c1f7f10 2024-09-11T12:54:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f516c1f56b0 con 0x7f516c1f7f10 2024-09-11T12:54:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5158011d70 con 0x7f516c1f7f10 2024-09-11T12:54:31.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f516c157c20 con 0x7f516c1f7f10 2024-09-11T12:54:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f516c15d270 con 0x7f516c1f7f10 2024-09-11T12:54:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f5158018070 con 0x7f516c1f7f10 2024-09-11T12:54:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 0x7f514c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.571+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f5158092d30 con 0x7f516c1f7f10 2024-09-11T12:54:31.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.575+0000 7f516b400640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 0x7f514c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:31.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.575+0000 7f516b400640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 0x7f514c073c60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f516c06e460 tx=0x7f515c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:31.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.575+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f515805e290 con 0x7f516c1f7f10 2024-09-11T12:54:31.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:31.779+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f516c15b920 con 0x7f516c1f7f10 2024-09-11T12:54:32.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5163400640 1 -- 172.21.15.78:0/2952745030 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v72) ==== 136+0+0 (secure 0 0 0) 0x7f51580627e0 con 0x7f516c1f7f10 2024-09-11T12:54:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 msgr2=0x7f514c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 0x7f514c073c60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f516c06e460 tx=0x7f515c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 msgr2=0x7f516c157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5158003f00 tx=0x7f5158001810 comp rx=0 tx=0).stop 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 shutdown_connections 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f516c1577a0 0x7f516c1f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f516c1f7f10 0x7f516c157260 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f514c0717a0 0x7f514c073c60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 --2- 172.21.15.78:0/2952745030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51500051d0 0x7f516c1f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 >> 172.21.15.78:0/2952745030 conn(0x7f516c0b3050 msgr2=0x7f516c0b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 shutdown_connections 2024-09-11T12:54:32.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.327+0000 7f5172200640 1 -- 172.21.15.78:0/2952745030 wait complete. 2024-09-11T12:54:32.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- 172.21.15.78:0/3286743918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3415e5b0 msgr2=0x7ffb3415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 --2- 172.21.15.78:0/3286743918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3415e5b0 0x7ffb3415e9f0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7ffb20009900 tx=0x7ffb2001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- 172.21.15.78:0/3286743918 shutdown_connections 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 --2- 172.21.15.78:0/3286743918 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3415e5b0 0x7ffb3415e9f0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 --2- 172.21.15.78:0/3286743918 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb34074240 0x7ffb3415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- 172.21.15.78:0/3286743918 >> 172.21.15.78:0/3286743918 conn(0x7ffb3406ca80 msgr2=0x7ffb340b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- 172.21.15.78:0/3286743918 shutdown_connections 2024-09-11T12:54:32.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- 172.21.15.78:0/3286743918 wait complete. 2024-09-11T12:54:32.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 Processor -- start 2024-09-11T12:54:32.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.491+0000 7ffb3b600640 1 -- start start 2024-09-11T12:54:32.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 0x7ffb3413a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb3415e5b0 0x7ffb341f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffb341680d0 con 0x7ffb3406fc50 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffb34167f50 con 0x7ffb34074240 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffb34168250 con 0x7ffb3415e5b0 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57636/0 (socket says 172.21.15.78:57636) 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 -- 172.21.15.78:0/1335187704 learned_addr learned my addr 172.21.15.78:0/1335187704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb33e00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 0x7ffb3413a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb39600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb3415e5b0 0x7ffb341f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 -- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb3415e5b0 msgr2=0x7ffb341f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb3415e5b0 0x7ffb341f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 -- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 msgr2=0x7ffb3413a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 0x7ffb3413a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 -- 172.21.15.78:0/1335187704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb34153780 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb33e00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 0x7ffb3413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb38c00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009fd0 tx=0x7ffb2400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb2400ce50 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb24011470 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb240118a0 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb34153a60 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb34154020 con 0x7ffb3406fc50 2024-09-11T12:54:32.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.495+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb3415ba80 con 0x7ffb3406fc50 2024-09-11T12:54:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.499+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7ffb24011a40 con 0x7ffb3406fc50 2024-09-11T12:54:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.499+0000 7ffb30c00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 0x7ffb14073e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.499+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ffb24092ba0 con 0x7ffb3406fc50 2024-09-11T12:54:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.499+0000 7ffb33e00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 0x7ffb14073e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.499+0000 7ffb33e00640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 0x7ffb14073e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffb200af000 tx=0x7ffb2001b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:32.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.503+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ffb24016030 con 0x7ffb3406fc50 2024-09-11T12:54:32.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.691+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ffb3413a640 con 0x7ffb14071940 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.691+0000 7ffb30c00640 1 -- 172.21.15.78:0/1335187704 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ffb3413a640 con 0x7ffb14071940 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:32.695 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 msgr2=0x7ffb14073e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 0x7ffb14073e00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffb200af000 tx=0x7ffb2001b0f0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 msgr2=0x7ffb34153130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7ffb24009fd0 tx=0x7ffb2400c270 comp rx=0 tx=0).stop 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 shutdown_connections 2024-09-11T12:54:32.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffb3415e5b0 0x7ffb341f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffb14071940 0x7ffb14073e00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffb34074240 0x7ffb3413a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 --2- 172.21.15.78:0/1335187704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffb3406fc50 0x7ffb34153130 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 >> 172.21.15.78:0/1335187704 conn(0x7ffb3406ca80 msgr2=0x7ffb3406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 shutdown_connections 2024-09-11T12:54:32.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.695+0000 7ffb3b600640 1 -- 172.21.15.78:0/1335187704 wait complete. 2024-09-11T12:54:32.717 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:54:32.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- 172.21.15.78:0/4001573275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580b3420 msgr2=0x7f505815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 --2- 172.21.15.78:0/4001573275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580b3420 0x7f505815a700 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f50580cee20 tx=0x7f504401c800 comp rx=0 tx=0).stop 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- 172.21.15.78:0/4001573275 shutdown_connections 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 --2- 172.21.15.78:0/4001573275 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580b3420 0x7f505815a700 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 --2- 172.21.15.78:0/4001573275 >> v2:172.21.15.157:3300/0 conn(0x7f50580cd770 0x7f50580cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 --2- 172.21.15.78:0/4001573275 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 0x7f50580cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- 172.21.15.78:0/4001573275 >> 172.21.15.78:0/4001573275 conn(0x7f5058071dd0 msgr2=0x7f50580721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- 172.21.15.78:0/4001573275 shutdown_connections 2024-09-11T12:54:32.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- 172.21.15.78:0/4001573275 wait complete. 2024-09-11T12:54:32.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 Processor -- start 2024-09-11T12:54:32.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.859+0000 7f505fe00640 1 -- start start 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 0x7f50581fb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580b3420 0x7f50581584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50580b1e10 con 0x7f50580cd770 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50580b1c90 con 0x7f50580b3420 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50580b1f90 con 0x7f50580b3040 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57664/0 (socket says 172.21.15.78:57664) 2024-09-11T12:54:32.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 -- 172.21.15.78:0/879028342 learned_addr learned my addr 172.21.15.78:0/879028342 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505d400640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 0x7f50581fb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505ca00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580b3420 0x7f50581584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 -- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 msgr2=0x7f50581fb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 0x7f50581fb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 -- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580b3420 msgr2=0x7f50581584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580b3420 0x7f50581584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 -- 172.21.15.78:0/879028342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50581fa7f0 con 0x7f50580cd770 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505de00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f505000ba10 tx=0x7f505000bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050019070 con 0x7f50580cd770 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f505000cbd0 con 0x7f50580cd770 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050014440 con 0x7f50580cd770 2024-09-11T12:54:32.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5058158db0 con 0x7f50580cd770 2024-09-11T12:54:32.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50581fad40 con 0x7f50580cd770 2024-09-11T12:54:32.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f505815a700 con 0x7f50580cd770 2024-09-11T12:54:32.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.863+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f505001c070 con 0x7f50580cd770 2024-09-11T12:54:32.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.867+0000 7f5055600640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 0x7f5038073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:32.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.867+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f5050091cd0 con 0x7f50580cd770 2024-09-11T12:54:32.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.867+0000 7f505d400640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 0x7f5038073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:32.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.867+0000 7f505d400640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 0x7f5038073c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f504c009990 tx=0x7f504c009110 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:32.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:32.871+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f505005d340 con 0x7f50580cd770 2024-09-11T12:54:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5058154ac0 con 0x7f50580cd770 2024-09-11T12:54:33.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f5055600640 1 -- 172.21.15.78:0/879028342 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f5050061890 con 0x7f50580cd770 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 msgr2=0x7f5038073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 0x7f5038073c60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f504c009990 tx=0x7f504c009110 comp rx=0 tx=0).stop 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 msgr2=0x7f5058140bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f505000ba10 tx=0x7f505000bed0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 shutdown_connections 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50580cd770 0x7f5058140bd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50380717a0 0x7f5038073c60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50580b3420 0x7f50581584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 --2- 172.21.15.78:0/879028342 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50580b3040 0x7f50581fb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 >> 172.21.15.78:0/879028342 conn(0x7f5058071dd0 msgr2=0x7f50580721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 shutdown_connections 2024-09-11T12:54:33.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.039+0000 7f505fe00640 1 -- 172.21.15.78:0/879028342 wait complete. 2024-09-11T12:54:33.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:33.062 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045) 2024-09-11T12:54:33.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:33.089 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/fake) 2024-09-11T12:54:33.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:33.160 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:54:33.212 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:54:33.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:33.213 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045) 2024-09-11T12:54:33.283 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:33.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol32140/.meta 2024-09-11T12:54:33.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol32140 2024-09-11T12:54:33.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- 172.21.15.78:0/1406328362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 msgr2=0x7fdc880cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1406328362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc880cdc10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009870 tx=0x7fdc7401c740 comp rx=0 tx=0).stop 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- 172.21.15.78:0/1406328362 shutdown_connections 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1406328362 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc880cdc10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- 172.21.15.78:0/1406328362 >> 172.21.15.78:0/1406328362 conn(0x7fdc88071dd0 msgr2=0x7fdc880721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- 172.21.15.78:0/1406328362 shutdown_connections 2024-09-11T12:54:33.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- 172.21.15.78:0/1406328362 wait complete. 2024-09-11T12:54:33.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 Processor -- start 2024-09-11T12:54:33.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- start start 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc8814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc881518b0 0x7fdc88138440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88168e90 con 0x7fdc880ce150 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88168d10 con 0x7fdc880b2120 2024-09-11T12:54:33.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc8d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc88169010 con 0x7fdc881518b0 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57672/0 (socket says 172.21.15.78:57672) 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 -- 172.21.15.78:0/1175514072 learned_addr learned my addr 172.21.15.78:0/1175514072 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc87200640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc881518b0 0x7fdc88138440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc86800640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc8814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 -- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc881518b0 msgr2=0x7fdc88138440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc881518b0 0x7fdc88138440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 -- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 msgr2=0x7fdc8814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc8814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 -- 172.21.15.78:0/1175514072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc88147290 con 0x7fdc880ce150 2024-09-11T12:54:33.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc85e00640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009790 tx=0x7fdc74005320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:33.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.539+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740b0070 con 0x7fdc880ce150 2024-09-11T12:54:33.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc740b1040 con 0x7fdc880ce150 2024-09-11T12:54:33.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc88149f90 con 0x7fdc880ce150 2024-09-11T12:54:33.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc740046d0 con 0x7fdc880ce150 2024-09-11T12:54:33.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8814a550 con 0x7fdc880ce150 2024-09-11T12:54:33.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc880cdc10 con 0x7fdc880ce150 2024-09-11T12:54:33.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fdc740bd020 con 0x7fdc880ce150 2024-09-11T12:54:33.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc7ea00640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 0x7fdc68073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:33.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fdc7412b9b0 con 0x7fdc880ce150 2024-09-11T12:54:33.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.543+0000 7fdc86800640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 0x7fdc68073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:33.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.547+0000 7fdc86800640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 0x7fdc68073c60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000f70 tx=0x7fdc78009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:33.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.547+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdc740c4020 con 0x7fdc880ce150 2024-09-11T12:54:33.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.739+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7fdc88142f20 con 0x7fdc680717a0 2024-09-11T12:54:33.747 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:33.743+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:33.748 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:33.743+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:33.748 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:33.743+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:33.748 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:33.743+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:33.748 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:33.743+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:33.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc7ea00640 1 -- 172.21.15.78:0/1175514072 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fdc88142f20 con 0x7fdc680717a0 2024-09-11T12:54:33.985 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045 2024-09-11T12:54:33.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 msgr2=0x7fdc68073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 0x7fdc68073c60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdc78000f70 tx=0x7fdc78009040 comp rx=0 tx=0).stop 2024-09-11T12:54:33.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 msgr2=0x7fdc881498d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fdc74009790 tx=0x7fdc74005320 comp rx=0 tx=0).stop 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 shutdown_connections 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc881518b0 0x7fdc88138440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc880ce150 0x7fdc881498d0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc680717a0 0x7fdc68073c60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 --2- 172.21.15.78:0/1175514072 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc880b2120 0x7fdc8814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 >> 172.21.15.78:0/1175514072 conn(0x7fdc88071dd0 msgr2=0x7fdc8806fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 shutdown_connections 2024-09-11T12:54:33.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:33.983+0000 7fdc8d400640 1 -- 172.21.15.78:0/1175514072 wait complete. 2024-09-11T12:54:34.006 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- 172.21.15.78:0/1737169326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b4c00 msgr2=0x7f9a6015ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 --2- 172.21.15.78:0/1737169326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b4c00 0x7f9a6015ae90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f9a600b5e00 tx=0x7f9a4c01c800 comp rx=0 tx=0).stop 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- 172.21.15.78:0/1737169326 shutdown_connections 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 --2- 172.21.15.78:0/1737169326 >> v2:172.21.15.157:3300/0 conn(0x7f9a600b2ec0 0x7f9a600b3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 --2- 172.21.15.78:0/1737169326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b4c00 0x7f9a6015ae90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- 172.21.15.78:0/1737169326 >> 172.21.15.78:0/1737169326 conn(0x7f9a60075a10 msgr2=0x7f9a600734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- 172.21.15.78:0/1737169326 shutdown_connections 2024-09-11T12:54:34.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- 172.21.15.78:0/1737169326 wait complete. 2024-09-11T12:54:34.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 Processor -- start 2024-09-11T12:54:34.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.147+0000 7f9a66200640 1 -- start start 2024-09-11T12:54:34.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 0x7f9a601446f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 0x7f9a601633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600cdf70 con 0x7f9a600b2ec0 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9a600cddf0 con 0x7f9a60154560 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9a600ce0f0 con 0x7f9a600b4c00 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51266/0 (socket says 172.21.15.78:51266) 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 -- 172.21.15.78:0/2126699144 learned_addr learned my addr 172.21.15.78:0/2126699144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 -- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 msgr2=0x7f9a601446f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5fe00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 0x7f9a601633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5ea00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 0x7f9a601446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 0x7f9a601446f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 -- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 msgr2=0x7f9a601633d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 0x7f9a601633d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 -- 172.21.15.78:0/2126699144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a601fb800 con 0x7f9a600b2ec0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5fe00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 0x7f9a601633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5ea00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 0x7f9a601446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a5f400640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9a50001ec0 tx=0x7f9a5000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50017070 con 0x7f9a600b2ec0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a50003600 con 0x7f9a600b2ec0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a50003a50 con 0x7f9a600b2ec0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a60151980 con 0x7f9a600b2ec0 2024-09-11T12:54:34.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a601628f0 con 0x7f9a600b2ec0 2024-09-11T12:54:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a600b3370 con 0x7f9a600b2ec0 2024-09-11T12:54:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.151+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f9a5001a070 con 0x7f9a600b2ec0 2024-09-11T12:54:34.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.155+0000 7f9a57400640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 0x7f9a40073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.155+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f9a50091230 con 0x7f9a600b2ec0 2024-09-11T12:54:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.155+0000 7f9a5ea00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 0x7f9a40073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.155+0000 7f9a5ea00640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 0x7f9a40073c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0af000 tx=0x7f9a4c01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.159+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9a5005cf80 con 0x7f9a600b2ec0 2024-09-11T12:54:34.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.351+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f9a60144c30 con 0x7f9a400717d0 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a57400640 1 -- 172.21.15.78:0/2126699144 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f9a60144c30 con 0x7f9a400717d0 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subvol32140" 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:34.356 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:54:34.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 msgr2=0x7f9a40073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 0x7f9a40073c90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c0af000 tx=0x7f9a4c01b040 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 msgr2=0x7f9a601fb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9a50001ec0 tx=0x7f9a5000c890 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 shutdown_connections 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9a400717d0 0x7f9a40073c90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9a60154560 0x7f9a601633d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9a600b4c00 0x7f9a601446f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 --2- 172.21.15.78:0/2126699144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9a600b2ec0 0x7f9a601fb2c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 >> 172.21.15.78:0/2126699144 conn(0x7f9a60075a10 msgr2=0x7f9a600afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 shutdown_connections 2024-09-11T12:54:34.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.355+0000 7f9a66200640 1 -- 172.21.15.78:0/2126699144 wait complete. 2024-09-11T12:54:34.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol32140 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 -- 172.21.15.78:0/3987833109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 msgr2=0x7fd5ec003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 --2- 172.21.15.78:0/3987833109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd5ec003e30 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd5f8005fd0 tx=0x7fd5f801cd30 comp rx=0 tx=0).stop 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd607400640 1 -- 172.21.15.78:0/3987833109 reap_dead start 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 -- 172.21.15.78:0/3987833109 shutdown_connections 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 -- 172.21.15.78:0/3987833109 >> 172.21.15.78:0/3987833109 conn(0x7fd60806e860 msgr2=0x7fd60806ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 -- 172.21.15.78:0/3987833109 shutdown_connections 2024-09-11T12:54:34.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.519+0000 7fd60dc00640 1 -- 172.21.15.78:0/3987833109 wait complete. 2024-09-11T12:54:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 Processor -- start 2024-09-11T12:54:34.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- start start 2024-09-11T12:54:34.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6080d12e0 0x7fd60813a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd60813af70 0x7fd60815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd608166750 con 0x7fd5ec003a10 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd6081603f0 con 0x7fd60813af70 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd6081668d0 con 0x7fd6080d12e0 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51288/0 (socket says 172.21.15.78:51288) 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 -- 172.21.15.78:0/254911030 learned_addr learned my addr 172.21.15.78:0/254911030 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606000640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6080d12e0 0x7fd60813a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd607400640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd60813af70 0x7fd60815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 -- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6080d12e0 msgr2=0x7fd60813a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6080d12e0 0x7fd60813a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 -- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd60813af70 msgr2=0x7fd60815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd60813af70 0x7fd60815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 -- 172.21.15.78:0/254911030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6080d30a0 con 0x7fd5ec003a10 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd606a00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd5f801cb60 tx=0x7fd5f8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f80ab040 con 0x7fd5ec003a10 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5f80a5970 con 0x7fd5ec003a10 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6080d17a0 con 0x7fd5ec003a10 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8004910 con 0x7fd5ec003a10 2024-09-11T12:54:34.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60806cad0 con 0x7fd5ec003a10 2024-09-11T12:54:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.523+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6080a9460 con 0x7fd5ec003a10 2024-09-11T12:54:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.527+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd5f80c0030 con 0x7fd5ec003a10 2024-09-11T12:54:34.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.527+0000 7fd5fea00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 0x7fd5e8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.527+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd5f812c400 con 0x7fd5ec003a10 2024-09-11T12:54:34.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.527+0000 7fd606000640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 0x7fd5e8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.527+0000 7fd606000640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 0x7fd5e8073c60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4005ee0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.531+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd5f80c7060 con 0x7fd5ec003a10 2024-09-11T12:54:34.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.723+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7fd6080b34b0 con 0x7fd5e80717a0 2024-09-11T12:54:34.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.747+0000 7fd5fea00640 1 -- 172.21.15.78:0/254911030 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+705 (secure 0 0 0) 0x7fd6080b34b0 con 0x7fd5e80717a0 2024-09-11T12:54:34.750 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "atime": "2024-09-11 12:54:33", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_pcent": "undefined", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_quota": "infinite", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_used": 0, 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "created_at": "2024-09-11 12:54:33", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "ctime": "2024-09-11 12:54:33", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "data_pool": "cephfs_data", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "earmark": "", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "features": [ 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-clone", 2024-09-11T12:54:34.751 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-autoprotect" 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "flavor": 1, 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 0, 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": 16877, 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "mon_addrs": [ 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.78:6789", 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6789", 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6790" 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "mtime": "2024-09-11 12:54:33", 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045", 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_namespace": "", 2024-09-11T12:54:34.752 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "complete", 2024-09-11T12:54:34.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "subvolume", 2024-09-11T12:54:34.753 INFO:teuthology.orchestra.run.smithi078.stdout: "uid": 0 2024-09-11T12:54:34.753 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 msgr2=0x7fd5e8073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 0x7fd5e8073c60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5f4005fd0 tx=0x7fd5f4005ee0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 msgr2=0x7fd60806d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd5f801cb60 tx=0x7fd5f8005620 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 shutdown_connections 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd5e80717a0 0x7fd5e8073c60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd60813af70 0x7fd60815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd6080d12e0 0x7fd60813a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 --2- 172.21.15.78:0/254911030 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5ec003a10 0x7fd60806d2d0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 >> 172.21.15.78:0/254911030 conn(0x7fd60806e860 msgr2=0x7fd60813f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:34.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 shutdown_connections 2024-09-11T12:54:34.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.751+0000 7fd60dc00640 1 -- 172.21.15.78:0/254911030 wait complete. 2024-09-11T12:54:34.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol32140 10485760 2024-09-11T12:54:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 -- 172.21.15.78:0/2965266372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 msgr2=0x7f0ba4003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/2965266372 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0ba4003e60 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0bb000c2e0 tx=0x7f0bb001cea0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc1e00640 1 -- 172.21.15.78:0/2965266372 reap_dead start 2024-09-11T12:54:34.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 -- 172.21.15.78:0/2965266372 shutdown_connections 2024-09-11T12:54:34.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 -- 172.21.15.78:0/2965266372 >> 172.21.15.78:0/2965266372 conn(0x7f0bbc0b3050 msgr2=0x7f0bbc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:34.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 -- 172.21.15.78:0/2965266372 shutdown_connections 2024-09-11T12:54:34.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.915+0000 7f0bc3e00640 1 -- 172.21.15.78:0/2965266372 wait complete. 2024-09-11T12:54:34.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 Processor -- start 2024-09-11T12:54:34.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- start start 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 0x7f0bbc071da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0bbc14c770 0x7f0bbc0c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc167f00 con 0x7f0ba4003a40 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0bbc0b5df0 con 0x7f0bbc0cbd60 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0bbc168080 con 0x7f0bbc14c770 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51294/0 (socket says 172.21.15.78:51294) 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 -- 172.21.15.78:0/734210464 learned_addr learned my addr 172.21.15.78:0/734210464 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:34.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc0a00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 0x7f0bbc071da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0bbc14c770 0x7f0bbc0c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 -- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0bbc14c770 msgr2=0x7f0bbc0c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0bbc14c770 0x7f0bbc0c9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 -- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 msgr2=0x7f0bbc071da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 0x7f0bbc071da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 -- 172.21.15.78:0/734210464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bbc0722e0 con 0x7f0ba4003a40 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc0a00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 0x7f0bbc071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc1400640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0bb001c960 tx=0x7f0bb0006d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00a5bc0 con 0x7f0ba4003a40 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bb00014c0 con 0x7f0ba4003a40 2024-09-11T12:54:34.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bb00060d0 con 0x7f0ba4003a40 2024-09-11T12:54:34.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bbc1fb240 con 0x7f0ba4003a40 2024-09-11T12:54:34.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.919+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bbc1fb800 con 0x7f0ba4003a40 2024-09-11T12:54:34.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bbc0cd490 con 0x7f0ba4003a40 2024-09-11T12:54:34.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f0bb0006270 con 0x7f0ba4003a40 2024-09-11T12:54:34.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bb9600640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 0x7f0b9c073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:34.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f0bb012b1d0 con 0x7f0ba4003a40 2024-09-11T12:54:34.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bc0a00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 0x7f0b9c073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:34.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.923+0000 7f0bc0a00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 0x7f0b9c073c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0cb970 tx=0x7f0ba80091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:34.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:34.927+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0bb00c0070 con 0x7f0ba4003a40 2024-09-11T12:54:35.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.123+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol32140", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f0bbc14fbc0 con 0x7f0b9c0717a0 2024-09-11T12:54:35.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bb9600640 1 -- 172.21.15.78:0/734210464 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f0bbc14fbc0 con 0x7f0b9c0717a0 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_used": 0 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_quota": 10485760 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_pcent": "0.00" 2024-09-11T12:54:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:35.154 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 msgr2=0x7f0b9c073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 0x7f0b9c073c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0cb970 tx=0x7f0ba80091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 msgr2=0x7f0bbc0c95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f0bb001c960 tx=0x7f0bb0006d30 comp rx=0 tx=0).stop 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 shutdown_connections 2024-09-11T12:54:35.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0bbc14c770 0x7f0bbc0c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0b9c0717a0 0x7f0b9c073c60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0bbc0cbd60 0x7f0bbc071da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 --2- 172.21.15.78:0/734210464 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0ba4003a40 0x7f0bbc0c95a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 >> 172.21.15.78:0/734210464 conn(0x7f0bbc0b3050 msgr2=0x7f0bbc14ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 shutdown_connections 2024-09-11T12:54:35.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.151+0000 7f0bc3e00640 1 -- 172.21.15.78:0/734210464 wait complete. 2024-09-11T12:54:35.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol32140 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- 172.21.15.78:0/3248772016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 msgr2=0x7fd07815a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 --2- 172.21.15.78:0/3248772016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07815a700 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd07806d1a0 tx=0x7fd06001c890 comp rx=0 tx=0).stop 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- 172.21.15.78:0/3248772016 shutdown_connections 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 --2- 172.21.15.78:0/3248772016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07815a700 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 --2- 172.21.15.78:0/3248772016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780cd770 0x7fd0780cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 --2- 172.21.15.78:0/3248772016 >> v2:172.21.15.157:3300/0 conn(0x7fd0780b3040 0x7fd0780cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- 172.21.15.78:0/3248772016 >> 172.21.15.78:0/3248772016 conn(0x7fd078071dd0 msgr2=0x7fd0780721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:35.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- 172.21.15.78:0/3248772016 shutdown_connections 2024-09-11T12:54:35.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- 172.21.15.78:0/3248772016 wait complete. 2024-09-11T12:54:35.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 Processor -- start 2024-09-11T12:54:35.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.315+0000 7fd07e200640 1 -- start start 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b3040 0x7fd07813f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 0x7fd0781f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780aedf0 con 0x7fd0780b3420 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0780aec70 con 0x7fd0780cd770 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0780aef70 con 0x7fd0780b3040 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51312/0 (socket says 172.21.15.78:51312) 2024-09-11T12:54:35.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 -- 172.21.15.78:0/1891394404 learned_addr learned my addr 172.21.15.78:0/1891394404 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd077400640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b3040 0x7fd07813f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd077e00640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 0x7fd0781f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 -- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b3040 msgr2=0x7fd07813f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b3040 0x7fd07813f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 -- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 msgr2=0x7fd0781f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 0x7fd0781f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 -- 172.21.15.78:0/1891394404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd078140050 con 0x7fd0780b3420 2024-09-11T12:54:35.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd077e00640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 0x7fd0781f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:35.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06fe00640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd0600b0040 tx=0x7fd060004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600b2040 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd060005070 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd078140330 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0600054c0 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd07813fcb0 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd07815a700 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.319+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7fd0600bd060 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.323+0000 7fd06f400640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 0x7fd054073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.323+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd06012b480 con 0x7fd0780b3420 2024-09-11T12:54:35.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.323+0000 7fd077400640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 0x7fd054073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.323+0000 7fd077400640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 0x7fd054073c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bb70 tx=0x7fd06800b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:35.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.327+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd0600164b0 con 0x7fd0780b3420 2024-09-11T12:54:35.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.527+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7fd0780702b0 con 0x7fd0540717a0 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd06f400640 1 -- 172.21.15.78:0/1891394404 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+698 (secure 0 0 0) 0x7fd0780702b0 con 0x7fd0540717a0 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "atime": "2024-09-11 12:54:33", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_pcent": "0.00", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_quota": 10485760, 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_used": 0, 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "created_at": "2024-09-11 12:54:33", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "ctime": "2024-09-11 12:54:35", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "data_pool": "cephfs_data", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "earmark": "", 2024-09-11T12:54:35.553 INFO:teuthology.orchestra.run.smithi078.stdout: "features": [ 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-clone", 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-autoprotect" 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flavor": 1, 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 0, 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": 16877, 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mon_addrs": [ 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.78:6789", 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6789", 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6790" 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "mtime": "2024-09-11 12:54:33", 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045", 2024-09-11T12:54:35.554 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_namespace": "", 2024-09-11T12:54:35.555 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "complete", 2024-09-11T12:54:35.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "subvolume", 2024-09-11T12:54:35.555 INFO:teuthology.orchestra.run.smithi078.stdout: "uid": 0 2024-09-11T12:54:35.555 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 msgr2=0x7fd054073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 0x7fd054073c60 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd06800bb70 tx=0x7fd06800b2e0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 msgr2=0x7fd07813e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fd0600b0040 tx=0x7fd060004040 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 shutdown_connections 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0540717a0 0x7fd054073c60 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0780cd770 0x7fd0781f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0780b3420 0x7fd07813e040 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 --2- 172.21.15.78:0/1891394404 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0780b3040 0x7fd07813f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 >> 172.21.15.78:0/1891394404 conn(0x7fd078071dd0 msgr2=0x7fd0780ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:35.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 shutdown_connections 2024-09-11T12:54:35.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.551+0000 7fd07e200640 1 -- 172.21.15.78:0/1891394404 wait complete. 2024-09-11T12:54:35.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol32140 --mode 777 --uid 1000 --gid 1000 2024-09-11T12:54:35.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- 172.21.15.78:0/3865512849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480745c0 msgr2=0x7f46480749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- 172.21.15.78:0/3865512849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480745c0 0x7f46480749a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4638009840 tx=0x7f463801c720 comp rx=0 tx=0).stop 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- 172.21.15.78:0/3865512849 shutdown_connections 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- 172.21.15.78:0/3865512849 >> v2:172.21.15.157:3300/0 conn(0x7f464815ed70 0x7f4648151390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- 172.21.15.78:0/3865512849 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480745c0 0x7f46480749a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- 172.21.15.78:0/3865512849 >> 172.21.15.78:0/3865512849 conn(0x7f46480d0840 msgr2=0x7f46480b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- 172.21.15.78:0/3865512849 shutdown_connections 2024-09-11T12:54:35.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- 172.21.15.78:0/3865512849 wait complete. 2024-09-11T12:54:35.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 Processor -- start 2024-09-11T12:54:35.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- start start 2024-09-11T12:54:35.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480745c0 0x7f4648147a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 0x7f4648159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f464806ef50 con 0x7f464815ed70 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f464806edd0 con 0x7f46480745c0 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f464d800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f464806f0d0 con 0x7f46480cd2c0 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51322/0 (socket says 172.21.15.78:51322) 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 -- 172.21.15.78:0/1444919877 learned_addr learned my addr 172.21.15.78:0/1444919877 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4646a00640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480745c0 0x7f4648147a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4646000640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 0x7f4648159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 -- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 msgr2=0x7f4648159ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 0x7f4648159ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 -- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480745c0 msgr2=0x7f4648147a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480745c0 0x7f4648147a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4647400640 1 -- 172.21.15.78:0/1444919877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46480b16d0 con 0x7f464815ed70 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.723+0000 7f4646000640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 0x7f4648159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:35.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f4647400640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f464000ead0 tx=0x7f464000ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:35.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640004b30 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4640004dc0 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640005280 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46480b1970 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464814c680 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46480749a0 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7f4640005420 con 0x7f464815ed70 2024-09-11T12:54:35.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 0x7f4628073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:35.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7f4640093470 con 0x7f464815ed70 2024-09-11T12:54:35.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.727+0000 7f4646a00640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 0x7f4628073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:35.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.731+0000 7f4646a00640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 0x7f4628073c60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae000 tx=0x7f463801af60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:35.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.731+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4640026020 con 0x7f464815ed70 2024-09-11T12:54:35.922 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.919+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol32140", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7f464806a160 con 0x7f46280717a0 2024-09-11T12:54:35.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.947+0000 7f463ea00640 1 -- 172.21.15.78:0/1444919877 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f464806a160 con 0x7f46280717a0 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 msgr2=0x7f4628073c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 0x7f4628073c60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f46380ae000 tx=0x7f463801af60 comp rx=0 tx=0).stop 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 msgr2=0x7f4648159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f464000ead0 tx=0x7f464000ef90 comp rx=0 tx=0).stop 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 shutdown_connections 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f464815ed70 0x7f4648159ff0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46480cd2c0 0x7f4648159ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46280717a0 0x7f4628073c60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 --2- 172.21.15.78:0/1444919877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46480745c0 0x7f4648147a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:35.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 >> 172.21.15.78:0/1444919877 conn(0x7f46480d0840 msgr2=0x7f46481510f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:35.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 shutdown_connections 2024-09-11T12:54:35.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:35.951+0000 7f464d800640 1 -- 172.21.15.78:0/1444919877 wait complete. 2024-09-11T12:54:35.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol32140 2024-09-11T12:54:36.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- 172.21.15.78:0/4259672657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80b3420 msgr2=0x7efbf80cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:36.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 --2- 172.21.15.78:0/4259672657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80b3420 0x7efbf80cf6e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7efbe400a950 tx=0x7efbe401db70 comp rx=0 tx=0).stop 2024-09-11T12:54:36.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- 172.21.15.78:0/4259672657 shutdown_connections 2024-09-11T12:54:36.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 --2- 172.21.15.78:0/4259672657 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80b3420 0x7efbf80cf6e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- 172.21.15.78:0/4259672657 >> 172.21.15.78:0/4259672657 conn(0x7efbf8071dd0 msgr2=0x7efbf80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:36.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- 172.21.15.78:0/4259672657 shutdown_connections 2024-09-11T12:54:36.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- 172.21.15.78:0/4259672657 wait complete. 2024-09-11T12:54:36.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 Processor -- start 2024-09-11T12:54:36.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.115+0000 7efbff000640 1 -- start start 2024-09-11T12:54:36.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 0x7efbf81591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 0x7efbf8160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80af070 con 0x7efbf80b3420 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80aeef0 con 0x7efbf80cd770 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efbf80af1f0 con 0x7efbf80b3040 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51330/0 (socket says 172.21.15.78:51330) 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 -- 172.21.15.78:0/1047771809 learned_addr learned my addr 172.21.15.78:0/1047771809 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7e00640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 0x7efbf81591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbfd000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 0x7efbf8160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 -- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 msgr2=0x7efbf81591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 0x7efbf81591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 -- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 msgr2=0x7efbf8160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:36.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 0x7efbf8160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 -- 172.21.15.78:0/1047771809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81596e0 con 0x7efbf80b3420 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7e00640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 0x7efbf81591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbfd000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 0x7efbf8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbf7400640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efbe400a8a0 tx=0x7efbe40038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe40b3040 con 0x7efbf80b3420 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbe4003b90 con 0x7efbf80b3420 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf8158100 con 0x7efbf80b3420 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe4005020 con 0x7efbf80b3420 2024-09-11T12:54:36.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf81586c0 con 0x7efbf80b3420 2024-09-11T12:54:36.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf80cf6e0 con 0x7efbf80b3420 2024-09-11T12:54:36.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.119+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 15) ==== 95034+0+0 (secure 0 0 0) 0x7efbe40051c0 con 0x7efbf80b3420 2024-09-11T12:54:36.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.123+0000 7efbefe00640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 0x7efbd8073c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:36.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.123+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5767+0+0 (secure 0 0 0) 0x7efbe412cb60 con 0x7efbf80b3420 2024-09-11T12:54:36.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.123+0000 7efbf7e00640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 0x7efbd8073c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:36.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.123+0000 7efbf7e00640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 0x7efbd8073c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efbe8004a40 tx=0x7efbe8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:36.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.127+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efbe4017760 con 0x7efbf80b3420 2024-09-11T12:54:36.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.319+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7efbf80ae220 con 0x7efbd80717d0 2024-09-11T12:54:36.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.343+0000 7efbefe00640 1 -- 172.21.15.78:0/1047771809 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7efbf80ae220 con 0x7efbd80717d0 2024-09-11T12:54:36.346 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 msgr2=0x7efbd8073c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 0x7efbd8073c90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efbe8004a40 tx=0x7efbe8009040 comp rx=0 tx=0).stop 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 msgr2=0x7efbf8160590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7efbe400a8a0 tx=0x7efbe40038b0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 shutdown_connections 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efbd80717d0 0x7efbd8073c90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efbf80cd770 0x7efbf8160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efbf80b3420 0x7efbf8160590 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 --2- 172.21.15.78:0/1047771809 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efbf80b3040 0x7efbf81591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:36.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 >> 172.21.15.78:0/1047771809 conn(0x7efbf8071dd0 msgr2=0x7efbf8143370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:36.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 shutdown_connections 2024-09-11T12:54:36.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:36.347+0000 7efbff000640 1 -- 172.21.15.78:0/1047771809 wait complete. 2024-09-11T12:54:36.372 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol32140 with 8 files each 1MB size under directory volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045 2024-09-11T12:54:36.373 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.373 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:36.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:36.383 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:36.455 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:36.455 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:36.455 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021989 s, 47.7 MB/s 2024-09-11T12:54:36.456 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.456 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:36.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:36.508 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:36.587 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:36.587 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:36.587 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271153 s, 38.7 MB/s 2024-09-11T12:54:36.588 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.588 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:36.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:36.640 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:36.712 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203173 s, 51.6 MB/s 2024-09-11T12:54:36.713 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.713 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:36.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:36.763 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:36.834 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180474 s, 58.1 MB/s 2024-09-11T12:54:36.835 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.835 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:36.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:36.884 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:36.953 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170754 s, 61.4 MB/s 2024-09-11T12:54:36.954 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:36.954 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:37.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:37.004 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:37.073 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170358 s, 61.6 MB/s 2024-09-11T12:54:37.074 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:37.074 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:37.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:37.125 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216284 s, 48.5 MB/s 2024-09-11T12:54:37.199 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:37.199 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:37.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:37.248 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-11T12:54:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-09-11T12:54:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-09-11T12:54:37.323 INFO:teuthology.orchestra.run.smithi078.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186774 s, 56.1 MB/s 2024-09-11T12:54:37.324 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol32140 subvol_snap58550 2024-09-11T12:54:37.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- 172.21.15.78:0/1173257994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c072350 msgr2=0x7fd43c06f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 --2- 172.21.15.78:0/1173257994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c072350 0x7fd43c06f5d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd434009c80 tx=0x7fd43401cd70 comp rx=0 tx=0).stop 2024-09-11T12:54:37.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- 172.21.15.78:0/1173257994 shutdown_connections 2024-09-11T12:54:37.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 --2- 172.21.15.78:0/1173257994 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c072350 0x7fd43c06f5d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 --2- 172.21.15.78:0/1173257994 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c074060 0x7fd43c071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- 172.21.15.78:0/1173257994 >> 172.21.15.78:0/1173257994 conn(0x7fd43c0b3050 msgr2=0x7fd43c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- 172.21.15.78:0/1173257994 shutdown_connections 2024-09-11T12:54:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- 172.21.15.78:0/1173257994 wait complete. 2024-09-11T12:54:37.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 Processor -- start 2024-09-11T12:54:37.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.511+0000 7fd441000640 1 -- start start 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c072350 0x7fd43c0ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 0x7fd43c0b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c0704d0 con 0x7fd43c151050 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd43c070350 con 0x7fd43c074060 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd43c070650 con 0x7fd43c072350 2024-09-11T12:54:37.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51342/0 (socket says 172.21.15.78:51342) 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 -- 172.21.15.78:0/2234749725 learned_addr learned my addr 172.21.15.78:0/2234749725 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd439e00640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c072350 0x7fd43c0ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd439400640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 0x7fd43c0b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 -- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c072350 msgr2=0x7fd43c0ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c072350 0x7fd43c0ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 -- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 msgr2=0x7fd43c0b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 0x7fd43c0b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 -- 172.21.15.78:0/2234749725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd43c1fac50 con 0x7fd43c151050 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd439400640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 0x7fd43c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd43a800640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd43401cba0 tx=0x7fd434009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340bb070 con 0x7fd43c151050 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd434004e00 con 0x7fd43c151050 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd43c1faf30 con 0x7fd43c151050 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4340b2040 con 0x7fd43c151050 2024-09-11T12:54:37.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd43c1fb4f0 con 0x7fd43c151050 2024-09-11T12:54:37.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.515+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd43c1fb6f0 con 0x7fd43c151050 2024-09-11T12:54:37.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.519+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7fd4340be070 con 0x7fd43c151050 2024-09-11T12:54:37.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.519+0000 7fd432000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 0x7fd41c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.519+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fd43412bec0 con 0x7fd43c151050 2024-09-11T12:54:37.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.519+0000 7fd439e00640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 0x7fd41c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.519+0000 7fd439e00640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 0x7fd41c073cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b810 tx=0x7fd42c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:37.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.523+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd43c1fb6f0 con 0x7fd43c151050 2024-09-11T12:54:37.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.715+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "subvol_snap58550", "target": ["mon-mgr", ""]}) -- 0x7fd43c06f5d0 con 0x7fd41c071830 2024-09-11T12:54:37.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.747+0000 7fd432000640 1 -- 172.21.15.78:0/2234749725 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd43c06f5d0 con 0x7fd41c071830 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.747+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 msgr2=0x7fd41c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.747+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 0x7fd41c073cf0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd42c00b810 tx=0x7fd42c008040 comp rx=0 tx=0).stop 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.747+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 msgr2=0x7fd43c1461c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.747+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd43401cba0 tx=0x7fd434009390 comp rx=0 tx=0).stop 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 shutdown_connections 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd43c151050 0x7fd43c1461c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd41c071830 0x7fd41c073cf0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd43c074060 0x7fd43c0b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 --2- 172.21.15.78:0/2234749725 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd43c072350 0x7fd43c0ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 >> 172.21.15.78:0/2234749725 conn(0x7fd43c0b3050 msgr2=0x7fd43c073950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 shutdown_connections 2024-09-11T12:54:37.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.751+0000 7fd441000640 1 -- 172.21.15.78:0/2234749725 wait complete. 2024-09-11T12:54:37.771 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol32140 subvol_snap58550 subvol_clone31610 2024-09-11T12:54:37.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 -- 172.21.15.78:0/1926897912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 msgr2=0x7f3d8c13ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 --2- 172.21.15.78:0/1926897912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c13ed30 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c0d2c20 tx=0x7f3d7801c800 comp rx=0 tx=0).stop 2024-09-11T12:54:37.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 -- 172.21.15.78:0/1926897912 shutdown_connections 2024-09-11T12:54:37.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 --2- 172.21.15.78:0/1926897912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c13ed30 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 --2- 172.21.15.78:0/1926897912 >> v2:172.21.15.157:3300/0 conn(0x7f3d8c13e450 0x7f3d8c0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 -- 172.21.15.78:0/1926897912 >> 172.21.15.78:0/1926897912 conn(0x7f3d8c0ce660 msgr2=0x7f3d8c0cea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:37.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 -- 172.21.15.78:0/1926897912 shutdown_connections 2024-09-11T12:54:37.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.915+0000 7f3d8b200640 1 -- 172.21.15.78:0/1926897912 wait complete. 2024-09-11T12:54:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 Processor -- start 2024-09-11T12:54:37.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- start start 2024-09-11T12:54:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 0x7f3d8c13ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 0x7f3d8c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c1687d0 con 0x7f3d8c13e830 2024-09-11T12:54:37.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3d8c168650 con 0x7f3d8c141780 2024-09-11T12:54:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3d8c168950 con 0x7f3d8c13e450 2024-09-11T12:54:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51356/0 (socket says 172.21.15.78:51356) 2024-09-11T12:54:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 -- 172.21.15.78:0/2266072575 learned_addr learned my addr 172.21.15.78:0/2266072575 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:37.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8a800640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 0x7f3d8c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89e00640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 0x7f3d8c13ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 -- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 msgr2=0x7f3d8c13ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 0x7f3d8c13ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 -- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 msgr2=0x7f3d8c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:37.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 0x7f3d8c1571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 -- 172.21.15.78:0/2266072575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d8c164f90 con 0x7f3d8c13e830 2024-09-11T12:54:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89e00640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 0x7f3d8c13ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8a800640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 0x7f3d8c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d89400640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c10 tx=0x7f3d780a56e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:37.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780bc070 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d78004430 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780b2040 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d8c165270 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.919+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d8c165830 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d8c0a9460 con 0x7f3d8c13e830 2024-09-11T12:54:37.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7f3d78003860 con 0x7f3d8c13e830 2024-09-11T12:54:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d82000640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 0x7f3d6c073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f3d7812bb40 con 0x7f3d8c13e830 2024-09-11T12:54:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d89e00640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 0x7f3d6c073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:37.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.923+0000 7f3d89e00640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 0x7f3d6c073cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005d10 tx=0x7f3d7c009550 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:37.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:37.927+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3d780163f0 con 0x7f3d8c13e830 2024-09-11T12:54:38.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.127+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "subvol_snap58550", "target_sub_name": "subvol_clone31610", "target": ["mon-mgr", ""]}) -- 0x7f3d8c15f040 con 0x7f3d6c071830 2024-09-11T12:54:38.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.203+0000 7f3d82000640 1 -- 172.21.15.78:0/2266072575 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d8c15f040 con 0x7f3d6c071830 2024-09-11T12:54:38.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.203+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 msgr2=0x7f3d6c073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.203+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 0x7f3d6c073cf0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f3d7c005d10 tx=0x7f3d7c009550 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.203+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 msgr2=0x7f3d8c156c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.203+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f3d78000c10 tx=0x7f3d780a56e0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 shutdown_connections 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3d6c071830 0x7f3d6c073cf0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3d8c141780 0x7f3d8c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3d8c13e830 0x7f3d8c156c80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 --2- 172.21.15.78:0/2266072575 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3d8c13e450 0x7f3d8c13ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 >> 172.21.15.78:0/2266072575 conn(0x7f3d8c0ce660 msgr2=0x7f3d8c15bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:38.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 shutdown_connections 2024-09-11T12:54:38.210 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f3d8b200640 1 -- 172.21.15.78:0/2266072575 wait complete. 2024-09-11T12:54:38.211 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.211 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.211 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.211 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.211 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.207+0000 7f1ed4400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.227 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.223+0000 7f1ed2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.227 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.223+0000 7f1ed2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.227 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.223+0000 7f1ed2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.227 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.223+0000 7f1ed2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.228 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:38.223+0000 7f1ed2600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:38.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone31610 2024-09-11T12:54:38.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 -- 172.21.15.78:0/3455688344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b81548f0 msgr2=0x7f33b80ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 --2- 172.21.15.78:0/3455688344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b81548f0 0x7f33b80ad870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009870 tx=0x7f33a801c750 comp rx=0 tx=0).stop 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 -- 172.21.15.78:0/3455688344 shutdown_connections 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 --2- 172.21.15.78:0/3455688344 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b81548f0 0x7f33b80ad870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 -- 172.21.15.78:0/3455688344 >> 172.21.15.78:0/3455688344 conn(0x7f33b80d3bf0 msgr2=0x7f33b80d4000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 -- 172.21.15.78:0/3455688344 shutdown_connections 2024-09-11T12:54:38.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.407+0000 7f33be000640 1 -- 172.21.15.78:0/3455688344 wait complete. 2024-09-11T12:54:38.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 Processor -- start 2024-09-11T12:54:38.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 -- start start 2024-09-11T12:54:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 0x7f33b8152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:38.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 0x7f33b81505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33b816bf40 con 0x7f33b81548f0 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33b816bdc0 con 0x7f33b80addb0 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33be000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33b816c0c0 con 0x7f33b815b380 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 0x7f33b8152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bcc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bcc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:57280/0 (socket says 172.21.15.78:57280) 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bcc00640 1 -- 172.21.15.78:0/514668055 learned_addr learned my addr 172.21.15.78:0/514668055 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:38.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bd600640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 0x7f33b81505b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 -- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 msgr2=0x7f33b81505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 0x7f33b81505b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 -- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 msgr2=0x7f33b8152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 -- 172.21.15.78:0/514668055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b81512e0 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bcc00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33bd600640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 0x7f33b81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33afe00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 0x7f33b8152ed0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f33b8074440 tx=0x7f33a000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.411+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0016020 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33a0012920 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0012d50 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b81515c0 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8135790 con 0x7f33b81548f0 2024-09-11T12:54:38.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b80ad870 con 0x7f33b81548f0 2024-09-11T12:54:38.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7f33a0019020 con 0x7f33b81548f0 2024-09-11T12:54:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33b5600640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 0x7f3394073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f33a0091d10 con 0x7f33b81548f0 2024-09-11T12:54:38.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.415+0000 7f33bcc00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 0x7f3394073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:38.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.419+0000 7f33bcc00640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 0x7f3394073cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33b81face0 tx=0x7f33a801af90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:38.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.423+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f33a0020030 con 0x7f33b81548f0 2024-09-11T12:54:38.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.647+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone31610", "target": ["mon-mgr", ""]}) -- 0x7f33b80d3390 con 0x7f3394071830 2024-09-11T12:54:38.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33b5600640 1 -- 172.21.15.78:0/514668055 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f33b80d3390 con 0x7f3394071830 2024-09-11T12:54:38.789 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:38.789 INFO:teuthology.orchestra.run.smithi078.stdout: "status": { 2024-09-11T12:54:38.789 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "complete" 2024-09-11T12:54:38.789 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:38.790 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 msgr2=0x7f3394073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 0x7f3394073cf0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f33b81face0 tx=0x7f33a801af90 comp rx=0 tx=0).stop 2024-09-11T12:54:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 msgr2=0x7f33b8152ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 0x7f33b8152ed0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f33b8074440 tx=0x7f33a000c960 comp rx=0 tx=0).stop 2024-09-11T12:54:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 shutdown_connections 2024-09-11T12:54:38.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b815b380 0x7f33b81505b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b81548f0 0x7f33b8152ed0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071830 0x7f3394073cf0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 --2- 172.21.15.78:0/514668055 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b80addb0 0x7f33b8152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.787+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 >> 172.21.15.78:0/514668055 conn(0x7f33b80d3bf0 msgr2=0x7f33b8161530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.791+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 shutdown_connections 2024-09-11T12:54:38.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:38.791+0000 7f33be000640 1 -- 172.21.15.78:0/514668055 wait complete. 2024-09-11T12:54:38.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:38.812 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone31610/.meta) 2024-09-11T12:54:38.833 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:54:38.833 INFO:teuthology.orchestra.run.smithi078.stdout:version = 2 2024-09-11T12:54:38.833 INFO:teuthology.orchestra.run.smithi078.stdout:type = clone 2024-09-11T12:54:38.833 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4 2024-09-11T12:54:38.833 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:54:38.834 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:38.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol32140 2024-09-11T12:54:39.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- 172.21.15.78:0/2671572469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800727a0 msgr2=0x7ff98015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 --2- 172.21.15.78:0/2671572469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800727a0 0x7ff98015af30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7ff978009bc0 tx=0x7ff97801cba0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- 172.21.15.78:0/2671572469 shutdown_connections 2024-09-11T12:54:39.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 --2- 172.21.15.78:0/2671572469 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800727a0 0x7ff98015af30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- 172.21.15.78:0/2671572469 >> 172.21.15.78:0/2671572469 conn(0x7ff9800d0410 msgr2=0x7ff9800d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:39.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- 172.21.15.78:0/2671572469 shutdown_connections 2024-09-11T12:54:39.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- 172.21.15.78:0/2671572469 wait complete. 2024-09-11T12:54:39.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 Processor -- start 2024-09-11T12:54:39.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- start start 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 0x7ff980137700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 0x7ff98015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff980168b60 con 0x7ff9800cc940 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9801689e0 con 0x7ff9800727a0 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff985800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff980168ce0 con 0x7ff980144110 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51408/0 (socket says 172.21.15.78:51408) 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 -- 172.21.15.78:0/4164635350 learned_addr learned my addr 172.21.15.78:0/4164635350 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97ea00640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 0x7ff980137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97f400640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 0x7ff98015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 -- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 msgr2=0x7ff98015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 0x7ff98015ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 -- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 msgr2=0x7ff980137700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 0x7ff980137700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 -- 172.21.15.78:0/4164635350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff980137e70 con 0x7ff9800cc940 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97f400640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 0x7ff98015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97ea00640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 0x7ff980137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:39.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff97e000640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c420 tx=0x7ff96c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.023+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c017070 con 0x7ff9800cc940 2024-09-11T12:54:39.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff96c003600 con 0x7ff9800cc940 2024-09-11T12:54:39.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c003a50 con 0x7ff9800cc940 2024-09-11T12:54:39.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9801fac80 con 0x7ff9800cc940 2024-09-11T12:54:39.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff980136be0 con 0x7ff9800cc940 2024-09-11T12:54:39.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9801faaa0 con 0x7ff9800cc940 2024-09-11T12:54:39.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7ff96c01a070 con 0x7ff9800cc940 2024-09-11T12:54:39.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff976a00640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 0x7ff960073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7ff96c091520 con 0x7ff9800cc940 2024-09-11T12:54:39.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.027+0000 7ff97ea00640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 0x7ff960073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.031+0000 7ff97ea00640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 0x7ff960073cf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f70 tx=0x7ff970009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.031+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff9801faaa0 con 0x7ff9800cc940 2024-09-11T12:54:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.223+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7ff98015af30 con 0x7ff960071830 2024-09-11T12:54:39.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.235+0000 7ff976a00640 1 -- 172.21.15.78:0/4164635350 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7ff98015af30 con 0x7ff960071830 2024-09-11T12:54:39.238 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045 2024-09-11T12:54:39.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.235+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 msgr2=0x7ff960073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.235+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 0x7ff960073cf0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff970000f70 tx=0x7ff970009040 comp rx=0 tx=0).stop 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.235+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 msgr2=0x7ff9801366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.235+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7ff96c00c420 tx=0x7ff96c00c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 shutdown_connections 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff980144110 0x7ff98015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9800cc940 0x7ff9801366a0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071830 0x7ff960073cf0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 --2- 172.21.15.78:0/4164635350 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9800727a0 0x7ff980137700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 >> 172.21.15.78:0/4164635350 conn(0x7ff9800d0410 msgr2=0x7ff9800d07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:39.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 shutdown_connections 2024-09-11T12:54:39.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.239+0000 7ff985800640 1 -- 172.21.15.78:0/4164635350 wait complete. 2024-09-11T12:54:39.259 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone31610 2024-09-11T12:54:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 -- 172.21.15.78:0/3636735459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 msgr2=0x7f5b4815c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/3636735459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b4815c7d0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f5b38009870 tx=0x7f5b3801c7f0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 -- 172.21.15.78:0/3636735459 shutdown_connections 2024-09-11T12:54:39.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/3636735459 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b4815c7d0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 -- 172.21.15.78:0/3636735459 >> 172.21.15.78:0/3636735459 conn(0x7f5b4806e860 msgr2=0x7f5b4806ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 -- 172.21.15.78:0/3636735459 shutdown_connections 2024-09-11T12:54:39.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.407+0000 7f5b4ec00640 1 -- 172.21.15.78:0/3636735459 wait complete. 2024-09-11T12:54:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 Processor -- start 2024-09-11T12:54:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- start start 2024-09-11T12:54:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b48141ec0 0x7f5b48137c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815c160 0x7f5b480cf370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48169980 con 0x7f5b48141ae0 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b48169b00 con 0x7f5b4815c160 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b48169c80 con 0x7f5b48141ec0 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51428/0 (socket says 172.21.15.78:51428) 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 -- 172.21.15.78:0/361922271 learned_addr learned my addr 172.21.15.78:0/361922271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:39.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 -- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b48141ec0 msgr2=0x7f5b48137c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b48141ec0 0x7f5b48137c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 -- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815c160 msgr2=0x7f5b480cf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815c160 0x7f5b480cf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 -- 172.21.15.78:0/361922271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4815b6a0 con 0x7f5b48141ae0 2024-09-11T12:54:39.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b47e00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5b3801a420 tx=0x7f5b380041a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b380b2040 con 0x7f5b48141ae0 2024-09-11T12:54:39.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b38005020 con 0x7f5b48141ae0 2024-09-11T12:54:39.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38005430 con 0x7f5b48141ae0 2024-09-11T12:54:39.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b480cfa30 con 0x7f5b48141ae0 2024-09-11T12:54:39.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b480cfff0 con 0x7f5b48141ae0 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.411+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b4815c7d0 con 0x7f5b48141ae0 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.415+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7f5b380a5810 con 0x7f5b48141ae0 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.415+0000 7f5b3fe00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 0x7f5b28073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.415+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f5b3812b9b0 con 0x7f5b48141ae0 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.415+0000 7f5b47400640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 0x7f5b28073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.415+0000 7f5b47400640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 0x7f5b28073cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b34007fd0 tx=0x7f5b340079a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.419+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5b4815b6a0 con 0x7f5b48141ae0 2024-09-11T12:54:39.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.611+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone31610", "target": ["mon-mgr", ""]}) -- 0x7f5b480a9dd0 con 0x7f5b28071830 2024-09-11T12:54:39.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b3fe00640 1 -- 172.21.15.78:0/361922271 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f5b340050d0 con 0x7f5b28071830 2024-09-11T12:54:39.620 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4 2024-09-11T12:54:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 msgr2=0x7f5b28073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 0x7f5b28073cf0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5b34007fd0 tx=0x7f5b340079a0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 msgr2=0x7f5b481f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f5b3801a420 tx=0x7f5b380041a0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 shutdown_connections 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b28071830 0x7f5b28073cf0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b4815c160 0x7f5b480cf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b48141ec0 0x7f5b48137c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 --2- 172.21.15.78:0/361922271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b48141ae0 0x7f5b481f7490 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 >> 172.21.15.78:0/361922271 conn(0x7f5b4806e860 msgr2=0x7f5b4813c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 shutdown_connections 2024-09-11T12:54:39.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.619+0000 7f5b4ec00640 1 -- 172.21.15.78:0/361922271 wait complete. 2024-09-11T12:54:39.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:39.642 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:39.653 INFO:teuthology.orchestra.run.smithi078.stdout:9 2024-09-11T12:54:39.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:39.653 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4) 2024-09-11T12:54:39.705 INFO:teuthology.orchestra.run.smithi078.stdout:9 2024-09-11T12:54:39.706 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone31610 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 -- 172.21.15.78:0/255710467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 msgr2=0x7f27b41416e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 --2- 172.21.15.78:0/255710467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b41416e0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f27b40b3e90 tx=0x7f27ac01caa0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 -- 172.21.15.78:0/255710467 shutdown_connections 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 --2- 172.21.15.78:0/255710467 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b41416e0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 --2- 172.21.15.78:0/255710467 >> v2:172.21.15.157:3300/0 conn(0x7f27b40cc5d0 0x7f27b40b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 -- 172.21.15.78:0/255710467 >> 172.21.15.78:0/255710467 conn(0x7f27b4071dd0 msgr2=0x7f27b40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 -- 172.21.15.78:0/255710467 shutdown_connections 2024-09-11T12:54:39.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.891+0000 7f27bac00640 1 -- 172.21.15.78:0/255710467 wait complete. 2024-09-11T12:54:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 Processor -- start 2024-09-11T12:54:39.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- start start 2024-09-11T12:54:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b40cc1f0 0x7f27b41533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b40cc5d0 0x7f27b41fb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4168e40 con 0x7f27b415ca30 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4168cc0 con 0x7f27b40cc5d0 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f27b4168fc0 con 0x7f27b40cc1f0 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51448/0 (socket says 172.21.15.78:51448) 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 -- 172.21.15.78:0/537137618 learned_addr learned my addr 172.21.15.78:0/537137618 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 -- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b40cc1f0 msgr2=0x7f27b41533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b40cc1f0 0x7f27b41533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 -- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b40cc5d0 msgr2=0x7f27b41fb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b40cc5d0 0x7f27b41fb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 -- 172.21.15.78:0/537137618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b41fbbd0 con 0x7f27b415ca30 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b8c00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f27ac01c510 tx=0x7f27ac005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0b2040 con 0x7f27b415ca30 2024-09-11T12:54:39.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ac0bb070 con 0x7f27b415ca30 2024-09-11T12:54:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac004550 con 0x7f27b415ca30 2024-09-11T12:54:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b41539c0 con 0x7f27b415ca30 2024-09-11T12:54:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.895+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b4153f80 con 0x7f27b415ca30 2024-09-11T12:54:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b40b2de0 con 0x7f27b415ca30 2024-09-11T12:54:39.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 16) ==== 95085+0+0 (secure 0 0 0) 0x7f27ac0046f0 con 0x7f27b415ca30 2024-09-11T12:54:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27b0c00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 0x7f2790073cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f27ac12caf0 con 0x7f27b415ca30 2024-09-11T12:54:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27b3e00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 0x7f2790073cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:39.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.899+0000 7f27b3e00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 0x7f2790073cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27a4007fd0 tx=0x7f27a40078e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:39.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:39.903+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f27ac016750 con 0x7f27b415ca30 2024-09-11T12:54:40.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.095+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone31610", "target": ["mon-mgr", ""]}) -- 0x7f27b40ce840 con 0x7f2790071830 2024-09-11T12:54:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.103+0000 7f27b0c00640 1 -- 172.21.15.78:0/537137618 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+743 (secure 0 0 0) 0x7f27b40ce840 con 0x7f2790071830 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "atime": "2024-09-11 12:54:33", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_pcent": "80.00", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_quota": 10485760, 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "bytes_used": 8388608, 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "created_at": "2024-09-11 12:54:38", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "ctime": "2024-09-11 12:54:38", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "data_pool": "cephfs_data", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "earmark": "", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "features": [ 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-clone", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-autoprotect", 2024-09-11T12:54:40.107 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot-retention" 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flavor": 2, 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "gid": 1000, 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "mode": 16895, 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "mon_addrs": [ 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.78:6789", 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6789", 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "172.21.15.157:6790" 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "mtime": "2024-09-11 12:54:37", 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "path": "/volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4", 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "pool_namespace": "", 2024-09-11T12:54:40.108 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "complete", 2024-09-11T12:54:40.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type": "clone", 2024-09-11T12:54:40.109 INFO:teuthology.orchestra.run.smithi078.stdout: "uid": 1000 2024-09-11T12:54:40.109 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 msgr2=0x7f2790073cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 0x7f2790073cf0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27a4007fd0 tx=0x7f27a40078e0 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 msgr2=0x7f27b4151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f27ac01c510 tx=0x7f27ac005a60 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 shutdown_connections 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f27b415ca30 0x7f27b4151af0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2790071830 0x7f2790073cf0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f27b40cc5d0 0x7f27b41fb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 --2- 172.21.15.78:0/537137618 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f27b40cc1f0 0x7f27b41533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 >> 172.21.15.78:0/537137618 conn(0x7f27b4071dd0 msgr2=0x7f27b4138650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 shutdown_connections 2024-09-11T12:54:40.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:40.107+0000 7f27bac00640 1 -- 172.21.15.78:0/537137618 wait complete. 2024-09-11T12:54:40.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.128 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.136 INFO:teuthology.orchestra.run.smithi078.stderr:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550: ceph.quota.max_bytes: No such attribute 2024-09-11T12:54:40.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:54:40.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.138 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.192 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:54:40.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.192 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.244 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.2 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.5 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.4 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.7 2024-09-11T12:54:40.301 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.6 2024-09-11T12:54:40.302 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.0 2024-09-11T12:54:40.302 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.3 2024-09-11T12:54:40.302 INFO:teuthology.orchestra.run.smithi078.stdout:volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.1 2024-09-11T12:54:40.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.303 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.355 INFO:teuthology.orchestra.run.smithi078.stdout:41ff 2024-09-11T12:54:40.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.356 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/) 2024-09-11T12:54:40.407 INFO:teuthology.orchestra.run.smithi078.stdout:41ff 2024-09-11T12:54:40.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.409 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.459 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.460 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/) 2024-09-11T12:54:40.511 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.512 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.563 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.564 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/) 2024-09-11T12:54:40.615 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.616 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550) 2024-09-11T12:54:40.667 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:40.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.668 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/) 2024-09-11T12:54:40.723 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:40.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.724 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.2) 2024-09-11T12:54:40.777 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:40.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.779 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.2) 2024-09-11T12:54:40.835 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:40.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.836 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.2) 2024-09-11T12:54:40.891 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.893 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.2) 2024-09-11T12:54:40.946 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:40.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:40.948 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.2) 2024-09-11T12:54:41.003 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.004 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.2) 2024-09-11T12:54:41.055 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.056 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.2) 2024-09-11T12:54:41.107 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:41.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.109 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.2) 2024-09-11T12:54:41.159 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:41.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.160 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.5) 2024-09-11T12:54:41.217 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:41.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.218 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.5) 2024-09-11T12:54:41.270 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:41.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.271 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.5) 2024-09-11T12:54:41.323 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:41.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.325 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.5) 2024-09-11T12:54:41.375 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:41.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.376 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.5) 2024-09-11T12:54:41.427 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.428 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.5) 2024-09-11T12:54:41.479 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.493 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.5) 2024-09-11T12:54:41.544 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:41.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.545 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.5) 2024-09-11T12:54:41.595 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:41.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.597 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.4) 2024-09-11T12:54:41.649 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:41.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.650 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.4) 2024-09-11T12:54:41.702 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:41.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.704 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.4) 2024-09-11T12:54:41.755 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:41.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.756 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.4) 2024-09-11T12:54:41.807 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:41.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.808 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.4) 2024-09-11T12:54:41.859 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.860 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.4) 2024-09-11T12:54:41.915 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:41.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:41.916 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.4) 2024-09-11T12:54:42.501 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:54:42.505 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:54:43.784 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:43.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:43.785 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.4) 2024-09-11T12:54:43.795 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:43.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:43.796 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.7) 2024-09-11T12:54:43.854 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:43.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:43.855 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.7) 2024-09-11T12:54:43.908 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:43.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:43.909 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.7) 2024-09-11T12:54:43.960 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:43.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:43.961 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.7) 2024-09-11T12:54:44.011 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:44.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.013 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.7) 2024-09-11T12:54:44.064 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.065 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.7) 2024-09-11T12:54:44.115 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.117 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.7) 2024-09-11T12:54:44.168 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:44.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.169 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.7) 2024-09-11T12:54:44.224 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:44.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.225 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.6) 2024-09-11T12:54:44.277 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:44.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.278 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.6) 2024-09-11T12:54:44.331 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:44.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.333 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.6) 2024-09-11T12:54:44.388 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:44.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.389 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.6) 2024-09-11T12:54:44.443 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:44.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.445 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.6) 2024-09-11T12:54:44.495 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.496 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.6) 2024-09-11T12:54:44.552 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.553 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.6) 2024-09-11T12:54:44.604 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:44.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.605 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.6) 2024-09-11T12:54:44.656 INFO:teuthology.orchestra.run.smithi078.stdout:1726059277 2024-09-11T12:54:44.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.657 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.0) 2024-09-11T12:54:44.713 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:44.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.714 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.0) 2024-09-11T12:54:44.764 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:44.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.765 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.0) 2024-09-11T12:54:44.815 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:44.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.816 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.0) 2024-09-11T12:54:44.867 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:44.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.868 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.0) 2024-09-11T12:54:44.919 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.920 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.0) 2024-09-11T12:54:44.972 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:44.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:44.973 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.0) 2024-09-11T12:54:45.030 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.031 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.0) 2024-09-11T12:54:45.088 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.089 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.3) 2024-09-11T12:54:45.146 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:45.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.147 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.3) 2024-09-11T12:54:45.196 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:45.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.198 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.3) 2024-09-11T12:54:45.251 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:45.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.253 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.3) 2024-09-11T12:54:45.304 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:45.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.305 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.3) 2024-09-11T12:54:45.356 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:45.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.358 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.3) 2024-09-11T12:54:45.407 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:45.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.409 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.3) 2024-09-11T12:54:45.463 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.465 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.3) 2024-09-11T12:54:45.516 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.518 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.1) 2024-09-11T12:54:45.570 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:45.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.571 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.1) 2024-09-11T12:54:45.621 INFO:teuthology.orchestra.run.smithi078.stdout:81b4 2024-09-11T12:54:45.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.622 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.1) 2024-09-11T12:54:45.672 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:45.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.673 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.1) 2024-09-11T12:54:45.724 INFO:teuthology.orchestra.run.smithi078.stdout:1000 2024-09-11T12:54:45.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.725 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.1) 2024-09-11T12:54:45.776 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:45.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.777 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.1) 2024-09-11T12:54:45.828 INFO:teuthology.orchestra.run.smithi078.stdout:1271 2024-09-11T12:54:45.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.829 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045/.snap/subvol_snap58550/subvolume_file.1) 2024-09-11T12:54:45.887 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:45.888 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone31610/2806224a-1aca-469c-b4a7-7eb50a2851f4/subvolume_file.1) 2024-09-11T12:54:45.940 INFO:teuthology.orchestra.run.smithi078.stdout:1726059276 2024-09-11T12:54:45.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol32140 fake subvol_clone76311 2024-09-11T12:54:46.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 -- 172.21.15.78:0/3059156715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 msgr2=0x7f26e00cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 --2- 172.21.15.78:0/3059156715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e00cf5a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f26d0009840 tx=0x7f26d001c820 comp rx=0 tx=0).stop 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 -- 172.21.15.78:0/3059156715 shutdown_connections 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 --2- 172.21.15.78:0/3059156715 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 0x7f26e015bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 --2- 172.21.15.78:0/3059156715 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e00cf5a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 -- 172.21.15.78:0/3059156715 >> 172.21.15.78:0/3059156715 conn(0x7f26e00b1c50 msgr2=0x7f26e0070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 -- 172.21.15.78:0/3059156715 shutdown_connections 2024-09-11T12:54:46.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.139+0000 7f26e5c00640 1 -- 172.21.15.78:0/3059156715 wait complete. 2024-09-11T12:54:46.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 Processor -- start 2024-09-11T12:54:46.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- start start 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e0149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 0x7f26e014d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0152390 con 0x7f26e00b36b0 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26e0152210 con 0x7f26e00b32d0 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f26e0152510 con 0x7f26e00b5f10 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60174/0 (socket says 172.21.15.78:60174) 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 -- 172.21.15.78:0/4184575061 learned_addr learned my addr 172.21.15.78:0/4184575061 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26dea00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e0149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26df400640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 0x7f26e014d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 -- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 msgr2=0x7f26e014d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 0x7f26e014d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 -- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 msgr2=0x7f26e0149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e0149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 -- 172.21.15.78:0/4184575061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26e006c650 con 0x7f26e00b36b0 2024-09-11T12:54:46.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d7e00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c4e0 tx=0x7f26c800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c80052d0 con 0x7f26e00b36b0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26c8005470 con 0x7f26e00b36b0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26e006c8f0 con 0x7f26e00b36b0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26c8012890 con 0x7f26e00b36b0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.143+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26e006ceb0 con 0x7f26e00b36b0 2024-09-11T12:54:46.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26e0152a40 con 0x7f26e00b36b0 2024-09-11T12:54:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f26c8003600 con 0x7f26e00b36b0 2024-09-11T12:54:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26d6a00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 0x7f26bc073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f26c8091d10 con 0x7f26e00b36b0 2024-09-11T12:54:46.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26dea00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 0x7f26bc073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.147+0000 7f26dea00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 0x7f26bc073e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26d00af000 tx=0x7f26d001b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:46.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.151+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f26c805d160 con 0x7f26e00b36b0 2024-09-11T12:54:46.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.359+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "fake", "target_sub_name": "subvol_clone76311", "target": ["mon-mgr", ""]}) -- 0x7f26e015dc40 con 0x7f26bc071950 2024-09-11T12:54:46.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.423+0000 7f26d6a00640 1 -- 172.21.15.78:0/4184575061 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26e015dc40 con 0x7f26bc071950 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 msgr2=0x7f26bc073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 0x7f26bc073e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f26d00af000 tx=0x7f26d001b060 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 msgr2=0x7f26e014ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f26c800c4e0 tx=0x7f26c800c9a0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 shutdown_connections 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26e00b5f10 0x7f26e014d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26e00b36b0 0x7f26e014ce10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26bc071950 0x7f26bc073e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 --2- 172.21.15.78:0/4184575061 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26e00b32d0 0x7f26e0149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 >> 172.21.15.78:0/4184575061 conn(0x7f26e00b1c50 msgr2=0x7f26e0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 shutdown_connections 2024-09-11T12:54:46.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.427+0000 7f26e5c00640 1 -- 172.21.15.78:0/4184575061 wait complete. 2024-09-11T12:54:46.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone76311 2024-09-11T12:54:46.596 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- 172.21.15.78:0/4213546249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0641518b0 msgr2=0x7fa0640b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- 172.21.15.78:0/4213546249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0641518b0 0x7fa0640b1a10 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fa054009900 tx=0x7fa05401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- 172.21.15.78:0/4213546249 shutdown_connections 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- 172.21.15.78:0/4213546249 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0640ce320 0x7fa06406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- 172.21.15.78:0/4213546249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0641518b0 0x7fa0640b1a10 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- 172.21.15.78:0/4213546249 >> 172.21.15.78:0/4213546249 conn(0x7fa064071dd0 msgr2=0x7fa0640721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- 172.21.15.78:0/4213546249 shutdown_connections 2024-09-11T12:54:46.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- 172.21.15.78:0/4213546249 wait complete. 2024-09-11T12:54:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 Processor -- start 2024-09-11T12:54:46.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- start start 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0640b1f50 0x7fa06414fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0641518b0 0x7fa064144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa064167960 con 0x7fa0640ce320 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0641677e0 con 0x7fa0640b1f50 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa06b400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa064167ae0 con 0x7fa0641518b0 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60180/0 (socket says 172.21.15.78:60180) 2024-09-11T12:54:46.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 -- 172.21.15.78:0/2226720251 learned_addr learned my addr 172.21.15.78:0/2226720251 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa068a00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0640b1f50 0x7fa06414fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa069400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0641518b0 0x7fa064144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 -- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0641518b0 msgr2=0x7fa064144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0641518b0 0x7fa064144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 -- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0640b1f50 msgr2=0x7fa06414fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0640b1f50 0x7fa06414fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.595+0000 7fa063e00640 1 -- 172.21.15.78:0/2226720251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa06414f110 con 0x7fa0640ce320 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa063e00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c4b0 tx=0x7fa05000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0500052d0 con 0x7fa0640ce320 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa050005470 con 0x7fa0640ce320 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa050012870 con 0x7fa0640ce320 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa06414f3f0 con 0x7fa0640ce320 2024-09-11T12:54:46.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa06414f9b0 con 0x7fa0640ce320 2024-09-11T12:54:46.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa06414fbb0 con 0x7fa0640ce320 2024-09-11T12:54:46.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7fa050003600 con 0x7fa0640ce320 2024-09-11T12:54:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 0x7fa044073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fa050091e70 con 0x7fa0640ce320 2024-09-11T12:54:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.599+0000 7fa068a00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 0x7fa044073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.603+0000 7fa068a00640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 0x7fa044073e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0540af000 tx=0x7fa05401b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:46.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.603+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa06414fbb0 con 0x7fa0640ce320 2024-09-11T12:54:46.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:46.803+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone76311", "target": ["mon-mgr", ""]}) -- 0x7fa064151030 con 0x7fa044071950 2024-09-11T12:54:48.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.795+0000 7fa060c00640 1 -- 172.21.15.78:0/2226720251 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7fa064151030 con 0x7fa044071950 2024-09-11T12:54:48.797 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:48.797 INFO:teuthology.orchestra.run.smithi078.stdout: "status": { 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "pending", 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: "source": { 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: "volume": "cephfs", 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: "subvolume": "subvol32140", 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot": "fake" 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:48.798 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.795+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 msgr2=0x7fa044073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.795+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 0x7fa044073e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa0540af000 tx=0x7fa05401b120 comp rx=0 tx=0).stop 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.795+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 msgr2=0x7fa0641509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.795+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fa05000c4b0 tx=0x7fa05000c970 comp rx=0 tx=0).stop 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 shutdown_connections 2024-09-11T12:54:48.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0641518b0 0x7fa064144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa0640ce320 0x7fa0641509a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa044071950 0x7fa044073e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 --2- 172.21.15.78:0/2226720251 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0640b1f50 0x7fa06414fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 >> 172.21.15.78:0/2226720251 conn(0x7fa064071dd0 msgr2=0x7fa0640afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 shutdown_connections 2024-09-11T12:54:48.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:48.799+0000 7fa06b400640 1 -- 172.21.15.78:0/2226720251 wait complete. 2024-09-11T12:54:49.830 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone76311 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 -- 172.21.15.78:0/2206620415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 msgr2=0x7fa50c003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2206620415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 0x7fa50c003e80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fa518009ee0 tx=0x7fa51801d310 comp rx=0 tx=0).stop 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52a400640 1 -- 172.21.15.78:0/2206620415 reap_dead start 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 -- 172.21.15.78:0/2206620415 shutdown_connections 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 -- 172.21.15.78:0/2206620415 >> 172.21.15.78:0/2206620415 conn(0x7fa524075c10 msgr2=0x7fa5240b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 -- 172.21.15.78:0/2206620415 shutdown_connections 2024-09-11T12:54:49.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.983+0000 7fa52ae00640 1 -- 172.21.15.78:0/2206620415 wait complete. 2024-09-11T12:54:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 Processor -- start 2024-09-11T12:54:49.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- start start 2024-09-11T12:54:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 0x7fa52413e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:49.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 0x7fa524156ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa52415dd80 con 0x7fa524152d80 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa52415dc00 con 0x7fa50c003a60 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa52415df00 con 0x7fa5241558a0 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60206/0 (socket says 172.21.15.78:60206) 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 -- 172.21.15.78:0/2896903215 learned_addr learned my addr 172.21.15.78:0/2896903215 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa529000640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 0x7fa524156ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa529a00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 0x7fa52413e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 -- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 msgr2=0x7fa524156ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 0x7fa524156ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:49.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 -- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 msgr2=0x7fa52413e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 0x7fa52413e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 -- 172.21.15.78:0/2896903215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa524156580 con 0x7fa524152d80 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa529000640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 0x7fa524156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52a400640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa524070bd0 tx=0x7fa51c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa510006bc0 con 0x7fa524152d80 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa51c00f890 con 0x7fa524152d80 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa524155d80 con 0x7fa524152d80 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c00fca0 con 0x7fa524152d80 2024-09-11T12:54:49.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.987+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52413ee00 con 0x7fa524152d80 2024-09-11T12:54:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa52415beb0 con 0x7fa524152d80 2024-09-11T12:54:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7fa51c003600 con 0x7fa524152d80 2024-09-11T12:54:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa521600640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 0x7fa504073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:49.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fa51c092d80 con 0x7fa524152d80 2024-09-11T12:54:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa529a00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 0x7fa504073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:49.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.991+0000 7fa529a00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 0x7fa504073e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa518009ee0 tx=0x7fa5180a7040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:49.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:49.995+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa51c05e710 con 0x7fa524152d80 2024-09-11T12:54:50.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:50.195+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone76311", "target": ["mon-mgr", ""]}) -- 0x7fa5240ced10 con 0x7fa504071950 2024-09-11T12:54:53.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa521600640 1 -- 172.21.15.78:0/2896903215 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fa5240ced10 con 0x7fa504071950 2024-09-11T12:54:53.792 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:53.793 INFO:teuthology.orchestra.run.smithi078.stdout: "status": { 2024-09-11T12:54:53.793 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "complete" 2024-09-11T12:54:53.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:53.793 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 msgr2=0x7fa504073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 0x7fa504073e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa518009ee0 tx=0x7fa5180a7040 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 msgr2=0x7fa5240cd730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fa524070bd0 tx=0x7fa51c011c40 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 shutdown_connections 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa524152d80 0x7fa5240cd730 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5241558a0 0x7fa524156ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa504071950 0x7fa504073e10 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 --2- 172.21.15.78:0/2896903215 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa50c003a60 0x7fa52413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 >> 172.21.15.78:0/2896903215 conn(0x7fa524075c10 msgr2=0x7fa5240b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:53.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 shutdown_connections 2024-09-11T12:54:53.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:53.791+0000 7fa52ae00640 1 -- 172.21.15.78:0/2896903215 wait complete. 2024-09-11T12:54:53.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:53.818 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone76311/.meta) 2024-09-11T12:54:53.839 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:54:53.839 INFO:teuthology.orchestra.run.smithi078.stdout:version = 2 2024-09-11T12:54:53.839 INFO:teuthology.orchestra.run.smithi078.stdout:type = clone 2024-09-11T12:54:53.840 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol_clone76311/44976963-b076-4ccc-a0ed-b562da785154 2024-09-11T12:54:53.840 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:54:53.840 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:53.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol32140 subvol_snap58550 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3147650829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc073e90 msgr2=0x7fa5bc071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3147650829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc073e90 0x7fa5bc071e10 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc06e460 tx=0x7fa5a401c890 comp rx=0 tx=0).stop 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3147650829 shutdown_connections 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3147650829 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc073e90 0x7fa5bc071e10 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3147650829 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc072350 0x7fa5bc15dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3147650829 >> v2:172.21.15.157:3300/0 conn(0x7fa5bc151050 0x7fa5bc073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3147650829 >> 172.21.15.78:0/3147650829 conn(0x7fa5bc0b3050 msgr2=0x7fa5bc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3147650829 shutdown_connections 2024-09-11T12:54:54.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3147650829 wait complete. 2024-09-11T12:54:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 Processor -- start 2024-09-11T12:54:54.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- start start 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc073e90 0x7fa5bc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc151050 0x7fa5bc15b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc15e7f0 con 0x7fa5bc072350 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5bc15e670 con 0x7fa5bc151050 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5c0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5bc15e970 con 0x7fa5bc073e90 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5b9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51190/0 (socket says 172.21.15.78:51190) 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5b9e00640 1 -- 172.21.15.78:0/3681481221 learned_addr learned my addr 172.21.15.78:0/3681481221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5b9400640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc073e90 0x7fa5bc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.031+0000 7fa5ba800640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc151050 0x7fa5bc15b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 -- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc073e90 msgr2=0x7fa5bc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc073e90 0x7fa5bc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 -- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc151050 msgr2=0x7fa5bc15b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc151050 0x7fa5bc15b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 -- 172.21.15.78:0/3681481221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5bc14c4d0 con 0x7fa5bc072350 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b9e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00db90 tx=0x7fa5ac00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac01b070 con 0x7fa5bc072350 2024-09-11T12:54:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5ac00ae50 con 0x7fa5bc072350 2024-09-11T12:54:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5bc14c7b0 con 0x7fa5bc072350 2024-09-11T12:54:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5ac00e9c0 con 0x7fa5bc072350 2024-09-11T12:54:54.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5bc1ed120 con 0x7fa5bc072350 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5bc071e10 con 0x7fa5bc072350 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7fa5ac01e070 con 0x7fa5bc072350 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.035+0000 7fa5b2000640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 0x7fa598073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.039+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fa5ac092770 con 0x7fa5bc072350 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.039+0000 7fa5b9400640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 0x7fa598073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.039+0000 7fa5b9400640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 0x7fa598073e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40af000 tx=0x7fa5a401b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.043+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa5ac05dc40 con 0x7fa5bc072350 2024-09-11T12:54:54.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.235+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "subvol_snap58550", "target": ["mon-mgr", ""]}) -- 0x7fa5bc070ce0 con 0x7fa598071950 2024-09-11T12:54:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.247+0000 7fa5b2000640 1 -- 172.21.15.78:0/3681481221 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7fa5bc070ce0 con 0x7fa598071950 2024-09-11T12:54:54.251 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:54:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "created_at": "2024-09-11 12:54:37.746758", 2024-09-11T12:54:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "data_pool": "cephfs_data", 2024-09-11T12:54:54.252 INFO:teuthology.orchestra.run.smithi078.stdout: "has_pending_clones": "no" 2024-09-11T12:54:54.252 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 msgr2=0x7fa598073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 0x7fa598073e10 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa5a40af000 tx=0x7fa5a401b0d0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 msgr2=0x7fa5bc1ed920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fa5ac00db90 tx=0x7fa5ac00a620 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 shutdown_connections 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa598071950 0x7fa598073e10 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5bc151050 0x7fa5bc15b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5bc073e90 0x7fa5bc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 --2- 172.21.15.78:0/3681481221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5bc072350 0x7fa5bc1ed920 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 >> 172.21.15.78:0/3681481221 conn(0x7fa5bc0b3050 msgr2=0x7fa5bc164870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 shutdown_connections 2024-09-11T12:54:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.251+0000 7fa5c0e00640 1 -- 172.21.15.78:0/3681481221 wait complete. 2024-09-11T12:54:54.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol32140 2024-09-11T12:54:54.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- 172.21.15.78:0/2272902156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 msgr2=0x7f43800b59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- 172.21.15.78:0/2272902156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f43800b59c0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f43800b60b0 tx=0x7f437801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- 172.21.15.78:0/2272902156 shutdown_connections 2024-09-11T12:54:54.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- 172.21.15.78:0/2272902156 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f43800b59c0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- 172.21.15.78:0/2272902156 >> v2:172.21.15.157:3300/0 conn(0x7f43800cf520 0x7f43800cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- 172.21.15.78:0/2272902156 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4380152990 0x7f438015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- 172.21.15.78:0/2272902156 >> 172.21.15.78:0/2272902156 conn(0x7f43800b3050 msgr2=0x7f43800b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- 172.21.15.78:0/2272902156 shutdown_connections 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- 172.21.15.78:0/2272902156 wait complete. 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 Processor -- start 2024-09-11T12:54:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- start start 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43800cf520 0x7f4380158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 0x7f438006fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f438015f6e0 con 0x7f43800b5530 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f438015f560 con 0x7f4380152990 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.419+0000 7f4387600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f438015f860 con 0x7f43800cf520 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51214/0 (socket says 172.21.15.78:51214) 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 -- 172.21.15.78:0/3367438708 learned_addr learned my addr 172.21.15.78:0/3367438708 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:54.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437fe00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43800cf520 0x7f4380158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4385600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 0x7f438006fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 -- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43800cf520 msgr2=0x7f4380158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43800cf520 0x7f4380158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 -- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 msgr2=0x7f438006fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 0x7f438006fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 -- 172.21.15.78:0/3367438708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43801593d0 con 0x7f43800b5530 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4385600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 0x7f438006fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4384c00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f437000dac0 tx=0x7f437000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370017070 con 0x7f43800b5530 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4370004b30 con 0x7f43800b5530 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370005020 con 0x7f43800b5530 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f438006eca0 con 0x7f43800b5530 2024-09-11T12:54:54.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f438006f260 con 0x7f43800b5530 2024-09-11T12:54:54.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43800cf960 con 0x7f43800b5530 2024-09-11T12:54:54.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f43700051c0 con 0x7f43800b5530 2024-09-11T12:54:54.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.423+0000 7f437cc00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 0x7f4360073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.427+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f4370092620 con 0x7f43800b5530 2024-09-11T12:54:54.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.427+0000 7f437fe00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 0x7f4360073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.427+0000 7f437fe00640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 0x7f4360073e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4380142920 tx=0x7f436c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.431+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4370020070 con 0x7f43800b5530 2024-09-11T12:54:54.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.623+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7f438006ea50 con 0x7f4360071950 2024-09-11T12:54:54.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.635+0000 7f437cc00640 1 -- 172.21.15.78:0/3367438708 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7f438006ea50 con 0x7f4360071950 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "fake" 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "subvol_snap58550" 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:54:54.640 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 msgr2=0x7f4360073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 0x7f4360073e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4380142920 tx=0x7f436c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 msgr2=0x7f4380141d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f437000dac0 tx=0x7f437000df80 comp rx=0 tx=0).stop 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 shutdown_connections 2024-09-11T12:54:54.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4360071950 0x7f4360073e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4380152990 0x7f438006fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f43800cf520 0x7f4380158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 --2- 172.21.15.78:0/3367438708 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f43800b5530 0x7f4380141d00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 >> 172.21.15.78:0/3367438708 conn(0x7f43800b3050 msgr2=0x7f4380165dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 shutdown_connections 2024-09-11T12:54:54.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.639+0000 7f4387600640 1 -- 172.21.15.78:0/3367438708 wait complete. 2024-09-11T12:54:54.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol32140 subvol_snap58550 2024-09-11T12:54:54.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.803+0000 7f0a1e400640 1 -- 172.21.15.78:0/1923522569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b5e90 msgr2=0x7f0a1815f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.803+0000 7f0a1e400640 1 --2- 172.21.15.78:0/1923522569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b5e90 0x7f0a1815f4e0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009870 tx=0x7f0a0801c7f0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- 172.21.15.78:0/1923522569 shutdown_connections 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 --2- 172.21.15.78:0/1923522569 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180b3730 0x7f0a1815fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 --2- 172.21.15.78:0/1923522569 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b5e90 0x7f0a1815f4e0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- 172.21.15.78:0/1923522569 >> 172.21.15.78:0/1923522569 conn(0x7f0a180b1c00 msgr2=0x7f0a18070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- 172.21.15.78:0/1923522569 shutdown_connections 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- 172.21.15.78:0/1923522569 wait complete. 2024-09-11T12:54:54.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 Processor -- start 2024-09-11T12:54:54.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- start start 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a180b5e90 0x7f0a18155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180cf3e0 0x7f0a18159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181687f0 con 0x7f0a180b3730 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a18168670 con 0x7f0a180b5e90 2024-09-11T12:54:54.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a1e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a18168970 con 0x7f0a180cf3e0 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51242/0 (socket says 172.21.15.78:51242) 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 -- 172.21.15.78:0/4123573452 learned_addr learned my addr 172.21.15.78:0/4123573452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17e00640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180cf3e0 0x7f0a18159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a0fe00640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a180b5e90 0x7f0a18155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 -- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180cf3e0 msgr2=0x7f0a18159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180cf3e0 0x7f0a18159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 -- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a180b5e90 msgr2=0x7f0a18155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a180b5e90 0x7f0a18155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 -- 172.21.15.78:0/4123573452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1813ea40 con 0x7f0a180b3730 2024-09-11T12:54:54.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.807+0000 7f0a17400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009790 tx=0x7f0a08004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a080bb070 con 0x7f0a180b3730 2024-09-11T12:54:54.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a08005320 con 0x7f0a180b3730 2024-09-11T12:54:54.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a080b2040 con 0x7f0a180b3730 2024-09-11T12:54:54.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a181f81c0 con 0x7f0a180b3730 2024-09-11T12:54:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1813e1a0 con 0x7f0a180b3730 2024-09-11T12:54:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a180b3020 con 0x7f0a180b3730 2024-09-11T12:54:54.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f0a080a5810 con 0x7f0a180b3730 2024-09-11T12:54:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.811+0000 7f0a0f400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 0x7f09f4073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:54.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.815+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f0a0812be60 con 0x7f0a180b3730 2024-09-11T12:54:54.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.815+0000 7f0a0fe00640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 0x7f09f4073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:54.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.815+0000 7f0a0fe00640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 0x7f09f4073e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a181598d0 tx=0x7f0a00009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:54.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:54.815+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0a08016410 con 0x7f0a180b3730 2024-09-11T12:54:55.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.015+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "subvol_snap58550", "target": ["mon-mgr", ""]}) -- 0x7f0a18164fc0 con 0x7f09f4071950 2024-09-11T12:54:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.039+0000 7f0a0f400640 1 -- 172.21.15.78:0/4123573452 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0a18164fc0 con 0x7f09f4071950 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 msgr2=0x7f09f4073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 0x7f09f4073e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a181598d0 tx=0x7f0a00009040 comp rx=0 tx=0).stop 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 msgr2=0x7f0a181f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009790 tx=0x7f0a08004220 comp rx=0 tx=0).stop 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 shutdown_connections 2024-09-11T12:54:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a180cf3e0 0x7f0a18159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f09f4071950 0x7f09f4073e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a180b5e90 0x7f0a18155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 --2- 172.21.15.78:0/4123573452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a180b3730 0x7f0a181f7c80 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 >> 172.21.15.78:0/4123573452 conn(0x7f0a180b1c00 msgr2=0x7f0a180d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 shutdown_connections 2024-09-11T12:54:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.043+0000 7f0a1e400640 1 -- 172.21.15.78:0/4123573452 wait complete. 2024-09-11T12:54:55.065 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol32140 fake 2024-09-11T12:54:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 -- 172.21.15.78:0/3333519519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8620004da0 msgr2=0x7f8620005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 --2- 172.21.15.78:0/3333519519 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8620004da0 0x7f8620005220 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8634001860 tx=0x7f863401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8641000640 1 -- 172.21.15.78:0/3333519519 reap_dead start 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 -- 172.21.15.78:0/3333519519 shutdown_connections 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 -- 172.21.15.78:0/3333519519 >> 172.21.15.78:0/3333519519 conn(0x7f863c0b3050 msgr2=0x7f863c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 -- 172.21.15.78:0/3333519519 shutdown_connections 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.207+0000 7f8643000640 1 -- 172.21.15.78:0/3333519519 wait complete. 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 Processor -- start 2024-09-11T12:54:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- start start 2024-09-11T12:54:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 0x7f863c1468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f863c073280 0x7f863c073730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f863c0cfcf0 con 0x7f863c1e45a0 2024-09-11T12:54:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f863c0cfb70 con 0x7f8620004da0 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f863c0cdba0 con 0x7f863c073280 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51268/0 (socket says 172.21.15.78:51268) 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 -- 172.21.15.78:0/3866694041 learned_addr learned my addr 172.21.15.78:0/3866694041 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863be00640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 0x7f863c1468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8641000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f863c073280 0x7f863c073730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 -- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f863c073280 msgr2=0x7f863c073730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f863c073280 0x7f863c073730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 -- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 msgr2=0x7f863c1468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 0x7f863c1468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 -- 172.21.15.78:0/3866694041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f863c146e10 con 0x7f863c1e45a0 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863be00640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 0x7f863c1468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f863b400640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8628011800 tx=0x7f8628011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:55.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8628004460 con 0x7f863c1e45a0 2024-09-11T12:54:55.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8628016070 con 0x7f863c1e45a0 2024-09-11T12:54:55.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f863c1e2970 con 0x7f863c1e45a0 2024-09-11T12:54:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862800fba0 con 0x7f863c1e45a0 2024-09-11T12:54:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.211+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f863c1e2f30 con 0x7f863c1e45a0 2024-09-11T12:54:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f863c0b59c0 con 0x7f863c1e45a0 2024-09-11T12:54:55.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f8628018030 con 0x7f863c1e45a0 2024-09-11T12:54:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f8633e00640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 0x7f861c073e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f8628092aa0 con 0x7f863c1e45a0 2024-09-11T12:54:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f863be00640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 0x7f861c073e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.215+0000 7f863be00640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 0x7f861c073e40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f862c001360 tx=0x7f862c006f60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:55.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.219+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f862805e5d0 con 0x7f863c1e45a0 2024-09-11T12:54:55.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.423+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol32140", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f863c15a740 con 0x7f861c071980 2024-09-11T12:54:55.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.447+0000 7f8633e00640 1 -- 172.21.15.78:0/3866694041 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f863c15a740 con 0x7f861c071980 2024-09-11T12:54:55.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 msgr2=0x7f861c073e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 0x7f861c073e40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f862c001360 tx=0x7f862c006f60 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 msgr2=0x7f863c1e4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8628011800 tx=0x7f8628011cc0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 shutdown_connections 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f863c073280 0x7f863c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f863c1e45a0 0x7f863c1e4a20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f861c071980 0x7f861c073e40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 --2- 172.21.15.78:0/3866694041 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8620004da0 0x7f863c1468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 >> 172.21.15.78:0/3866694041 conn(0x7f863c0b3050 msgr2=0x7f863c15b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 shutdown_connections 2024-09-11T12:54:55.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.451+0000 7f8643000640 1 -- 172.21.15.78:0/3866694041 wait complete. 2024-09-11T12:54:55.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:55.473 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol32140/.meta) 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol32140/1895bdbb-1c5e-4cae-9354-2881e4678045 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:54:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:55.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol32140 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- 172.21.15.78:0/3099034144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc0731a0 msgr2=0x7fb2fc0735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- 172.21.15.78:0/3099034144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc0735e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009930 tx=0x7fb2e801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- 172.21.15.78:0/3099034144 shutdown_connections 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- 172.21.15.78:0/3099034144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc0735e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- 172.21.15.78:0/3099034144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 0x7fb2fc0cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- 172.21.15.78:0/3099034144 >> 172.21.15.78:0/3099034144 conn(0x7fb2fc075c10 msgr2=0x7fb2fc0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:55.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- 172.21.15.78:0/3099034144 shutdown_connections 2024-09-11T12:54:55.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- 172.21.15.78:0/3099034144 wait complete. 2024-09-11T12:54:55.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 Processor -- start 2024-09-11T12:54:55.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- start start 2024-09-11T12:54:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 0x7fb2fc13a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc15db00 0x7fb2fc138e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc168130 con 0x7fb2fc0731a0 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2fc167fb0 con 0x7fb2fc15db00 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb301000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2fc1682b0 con 0x7fb2fc140190 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb2fb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51278/0 (socket says 172.21.15.78:51278) 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb2fb400640 1 -- 172.21.15.78:0/3617279458 learned_addr learned my addr 172.21.15.78:0/3617279458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb2faa00640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 0x7fb2fc13a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.683+0000 7fb2fbe00640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc15db00 0x7fb2fc138e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 -- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 msgr2=0x7fb2fc13a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 0x7fb2fc13a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 -- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc15db00 msgr2=0x7fb2fc138e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc15db00 0x7fb2fc138e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 -- 172.21.15.78:0/3617279458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2fc159c70 con 0x7fb2fc0731a0 2024-09-11T12:54:55.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2fb400640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009d90 tx=0x7fb2ec00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec003b80 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2ec003d20 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2fc141970 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec004280 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2fc141f30 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2fc0735e0 con 0x7fb2fc0731a0 2024-09-11T12:54:55.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7fb2ec004420 con 0x7fb2fc0731a0 2024-09-11T12:54:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 0x7fb2dc073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fb2ec092fd0 con 0x7fb2fc0731a0 2024-09-11T12:54:55.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.687+0000 7fb2faa00640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 0x7fb2dc073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:55.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.691+0000 7fb2faa00640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 0x7fb2dc073e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80af000 tx=0x7fb2e801b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:55.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.691+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb2ec05e960 con 0x7fb2fc0731a0 2024-09-11T12:54:55.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.883+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol32140", "target": ["mon-mgr", ""]}) -- 0x7fb2fc13da10 con 0x7fb2dc071950 2024-09-11T12:54:55.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.919+0000 7fb2f3400640 1 -- 172.21.15.78:0/3617279458 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb2fc13da10 con 0x7fb2dc071950 2024-09-11T12:54:55.924 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 msgr2=0x7fb2dc073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 0x7fb2dc073e10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80af000 tx=0x7fb2e801b120 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 msgr2=0x7fb2fc159730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec009d90 tx=0x7fb2ec00c6a0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 shutdown_connections 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2dc071950 0x7fb2dc073e10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2fc15db00 0x7fb2fc138e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2fc140190 0x7fb2fc13a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 --2- 172.21.15.78:0/3617279458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2fc0731a0 0x7fb2fc159730 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 >> 172.21.15.78:0/3617279458 conn(0x7fb2fc075c10 msgr2=0x7fb2fc15dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 shutdown_connections 2024-09-11T12:54:55.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:55.923+0000 7fb301000640 1 -- 172.21.15.78:0/3617279458 wait complete. 2024-09-11T12:54:55.927 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.923+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.927 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.923+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.927 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.923+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.927 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.923+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.927 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.923+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.943 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.939+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.943 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.939+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.943 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.939+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.943 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.939+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.943 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:54:55.939+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:54:55.948 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone31610 2024-09-11T12:54:56.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 -- 172.21.15.78:0/713953378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 msgr2=0x7f6adc072da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 --2- 172.21.15.78:0/713953378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc072da0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009840 tx=0x7f6ac801c790 comp rx=0 tx=0).stop 2024-09-11T12:54:56.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 -- 172.21.15.78:0/713953378 shutdown_connections 2024-09-11T12:54:56.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 --2- 172.21.15.78:0/713953378 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 0x7f6adc06f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 --2- 172.21.15.78:0/713953378 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc072da0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 -- 172.21.15.78:0/713953378 >> 172.21.15.78:0/713953378 conn(0x7f6adc073c40 msgr2=0x7f6adc074050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:56.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 -- 172.21.15.78:0/713953378 shutdown_connections 2024-09-11T12:54:56.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.115+0000 7f6ae1200640 1 -- 172.21.15.78:0/713953378 wait complete. 2024-09-11T12:54:56.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 Processor -- start 2024-09-11T12:54:56.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 -- start start 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 0x7f6adc1f8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc1fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc06fe50 con 0x7f6adc0cd810 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6adc06fcd0 con 0x7f6adc0cca40 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6adc06ffd0 con 0x7f6adc0732e0 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51318/0 (socket says 172.21.15.78:51318) 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 -- 172.21.15.78:0/988523298 learned_addr learned my addr 172.21.15.78:0/988523298 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ad9400640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc1fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ad9e00640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 0x7f6adc1f8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 -- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 msgr2=0x7f6adc1f8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 0x7f6adc1f8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 -- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 msgr2=0x7f6adc1fac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc1fac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 -- 172.21.15.78:0/988523298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6adc1f8560 con 0x7f6adc0cd810 2024-09-11T12:54:56.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ada800640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400eb20 tx=0x7f6ad400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4004b30 con 0x7f6adc0cd810 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ad4004cf0 con 0x7f6adc0cd810 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ad4005280 con 0x7f6adc0cd810 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.119+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6adc1fb570 con 0x7f6adc0cd810 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6adc1f7820 con 0x7f6adc0cd810 2024-09-11T12:54:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6adc1f7610 con 0x7f6adc0cd810 2024-09-11T12:54:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f6ad4005420 con 0x7f6adc0cd810 2024-09-11T12:54:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ad2000640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 0x7f6abc073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f6ad4092ea0 con 0x7f6adc0cd810 2024-09-11T12:54:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ad9e00640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 0x7f6abc073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.123+0000 7f6ad9e00640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 0x7f6abc073e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001870 tx=0x7f6acc001380 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:56.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.127+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6adc1f8560 con 0x7f6adc0cd810 2024-09-11T12:54:56.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.323+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone31610", "target": ["mon-mgr", ""]}) -- 0x7f6adc158b20 con 0x7f6abc071950 2024-09-11T12:54:56.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ad2000640 1 -- 172.21.15.78:0/988523298 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6adc158b20 con 0x7f6abc071950 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 msgr2=0x7f6abc073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 0x7f6abc073e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f6acc001870 tx=0x7f6acc001380 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 msgr2=0x7f6adc15c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400eb20 tx=0x7f6ad400c2d0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 shutdown_connections 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6adc0cd810 0x7f6adc15c9e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6abc071950 0x7f6abc073e10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6adc0cca40 0x7f6adc1fac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 --2- 172.21.15.78:0/988523298 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6adc0732e0 0x7f6adc1f8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 >> 172.21.15.78:0/988523298 conn(0x7f6adc073c40 msgr2=0x7f6adc072460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 shutdown_connections 2024-09-11T12:54:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.339+0000 7f6ae1200640 1 -- 172.21.15.78:0/988523298 wait complete. 2024-09-11T12:54:56.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone76311 2024-09-11T12:54:56.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- 172.21.15.78:0/81961538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 msgr2=0x7f62940b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 --2- 172.21.15.78:0/81961538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f62940b2de0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f62940b3e90 tx=0x7f628401cae0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- 172.21.15.78:0/81961538 shutdown_connections 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 --2- 172.21.15.78:0/81961538 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 0x7f62941416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 --2- 172.21.15.78:0/81961538 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f62940b2de0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- 172.21.15.78:0/81961538 >> 172.21.15.78:0/81961538 conn(0x7f6294071dd0 msgr2=0x7f62940721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- 172.21.15.78:0/81961538 shutdown_connections 2024-09-11T12:54:56.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- 172.21.15.78:0/81961538 wait complete. 2024-09-11T12:54:56.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 Processor -- start 2024-09-11T12:54:56.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.519+0000 7f629a000640 1 -- start start 2024-09-11T12:54:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f629414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 0x7f629415a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6294168ef0 con 0x7f62940cc5d0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6294168d70 con 0x7f62940cc1f0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6294169070 con 0x7f629415ca30 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51320/0 (socket says 172.21.15.78:51320) 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 -- 172.21.15.78:0/1706086704 learned_addr learned my addr 172.21.15.78:0/1706086704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6293400640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f629414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6293e00640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 0x7f629415a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 -- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 msgr2=0x7f629415a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 0x7f629415a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 -- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 msgr2=0x7f629414f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f629414f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 -- 172.21.15.78:0/1706086704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629413b7e0 con 0x7f62940cc5d0 2024-09-11T12:54:56.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6293400640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f629414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f6292a00640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f628000c460 tx=0x7f628000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280005480 con 0x7f62940cc5d0 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6280003600 con 0x7f62940cc5d0 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62941399d0 con 0x7f62940cc5d0 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280003a50 con 0x7f62940cc5d0 2024-09-11T12:54:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6294139f90 con 0x7f62940cc5d0 2024-09-11T12:54:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62940b2de0 con 0x7f62940cc5d0 2024-09-11T12:54:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.523+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f6280003bf0 con 0x7f62940cc5d0 2024-09-11T12:54:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.527+0000 7f628b400640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 0x7f6274073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.527+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f6280091cf0 con 0x7f62940cc5d0 2024-09-11T12:54:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.527+0000 7f6293400640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 0x7f6274073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:56.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.527+0000 7f6293400640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 0x7f6274073e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62840af000 tx=0x7f628401b320 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:56.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.531+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f628005d140 con 0x7f62940cc5d0 2024-09-11T12:54:56.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.723+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone76311", "target": ["mon-mgr", ""]}) -- 0x7f629414fe50 con 0x7f6274071950 2024-09-11T12:54:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.735+0000 7f628b400640 1 -- 172.21.15.78:0/1706086704 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f629414fe50 con 0x7f6274071950 2024-09-11T12:54:56.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 msgr2=0x7f6274073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 0x7f6274073e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f62840af000 tx=0x7f628401b320 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 msgr2=0x7f629413af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f628000c460 tx=0x7f628000c920 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 shutdown_connections 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f629415ca30 0x7f629415a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62940cc5d0 0x7f629413af00 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6274071950 0x7f6274073e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 --2- 172.21.15.78:0/1706086704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62940cc1f0 0x7f629414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 >> 172.21.15.78:0/1706086704 conn(0x7f6294071dd0 msgr2=0x7f62941407f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 shutdown_connections 2024-09-11T12:54:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56.739+0000 7f629a000640 1 -- 172.21.15.78:0/1706086704 wait complete. 2024-09-11T12:54:56.764 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:54:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:54:56.827 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:54:56.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:54:56.834 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:54:56.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:56.935 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:54:57.014 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:54:57.014 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:57.014 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:57.014 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:57.023 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:57.023 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:57.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:54:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 -- 172.21.15.78:0/4277032585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff670004810 msgr2=0x7ff670004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 --2- 172.21.15.78:0/4277032585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff670004810 0x7ff670004c90 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff674009b30 tx=0x7ff67401d240 comp rx=0 tx=0).stop 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff691000640 1 -- 172.21.15.78:0/4277032585 reap_dead start 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 -- 172.21.15.78:0/4277032585 shutdown_connections 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 -- 172.21.15.78:0/4277032585 >> 172.21.15.78:0/4277032585 conn(0x7ff68c06ca80 msgr2=0x7ff68c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 -- 172.21.15.78:0/4277032585 shutdown_connections 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.255+0000 7ff693000640 1 -- 172.21.15.78:0/4277032585 wait complete. 2024-09-11T12:54:57.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 Processor -- start 2024-09-11T12:54:57.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- start start 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 0x7ff68c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff68c13bd40 0x7ff68c0ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c070d00 con 0x7ff68c13b8c0 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c070b80 con 0x7ff670004810 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff68c13e740 con 0x7ff68c13bd40 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51332/0 (socket says 172.21.15.78:51332) 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 -- 172.21.15.78:0/2873109111 learned_addr learned my addr 172.21.15.78:0/2873109111 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff68be00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 0x7ff68c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff691000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff68c13bd40 0x7ff68c0ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 -- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff68c13bd40 msgr2=0x7ff68c0ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff68c13bd40 0x7ff68c0ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 -- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 msgr2=0x7ff68c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 0x7ff68c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 -- 172.21.15.78:0/2873109111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c15b300 con 0x7ff68c13b8c0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff68be00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 0x7ff68c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff683e00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff674009b30 tx=0x7ff6740acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6740bd020 con 0x7ff68c13b8c0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff674005020 con 0x7ff68c13b8c0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6740b1040 con 0x7ff68c13b8c0 2024-09-11T12:54:57.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c15b5d0 con 0x7ff68c13b8c0 2024-09-11T12:54:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c15bb90 con 0x7ff68c13b8c0 2024-09-11T12:54:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.259+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68c13ecd0 con 0x7ff68c13b8c0 2024-09-11T12:54:57.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.263+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7ff6740c0070 con 0x7ff68c13b8c0 2024-09-11T12:54:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.263+0000 7ff688c00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 0x7ff668073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.263+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7ff67412cd80 con 0x7ff68c13b8c0 2024-09-11T12:54:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.263+0000 7ff68be00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 0x7ff668073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.263+0000 7ff68be00640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 0x7ff668073e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00a680 tx=0x7ff67c00c040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:57.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.267+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff674016f80 con 0x7ff68c13b8c0 2024-09-11T12:54:57.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.443+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff68c159230 con 0x7ff68c13b8c0 2024-09-11T12:54:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff688c00640 1 -- 172.21.15.78:0/2873109111 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) ==== 80+0+0 (secure 0 0 0) 0x7ff6740f81d0 con 0x7ff68c13b8c0 2024-09-11T12:54:57.448 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:54:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 msgr2=0x7ff668073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 0x7ff668073e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff67c00a680 tx=0x7ff67c00c040 comp rx=0 tx=0).stop 2024-09-11T12:54:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 msgr2=0x7ff68c0ce7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff674009b30 tx=0x7ff6740acd20 comp rx=0 tx=0).stop 2024-09-11T12:54:57.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 shutdown_connections 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff68c13bd40 0x7ff68c0ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff68c13b8c0 0x7ff68c0ce7c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff668071950 0x7ff668073e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 --2- 172.21.15.78:0/2873109111 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff670004810 0x7ff68c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 >> 172.21.15.78:0/2873109111 conn(0x7ff68c06ca80 msgr2=0x7ff68c153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 shutdown_connections 2024-09-11T12:54:57.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.447+0000 7ff693000640 1 -- 172.21.15.78:0/2873109111 wait complete. 2024-09-11T12:54:57.469 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:54:57.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:57.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:54:57.515 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:54:57.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:57.516 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:54:57.569 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:57.569 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:57.569 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:57.578 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:57.578 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:57.633 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 -- 172.21.15.78:0/664745857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080736d0 msgr2=0x7f4e0815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/664745857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080736d0 0x7f4e0815af90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f4e08074d50 tx=0x7f4df401c770 comp rx=0 tx=0).stop 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 -- 172.21.15.78:0/664745857 shutdown_connections 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/664745857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080736d0 0x7f4e0815af90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/664745857 >> v2:172.21.15.157:3300/0 conn(0x7f4e08157ae0 0x7f4e08070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.771+0000 7f4e0fa00640 1 -- 172.21.15.78:0/664745857 >> 172.21.15.78:0/664745857 conn(0x7f4e0806ca80 msgr2=0x7f4e080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:57.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- 172.21.15.78:0/664745857 shutdown_connections 2024-09-11T12:54:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- 172.21.15.78:0/664745857 wait complete. 2024-09-11T12:54:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 Processor -- start 2024-09-11T12:54:57.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- start start 2024-09-11T12:54:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e080736d0 0x7f4e081646f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e08157ae0 0x7f4e08071df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e08166f10 con 0x7f4e080b4db0 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e080b5710 con 0x7f4e080736d0 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e08167090 con 0x7f4e08157ae0 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51346/0 (socket says 172.21.15.78:51346) 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 -- 172.21.15.78:0/1933261985 learned_addr learned my addr 172.21.15.78:0/1933261985 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0d000640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e080736d0 0x7f4e081646f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0da00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e08157ae0 0x7f4e08071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 -- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e08157ae0 msgr2=0x7f4e08071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e08157ae0 0x7f4e08071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 -- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e080736d0 msgr2=0x7f4e081646f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e080736d0 0x7f4e081646f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 -- 172.21.15.78:0/1933261985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e081f8b30 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e07e00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c10 tx=0x7f4df40a6b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df40a6d00 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df40a6ea0 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e08151110 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df4004290 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.775+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e081516d0 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0815af90 con 0x7f4e080b4db0 2024-09-11T12:54:57.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7f4df40b8030 con 0x7f4e080b4db0 2024-09-11T12:54:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e04c00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 0x7f4de8073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f4df412b7c0 con 0x7f4e080b4db0 2024-09-11T12:54:57.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e0d000640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 0x7f4de8073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:57.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.779+0000 7f4e0d000640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 0x7f4de8073e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00bc50 tx=0x7f4dfc00b3e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:57.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.783+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4df40bf070 con 0x7f4e080b4db0 2024-09-11T12:54:57.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.959+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4e08159db0 con 0x7f4e080b4db0 2024-09-11T12:54:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.959+0000 7f4e04c00640 1 -- 172.21.15.78:0/1933261985 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v103) ==== 80+0+0 (secure 0 0 0) 0x7f4df40ac030 con 0x7f4e080b4db0 2024-09-11T12:54:57.963 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:54:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 msgr2=0x7f4de8073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 0x7f4de8073e10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc00bc50 tx=0x7f4dfc00b3e0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 msgr2=0x7f4e08149a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:57.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f4df4000c10 tx=0x7f4df40a6b10 comp rx=0 tx=0).stop 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 shutdown_connections 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e08157ae0 0x7f4e08071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e080b4db0 0x7f4e08149a00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4de8071950 0x7f4de8073e10 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 --2- 172.21.15.78:0/1933261985 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e080736d0 0x7f4e081646f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 >> 172.21.15.78:0/1933261985 conn(0x7f4e0806ca80 msgr2=0x7f4e080cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 shutdown_connections 2024-09-11T12:54:57.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:57.963+0000 7f4e0fa00640 1 -- 172.21.15.78:0/1933261985 wait complete. 2024-09-11T12:54:57.983 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:54:57.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:57.984 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:54:58.012 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:54:58.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:54:58.012 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:54:58.068 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:58.068 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:54:58.068 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:58.077 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:54:58.077 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:54:58.077 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:54:58.078 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:54:58.120 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:54:58.120 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- 172.21.15.78:0/2446768203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015dee0 msgr2=0x7fda600b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 --2- 172.21.15.78:0/2446768203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015dee0 0x7fda600b54b0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fda600d1760 tx=0x7fda4c01c800 comp rx=0 tx=0).stop 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- 172.21.15.78:0/2446768203 shutdown_connections 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 --2- 172.21.15.78:0/2446768203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015dee0 0x7fda600b54b0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 --2- 172.21.15.78:0/2446768203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 0x7fda6006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 --2- 172.21.15.78:0/2446768203 >> v2:172.21.15.157:3300/0 conn(0x7fda6015db00 0x7fda60073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- 172.21.15.78:0/2446768203 >> 172.21.15.78:0/2446768203 conn(0x7fda6006c650 msgr2=0x7fda600d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- 172.21.15.78:0/2446768203 shutdown_connections 2024-09-11T12:54:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- 172.21.15.78:0/2446768203 wait complete. 2024-09-11T12:54:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 Processor -- start 2024-09-11T12:54:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.259+0000 7fda64e00640 1 -- start start 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 0x7fda6015c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda6015dee0 0x7fda60140bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fda60161fb0 con 0x7fda6015db00 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fda60161e30 con 0x7fda6015dee0 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fda60162130 con 0x7fda60074060 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51364/0 (socket says 172.21.15.78:51364) 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 -- 172.21.15.78:0/3979645450 learned_addr learned my addr 172.21.15.78:0/3979645450 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5de00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 0x7fda6015c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5e800640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda6015dee0 0x7fda60140bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 -- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 msgr2=0x7fda6015c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 0x7fda6015c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 -- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda6015dee0 msgr2=0x7fda60140bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda6015dee0 0x7fda60140bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 -- 172.21.15.78:0/3979645450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda60162b20 con 0x7fda6015db00 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda5d400640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0b0040 tx=0x7fda4c0a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0b2040 con 0x7fda6015db00 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda4c0043c0 con 0x7fda6015db00 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda4c0052d0 con 0x7fda6015db00 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda60162e00 con 0x7fda6015db00 2024-09-11T12:54:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6015c450 con 0x7fda6015db00 2024-09-11T12:54:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.263+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda601625e0 con 0x7fda6015db00 2024-09-11T12:54:58.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.267+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 17) ==== 95187+0+0 (secure 0 0 0) 0x7fda4c005470 con 0x7fda6015db00 2024-09-11T12:54:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.267+0000 7fda56000640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 0x7fda40073e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.267+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7fda4c12bb50 con 0x7fda6015db00 2024-09-11T12:54:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.267+0000 7fda5de00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 0x7fda40073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.267+0000 7fda5de00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 0x7fda40073e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda6015c0a0 tx=0x7fda5000b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:58.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.271+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fda4c0164b0 con 0x7fda6015db00 2024-09-11T12:54:58.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.475+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda600b9ac0 con 0x7fda6015db00 2024-09-11T12:54:58.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.475+0000 7fda56000640 1 -- 172.21.15.78:0/3979645450 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5228 (secure 0 0 0) 0x7fda4c0f7070 con 0x7fda6015db00 2024-09-11T12:54:58.478 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:58.478 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T12:54:32:323945+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T12:54:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 msgr2=0x7fda40073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 0x7fda40073e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fda6015c0a0 tx=0x7fda5000b390 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 msgr2=0x7fda601401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fda4c0b0040 tx=0x7fda4c0a5800 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 shutdown_connections 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fda40071950 0x7fda40073e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fda6015dee0 0x7fda60140bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fda6015db00 0x7fda601401a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 --2- 172.21.15.78:0/3979645450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fda60074060 0x7fda6015c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 >> 172.21.15.78:0/3979645450 conn(0x7fda6006c650 msgr2=0x7fda600d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 shutdown_connections 2024-09-11T12:54:58.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.479+0000 7fda64e00640 1 -- 172.21.15.78:0/3979645450 wait complete. 2024-09-11T12:54:58.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:58.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- 172.21.15.78:0/2718408064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4b60 msgr2=0x7f38dc15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.645 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 --2- 172.21.15.78:0/2718408064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4b60 0x7f38dc15ba90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009900 tx=0x7f38d401c8a0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- 172.21.15.78:0/2718408064 shutdown_connections 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 --2- 172.21.15.78:0/2718408064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4b60 0x7f38dc15ba90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 --2- 172.21.15.78:0/2718408064 >> v2:172.21.15.157:3300/0 conn(0x7f38dc0ccb20 0x7f38dc15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 --2- 172.21.15.78:0/2718408064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0b4780 0x7f38dc073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- 172.21.15.78:0/2718408064 >> 172.21.15.78:0/2718408064 conn(0x7f38dc06ca80 msgr2=0x7f38dc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- 172.21.15.78:0/2718408064 shutdown_connections 2024-09-11T12:54:58.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- 172.21.15.78:0/2718408064 wait complete. 2024-09-11T12:54:58.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 Processor -- start 2024-09-11T12:54:58.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 -- start start 2024-09-11T12:54:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.643+0000 7f38e1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38dc0b4b60 0x7f38dc1f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0ccb20 0x7f38dc164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc06f190 con 0x7f38dc0b4780 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38dc06f010 con 0x7f38dc0b4b60 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f38dc06f310 con 0x7f38dc0ccb20 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51388/0 (socket says 172.21.15.78:51388) 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 -- 172.21.15.78:0/3698432144 learned_addr learned my addr 172.21.15.78:0/3698432144 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9400640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38dc0b4b60 0x7f38dc1f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38da800640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0ccb20 0x7f38dc164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 -- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0ccb20 msgr2=0x7f38dc164e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0ccb20 0x7f38dc164e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 -- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38dc0b4b60 msgr2=0x7f38dc1f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38dc0b4b60 0x7f38dc1f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 -- 172.21.15.78:0/3698432144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38dc1f8250 con 0x7f38dc0b4780 2024-09-11T12:54:58.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d9e00640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00ea20 tx=0x7f38cc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc00cde0 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38cc004b30 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38cc005020 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38dc15d210 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38dc15d7d0 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.647+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38dc073f70 con 0x7f38dc0b4780 2024-09-11T12:54:58.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f38cc0051c0 con 0x7f38dc0b4780 2024-09-11T12:54:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d2000640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 0x7f38bc073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f38cc093240 con 0x7f38dc0b4780 2024-09-11T12:54:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d9400640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 0x7f38bc073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:58.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d9400640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 0x7f38bc073f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38dc155d60 tx=0x7f38c800a2d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:58.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.651+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f38dc1f8250 con 0x7f38dc0b4780 2024-09-11T12:54:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.851+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38dc1504a0 con 0x7f38dc0b4780 2024-09-11T12:54:58.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.851+0000 7f38d2000640 1 -- 172.21.15.78:0/3698432144 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5228 (secure 0 0 0) 0x7f38cc016030 con 0x7f38dc0b4780 2024-09-11T12:54:58.856 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:58.856 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T12:54:32:323945+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:58.856 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 msgr2=0x7f38bc073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 0x7f38bc073f30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f38dc155d60 tx=0x7f38c800a2d0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 msgr2=0x7f38dc15cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00ea20 tx=0x7f38cc00eee0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 shutdown_connections 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38dc0ccb20 0x7f38dc164e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38bc071a70 0x7f38bc073f30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38dc0b4b60 0x7f38dc1f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 --2- 172.21.15.78:0/3698432144 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38dc0b4780 0x7f38dc15cc10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 >> 172.21.15.78:0/3698432144 conn(0x7f38dc06ca80 msgr2=0x7f38dc166000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 shutdown_connections 2024-09-11T12:54:58.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:58.855+0000 7f38e1200640 1 -- 172.21.15.78:0/3698432144 wait complete. 2024-09-11T12:54:58.877 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:54:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 -- 172.21.15.78:0/2016129877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea80cf760 msgr2=0x7efea8074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 --2- 172.21.15.78:0/2016129877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea80cf760 0x7efea8074260 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7efea80cc800 tx=0x7efe9801c890 comp rx=0 tx=0).stop 2024-09-11T12:54:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 -- 172.21.15.78:0/2016129877 shutdown_connections 2024-09-11T12:54:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 --2- 172.21.15.78:0/2016129877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea806fc50 0x7efea80700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 --2- 172.21.15.78:0/2016129877 >> v2:172.21.15.157:3300/0 conn(0x7efea815aec0 0x7efea8070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 --2- 172.21.15.78:0/2016129877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea80cf760 0x7efea8074260 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 -- 172.21.15.78:0/2016129877 >> 172.21.15.78:0/2016129877 conn(0x7efea806ca80 msgr2=0x7efea80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 -- 172.21.15.78:0/2016129877 shutdown_connections 2024-09-11T12:54:59.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.015+0000 7efeadc00640 1 -- 172.21.15.78:0/2016129877 wait complete. 2024-09-11T12:54:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 Processor -- start 2024-09-11T12:54:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- start start 2024-09-11T12:54:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea80cf760 0x7efea8142ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea815aec0 0x7efea8143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efea815f290 con 0x7efea806fc50 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efea815f110 con 0x7efea815aec0 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efea8151010 con 0x7efea80cf760 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51412/0 (socket says 172.21.15.78:51412) 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 -- 172.21.15.78:0/3691831910 learned_addr learned my addr 172.21.15.78:0/3691831910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6000640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea80cf760 0x7efea8142ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea7400640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea815aec0 0x7efea8143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 -- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea80cf760 msgr2=0x7efea8142ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea80cf760 0x7efea8142ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 -- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea815aec0 msgr2=0x7efea8143150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea815aec0 0x7efea8143150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 -- 172.21.15.78:0/3691831910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efea81fa560 con 0x7efea806fc50 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efea6a00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7efe980b0040 tx=0x7efe98004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980b2040 con 0x7efea806fc50 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe98005030 con 0x7efea806fc50 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efea8143ca0 con 0x7efea806fc50 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe980044f0 con 0x7efea806fc50 2024-09-11T12:54:59.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.019+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efea81faae0 con 0x7efea806fc50 2024-09-11T12:54:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efea8159c00 con 0x7efea806fc50 2024-09-11T12:54:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7efe98005480 con 0x7efea806fc50 2024-09-11T12:54:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efe9ea00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 0x7efe88073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7efe9812bf90 con 0x7efea806fc50 2024-09-11T12:54:59.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efea6000640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 0x7efe88073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.023+0000 7efea6000640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 0x7efe88073f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efea81341f0 tx=0x7efe9400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.027+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efe980164b0 con 0x7efea806fc50 2024-09-11T12:54:59.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.195+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efea8074240 con 0x7efea806fc50 2024-09-11T12:54:59.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.195+0000 7efe9ea00640 1 -- 172.21.15.78:0/3691831910 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16775 (secure 0 0 0) 0x7efe980f7220 con 0x7efea806fc50 2024-09-11T12:54:59.199 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:59.199 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":103,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:36.348250+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:54:20.362554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-11T12:54:21.629928+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.26","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 msgr2=0x7efe88073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 0x7efe88073f60 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7efea81341f0 tx=0x7efe9400b2e0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 msgr2=0x7efea81fb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7efe980b0040 tx=0x7efe98004040 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 shutdown_connections 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efe88071aa0 0x7efe88073f60 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efea815aec0 0x7efea8143150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efea80cf760 0x7efea8142ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 --2- 172.21.15.78:0/3691831910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efea806fc50 0x7efea81fb000 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 >> 172.21.15.78:0/3691831910 conn(0x7efea806ca80 msgr2=0x7efea815c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 shutdown_connections 2024-09-11T12:54:59.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.199+0000 7efeadc00640 1 -- 172.21.15.78:0/3691831910 wait complete. 2024-09-11T12:54:59.221 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:54:59.221 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 -- 172.21.15.78:0/2896395878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815bee0 msgr2=0x7f697815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 --2- 172.21.15.78:0/2896395878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815bee0 0x7f697815c390 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6970009840 tx=0x7f697001c820 comp rx=0 tx=0).stop 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 -- 172.21.15.78:0/2896395878 shutdown_connections 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 --2- 172.21.15.78:0/2896395878 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815bee0 0x7f697815c390 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 --2- 172.21.15.78:0/2896395878 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f697815eda0 0x7f697815cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 -- 172.21.15.78:0/2896395878 >> 172.21.15.78:0/2896395878 conn(0x7f697806ca80 msgr2=0x7f69780b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 -- 172.21.15.78:0/2896395878 shutdown_connections 2024-09-11T12:54:59.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.359+0000 7f697e200640 1 -- 172.21.15.78:0/2896395878 wait complete. 2024-09-11T12:54:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 Processor -- start 2024-09-11T12:54:59.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- start start 2024-09-11T12:54:59.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69780b4520 0x7f6978152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 0x7f697814b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f69780cd360 con 0x7f697815bee0 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f69780cd1e0 con 0x7f697815eda0 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f69780cd4e0 con 0x7f69780b4520 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51426/0 (socket says 172.21.15.78:51426) 2024-09-11T12:54:59.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 -- 172.21.15.78:0/620402901 learned_addr learned my addr 172.21.15.78:0/620402901 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6977e00640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 0x7f697814b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6977400640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69780b4520 0x7f6978152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 -- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69780b4520 msgr2=0x7f6978152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69780b4520 0x7f6978152fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 -- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 msgr2=0x7f697814b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 0x7f697814b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 -- 172.21.15.78:0/620402901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f697814dae0 con 0x7f697815bee0 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6977e00640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 0x7f697814b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f6976a00640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f696400c500 tx=0x7f696400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964017070 con 0x7f697815bee0 2024-09-11T12:54:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6978157bb0 con 0x7f697815bee0 2024-09-11T12:54:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69640053c0 con 0x7f697815bee0 2024-09-11T12:54:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6964012890 con 0x7f697815bee0 2024-09-11T12:54:59.370 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.363+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6978158170 con 0x7f697815bee0 2024-09-11T12:54:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697815c390 con 0x7f697815bee0 2024-09-11T12:54:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f696401a070 con 0x7f697815bee0 2024-09-11T12:54:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f696f400640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 0x7f6958073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5767+0+0 (secure 0 0 0) 0x7f6964091800 con 0x7f697815bee0 2024-09-11T12:54:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f6977400640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 0x7f6958073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.367+0000 7f6977400640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 0x7f6958073f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69781f8830 tx=0x7f6968010040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.371+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f696405ca10 con 0x7f697815bee0 2024-09-11T12:54:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.571+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f697815e030 con 0x7f697815bee0 2024-09-11T12:54:59.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.571+0000 7f696f400640 1 -- 172.21.15.78:0/620402901 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5228 (secure 0 0 0) 0x7f69640652a0 con 0x7f697815bee0 2024-09-11T12:54:59.575 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:59.575 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T12:54:32:323945+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:59.575 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 msgr2=0x7f6958073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 0x7f6958073f60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f69781f8830 tx=0x7f6968010040 comp rx=0 tx=0).stop 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 msgr2=0x7f697814d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f696400c500 tx=0x7f696400c9c0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 shutdown_connections 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6958071aa0 0x7f6958073f60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f697815eda0 0x7f697814b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f697815bee0 0x7f697814d370 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 --2- 172.21.15.78:0/620402901 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f69780b4520 0x7f6978152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 >> 172.21.15.78:0/620402901 conn(0x7f697806ca80 msgr2=0x7f697815b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 shutdown_connections 2024-09-11T12:54:59.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.575+0000 7f697e200640 1 -- 172.21.15.78:0/620402901 wait complete. 2024-09-11T12:54:59.602 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- 172.21.15.78:0/3569761945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f40d0bf0 msgr2=0x7fc1f40d10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/3569761945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f40d0bf0 0x7fc1f40d10a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc1ec009c80 tx=0x7fc1ec01cda0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- 172.21.15.78:0/3569761945 shutdown_connections 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/3569761945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f40d0bf0 0x7fc1f40d10a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/3569761945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40b14b0 0x7fc1f415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- 172.21.15.78:0/3569761945 >> 172.21.15.78:0/3569761945 conn(0x7fc1f4074b90 msgr2=0x7fc1f40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- 172.21.15.78:0/3569761945 shutdown_connections 2024-09-11T12:54:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- 172.21.15.78:0/3569761945 wait complete. 2024-09-11T12:54:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 Processor -- start 2024-09-11T12:54:59.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.751+0000 7fc1f9a00640 1 -- start start 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 0x7fc1f406c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40d0bf0 0x7fc1f4146ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4168f50 con 0x7fc1f406fdf0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f4168dd0 con 0x7fc1f40b14b0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc1f41690d0 con 0x7fc1f40d0bf0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51452/0 (socket says 172.21.15.78:51452) 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 -- 172.21.15.78:0/166776909 learned_addr learned my addr 172.21.15.78:0/166776909 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f3400640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40d0bf0 0x7fc1f4146ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2000640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 0x7fc1f406c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 -- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40d0bf0 msgr2=0x7fc1f4146ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40d0bf0 0x7fc1f4146ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 -- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 msgr2=0x7fc1f406c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 0x7fc1f406c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 -- 172.21.15.78:0/166776909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1f406d030 con 0x7fc1f406fdf0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2000640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 0x7fc1f406c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f2a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c9a0 tx=0x7fc1e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4007c10 con 0x7fc1f406fdf0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1e400f450 con 0x7fc1f406fdf0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1e400f8a0 con 0x7fc1f406fdf0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1f406d310 con 0x7fc1f406fdf0 2024-09-11T12:54:59.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1f406d5f0 con 0x7fc1f406fdf0 2024-09-11T12:54:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.755+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1f415c100 con 0x7fc1f406fdf0 2024-09-11T12:54:59.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.759+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fc1e400fa40 con 0x7fc1f406fdf0 2024-09-11T12:54:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.759+0000 7fc1eaa00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 0x7fc1d4073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:54:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.759+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5823+0+0 (secure 0 0 0) 0x7fc1e4092e50 con 0x7fc1f406fdf0 2024-09-11T12:54:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.759+0000 7fc1f2000640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 0x7fc1d4073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:54:59.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.759+0000 7fc1f2000640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 0x7fc1d4073f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1f406d980 tx=0x7fc1e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:54:59.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.763+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc1e4096030 con 0x7fc1f406fdf0 2024-09-11T12:54:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.959+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1f415b3c0 con 0x7fc1f406fdf0 2024-09-11T12:54:59.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1eaa00640 1 -- 172.21.15.78:0/166776909 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) ==== 94+0+5228 (secure 0 0 0) 0x7fc1e4014030 con 0x7fc1f406fdf0 2024-09-11T12:54:59.964 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:54:59.965 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":72,"btime":"2024-09-11T12:54:32:323945+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:54:22.659605+0000","modified":"2024-09-11T12:54:25.688540+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6894},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6894":{"gid":6894,"name":"c","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/785402870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":785402870},{"type":"v1","addr":"172.21.15.78:6835","nonce":785402870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6894,"qdb_cluster":[6894]},"id":8}]} 2024-09-11T12:54:59.965 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 72 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 msgr2=0x7fc1d4073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 0x7fc1d4073f30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc1f406d980 tx=0x7fc1e0009040 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 msgr2=0x7fc1f414d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400c9a0 tx=0x7fc1e400ce60 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 shutdown_connections 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc1f40d0bf0 0x7fc1f4146ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc1d4071a70 0x7fc1d4073f30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc1f40b14b0 0x7fc1f406c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 --2- 172.21.15.78:0/166776909 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc1f406fdf0 0x7fc1f414d810 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 >> 172.21.15.78:0/166776909 conn(0x7fc1f4074b90 msgr2=0x7fc1f415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:54:59.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 shutdown_connections 2024-09-11T12:54:59.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:54:59.963+0000 7fc1f9a00640 1 -- 172.21.15.78:0/166776909 wait complete. 2024-09-11T12:54:59.986 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 -- 172.21.15.78:0/3363398646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf815ba80 msgr2=0x7fbcf806fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 --2- 172.21.15.78:0/3363398646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf815ba80 0x7fbcf806fc50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fbcf815b1b0 tx=0x7fbce401c7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 -- 172.21.15.78:0/3363398646 shutdown_connections 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 --2- 172.21.15.78:0/3363398646 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf815ba80 0x7fbcf806fc50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 -- 172.21.15.78:0/3363398646 >> 172.21.15.78:0/3363398646 conn(0x7fbcf806ca80 msgr2=0x7fbcf80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:00.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 -- 172.21.15.78:0/3363398646 shutdown_connections 2024-09-11T12:55:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.123+0000 7fbcfe600640 1 -- 172.21.15.78:0/3363398646 wait complete. 2024-09-11T12:55:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 Processor -- start 2024-09-11T12:55:00.128 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- start start 2024-09-11T12:55:00.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf8074240 0x7fbcf814f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf815ba80 0x7fbcf81f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf815ee40 con 0x7fbcf8070190 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf815ecc0 con 0x7fbcf8074240 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf8168f60 con 0x7fbcf815ba80 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51474/0 (socket says 172.21.15.78:51474) 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 -- 172.21.15.78:0/5785309 learned_addr learned my addr 172.21.15.78:0/5785309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf6a00640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf8074240 0x7fbcf814f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7e00640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf815ba80 0x7fbcf81f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 -- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf815ba80 msgr2=0x7fbcf81f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf815ba80 0x7fbcf81f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 -- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf8074240 msgr2=0x7fbcf814f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf8074240 0x7fbcf814f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 -- 172.21.15.78:0/5785309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf80b18b0 con 0x7fbcf8070190 2024-09-11T12:55:00.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcf7400640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c7c0 tx=0x7fbce800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8016020 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce800ab40 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf8154390 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8012420 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.127+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf8154950 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf81fac30 con 0x7fbcf8070190 2024-09-11T12:55:00.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fbce8019020 con 0x7fbcf8070190 2024-09-11T12:55:00.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcef400640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 0x7fbcd8073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5823+0+0 (secure 0 0 0) 0x7fbce8092540 con 0x7fbcf8070190 2024-09-11T12:55:00.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcf6a00640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 0x7fbcd8073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.131+0000 7fbcf6a00640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 0x7fbcd8073f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbce40af000 tx=0x7fbce401b030 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.135+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbce8020030 con 0x7fbcf8070190 2024-09-11T12:55:00.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.303+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbcf806fc50 con 0x7fbcf8070190 2024-09-11T12:55:00.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.303+0000 7fbcef400640 1 -- 172.21.15.78:0/5785309 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16886 (secure 0 0 0) 0x7fbce805d6d0 con 0x7fbcf8070190 2024-09-11T12:55:00.306 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:00.307 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":104,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:54:59.572433+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:54:20.362554+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-11T12:54:21.629928+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":104,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.26","mappings":[{"from":0,"to":2}]},{"pgid":"17.37","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:00.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 msgr2=0x7fbcd8073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 0x7fbcd8073f60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbce40af000 tx=0x7fbce401b030 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 msgr2=0x7fbcf80b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fbce800c7c0 tx=0x7fbce800cc80 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 shutdown_connections 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbcf815ba80 0x7fbcf81f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbcd8071aa0 0x7fbcd8073f60 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbcf8074240 0x7fbcf814f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 --2- 172.21.15.78:0/5785309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbcf8070190 0x7fbcf80b1370 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 >> 172.21.15.78:0/5785309 conn(0x7fbcf806ca80 msgr2=0x7fbcf80cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 shutdown_connections 2024-09-11T12:55:00.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.307+0000 7fbcfe600640 1 -- 172.21.15.78:0/5785309 wait complete. 2024-09-11T12:55:00.328 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 -- 172.21.15.78:0/3285698739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c074260 msgr2=0x7fc97c15aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 --2- 172.21.15.78:0/3285698739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c074260 0x7fc97c15aec0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fc97c072f50 tx=0x7fc96801cd80 comp rx=0 tx=0).stop 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 -- 172.21.15.78:0/3285698739 shutdown_connections 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 --2- 172.21.15.78:0/3285698739 >> v2:172.21.15.157:3300/0 conn(0x7fc97c06fc50 0x7fc97c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 --2- 172.21.15.78:0/3285698739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c074260 0x7fc97c15aec0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 -- 172.21.15.78:0/3285698739 >> 172.21.15.78:0/3285698739 conn(0x7fc97c06ca80 msgr2=0x7fc97c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 -- 172.21.15.78:0/3285698739 shutdown_connections 2024-09-11T12:55:00.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.467+0000 7fc981a00640 1 -- 172.21.15.78:0/3285698739 wait complete. 2024-09-11T12:55:00.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 Processor -- start 2024-09-11T12:55:00.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- start start 2024-09-11T12:55:00.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc97c074260 0x7fc97c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 0x7fc97c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c151010 con 0x7fc97c06fc50 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc97c15f360 con 0x7fc97c074260 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc97c151190 con 0x7fc97c0cf760 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51506/0 (socket says 172.21.15.78:51506) 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 -- 172.21.15.78:0/538257162 learned_addr learned my addr 172.21.15.78:0/538257162 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97a000640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc97c074260 0x7fc97c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97b400640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 0x7fc97c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 -- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 msgr2=0x7fc97c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 0x7fc97c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 -- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc97c074260 msgr2=0x7fc97c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc97c074260 0x7fc97c154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 -- 172.21.15.78:0/538257162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc97c1f6e40 con 0x7fc97c06fc50 2024-09-11T12:55:00.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97b400640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 0x7fc97c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:00.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc97aa00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc96c001ec0 tx=0x7fc96c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c017070 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc96c003600 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc97c1fa1c0 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c003a50 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.471+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc97c1f7680 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc97c0700a0 con 0x7fc97c06fc50 2024-09-11T12:55:00.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fc96c01a070 con 0x7fc97c06fc50 2024-09-11T12:55:00.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc972a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 0x7fc95c073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5823+0+0 (secure 0 0 0) 0x7fc96c0917a0 con 0x7fc97c06fc50 2024-09-11T12:55:00.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc97a000640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 0x7fc95c073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.475+0000 7fc97a000640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 0x7fc95c073f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9680af000 tx=0x7fc96801b5c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.479+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc97c1f6e40 con 0x7fc97c06fc50 2024-09-11T12:55:00.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.683+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fc97c15b630 con 0x7fc97c06fc50 2024-09-11T12:55:00.685 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:55:00.683+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:55:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.687+0000 7fc972a00640 1 -- 172.21.15.78:0/538257162 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v73) ==== 161+0+0 (secure 0 0 0) 0x7fc96c0652b0 con 0x7fc97c06fc50 2024-09-11T12:55:00.689 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:55:00.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 msgr2=0x7fc95c073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 0x7fc95c073f30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9680af000 tx=0x7fc96801b5c0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 msgr2=0x7fc97c1506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fc96c001ec0 tx=0x7fc96c00c890 comp rx=0 tx=0).stop 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 shutdown_connections 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc97c0cf760 0x7fc97c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc95c071a70 0x7fc95c073f30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc97c074260 0x7fc97c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 --2- 172.21.15.78:0/538257162 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc97c06fc50 0x7fc97c1506f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 >> 172.21.15.78:0/538257162 conn(0x7fc97c06ca80 msgr2=0x7fc97c0cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 shutdown_connections 2024-09-11T12:55:00.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.691+0000 7fc981a00640 1 -- 172.21.15.78:0/538257162 wait complete. 2024-09-11T12:55:00.713 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:55:00.758 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:55:00.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- 172.21.15.78:0/2083335663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77640b35d0 msgr2=0x7f776415aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- 172.21.15.78:0/2083335663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77640b35d0 0x7f776415aa70 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f7764073dd0 tx=0x7f775c01cac0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- 172.21.15.78:0/2083335663 shutdown_connections 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- 172.21.15.78:0/2083335663 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f77640b35d0 0x7f776415aa70 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- 172.21.15.78:0/2083335663 >> v2:172.21.15.157:3300/0 conn(0x7f7764150e50 0x7f776415a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- 172.21.15.78:0/2083335663 >> 172.21.15.78:0/2083335663 conn(0x7f7764075a10 msgr2=0x7f77640734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- 172.21.15.78:0/2083335663 shutdown_connections 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- 172.21.15.78:0/2083335663 wait complete. 2024-09-11T12:55:00.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 Processor -- start 2024-09-11T12:55:00.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- start start 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77640b35d0 0x7f776414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7764150a70 0x7f7764153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7764166ab0 con 0x7f7764153fa0 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7764166930 con 0x7f7764150a70 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776c400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7764166c30 con 0x7f77640b35d0 2024-09-11T12:55:00.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51552/0 (socket says 172.21.15.78:51552) 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 -- 172.21.15.78:0/1175614877 learned_addr learned my addr 172.21.15.78:0/1175614877 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f7769000640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7764150a70 0x7f7764153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f7769a00640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77640b35d0 0x7f776414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 -- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77640b35d0 msgr2=0x7f776414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77640b35d0 0x7f776414f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 -- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7764150a70 msgr2=0x7f7764153a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7764150a70 0x7f7764153a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.863+0000 7f776a400640 1 -- 172.21.15.78:0/1175614877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f776414e2c0 con 0x7f7764153fa0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f776a400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f775c009750 tx=0x7f775c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0bb070 con 0x7f7764153fa0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f775c004e00 con 0x7f7764153fa0 2024-09-11T12:55:00.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f775c0b2040 con 0x7f7764153fa0 2024-09-11T12:55:00.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f776414e5a0 con 0x7f7764153fa0 2024-09-11T12:55:00.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f776414eb60 con 0x7f7764153fa0 2024-09-11T12:55:00.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f776414e050 con 0x7f7764153fa0 2024-09-11T12:55:00.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f775c0a5af0 con 0x7f7764153fa0 2024-09-11T12:55:00.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 0x7f7744073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:00.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5881+0+0 (secure 0 0 0) 0x7f775c12c2d0 con 0x7f7764153fa0 2024-09-11T12:55:00.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.867+0000 7f7769a00640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 0x7f7744073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:00.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.871+0000 7f7769a00640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 0x7f7744073f60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7758000f70 tx=0x7f7758009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:00.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:00.871+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f775c0166e0 con 0x7f7764153fa0 2024-09-11T12:55:01.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.075+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7764072930 con 0x7f7764153fa0 2024-09-11T12:55:01.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.687+0000 7f7761600640 1 -- 172.21.15.78:0/1175614877 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v74) ==== 104+0+0 (secure 0 0 0) 0x7f775c12c0f0 con 0x7f7764153fa0 2024-09-11T12:55:01.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 msgr2=0x7f7744073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 0x7f7744073f60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7758000f70 tx=0x7f7758009040 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 msgr2=0x7f7764151240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f775c009750 tx=0x7f775c003f60 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 shutdown_connections 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7764153fa0 0x7f7764151240 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7744071aa0 0x7f7744073f60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7764150a70 0x7f7764153a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 --2- 172.21.15.78:0/1175614877 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f77640b35d0 0x7f776414f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 >> 172.21.15.78:0/1175614877 conn(0x7f7764075a10 msgr2=0x7f776415ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:01.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 shutdown_connections 2024-09-11T12:55:01.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.691+0000 7f776c400640 1 -- 172.21.15.78:0/1175614877 wait complete. 2024-09-11T12:55:01.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:55:01.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- 172.21.15.78:0/1292361980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5790 msgr2=0x7fa8600723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 --2- 172.21.15.78:0/1292361980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5790 0x7fa8600723f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fa860073a30 tx=0x7fa85001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- 172.21.15.78:0/1292361980 shutdown_connections 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 --2- 172.21.15.78:0/1292361980 >> v2:172.21.15.157:3300/0 conn(0x7fa860151340 0x7fa8601517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 --2- 172.21.15.78:0/1292361980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa8600b5b70 0x7fa860072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 --2- 172.21.15.78:0/1292361980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5790 0x7fa8600723f0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- 172.21.15.78:0/1292361980 >> 172.21.15.78:0/1292361980 conn(0x7fa8600b3050 msgr2=0x7fa8600b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- 172.21.15.78:0/1292361980 shutdown_connections 2024-09-11T12:55:01.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- 172.21.15.78:0/1292361980 wait complete. 2024-09-11T12:55:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 Processor -- start 2024-09-11T12:55:01.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.899+0000 7fa865000640 1 -- start start 2024-09-11T12:55:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600b5790 0x7fa860144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa860151340 0x7fa86014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa860070980 con 0x7fa8600b5b70 2024-09-11T12:55:01.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa860070800 con 0x7fa8600b5790 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa86015afc0 con 0x7fa860151340 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51568/0 (socket says 172.21.15.78:51568) 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 -- 172.21.15.78:0/3199886337 learned_addr learned my addr 172.21.15.78:0/3199886337 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85de00640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600b5790 0x7fa860144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85e800640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa860151340 0x7fa86014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 -- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa860151340 msgr2=0x7fa86014a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa860151340 0x7fa86014a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 -- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600b5790 msgr2=0x7fa860144c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600b5790 0x7fa860144c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 -- 172.21.15.78:0/3199886337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa860145ca0 con 0x7fa8600b5b70 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa85d400640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00d780 tx=0x7fa84c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:01.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c017070 con 0x7fa8600b5b70 2024-09-11T12:55:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c00de90 con 0x7fa8600b5b70 2024-09-11T12:55:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c005020 con 0x7fa8600b5b70 2024-09-11T12:55:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8601476e0 con 0x7fa8600b5b70 2024-09-11T12:55:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa86014e280 con 0x7fa8600b5b70 2024-09-11T12:55:01.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8601517d0 con 0x7fa8600b5b70 2024-09-11T12:55:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.903+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fa84c0051c0 con 0x7fa8600b5b70 2024-09-11T12:55:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.907+0000 7fa856000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 0x7fa840073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.907+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5881+0+0 (secure 0 0 0) 0x7fa84c092630 con 0x7fa8600b5b70 2024-09-11T12:55:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.907+0000 7fa85de00640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 0x7fa840073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:01.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.907+0000 7fa85de00640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 0x7fa840073f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8500af000 tx=0x7fa85001b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:01.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:01.911+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa84c05d700 con 0x7fa8600b5b70 2024-09-11T12:55:02.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.075+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa860165720 con 0x7fa8600b5b70 2024-09-11T12:55:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa856000640 1 -- 172.21.15.78:0/3199886337 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v106) ==== 181+0+0 (secure 0 0 0) 0x7fa84c061c50 con 0x7fa8600b5b70 2024-09-11T12:55:02.696 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:55:02.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 msgr2=0x7fa840073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:02.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 0x7fa840073f30 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa8500af000 tx=0x7fa85001b100 comp rx=0 tx=0).stop 2024-09-11T12:55:02.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 msgr2=0x7fa8601f5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00d780 tx=0x7fa84c00dc40 comp rx=0 tx=0).stop 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 shutdown_connections 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa860151340 0x7fa86014a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa8600b5b70 0x7fa8601f5920 secure :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fa84c00d780 tx=0x7fa84c00dc40 comp rx=0 tx=0).stop 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa840071a70 0x7fa840073f30 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 --2- 172.21.15.78:0/3199886337 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa8600b5790 0x7fa860144c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.695+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 >> 172.21.15.78:0/3199886337 conn(0x7fa8600b3050 msgr2=0x7fa8600cf530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:02.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.699+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 shutdown_connections 2024-09-11T12:55:02.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.699+0000 7fa865000640 1 -- 172.21.15.78:0/3199886337 wait complete. 2024-09-11T12:55:02.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- 172.21.15.78:0/155223951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f00ac940 msgr2=0x7f99f00aa090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- 172.21.15.78:0/155223951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f00ac940 0x7f99f00aa090 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009900 tx=0x7f99e801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- 172.21.15.78:0/155223951 shutdown_connections 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- 172.21.15.78:0/155223951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f00ac940 0x7f99f00aa090 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- 172.21.15.78:0/155223951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f015bd70 0x7f99f00ac400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- 172.21.15.78:0/155223951 >> 172.21.15.78:0/155223951 conn(0x7f99f006e860 msgr2=0x7f99f00ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:02.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- 172.21.15.78:0/155223951 shutdown_connections 2024-09-11T12:55:02.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- 172.21.15.78:0/155223951 wait complete. 2024-09-11T12:55:02.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 Processor -- start 2024-09-11T12:55:02.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- start start 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f00ac940 0x7f99f0137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f013e690 0x7f99f01394c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99f0168670 con 0x7f99f015bd70 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99f01684f0 con 0x7f99f013e690 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99f4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f99f01687f0 con 0x7f99f00ac940 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99efe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99efe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51586/0 (socket says 172.21.15.78:51586) 2024-09-11T12:55:02.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.875+0000 7f99efe00640 1 -- 172.21.15.78:0/2043734062 learned_addr learned my addr 172.21.15.78:0/2043734062 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:02.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99ef400640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f00ac940 0x7f99f0137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:02.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99eea00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f013e690 0x7f99f01394c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:02.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 -- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f00ac940 msgr2=0x7f99f0137fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:02.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f00ac940 0x7f99f0137fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 -- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f013e690 msgr2=0x7f99f01394c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f013e690 0x7f99f01394c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 -- 172.21.15.78:0/2043734062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f0138700 con 0x7f99f015bd70 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99efe00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f99e80b0040 tx=0x7f99e80a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80bb070 con 0x7f99f015bd70 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99e8005070 con 0x7f99f015bd70 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b2040 con 0x7f99f015bd70 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f01389e0 con 0x7f99f015bd70 2024-09-11T12:55:02.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f0138fa0 con 0x7f99f015bd70 2024-09-11T12:55:02.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f00aa090 con 0x7f99f015bd70 2024-09-11T12:55:02.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.879+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f99e80054c0 con 0x7f99f015bd70 2024-09-11T12:55:02.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.883+0000 7f99e7400640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 0x7f99d0073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:02.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.883+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5443+0+0 (secure 0 0 0) 0x7f99e812be90 con 0x7f99f015bd70 2024-09-11T12:55:02.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.883+0000 7f99ef400640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 0x7f99d0073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:02.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.883+0000 7f99ef400640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 0x7f99d0073f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99f013df50 tx=0x7f99e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:02.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:02.887+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f99e80c4080 con 0x7f99f015bd70 2024-09-11T12:55:03.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.055+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f99f00cee40 con 0x7f99f015bd70 2024-09-11T12:55:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.715+0000 7f99e7400640 1 -- 172.21.15.78:0/2043734062 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) ==== 169+0+0 (secure 0 0 0) 0x7f99e80164d0 con 0x7f99f015bd70 2024-09-11T12:55:03.719 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 msgr2=0x7f99d0073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 0x7f99d0073f30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f99f013df50 tx=0x7f99e0009040 comp rx=0 tx=0).stop 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 msgr2=0x7f99f0153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f99e80b0040 tx=0x7f99e80a57f0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 shutdown_connections 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99f015bd70 0x7f99f0153f60 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f99d0071a70 0x7f99d0073f30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f99f013e690 0x7f99f01394c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 --2- 172.21.15.78:0/2043734062 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99f00ac940 0x7f99f0137fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 >> 172.21.15.78:0/2043734062 conn(0x7f99f006e860 msgr2=0x7f99f013b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:03.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 shutdown_connections 2024-09-11T12:55:03.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.719+0000 7f99f4c00640 1 -- 172.21.15.78:0/2043734062 wait complete. 2024-09-11T12:55:03.746 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 -- 172.21.15.78:0/2151723337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 msgr2=0x7fcb6c003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/2151723337 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb6c003e60 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fcb78006fd0 tx=0x7fcb7801ce80 comp rx=0 tx=0).stop 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8dc00640 1 -- 172.21.15.78:0/2151723337 reap_dead start 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 -- 172.21.15.78:0/2151723337 shutdown_connections 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 -- 172.21.15.78:0/2151723337 >> 172.21.15.78:0/2151723337 conn(0x7fcb8806ca80 msgr2=0x7fcb880b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 -- 172.21.15.78:0/2151723337 shutdown_connections 2024-09-11T12:55:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.887+0000 7fcb8fc00640 1 -- 172.21.15.78:0/2151723337 wait complete. 2024-09-11T12:55:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 Processor -- start 2024-09-11T12:55:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- start start 2024-09-11T12:55:03.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:03.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88151d80 0x7fcb88155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:03.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 0x7fcb88152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:03.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88167b10 con 0x7fcb6c003a40 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb88167990 con 0x7fcb88151d80 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb88167c90 con 0x7fcb881529a0 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53386/0 (socket says 172.21.15.78:53386) 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 -- 172.21.15.78:0/132373247 learned_addr learned my addr 172.21.15.78:0/132373247 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8dc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 0x7fcb88152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 -- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 msgr2=0x7fcb88152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 0x7fcb88152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 -- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88151d80 msgr2=0x7fcb88155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88151d80 0x7fcb88155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 -- 172.21.15.78:0/132373247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb88155480 con 0x7fcb6c003a40 2024-09-11T12:55:03.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8dc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 0x7fcb88152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8d200640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcb78098830 tx=0x7fcb78001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78001500 con 0x7fcb6c003a40 2024-09-11T12:55:03.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb780016a0 con 0x7fcb6c003a40 2024-09-11T12:55:03.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb88155ca0 con 0x7fcb6c003a40 2024-09-11T12:55:03.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78006310 con 0x7fcb6c003a40 2024-09-11T12:55:03.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb881551a0 con 0x7fcb6c003a40 2024-09-11T12:55:03.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.891+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb8806ecf0 con 0x7fcb6c003a40 2024-09-11T12:55:03.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.895+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fcb780064b0 con 0x7fcb6c003a40 2024-09-11T12:55:03.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.895+0000 7fcb84c00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 0x7fcb68073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:03.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.895+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7fcb7812bc80 con 0x7fcb6c003a40 2024-09-11T12:55:03.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.895+0000 7fcb87e00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 0x7fcb68073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:03.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.895+0000 7fcb87e00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 0x7fcb68073f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb880cdf80 tx=0x7fcb74005b00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:03.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:03.899+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcb88155ca0 con 0x7fcb6c003a40 2024-09-11T12:55:04.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.067+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fcb88156aa0 con 0x7fcb6c003a40 2024-09-11T12:55:04.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb84c00640 1 -- 172.21.15.78:0/132373247 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fcb78016c30 con 0x7fcb6c003a40 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 msgr2=0x7fcb68073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 0x7fcb68073f60 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcb880cdf80 tx=0x7fcb74005b00 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 msgr2=0x7fcb88148b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fcb78098830 tx=0x7fcb78001cc0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 shutdown_connections 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb881529a0 0x7fcb88152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb68071aa0 0x7fcb68073f60 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb88151d80 0x7fcb88155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 --2- 172.21.15.78:0/132373247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb6c003a40 0x7fcb88148b10 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 >> 172.21.15.78:0/132373247 conn(0x7fcb8806ca80 msgr2=0x7fcb880b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:04.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 shutdown_connections 2024-09-11T12:55:04.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.071+0000 7fcb8fc00640 1 -- 172.21.15.78:0/132373247 wait complete. 2024-09-11T12:55:04.095 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-11T12:55:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 -- 172.21.15.78:0/444676430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e00b14b0 msgr2=0x7f20e015c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 --2- 172.21.15.78:0/444676430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e00b14b0 0x7f20e015c100 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009900 tx=0x7f20d401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 -- 172.21.15.78:0/444676430 shutdown_connections 2024-09-11T12:55:04.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 --2- 172.21.15.78:0/444676430 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e00b14b0 0x7f20e015c100 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 -- 172.21.15.78:0/444676430 >> 172.21.15.78:0/444676430 conn(0x7f20e0074b90 msgr2=0x7f20e00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 -- 172.21.15.78:0/444676430 shutdown_connections 2024-09-11T12:55:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 -- 172.21.15.78:0/444676430 wait complete. 2024-09-11T12:55:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.235+0000 7f20e8000640 1 Processor -- start 2024-09-11T12:55:04.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- start start 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e00b14b0 0x7f20e013f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 0x7f20e014a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20e016a400 con 0x7f20e006fdf0 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20e016a280 con 0x7f20e00b14b0 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20e016a580 con 0x7f20e00d0bf0 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53412/0 (socket says 172.21.15.78:53412) 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 -- 172.21.15.78:0/769874421 learned_addr learned my addr 172.21.15.78:0/769874421 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:04.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e4c00640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e00b14b0 0x7f20e013f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e6000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 0x7f20e014a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 -- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 msgr2=0x7f20e014a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 0x7f20e014a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 -- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e00b14b0 msgr2=0x7f20e013f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e00b14b0 0x7f20e013f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 -- 172.21.15.78:0/769874421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e0145a60 con 0x7f20e006fdf0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e6000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 0x7f20e014a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e5600640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f20d40194f0 tx=0x7f20d40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d40b2040 con 0x7f20e006fdf0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20d4005020 con 0x7f20e006fdf0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4005450 con 0x7f20e006fdf0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e01f9450 con 0x7f20e006fdf0 2024-09-11T12:55:04.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e01f9a10 con 0x7f20e006fdf0 2024-09-11T12:55:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.239+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e015c100 con 0x7f20e006fdf0 2024-09-11T12:55:04.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.243+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f20d40046d0 con 0x7f20e006fdf0 2024-09-11T12:55:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.243+0000 7f20dd600640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 0x7f20c0073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.243+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f20d412c350 con 0x7f20e006fdf0 2024-09-11T12:55:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.243+0000 7f20e4c00640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 0x7f20c0073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.243+0000 7f20e4c00640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 0x7f20c0073f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b3630 tx=0x7f20cc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:04.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.247+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f20d40164d0 con 0x7f20e006fdf0 2024-09-11T12:55:04.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.479+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f20e014bd50 con 0x7f20e006fdf0 2024-09-11T12:55:04.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20dd600640 1 -- 172.21.15.78:0/769874421 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v299) ==== 149+0+0 (secure 0 0 0) 0x7f20d40f7ee0 con 0x7f20e006fdf0 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 msgr2=0x7f20c0073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 0x7f20c0073f30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f20e00b3630 tx=0x7f20cc009040 comp rx=0 tx=0).stop 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 msgr2=0x7f20e0145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f20d40194f0 tx=0x7f20d40040c0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 shutdown_connections 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20e00d0bf0 0x7f20e014a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20c0071a70 0x7f20c0073f30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20e00b14b0 0x7f20e013f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 --2- 172.21.15.78:0/769874421 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20e006fdf0 0x7f20e0145520 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 >> 172.21.15.78:0/769874421 conn(0x7f20e0074b90 msgr2=0x7f20e0153ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:04.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 shutdown_connections 2024-09-11T12:55:04.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.719+0000 7f20e8000640 1 -- 172.21.15.78:0/769874421 wait complete. 2024-09-11T12:55:04.742 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-11T12:55:04.743 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-11T12:55:04.743 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:55:04.743 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- 172.21.15.78:0/1818769415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45200707f0 msgr2=0x7f4520073ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- 172.21.15.78:0/1818769415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45200707f0 0x7f4520073ed0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f45200b2770 tx=0x7f451001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- 172.21.15.78:0/1818769415 shutdown_connections 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- 172.21.15.78:0/1818769415 >> v2:172.21.15.157:3300/0 conn(0x7f452015af10 0x7f452015db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- 172.21.15.78:0/1818769415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f45200707f0 0x7f4520073ed0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- 172.21.15.78:0/1818769415 >> 172.21.15.78:0/1818769415 conn(0x7f452006ca80 msgr2=0x7f45200b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- 172.21.15.78:0/1818769415 shutdown_connections 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- 172.21.15.78:0/1818769415 wait complete. 2024-09-11T12:55:04.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 Processor -- start 2024-09-11T12:55:04.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- start start 2024-09-11T12:55:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45200707f0 0x7f452013a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452015af10 0x7f45201550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f45200b5200 con 0x7f4520071d60 2024-09-11T12:55:04.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f45200b5080 con 0x7f452015af10 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.883+0000 7f4527000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f45200b5380 con 0x7f45200707f0 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53432/0 (socket says 172.21.15.78:53432) 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 -- 172.21.15.78:0/3238166016 learned_addr learned my addr 172.21.15.78:0/3238166016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451fe00640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45200707f0 0x7f452013a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4525000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452015af10 0x7f45201550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 -- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45200707f0 msgr2=0x7f452013a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45200707f0 0x7f452013a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 -- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452015af10 msgr2=0x7f45201550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452015af10 0x7f45201550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 -- 172.21.15.78:0/3238166016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452013d1e0 con 0x7f4520071d60 2024-09-11T12:55:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f451f400640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f450c00b550 tx=0x7f450c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c014070 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f450c010980 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c010dd0 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452013d4c0 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f452013da80 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f452013dc80 con 0x7f4520071d60 2024-09-11T12:55:04.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.887+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f450c017070 con 0x7f4520071d60 2024-09-11T12:55:04.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.891+0000 7f4517e00640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 0x7f4500073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:04.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.891+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f450c0920c0 con 0x7f4520071d60 2024-09-11T12:55:04.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.891+0000 7f451fe00640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 0x7f4500073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:04.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.891+0000 7f451fe00640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 0x7f4500073f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:04.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:04.891+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f452013dc80 con 0x7f4520071d60 2024-09-11T12:55:05.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.059+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4520073ed0 con 0x7f4520071d60 2024-09-11T12:55:05.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4517e00640 1 -- 172.21.15.78:0/3238166016 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+13034 (secure 0 0 0) 0x7f450c05dc10 con 0x7f4520071d60 2024-09-11T12:55:05.064 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:05.064 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":108,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:03.718852+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":102,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/785402870":"2024-09-12T12:55:00.686842+0000","172.21.15.78:6833/785402870":"2024-09-12T12:55:00.686842+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 msgr2=0x7f4500073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 0x7f4500073f30 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45100af000 tx=0x7f451001b100 comp rx=0 tx=0).stop 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 msgr2=0x7f4520154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f450c00b550 tx=0x7f450c00ba10 comp rx=0 tx=0).stop 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 shutdown_connections 2024-09-11T12:55:05.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4500071a70 0x7f4500073f30 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f452015af10 0x7f45201550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4520071d60 0x7f4520154ba0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 --2- 172.21.15.78:0/3238166016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f45200707f0 0x7f452013a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 >> 172.21.15.78:0/3238166016 conn(0x7f452006ca80 msgr2=0x7f4520072190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 shutdown_connections 2024-09-11T12:55:05.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.063+0000 7f4527000640 1 -- 172.21.15.78:0/3238166016 wait complete. 2024-09-11T12:55:05.087 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:55:05.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:55:05.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.227+0000 7f15e9400640 1 -- 172.21.15.78:0/3092746639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 msgr2=0x7f15c8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.227+0000 7f15e9400640 1 --2- 172.21.15.78:0/3092746639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15c8004c90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0016f0 tx=0x7f15cc01c570 comp rx=0 tx=0).stop 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/3092746639 shutdown_connections 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 --2- 172.21.15.78:0/3092746639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15c8004c90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/3092746639 >> 172.21.15.78:0/3092746639 conn(0x7f15e40d0410 msgr2=0x7f15e40d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/3092746639 shutdown_connections 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/3092746639 wait complete. 2024-09-11T12:55:05.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 Processor -- start 2024-09-11T12:55:05.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- start start 2024-09-11T12:55:05.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 0x7f15e4139fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 0x7f15e413a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15e415b3d0 con 0x7f15c8004810 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15e415b250 con 0x7f15e40adcf0 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15e40cc230 con 0x7f15e40ad870 2024-09-11T12:55:05.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53456/0 (socket says 172.21.15.78:53456) 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 -- 172.21.15.78:0/1158601717 learned_addr learned my addr 172.21.15.78:0/1158601717 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15d9e00640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 0x7f15e4139fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e3200640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 0x7f15e413a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 -- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 msgr2=0x7f15e4139fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 0x7f15e4139fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 -- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 msgr2=0x7f15e413a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 0x7f15e413a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 -- 172.21.15.78:0/1158601717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e4151b10 con 0x7f15c8004810 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e3200640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 0x7f15e413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15d9e00640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 0x7f15e4139fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e2800640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f15d4001730 tx=0x7f15d400e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d4012850 con 0x7f15c8004810 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15d4012ae0 con 0x7f15c8004810 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e4151df0 con 0x7f15c8004810 2024-09-11T12:55:05.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d400ee60 con 0x7f15c8004810 2024-09-11T12:55:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.231+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e413c080 con 0x7f15c8004810 2024-09-11T12:55:05.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15e40d1500 con 0x7f15c8004810 2024-09-11T12:55:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f15d400d030 con 0x7f15c8004810 2024-09-11T12:55:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15db400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 0x7f15c0073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f15d40953b0 con 0x7f15c8004810 2024-09-11T12:55:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15d9e00640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 0x7f15c0073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.235+0000 7f15d9e00640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 0x7f15c0073f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0016c0 tx=0x7f15cc019360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:05.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.239+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f15d4016030 con 0x7f15c8004810 2024-09-11T12:55:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15e40cc720 con 0x7f15c8004810 2024-09-11T12:55:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15db400640 1 -- 172.21.15.78:0/1158601717 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f15d4060e50 con 0x7f15c8004810 2024-09-11T12:55:05.410 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:55:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 msgr2=0x7f15c0073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 0x7f15c0073f60 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0016c0 tx=0x7f15cc019360 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 msgr2=0x7f15e41511e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f15d4001730 tx=0x7f15d400e6b0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 shutdown_connections 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15c0071aa0 0x7f15c0073f60 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15e40adcf0 0x7f15e413a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15e40ad870 0x7f15e4139fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 --2- 172.21.15.78:0/1158601717 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15c8004810 0x7f15e41511e0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 >> 172.21.15.78:0/1158601717 conn(0x7f15e40d0410 msgr2=0x7f15e40cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 shutdown_connections 2024-09-11T12:55:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.407+0000 7f15e9400640 1 -- 172.21.15.78:0/1158601717 wait complete. 2024-09-11T12:55:05.431 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-11T12:55:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- 172.21.15.78:0/347826115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc0a75d0 msgr2=0x7f75fc0b0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- 172.21.15.78:0/347826115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc0a75d0 0x7f75fc0b0dc0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f75fc074430 tx=0x7f75e801c800 comp rx=0 tx=0).stop 2024-09-11T12:55:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- 172.21.15.78:0/347826115 shutdown_connections 2024-09-11T12:55:05.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- 172.21.15.78:0/347826115 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc0a75d0 0x7f75fc0b0dc0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- 172.21.15.78:0/347826115 >> v2:172.21.15.157:3300/0 conn(0x7f75fc15f080 0x7f75fc06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- 172.21.15.78:0/347826115 >> 172.21.15.78:0/347826115 conn(0x7f75fc0cf830 msgr2=0x7f75fc0cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- 172.21.15.78:0/347826115 shutdown_connections 2024-09-11T12:55:05.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- 172.21.15.78:0/347826115 wait complete. 2024-09-11T12:55:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 Processor -- start 2024-09-11T12:55:05.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- start start 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75fc0a75d0 0x7f75fc14bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc0b1300 0x7f75fc14e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc169000 con 0x7f75fc15f080 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f75fc168e80 con 0x7f75fc0b1300 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f7602800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f75fc169180 con 0x7f75fc0a75d0 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f75fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53468/0 (socket says 172.21.15.78:53468) 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f75fbe00640 1 -- 172.21.15.78:0/292849043 learned_addr learned my addr 172.21.15.78:0/292849043 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f75fb400640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75fc0a75d0 0x7f75fc14bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.571+0000 7f75faa00640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc0b1300 0x7f75fc14e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 -- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75fc0a75d0 msgr2=0x7f75fc14bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75fc0a75d0 0x7f75fc14bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 -- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc0b1300 msgr2=0x7f75fc14e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc0b1300 0x7f75fc14e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 -- 172.21.15.78:0/292849043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75fc14ef20 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75fbe00640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f75f4001ec0 tx=0x7f75f400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4017070 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f40053c0 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f4003860 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75fc1555b0 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75fc14cec0 con 0x7f75fc15f080 2024-09-11T12:55:05.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75fc0d15a0 con 0x7f75fc15f080 2024-09-11T12:55:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f75f401a070 con 0x7f75fc15f080 2024-09-11T12:55:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 0x7f75dc073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.575+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f75f40915b0 con 0x7f75fc15f080 2024-09-11T12:55:05.580 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.579+0000 7f75fb400640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 0x7f75dc073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:05.581 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.579+0000 7f75fb400640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 0x7f75dc073f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009e60 tx=0x7f75ec0095b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:05.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.583+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f75fc0b1710 con 0x7f75fc15f080 2024-09-11T12:55:05.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:05.803+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f75fc0d0730 con 0x7f75fc15f080 2024-09-11T12:55:06.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f75f3400640 1 -- 172.21.15.78:0/292849043 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v301) ==== 151+0+0 (secure 0 0 0) 0x7f75f4065340 con 0x7f75fc15f080 2024-09-11T12:55:06.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 msgr2=0x7f75dc073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:06.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 0x7f75dc073f30 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f75ec009e60 tx=0x7f75ec0095b0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 msgr2=0x7f75fc0b2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:06.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f75f4001ec0 tx=0x7f75f400c890 comp rx=0 tx=0).stop 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 shutdown_connections 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f75fc15f080 0x7f75fc0b2b80 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f75dc071a70 0x7f75dc073f30 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75fc0b1300 0x7f75fc14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 --2- 172.21.15.78:0/292849043 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75fc0a75d0 0x7f75fc14bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 >> 172.21.15.78:0/292849043 conn(0x7f75fc0cf830 msgr2=0x7f75fc14f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 shutdown_connections 2024-09-11T12:55:06.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.727+0000 7f7602800640 1 -- 172.21.15.78:0/292849043 wait complete. 2024-09-11T12:55:06.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:55:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 -- 172.21.15.78:0/2949458815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6250003a10 msgr2=0x7f6250003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 --2- 172.21.15.78:0/2949458815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6250003a10 0x7f6250003e30 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f625c009ef0 tx=0x7f625c01d5b0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f626e600640 1 -- 172.21.15.78:0/2949458815 reap_dead start 2024-09-11T12:55:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 -- 172.21.15.78:0/2949458815 shutdown_connections 2024-09-11T12:55:06.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 -- 172.21.15.78:0/2949458815 >> 172.21.15.78:0/2949458815 conn(0x7f626806f3a0 msgr2=0x7f62680cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 -- 172.21.15.78:0/2949458815 shutdown_connections 2024-09-11T12:55:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.899+0000 7f6270600640 1 -- 172.21.15.78:0/2949458815 wait complete. 2024-09-11T12:55:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 Processor -- start 2024-09-11T12:55:06.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- start start 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6250003a10 0x7f6268162f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6268153750 0x7f62680c2f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6268165940 con 0x7f62681532d0 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62681657c0 con 0x7f6268153750 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6268165ac0 con 0x7f6250003a10 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:06.907 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53480/0 (socket says 172.21.15.78:53480) 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 -- 172.21.15.78:0/1498611287 learned_addr learned my addr 172.21.15.78:0/1498611287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626dc00640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6250003a10 0x7f6268162f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626e600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6268153750 0x7f62680c2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 -- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6250003a10 msgr2=0x7f6268162f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6250003a10 0x7f6268162f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 -- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6268153750 msgr2=0x7f62680c2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6268153750 0x7f62680c2f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 -- 172.21.15.78:0/1498611287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62681f8840 con 0x7f62681532d0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f626d200640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6254011ab0 tx=0x7f6254011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254004020 con 0x7f62681532d0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62540042b0 con 0x7f62681532d0 2024-09-11T12:55:06.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62681f8b20 con 0x7f62681532d0 2024-09-11T12:55:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254003860 con 0x7f62681532d0 2024-09-11T12:55:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.903+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62681634c0 con 0x7f62681532d0 2024-09-11T12:55:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62680cf7a0 con 0x7f62681532d0 2024-09-11T12:55:06.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f6254003a00 con 0x7f62681532d0 2024-09-11T12:55:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f6265600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 0x7f6248073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f6254092fc0 con 0x7f62681532d0 2024-09-11T12:55:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f626dc00640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 0x7f6248073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:06.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.907+0000 7f626dc00640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 0x7f6248073f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f625c0af000 tx=0x7f625c01a350 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:06.911+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f625405e550 con 0x7f62681532d0 2024-09-11T12:55:07.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f6268154260 con 0x7f62681532d0 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6265600640 1 -- 172.21.15.78:0/1498611287 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+1998 (secure 0 0 0) 0x7f6254062aa0 con 0x7f62681532d0 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:55:07.090 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:55:07.091 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:55:07.092 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:55:07.092 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:55:07.092 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:55:07.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 msgr2=0x7f6248073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 0x7f6248073f30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f625c0af000 tx=0x7f625c01a350 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 msgr2=0x7f62681f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f6254011ab0 tx=0x7f6254011f70 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 shutdown_connections 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6248071a70 0x7f6248073f30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6268153750 0x7f62680c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62681532d0 0x7f62681f7df0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 --2- 172.21.15.78:0/1498611287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6250003a10 0x7f6268162f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 >> 172.21.15.78:0/1498611287 conn(0x7f626806f3a0 msgr2=0x7f6268151f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 shutdown_connections 2024-09-11T12:55:07.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.087+0000 7f6270600640 1 -- 172.21.15.78:0/1498611287 wait complete. 2024-09-11T12:55:07.110 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:55:07.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 -- 172.21.15.78:0/1342304750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004d50 msgr2=0x7f20a40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1342304750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20a4004d50 0x7f20a40051d0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f20b800c670 tx=0x7f20b801d320 comp rx=0 tx=0).stop 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c4a00640 1 -- 172.21.15.78:0/1342304750 reap_dead start 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 -- 172.21.15.78:0/1342304750 shutdown_connections 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 -- 172.21.15.78:0/1342304750 >> 172.21.15.78:0/1342304750 conn(0x7f20c0071dd0 msgr2=0x7f20c00721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 -- 172.21.15.78:0/1342304750 shutdown_connections 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.251+0000 7f20c6a00640 1 -- 172.21.15.78:0/1342304750 wait complete. 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 Processor -- start 2024-09-11T12:55:07.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- start start 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20a4004d50 0x7f20c0145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c014ec60 0x7f20c014f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0151e10 con 0x7f20c0148740 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f20c0151c90 con 0x7f20c014ec60 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f20c00cdc10 con 0x7f20a4004d50 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20bfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20a4004d50 0x7f20c0145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53496/0 (socket says 172.21.15.78:53496) 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 -- 172.21.15.78:0/1661279822 learned_addr learned my addr 172.21.15.78:0/1661279822 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:07.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20bf400640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c014ec60 0x7f20c014f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 -- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20a4004d50 msgr2=0x7f20c0145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20a4004d50 0x7f20c0145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 -- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c014ec60 msgr2=0x7f20c014f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c014ec60 0x7f20c014f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 -- 172.21.15.78:0/1661279822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c014f740 con 0x7f20c0148740 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c4a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f20b801cde0 tx=0x7f20b80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b80bc070 con 0x7f20c0148740 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b8003b80 con 0x7f20c0148740 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b80b2040 con 0x7f20c0148740 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c014fa20 con 0x7f20c0148740 2024-09-11T12:55:07.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.255+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c014ffe0 con 0x7f20c0148740 2024-09-11T12:55:07.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c00b1a10 con 0x7f20c0148740 2024-09-11T12:55:07.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f20b80ab070 con 0x7f20c0148740 2024-09-11T12:55:07.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20b7e00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 0x7f20a0073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f20b812c080 con 0x7f20c0148740 2024-09-11T12:55:07.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20bfe00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 0x7f20a0073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.259+0000 7f20bfe00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 0x7f20a0073f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d80 tx=0x7f20b00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:07.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.263+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f20b80170d0 con 0x7f20c0148740 2024-09-11T12:55:07.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.427+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f20c00b0270 con 0x7f20c0148740 2024-09-11T12:55:07.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.431+0000 7f20b7e00640 1 -- 172.21.15.78:0/1661279822 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f20b80f7500 con 0x7f20c0148740 2024-09-11T12:55:07.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 msgr2=0x7f20a0073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 0x7f20a0073f60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f20b0009d80 tx=0x7f20b00091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 msgr2=0x7f20c0148bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f20b801cde0 tx=0x7f20b80014a0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 shutdown_connections 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20c0148740 0x7f20c0148bc0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f20a0071aa0 0x7f20a0073f60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20c014ec60 0x7f20c014f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 --2- 172.21.15.78:0/1661279822 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20a4004d50 0x7f20c0145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 >> 172.21.15.78:0/1661279822 conn(0x7f20c0071dd0 msgr2=0x7f20c015b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 shutdown_connections 2024-09-11T12:55:07.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.435+0000 7f20c6a00640 1 -- 172.21.15.78:0/1661279822 wait complete. 2024-09-11T12:55:07.455 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:55:07.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:07.542 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:07.618 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:55:07.618 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:07.618 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:07.626 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:55:07.627 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:07.627 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:07.636 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:55:07.636 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:07.636 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:07.675 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:55:07.675 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:07.675 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:07.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:55:07.687 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:55:07.687 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- 172.21.15.78:0/209371141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 msgr2=0x7f2de415e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 --2- 172.21.15.78:0/209371141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de415e3d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2ddc009bc0 tx=0x7f2ddc01cb10 comp rx=0 tx=0).stop 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- 172.21.15.78:0/209371141 shutdown_connections 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 --2- 172.21.15.78:0/209371141 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de415e3d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- 172.21.15.78:0/209371141 >> 172.21.15.78:0/209371141 conn(0x7f2de40b19a0 msgr2=0x7f2de4070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- 172.21.15.78:0/209371141 shutdown_connections 2024-09-11T12:55:07.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- 172.21.15.78:0/209371141 wait complete. 2024-09-11T12:55:07.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 Processor -- start 2024-09-11T12:55:07.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.855+0000 7f2de3200640 1 -- start start 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de40b2dc0 0x7f2de41fa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de415c390 0x7f2de41572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4169d40 con 0x7f2de40b31f0 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2de4169bc0 con 0x7f2de40b2dc0 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2de4169ec0 con 0x7f2de415c390 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53514/0 (socket says 172.21.15.78:53514) 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 -- 172.21.15.78:0/1354584472 learned_addr learned my addr 172.21.15.78:0/1354584472 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:07.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de1e00640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de40b2dc0 0x7f2de41fa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de2800640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de415c390 0x7f2de41572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 -- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de415c390 msgr2=0x7f2de41572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de415c390 0x7f2de41572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 -- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de40b2dc0 msgr2=0x7f2de41fa2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de40b2dc0 0x7f2de41fa2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:07.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 -- 172.21.15.78:0/1354584472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2de41fa7e0 con 0x7f2de40b31f0 2024-09-11T12:55:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2dd9400640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00c450 tx=0x7f2dcc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc005480 con 0x7f2de40b31f0 2024-09-11T12:55:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2dcc003600 con 0x7f2de40b31f0 2024-09-11T12:55:07.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc003a30 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2de406e620 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2de40af250 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.859+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2de415c770 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.863+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f2dcc003bd0 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.863+0000 7f2ddaa00640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 0x7f2dc0073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.863+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f2dcc091be0 con 0x7f2de40b31f0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.863+0000 7f2de1e00640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 0x7f2dc0073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.863+0000 7f2de1e00640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 0x7f2dc0073f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4007fd0 tx=0x7f2dd4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:07.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:07.867+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2dcc05d1b0 con 0x7f2de40b31f0 2024-09-11T12:55:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2de4074e00 con 0x7f2de40b31f0 2024-09-11T12:55:08.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2ddaa00640 1 -- 172.21.15.78:0/1354584472 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) ==== 94+0+3917 (secure 0 0 0) 0x7f2dcc061700 con 0x7f2de40b31f0 2024-09-11T12:55:08.069 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:08.069 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":74,"btime":"2024-09-11T12:55:01:690745+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/3455804704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3455804704},{"type":"v1","addr":"172.21.15.157:6801","nonce":3455804704}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":5975,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1689182579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1689182579},{"type":"v1","addr":"172.21.15.157:6835","nonce":1689182579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":66},{"gid":6562,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1033428084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1033428084},{"type":"v1","addr":"172.21.15.78:6834","nonce":1033428084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2638586988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2638586988},{"type":"v1","addr":"172.21.15.78:6835","nonce":2638586988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":74}],"filesystems":[]} 2024-09-11T12:55:08.069 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 74 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 msgr2=0x7f2dc0073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 0x7f2dc0073f30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2dd4007fd0 tx=0x7f2dd4009040 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 msgr2=0x7f2de406dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc00c450 tx=0x7f2dcc00c910 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 shutdown_connections 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2de415c390 0x7f2de41572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2de40b31f0 0x7f2de406dd40 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2dc0071a70 0x7f2dc0073f30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 --2- 172.21.15.78:0/1354584472 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2de40b2dc0 0x7f2de41fa2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 >> 172.21.15.78:0/1354584472 conn(0x7f2de40b19a0 msgr2=0x7f2de406fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:08.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 shutdown_connections 2024-09-11T12:55:08.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.067+0000 7f2de3200640 1 -- 172.21.15.78:0/1354584472 wait complete. 2024-09-11T12:55:08.089 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:55:08.089 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:55:08.090 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:55:08.090 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:08.090 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:55:08.090 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:55:08.090 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:55:08.090 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:08.091 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:55:08.091 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:55:08.091 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:55:08.091 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:08.091 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:55:08.091 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:55:08.091 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:55:08.092 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:08.102 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:55:08.102 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:55:08.103 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:55:08.103 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:55:08.105 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:55:08.105 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:55:08.106 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:55:08.106 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:55:08.150 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:55:08.150 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:08.152 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:55:08.152 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:08.153 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:55:08.154 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:08.155 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:55:08.155 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:08.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:55:08.236 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:55:08.236 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:55:08.244 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:55:08.244 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:55:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- 172.21.15.78:0/2805055528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9013d840 msgr2=0x7f6c90140980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- 172.21.15.78:0/2805055528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9013d840 0x7f6c90140980 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6c80009c80 tx=0x7f6c8001cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.304 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- 172.21.15.78:0/2805055528 shutdown_connections 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- 172.21.15.78:0/2805055528 >> v2:172.21.15.157:3300/0 conn(0x7f6c9006fc50 0x7f6c90070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- 172.21.15.78:0/2805055528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c9013d840 0x7f6c90140980 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- 172.21.15.78:0/2805055528 >> 172.21.15.78:0/2805055528 conn(0x7f6c9006ca80 msgr2=0x7f6c900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- 172.21.15.78:0/2805055528 shutdown_connections 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- 172.21.15.78:0/2805055528 wait complete. 2024-09-11T12:55:08.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 Processor -- start 2024-09-11T12:55:08.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- start start 2024-09-11T12:55:08.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 0x7f6c90164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 0x7f6c9015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c9013d840 0x7f6c9013ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90165f90 con 0x7f6c9013d840 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6c90165e10 con 0x7f6c9006fc50 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c94e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6c90166110 con 0x7f6c900b4520 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 0x7f6c90164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 0x7f6c9015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 0x7f6c90164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:32908/0 (socket says 172.21.15.78:32908) 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8de00640 1 -- 172.21.15.78:0/963758732 learned_addr learned my addr 172.21.15.78:0/963758732 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8e800640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c9013d840 0x7f6c9013ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 -- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 msgr2=0x7f6c90164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 0x7f6c90164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 -- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c9013d840 msgr2=0x7f6c9013ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c9013d840 0x7f6c9013ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.303+0000 7f6c8d400640 1 -- 172.21.15.78:0/963758732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c90164f60 con 0x7f6c900b4520 2024-09-11T12:55:08.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c8d400640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 0x7f6c9015f5b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00e9c0 tx=0x7f6c7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c00ccc0 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c7c004030 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c90141300 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c004480 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c901418c0 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c90070090 con 0x7f6c900b4520 2024-09-11T12:55:08.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f6c7c003600 con 0x7f6c900b4520 2024-09-11T12:55:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 0x7f6c70073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(108..108 src has 1..108) ==== 4943+0+0 (secure 0 0 0) 0x7f6c7c092be0 con 0x7f6c900b4520 2024-09-11T12:55:08.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.307+0000 7f6c8de00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 0x7f6c70073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.311+0000 7f6c8de00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 0x7f6c70073f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c8001b450 tx=0x7f6c8001b5e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:08.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.311+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6c7c05dfd0 con 0x7f6c900b4520 2024-09-11T12:55:08.492 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.491+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f6c9015aae0 con 0x7f6c900b4520 2024-09-11T12:55:08.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.739+0000 7f6c86000640 1 -- 172.21.15.78:0/963758732 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) ==== 97+0+0 (secure 0 0 0) 0x7f6c7c062520 con 0x7f6c900b4520 2024-09-11T12:55:08.743 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 msgr2=0x7f6c70073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 0x7f6c70073f30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c8001b450 tx=0x7f6c8001b5e0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 msgr2=0x7f6c9015f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 0x7f6c9015f5b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00e9c0 tx=0x7f6c7c00ee80 comp rx=0 tx=0).stop 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 shutdown_connections 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6c9013d840 0x7f6c9013ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6c900b4520 0x7f6c9015f5b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6c70071a70 0x7f6c70073f30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 --2- 172.21.15.78:0/963758732 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6c9006fc50 0x7f6c90164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 >> 172.21.15.78:0/963758732 conn(0x7f6c9006ca80 msgr2=0x7f6c90071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 shutdown_connections 2024-09-11T12:55:08.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.743+0000 7f6c94e00640 1 -- 172.21.15.78:0/963758732 wait complete. 2024-09-11T12:55:08.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:55:08.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- 172.21.15.78:0/398704798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 msgr2=0x7ffafc15cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 --2- 172.21.15.78:0/398704798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc15cf60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffafc0b58e0 tx=0x7ffae801c800 comp rx=0 tx=0).stop 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- 172.21.15.78:0/398704798 shutdown_connections 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 --2- 172.21.15.78:0/398704798 >> v2:172.21.15.157:3300/0 conn(0x7ffafc0b4520 0x7ffafc0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 --2- 172.21.15.78:0/398704798 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc15cf60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- 172.21.15.78:0/398704798 >> 172.21.15.78:0/398704798 conn(0x7ffafc06ca80 msgr2=0x7ffafc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- 172.21.15.78:0/398704798 shutdown_connections 2024-09-11T12:55:08.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- 172.21.15.78:0/398704798 wait complete. 2024-09-11T12:55:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 Processor -- start 2024-09-11T12:55:08.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.907+0000 7ffb02a00640 1 -- start start 2024-09-11T12:55:08.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffafc0aed20 0x7ffafc0cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 0x7ffafc154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0cd250 con 0x7ffafc15cad0 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffafc0cd0d0 con 0x7ffafc0b4520 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffafc0cd3d0 con 0x7ffafc0aed20 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53574/0 (socket says 172.21.15.78:53574) 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 -- 172.21.15.78:0/2664317972 learned_addr learned my addr 172.21.15.78:0/2664317972 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffafb400640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 0x7ffafc154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffafbe00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffafc0aed20 0x7ffafc0cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 -- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffafc0aed20 msgr2=0x7ffafc0cbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffafc0aed20 0x7ffafc0cbcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 -- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 msgr2=0x7ffafc154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:08.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 0x7ffafc154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 -- 172.21.15.78:0/2664317972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffafc140050 con 0x7ffafc15cad0 2024-09-11T12:55:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffafb400640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 0x7ffafc154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb00a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e8c0 tx=0x7ffaf400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:08.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf400ccc0 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffaf400ce60 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf4005280 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffafc1558f0 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffafc13e5a0 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffafc0b4930 con 0x7ffafc15cad0 2024-09-11T12:55:08.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.911+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7ffaf4005420 con 0x7ffafc15cad0 2024-09-11T12:55:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.915+0000 7ffaf3e00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 0x7ffadc073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.915+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7ffaf4092c00 con 0x7ffafc15cad0 2024-09-11T12:55:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.915+0000 7ffafbe00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 0x7ffadc073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:08.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.915+0000 7ffafbe00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 0x7ffadc073f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffaec007fd0 tx=0x7ffaec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:08.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:08.919+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ffaf405e720 con 0x7ffafc15cad0 2024-09-11T12:55:09.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ffafc15b960 con 0x7ffafc15cad0 2024-09-11T12:55:09.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffaf3e00640 1 -- 172.21.15.78:0/2664317972 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7ffaf405e020 con 0x7ffafc15cad0 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:09.146 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:55:09.148 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:55:09.149 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:55:09.151 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:55:09.153 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:55:09.155 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.156 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:09.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:55:09.158 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.159 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:55:09.160 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 msgr2=0x7ffadc073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 0x7ffadc073f60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ffaec007fd0 tx=0x7ffaec009040 comp rx=0 tx=0).stop 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 msgr2=0x7ffafc159c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7ffaf400e8c0 tx=0x7ffaf400ed80 comp rx=0 tx=0).stop 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 shutdown_connections 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffafc15cad0 0x7ffafc159c10 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ffadc071aa0 0x7ffadc073f60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffafc0b4520 0x7ffafc154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 --2- 172.21.15.78:0/2664317972 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffafc0aed20 0x7ffafc0cbcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.143+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 >> 172.21.15.78:0/2664317972 conn(0x7ffafc06ca80 msgr2=0x7ffafc06f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:09.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.147+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 shutdown_connections 2024-09-11T12:55:09.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.147+0000 7ffb02a00640 1 -- 172.21.15.78:0/2664317972 wait complete. 2024-09-11T12:55:09.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- 172.21.15.78:0/2170310466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed80736d0 msgr2=0x7f7ed8157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 --2- 172.21.15.78:0/2170310466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed80736d0 0x7f7ed8157ae0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009900 tx=0x7f7ec401c850 comp rx=0 tx=0).stop 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- 172.21.15.78:0/2170310466 shutdown_connections 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 --2- 172.21.15.78:0/2170310466 >> v2:172.21.15.157:3300/0 conn(0x7f7ed806eb40 0x7f7ed806ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 --2- 172.21.15.78:0/2170310466 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed80736d0 0x7f7ed8157ae0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- 172.21.15.78:0/2170310466 >> 172.21.15.78:0/2170310466 conn(0x7f7ed806ca80 msgr2=0x7f7ed80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- 172.21.15.78:0/2170310466 shutdown_connections 2024-09-11T12:55:09.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- 172.21.15.78:0/2170310466 wait complete. 2024-09-11T12:55:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 Processor -- start 2024-09-11T12:55:09.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.307+0000 7f7edf000640 1 -- start start 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed806eb40 0x7f7ed8154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed8070a00 0x7f7ed81fb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80b51a0 con 0x7f7ed80736d0 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed80b5020 con 0x7f7ed806eb40 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ed80b5320 con 0x7f7ed8070a00 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53582/0 (socket says 172.21.15.78:53582) 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 -- 172.21.15.78:0/3558347857 learned_addr learned my addr 172.21.15.78:0/3558347857 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7ed7400640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed8070a00 0x7f7ed81fb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7ed7e00640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed806eb40 0x7f7ed8154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 -- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed8070a00 msgr2=0x7f7ed81fb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed8070a00 0x7f7ed81fb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 -- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed806eb40 msgr2=0x7f7ed8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed806eb40 0x7f7ed8154b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 -- 172.21.15.78:0/3558347857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed80b19c0 con 0x7f7ed80736d0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edd000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009d30 tx=0x7f7ed000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0003b80 con 0x7f7ed80736d0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ed0003d20 con 0x7f7ed80736d0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed80b1c50 con 0x7f7ed80736d0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0004280 con 0x7f7ed80736d0 2024-09-11T12:55:09.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed80afa60 con 0x7f7ed80736d0 2024-09-11T12:55:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.311+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed813d040 con 0x7f7ed80736d0 2024-09-11T12:55:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.315+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f7ed0004420 con 0x7f7ed80736d0 2024-09-11T12:55:09.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.315+0000 7f7ecfe00640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 0x7f7eb8073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.315+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7f7ed0092e70 con 0x7f7ed80736d0 2024-09-11T12:55:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.315+0000 7f7ed7e00640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 0x7f7eb8073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.315+0000 7f7ed7e00640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 0x7f7eb8073f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000f70 tx=0x7f7ec8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:09.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.319+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7ed813d040 con 0x7f7ed80736d0 2024-09-11T12:55:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7ed806ef80 con 0x7f7ed80736d0 2024-09-11T12:55:09.488 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7ecfe00640 1 -- 172.21.15.78:0/3558347857 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v112) ==== 84+0+65 (secure 0 0 0) 0x7f7ed005e2b0 con 0x7f7ed80736d0 2024-09-11T12:55:09.488 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:09.488 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:55:09.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:09.489 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:55:09.489 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:55:09.489 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:09.489 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 msgr2=0x7f7eb8073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 0x7f7eb8073f60 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8000f70 tx=0x7f7ec8009040 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 msgr2=0x7f7ed80b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0009d30 tx=0x7f7ed000c6a0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 shutdown_connections 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ed80736d0 0x7f7ed80b1480 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ed8070a00 0x7f7ed81fb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7eb8071aa0 0x7f7eb8073f60 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 --2- 172.21.15.78:0/3558347857 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ed806eb40 0x7f7ed8154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 >> 172.21.15.78:0/3558347857 conn(0x7f7ed806ca80 msgr2=0x7f7ed815cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 shutdown_connections 2024-09-11T12:55:09.491 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.487+0000 7f7edf000640 1 -- 172.21.15.78:0/3558347857 wait complete. 2024-09-11T12:55:09.510 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:55:09.510 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.651+0000 7ff049800640 1 -- 172.21.15.78:0/1625131462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff044152990 msgr2=0x7ff04415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.651+0000 7ff049800640 1 --2- 172.21.15.78:0/1625131462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff044152990 0x7ff04415d060 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ff034009bc0 tx=0x7ff03401cba0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.651+0000 7ff049800640 1 -- 172.21.15.78:0/1625131462 shutdown_connections 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.651+0000 7ff049800640 1 --2- 172.21.15.78:0/1625131462 >> v2:172.21.15.157:3300/0 conn(0x7ff0440b5c40 0x7ff0440cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 --2- 172.21.15.78:0/1625131462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff044152990 0x7ff04415d060 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- 172.21.15.78:0/1625131462 >> 172.21.15.78:0/1625131462 conn(0x7ff0440b3050 msgr2=0x7ff0440b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- 172.21.15.78:0/1625131462 shutdown_connections 2024-09-11T12:55:09.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- 172.21.15.78:0/1625131462 wait complete. 2024-09-11T12:55:09.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 Processor -- start 2024-09-11T12:55:09.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- start start 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0440cf520 0x7ff04414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044152990 0x7ff04414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff044168a80 con 0x7ff0440b5c40 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff044168900 con 0x7ff044152990 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff044168c00 con 0x7ff0440cf520 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53598/0 (socket says 172.21.15.78:53598) 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 -- 172.21.15.78:0/2673050729 learned_addr learned my addr 172.21.15.78:0/2673050729 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042000640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0440cf520 0x7ff04414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff043400640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044152990 0x7ff04414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 -- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0440cf520 msgr2=0x7ff04414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0440cf520 0x7ff04414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 -- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044152990 msgr2=0x7ff04414cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044152990 0x7ff04414cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 -- 172.21.15.78:0/2673050729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04414d090 con 0x7ff0440b5c40 2024-09-11T12:55:09.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff042a00640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff03401c610 tx=0x7ff034005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340b2040 con 0x7ff0440b5c40 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0340a5870 con 0x7ff0440b5c40 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04414f540 con 0x7ff0440b5c40 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.655+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034004240 con 0x7ff0440b5c40 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04414fb00 con 0x7ff0440b5c40 2024-09-11T12:55:09.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0440cf960 con 0x7ff0440b5c40 2024-09-11T12:55:09.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7ff0340bd030 con 0x7ff0440b5c40 2024-09-11T12:55:09.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff03aa00640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 0x7ff024073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:09.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4857+0+0 (secure 0 0 0) 0x7ff03412c180 con 0x7ff0440b5c40 2024-09-11T12:55:09.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.659+0000 7ff042000640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 0x7ff024073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:09.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.663+0000 7ff042000640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 0x7ff024073f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0441531b0 tx=0x7ff030001420 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:09.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.663+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff0340c4020 con 0x7ff0440b5c40 2024-09-11T12:55:09.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:09.835+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff04414ff90 con 0x7ff0440b5c40 2024-09-11T12:55:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.747+0000 7ff03aa00640 1 -- 172.21.15.78:0/2673050729 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v113) ==== 138+0+0 (secure 0 0 0) 0x7ff034016790 con 0x7ff0440b5c40 2024-09-11T12:55:10.750 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 msgr2=0x7ff024073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 0x7ff024073f60 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff0441531b0 tx=0x7ff030001420 comp rx=0 tx=0).stop 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 msgr2=0x7ff0441571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ff03401c610 tx=0x7ff034005a60 comp rx=0 tx=0).stop 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 shutdown_connections 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff024071aa0 0x7ff024073f60 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff044152990 0x7ff04414cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff0440cf520 0x7ff04414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 --2- 172.21.15.78:0/2673050729 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff0440b5c40 0x7ff0441571c0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 >> 172.21.15.78:0/2673050729 conn(0x7ff0440b3050 msgr2=0x7ff0440b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 shutdown_connections 2024-09-11T12:55:10.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.751+0000 7ff049800640 1 -- 172.21.15.78:0/2673050729 wait complete. 2024-09-11T12:55:10.782 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:55:10.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- 172.21.15.78:0/3505640571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc0cf790 msgr2=0x7f6fdc15e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 --2- 172.21.15.78:0/3505640571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc0cf790 0x7f6fdc15e6f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0ccd90 tx=0x7f6fc801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- 172.21.15.78:0/3505640571 shutdown_connections 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 --2- 172.21.15.78:0/3505640571 >> v2:172.21.15.157:3300/0 conn(0x7f6fdc1407e0 0x7f6fdc070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 --2- 172.21.15.78:0/3505640571 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc0cf790 0x7f6fdc15e6f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- 172.21.15.78:0/3505640571 >> 172.21.15.78:0/3505640571 conn(0x7f6fdc0b2ec0 msgr2=0x7f6fdc0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- 172.21.15.78:0/3505640571 shutdown_connections 2024-09-11T12:55:10.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- 172.21.15.78:0/3505640571 wait complete. 2024-09-11T12:55:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 Processor -- start 2024-09-11T12:55:10.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.931+0000 7f6fe3000640 1 -- start start 2024-09-11T12:55:10.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fdc0cf790 0x7f6fdc13b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fdc140400 0x7f6fdc160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc073a40 con 0x7f6fdc1407e0 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fdc0738c0 con 0x7f6fdc140400 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6fdc073bc0 con 0x7f6fdc0cf790 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53624/0 (socket says 172.21.15.78:53624) 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 -- 172.21.15.78:0/4045041628 learned_addr learned my addr 172.21.15.78:0/4045041628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fdbe00640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fdc0cf790 0x7f6fdc13b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fdb400640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fdc140400 0x7f6fdc160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 -- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fdc0cf790 msgr2=0x7f6fdc13b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fdc0cf790 0x7f6fdc13b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 -- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fdc140400 msgr2=0x7f6fdc160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fdc140400 0x7f6fdc160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 -- 172.21.15.78:0/4045041628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fdc074590 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe1000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400de20 tx=0x7f6fd400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4004d30 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fd400be40 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fdc0b3ed0 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400e8c0 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fdc147190 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.935+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fdc15e6f0 con 0x7f6fdc1407e0 2024-09-11T12:55:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.939+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f6fd400ea60 con 0x7f6fdc1407e0 2024-09-11T12:55:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.939+0000 7f6fd3e00640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 0x7f6fbc073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.939+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5227+0+0 (secure 0 0 0) 0x7f6fd4093200 con 0x7f6fdc1407e0 2024-09-11T12:55:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.939+0000 7f6fdbe00640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 0x7f6fbc073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.939+0000 7f6fdbe00640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 0x7f6fbc073f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc007fd0 tx=0x7f6fcc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:10.943+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6fd405e720 con 0x7f6fdc1407e0 2024-09-11T12:55:11.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.111+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f6fdc15b0a0 con 0x7f6fdc1407e0 2024-09-11T12:55:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.751+0000 7f6fd3e00640 1 -- 172.21.15.78:0/4045041628 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v114) ==== 170+0+0 (secure 0 0 0) 0x7f6fd4062c70 con 0x7f6fdc1407e0 2024-09-11T12:55:11.753 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:55:11.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.751+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 msgr2=0x7f6fbc073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:11.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.751+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 0x7f6fbc073f60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc007fd0 tx=0x7f6fcc009040 comp rx=0 tx=0).stop 2024-09-11T12:55:11.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.751+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 msgr2=0x7f6fdc1637d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:11.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.751+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400de20 tx=0x7f6fd400b730 comp rx=0 tx=0).stop 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 shutdown_connections 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fdc1407e0 0x7f6fdc1637d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fbc071aa0 0x7f6fbc073f60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fdc140400 0x7f6fdc160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 --2- 172.21.15.78:0/4045041628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fdc0cf790 0x7f6fdc13b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 >> 172.21.15.78:0/4045041628 conn(0x7f6fdc0b2ec0 msgr2=0x7f6fdc142e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:11.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 shutdown_connections 2024-09-11T12:55:11.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.755+0000 7f6fe3000640 1 -- 172.21.15.78:0/4045041628 wait complete. 2024-09-11T12:55:11.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- 172.21.15.78:0/603466092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0738b0 msgr2=0x7f6bec140c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/603466092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0738b0 0x7f6bec140c80 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f6bec072dc0 tx=0x7f6be401cd50 comp rx=0 tx=0).stop 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- 172.21.15.78:0/603466092 shutdown_connections 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/603466092 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0738b0 0x7f6bec140c80 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/603466092 >> v2:172.21.15.157:3300/0 conn(0x7f6bec140140 0x7f6bec0b33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- 172.21.15.78:0/603466092 >> 172.21.15.78:0/603466092 conn(0x7f6bec075c10 msgr2=0x7f6bec0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- 172.21.15.78:0/603466092 shutdown_connections 2024-09-11T12:55:11.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- 172.21.15.78:0/603466092 wait complete. 2024-09-11T12:55:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 Processor -- start 2024-09-11T12:55:11.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.939+0000 7f6bf0a00640 1 -- start start 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec0738b0 0x7f6bec0a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bec140140 0x7f6bec13dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec166e10 con 0x7f6bec0cc400 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6bec166c90 con 0x7f6bec0738b0 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6bec166f90 con 0x7f6bec140140 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:11.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53640/0 (socket says 172.21.15.78:53640) 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 -- 172.21.15.78:0/694462595 learned_addr learned my addr 172.21.15.78:0/694462595 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bebe00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bec140140 0x7f6bec13dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6beb400640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec0738b0 0x7f6bec0a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 -- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bec140140 msgr2=0x7f6bec13dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bec140140 0x7f6bec13dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 -- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec0738b0 msgr2=0x7f6bec0a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec0738b0 0x7f6bec0a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:11.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 -- 172.21.15.78:0/694462595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bec13f1b0 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3e00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c4c0 tx=0x7f6bd4001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4017070 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bd4003600 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4003a50 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bec0ab270 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bec1f8970 con 0x7f6bec0cc400 2024-09-11T12:55:11.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.943+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bec140c80 con 0x7f6bec0cc400 2024-09-11T12:55:11.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.947+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f6bd4003bf0 con 0x7f6bec0cc400 2024-09-11T12:55:11.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.947+0000 7f6be3400640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 0x7f6bc8073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:11.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.947+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5609+0+0 (secure 0 0 0) 0x7f6bd4091a30 con 0x7f6bec0cc400 2024-09-11T12:55:11.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.947+0000 7f6beb400640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 0x7f6bc8073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:11.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.947+0000 7f6beb400640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 0x7f6bc8073f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:11.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:11.951+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6bec13f1b0 con 0x7f6bec0cc400 2024-09-11T12:55:12.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.147+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6bec1f4d30 con 0x7f6bec0cc400 2024-09-11T12:55:12.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:55:12.560 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:55:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 18 and data pool 19 2024-09-11T12:55:12.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.775+0000 7f6be3400640 1 -- 172.21.15.78:0/694462595 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v77) ==== 174+0+0 (secure 0 0 0) 0x7f6bd405d3f0 con 0x7f6bec0cc400 2024-09-11T12:55:12.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 msgr2=0x7f6bc8073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 0x7f6bc8073f30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc008040 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 msgr2=0x7f6bec0aad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f6bd400c4c0 tx=0x7f6bd4001e80 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 shutdown_connections 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6bec140140 0x7f6bec13dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6bec0cc400 0x7f6bec0aad30 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6bc8071a70 0x7f6bc8073f30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 --2- 172.21.15.78:0/694462595 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6bec0738b0 0x7f6bec0a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 >> 172.21.15.78:0/694462595 conn(0x7f6bec075c10 msgr2=0x7f6bec142150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 shutdown_connections 2024-09-11T12:55:12.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.779+0000 7f6bf0a00640 1 -- 172.21.15.78:0/694462595 wait complete. 2024-09-11T12:55:12.820 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.959+0000 7fd27de00640 1 -- 172.21.15.78:0/2279460351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25c005220 msgr2=0x7fd25c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.959+0000 7fd27de00640 1 --2- 172.21.15.78:0/2279460351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25c005220 0x7fd25c00b9d0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fd270001860 tx=0x7fd27001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.959+0000 7fd27de00640 1 -- 172.21.15.78:0/2279460351 shutdown_connections 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.959+0000 7fd27de00640 1 --2- 172.21.15.78:0/2279460351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd25c005220 0x7fd25c00b9d0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.959+0000 7fd27de00640 1 -- 172.21.15.78:0/2279460351 >> 172.21.15.78:0/2279460351 conn(0x7fd278074b90 msgr2=0x7fd2780d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- 172.21.15.78:0/2279460351 shutdown_connections 2024-09-11T12:55:12.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- 172.21.15.78:0/2279460351 wait complete. 2024-09-11T12:55:12.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 Processor -- start 2024-09-11T12:55:12.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- start start 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 0x7fd278155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2781f24c0 0x7fd2781f2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2780d0b30 con 0x7fd27813b180 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd27815dad0 con 0x7fd25c005220 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd27de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2780d0cb0 con 0x7fd2781f24c0 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53666/0 (socket says 172.21.15.78:53666) 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 0x7fd278155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276000640 1 -- 172.21.15.78:0/1463172029 learned_addr learned my addr 172.21.15.78:0/1463172029 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd277400640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2781f24c0 0x7fd2781f2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:12.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 -- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2781f24c0 msgr2=0x7fd2781f2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2781f24c0 0x7fd2781f2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 -- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 msgr2=0x7fd27813b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 -- 172.21.15.78:0/1463172029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd278157b20 con 0x7fd25c005220 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276000640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd276a00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 0x7fd278155d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd268011ae0 tx=0x7fd26800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.963+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268004d30 con 0x7fd25c005220 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd26800fe60 con 0x7fd25c005220 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd278157e00 con 0x7fd25c005220 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268003bc0 con 0x7fd25c005220 2024-09-11T12:55:12.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2781583c0 con 0x7fd25c005220 2024-09-11T12:55:12.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2780d1210 con 0x7fd25c005220 2024-09-11T12:55:12.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fd268004050 con 0x7fd25c005220 2024-09-11T12:55:12.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd26f400640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 0x7fd258073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:12.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7fd268093be0 con 0x7fd25c005220 2024-09-11T12:55:12.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.967+0000 7fd276000640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 0x7fd258073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:12.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.971+0000 7fd276000640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 0x7fd258073f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd264001060 tx=0x7fd264000f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:12.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:12.971+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd26805ede0 con 0x7fd25c005220 2024-09-11T12:55:13.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.139+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd27815e590 con 0x7fd25c005220 2024-09-11T12:55:13.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.139+0000 7fd26f400640 1 -- 172.21.15.78:0/1463172029 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16752 (secure 0 0 0) 0x7fd268063330 con 0x7fd25c005220 2024-09-11T12:55:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:13.143 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":115,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:12.153470+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:09.848106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:11.124742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:13.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 msgr2=0x7fd258073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 0x7fd258073f60 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd264001060 tx=0x7fd264000f90 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 msgr2=0x7fd278155d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 0x7fd278155d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd268011ae0 tx=0x7fd26800f3c0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 shutdown_connections 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2781f24c0 0x7fd2781f2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd27813b180 0x7fd27813b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd258071aa0 0x7fd258073f60 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 --2- 172.21.15.78:0/1463172029 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd25c005220 0x7fd278155d90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 >> 172.21.15.78:0/1463172029 conn(0x7fd278074b90 msgr2=0x7fd27815eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 shutdown_connections 2024-09-11T12:55:13.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.143+0000 7fd27de00640 1 -- 172.21.15.78:0/1463172029 wait complete. 2024-09-11T12:55:13.165 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 -- 172.21.15.78:0/987238146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4073ed0 msgr2=0x7f8cf415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 --2- 172.21.15.78:0/987238146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415af10 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f8cf4070bd0 tx=0x7f8ce801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 -- 172.21.15.78:0/987238146 shutdown_connections 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 --2- 172.21.15.78:0/987238146 >> v2:172.21.15.157:3300/0 conn(0x7f8cf40b4db0 0x7f8cf40b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 --2- 172.21.15.78:0/987238146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415af10 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 -- 172.21.15.78:0/987238146 >> 172.21.15.78:0/987238146 conn(0x7f8cf406ca80 msgr2=0x7f8cf40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 -- 172.21.15.78:0/987238146 shutdown_connections 2024-09-11T12:55:13.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.303+0000 7f8cfc000640 1 -- 172.21.15.78:0/987238146 wait complete. 2024-09-11T12:55:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 Processor -- start 2024-09-11T12:55:13.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- start start 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cf40707f0 0x7f8cf40adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf4072530 con 0x7f8cf40b4db0 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf40723b0 con 0x7f8cf4073ed0 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf40726b0 con 0x7f8cf40707f0 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53680/0 (socket says 172.21.15.78:53680) 2024-09-11T12:55:13.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 -- 172.21.15.78:0/2319064825 learned_addr learned my addr 172.21.15.78:0/2319064825 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf9600640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cf40707f0 0x7f8cf40adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf8c00640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 -- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cf40707f0 msgr2=0x7f8cf40adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cf40707f0 0x7f8cf40adc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 -- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 msgr2=0x7f8cf415c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 -- 172.21.15.78:0/2319064825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf41f8100 con 0x7f8cf40b4db0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf8c00640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfa000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00ea20 tx=0x7f8cec00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00cde0 con 0x7f8cf40b4db0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cec004c40 con 0x7f8cf40b4db0 2024-09-11T12:55:13.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cec005280 con 0x7f8cf40b4db0 2024-09-11T12:55:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf41f83e0 con 0x7f8cf40b4db0 2024-09-11T12:55:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf4149430 con 0x7f8cf40b4db0 2024-09-11T12:55:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.307+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf415aef0 con 0x7f8cf40b4db0 2024-09-11T12:55:13.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.311+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f8cec005420 con 0x7f8cf40b4db0 2024-09-11T12:55:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.311+0000 7f8cf1600640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 0x7f8cd4073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.311+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f8cec092ec0 con 0x7f8cf40b4db0 2024-09-11T12:55:13.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.311+0000 7f8cf9600640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 0x7f8cd4073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.311+0000 7f8cf9600640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 0x7f8cd4073f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80af000 tx=0x7f8ce801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:13.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.315+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8cec016030 con 0x7f8cf40b4db0 2024-09-11T12:55:13.483 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.479+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8cf40b3970 con 0x7f8cf40b4db0 2024-09-11T12:55:13.484 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cf1600640 1 -- 172.21.15.78:0/2319064825 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) ==== 74+0+16752 (secure 0 0 0) 0x7f8cec05e690 con 0x7f8cf40b4db0 2024-09-11T12:55:13.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:13.484 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":115,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:12.153470+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:09.848106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:11.124742+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 msgr2=0x7f8cd4073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.486 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 0x7f8cd4073f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8ce80af000 tx=0x7f8ce801b100 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 msgr2=0x7f8cf40cecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00ea20 tx=0x7f8cec00eee0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 shutdown_connections 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8cf40b4db0 0x7f8cf40cecf0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8cd4071a70 0x7f8cd4073f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8cf4073ed0 0x7f8cf415c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 --2- 172.21.15.78:0/2319064825 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8cf40707f0 0x7f8cf40adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 >> 172.21.15.78:0/2319064825 conn(0x7f8cf406ca80 msgr2=0x7f8cf4158100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 shutdown_connections 2024-09-11T12:55:13.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.483+0000 7f8cfc000640 1 -- 172.21.15.78:0/2319064825 wait complete. 2024-09-11T12:55:13.506 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:55:13.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- 172.21.15.78:0/2313780168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 msgr2=0x7f026015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 --2- 172.21.15.78:0/2313780168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f026015a700 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f026006d1a0 tx=0x7f024c01c7d0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- 172.21.15.78:0/2313780168 shutdown_connections 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 --2- 172.21.15.78:0/2313780168 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f026015a700 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 --2- 172.21.15.78:0/2313780168 >> v2:172.21.15.157:3300/0 conn(0x7f02600b3040 0x7f02600cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- 172.21.15.78:0/2313780168 >> 172.21.15.78:0/2313780168 conn(0x7f0260071dd0 msgr2=0x7f02600721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- 172.21.15.78:0/2313780168 shutdown_connections 2024-09-11T12:55:13.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- 172.21.15.78:0/2313780168 wait complete. 2024-09-11T12:55:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 Processor -- start 2024-09-11T12:55:13.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.647+0000 7f0265800640 1 -- start start 2024-09-11T12:55:13.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 0x7f026013f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02600cd770 0x7f026013ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f02600af0f0 con 0x7f02600b3420 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f02600aef70 con 0x7f02600cd770 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f02600af270 con 0x7f02600b3040 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:53686/0 (socket says 172.21.15.78:53686) 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 -- 172.21.15.78:0/135940325 learned_addr learned my addr 172.21.15.78:0/135940325 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025ea00640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 0x7f026013f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025f400640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02600cd770 0x7f026013ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 -- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 msgr2=0x7f026013f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 0x7f026013f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 -- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02600cd770 msgr2=0x7f026013ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:13.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02600cd770 0x7f026013ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 -- 172.21.15.78:0/135940325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026013f230 con 0x7f02600b3420 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025ea00640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 0x7f026013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f025e000640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f024c000c10 tx=0x7f024c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c0b2040 con 0x7f02600b3420 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f024c004430 con 0x7f02600b3420 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0260143910 con 0x7f02600b3420 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f024c003820 con 0x7f02600b3420 2024-09-11T12:55:13.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0260157190 con 0x7f02600b3420 2024-09-11T12:55:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02600cf6e0 con 0x7f02600b3420 2024-09-11T12:55:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.651+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f024c0039c0 con 0x7f02600b3420 2024-09-11T12:55:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.655+0000 7f0256a00640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 0x7f0240073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:13.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.655+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5709+0+0 (secure 0 0 0) 0x7f024c12c770 con 0x7f02600b3420 2024-09-11T12:55:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.655+0000 7f025ea00640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 0x7f0240073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:13.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.655+0000 7f025ea00640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 0x7f0240073f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0260142e30 tx=0x7f0250009520 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:13.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.659+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f024c0163f0 con 0x7f02600b3420 2024-09-11T12:55:13.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:13.863+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f026013fbd0 con 0x7f02600b3420 2024-09-11T12:55:14.750 INFO:tasks.ceph.mds.b.smithi157.stderr:2024-09-11T12:55:14.749+0000 7f3aafc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:55:14.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.787+0000 7f0256a00640 1 -- 172.21.15.78:0/135940325 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v80) ==== 118+0+0 (secure 0 0 0) 0x7f024c0f80b0 con 0x7f02600b3420 2024-09-11T12:55:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 msgr2=0x7f0240073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 0x7f0240073f60 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0260142e30 tx=0x7f0250009520 comp rx=0 tx=0).stop 2024-09-11T12:55:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 msgr2=0x7f0260157990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:14.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f024c000c10 tx=0x7f024c004040 comp rx=0 tx=0).stop 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 shutdown_connections 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0240071aa0 0x7f0240073f60 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f02600cd770 0x7f026013ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f02600b3420 0x7f0260157990 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 --2- 172.21.15.78:0/135940325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f02600b3040 0x7f026013f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 >> 172.21.15.78:0/135940325 conn(0x7f0260071dd0 msgr2=0x7f0260070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 shutdown_connections 2024-09-11T12:55:14.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.791+0000 7f0265800640 1 -- 172.21.15.78:0/135940325 wait complete. 2024-09-11T12:55:14.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:55:14.818 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- 172.21.15.78:0/1825445805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 msgr2=0x7ff36c0b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 --2- 172.21.15.78:0/1825445805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c0b1a10 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff360009c80 tx=0x7ff36001cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- 172.21.15.78:0/1825445805 shutdown_connections 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 --2- 172.21.15.78:0/1825445805 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0b1f50 0x7ff36c0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 --2- 172.21.15.78:0/1825445805 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c0b1a10 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- 172.21.15.78:0/1825445805 >> 172.21.15.78:0/1825445805 conn(0x7ff36c071dd0 msgr2=0x7ff36c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:14.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- 172.21.15.78:0/1825445805 shutdown_connections 2024-09-11T12:55:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- 172.21.15.78:0/1825445805 wait complete. 2024-09-11T12:55:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 Processor -- start 2024-09-11T12:55:14.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.963+0000 7ff374000640 1 -- start start 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff36c0b1f50 0x7ff36c141960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0ce150 0x7ff36c1542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c167960 con 0x7ff36c1518b0 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c1677e0 con 0x7ff36c0b1f50 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff36c167ae0 con 0x7ff36c0ce150 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:14.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57798/0 (socket says 172.21.15.78:57798) 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 -- 172.21.15.78:0/649650235 learned_addr learned my addr 172.21.15.78:0/649650235 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff371600640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff36c0b1f50 0x7ff36c141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff370c00640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0ce150 0x7ff36c1542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 -- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0ce150 msgr2=0x7ff36c1542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0ce150 0x7ff36c1542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 -- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff36c0b1f50 msgr2=0x7ff36c141960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff36c0b1f50 0x7ff36c141960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 -- 172.21.15.78:0/649650235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c141ea0 con 0x7ff36c1518b0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff372000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c3f0 tx=0x7ff36400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364017070 con 0x7ff36c1518b0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3640053c0 con 0x7ff36c1518b0 2024-09-11T12:55:14.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364003860 con 0x7ff36c1518b0 2024-09-11T12:55:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36c154bd0 con 0x7ff36c1518b0 2024-09-11T12:55:14.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36c14deb0 con 0x7ff36c1518b0 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.967+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36c0cdc10 con 0x7ff36c1518b0 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.971+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7ff36401a070 con 0x7ff36c1518b0 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.971+0000 7ff369600640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 0x7ff34c073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.971+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7ff364091870 con 0x7ff36c1518b0 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.971+0000 7ff371600640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 0x7ff34c073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:14.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.971+0000 7ff371600640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 0x7ff34c073f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3600af000 tx=0x7ff36001b5e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:14.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:14.975+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff36405d150 con 0x7ff36c1518b0 2024-09-11T12:55:15.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.171+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ff36c15db00 con 0x7ff36c1518b0 2024-09-11T12:55:15.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.787+0000 7ff369600640 1 -- 172.21.15.78:0/649650235 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v81) ==== 122+0+0 (secure 0 0 0) 0x7ff364065390 con 0x7ff36c1518b0 2024-09-11T12:55:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 msgr2=0x7ff34c073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 0x7ff34c073f30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff3600af000 tx=0x7ff36001b5e0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 msgr2=0x7ff36c1465b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff36400c3f0 tx=0x7ff36400c8b0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 shutdown_connections 2024-09-11T12:55:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff36c1518b0 0x7ff36c1465b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff36c0ce150 0x7ff36c1542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff34c071a70 0x7ff34c073f30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 --2- 172.21.15.78:0/649650235 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff36c0b1f50 0x7ff36c141960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 >> 172.21.15.78:0/649650235 conn(0x7ff36c071dd0 msgr2=0x7ff36c15d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 shutdown_connections 2024-09-11T12:55:15.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.791+0000 7ff374000640 1 -- 172.21.15.78:0/649650235 wait complete. 2024-09-11T12:55:15.816 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:55:15.816 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:15.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 -- 172.21.15.78:0/744129282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 msgr2=0x7f08080736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:15.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 --2- 172.21.15.78:0/744129282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f08080736d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f080000ad90 tx=0x7f080001dc20 comp rx=0 tx=0).stop 2024-09-11T12:55:15.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 -- 172.21.15.78:0/744129282 shutdown_connections 2024-09-11T12:55:15.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 --2- 172.21.15.78:0/744129282 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f08080736d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 -- 172.21.15.78:0/744129282 >> 172.21.15.78:0/744129282 conn(0x7f080806ca80 msgr2=0x7f08080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:15.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 -- 172.21.15.78:0/744129282 shutdown_connections 2024-09-11T12:55:15.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.955+0000 7f080f800640 1 -- 172.21.15.78:0/744129282 wait complete. 2024-09-11T12:55:15.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 Processor -- start 2024-09-11T12:55:15.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- start start 2024-09-11T12:55:15.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808070a00 0x7f0808136240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:15.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0808157ae0 0x7f08081553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0808167f20 con 0x7f08080b54c0 2024-09-11T12:55:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f080806f700 con 0x7f0808070a00 2024-09-11T12:55:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08081680a0 con 0x7f0808157ae0 2024-09-11T12:55:15.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57804/0 (socket says 172.21.15.78:57804) 2024-09-11T12:55:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 -- 172.21.15.78:0/2940244635 learned_addr learned my addr 172.21.15.78:0/2940244635 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080ce00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808070a00 0x7f0808136240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080d800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0808157ae0 0x7f08081553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:15.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 -- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0808157ae0 msgr2=0x7f08081553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0808157ae0 0x7f08081553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 -- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808070a00 msgr2=0x7f0808136240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808070a00 0x7f0808136240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 -- 172.21.15.78:0/2940244635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080815e5c0 con 0x7f08080b54c0 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f07ffe00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c450 tx=0x7f07f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:15.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0012830 con 0x7f08080b54c0 2024-09-11T12:55:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07f0017070 con 0x7f08080b54c0 2024-09-11T12:55:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08081f8e40 con 0x7f08080b54c0 2024-09-11T12:55:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f0012d80 con 0x7f08080b54c0 2024-09-11T12:55:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08081f9400 con 0x7f08080b54c0 2024-09-11T12:55:15.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.959+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08080736d0 con 0x7f08080b54c0 2024-09-11T12:55:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.963+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f07f001c460 con 0x7f08080b54c0 2024-09-11T12:55:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.963+0000 7f0805600640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 0x7f07e4073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.963+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f07f00922a0 con 0x7f08080b54c0 2024-09-11T12:55:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.963+0000 7f080ce00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 0x7f07e4073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:15.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.963+0000 7f080ce00640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 0x7f07e4073f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000f70 tx=0x7f07f8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:15.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:15.967+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f07f005dbc0 con 0x7f08080b54c0 2024-09-11T12:55:16.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.163+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08081523d0 con 0x7f08080b54c0 2024-09-11T12:55:16.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.163+0000 7f0805600640 1 -- 172.21.15.78:0/2940244635 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5231 (secure 0 0 0) 0x7f07f005d4c0 con 0x7f08080b54c0 2024-09-11T12:55:16.167 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:16.168 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"btime":"2024-09-11T12:55:15:790926+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:16.168 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 81 2024-09-11T12:55:16.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 msgr2=0x7f07e4073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 0x7f07e4073f30 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f07f8000f70 tx=0x7f07f8009040 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 msgr2=0x7f080815e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f07f000c450 tx=0x7f07f000c910 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 shutdown_connections 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0808157ae0 0x7f08081553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08080b54c0 0x7f080815e080 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f07e4071a70 0x7f07e4073f30 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 --2- 172.21.15.78:0/2940244635 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0808070a00 0x7f0808136240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 >> 172.21.15.78:0/2940244635 conn(0x7f080806ca80 msgr2=0x7f0808072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:16.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 shutdown_connections 2024-09-11T12:55:16.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.167+0000 7f080f800640 1 -- 172.21.15.78:0/2940244635 wait complete. 2024-09-11T12:55:16.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 -- 172.21.15.78:0/1187381361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b80d15d0 msgr2=0x7f26b815ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 --2- 172.21.15.78:0/1187381361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b80d15d0 0x7f26b815ca00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f26b0009bc0 tx=0x7f26b001cb70 comp rx=0 tx=0).stop 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 -- 172.21.15.78:0/1187381361 shutdown_connections 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 --2- 172.21.15.78:0/1187381361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b80d15d0 0x7f26b815ca00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 --2- 172.21.15.78:0/1187381361 >> v2:172.21.15.157:3300/0 conn(0x7f26b815c1d0 0x7f26b806e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 -- 172.21.15.78:0/1187381361 >> 172.21.15.78:0/1187381361 conn(0x7f26b80ceab0 msgr2=0x7f26b80ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 -- 172.21.15.78:0/1187381361 shutdown_connections 2024-09-11T12:55:16.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.327+0000 7f26bfa00640 1 -- 172.21.15.78:0/1187381361 wait complete. 2024-09-11T12:55:16.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 Processor -- start 2024-09-11T12:55:16.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- start start 2024-09-11T12:55:16.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b80d15d0 0x7f26b8155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b815c1d0 0x7f26b814b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f26b80a9d10 con 0x7f26b815c5b0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f26b80a9b90 con 0x7f26b80d15d0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f26b80a9e90 con 0x7f26b815c1d0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57818/0 (socket says 172.21.15.78:57818) 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 -- 172.21.15.78:0/994635081 learned_addr learned my addr 172.21.15.78:0/994635081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bd000640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b80d15d0 0x7f26b8155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26afe00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b815c1d0 0x7f26b814b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 -- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b815c1d0 msgr2=0x7f26b814b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b815c1d0 0x7f26b814b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 -- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b80d15d0 msgr2=0x7f26b8155d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b80d15d0 0x7f26b8155d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 -- 172.21.15.78:0/994635081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b814a520 con 0x7f26b815c5b0 2024-09-11T12:55:16.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bda00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b0040 tx=0x7f26b0005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b00b2040 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26b0004040 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b814b7d0 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26b0004470 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.331+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b80accf0 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b815ca00 con 0x7f26b815c5b0 2024-09-11T12:55:16.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f26b00bd030 con 0x7f26b815c5b0 2024-09-11T12:55:16.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26b5600640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 0x7f2694073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f26b012c1e0 con 0x7f26b815c5b0 2024-09-11T12:55:16.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26bd000640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 0x7f2694073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.335+0000 7f26bd000640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 0x7f2694073f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009be0 tx=0x7f26a80091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:16.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.339+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f26b0016790 con 0x7f26b815c5b0 2024-09-11T12:55:16.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.507+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f26b815b790 con 0x7f26b815c5b0 2024-09-11T12:55:16.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26b5600640 1 -- 172.21.15.78:0/994635081 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) ==== 74+0+16739 (secure 0 0 0) 0x7f26b00f7430 con 0x7f26b815c5b0 2024-09-11T12:55:16.512 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:16.513 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":116,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:13.778187+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:09.848106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:11.124742+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 msgr2=0x7f2694073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 0x7f2694073f30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009be0 tx=0x7f26a80091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 msgr2=0x7f26b8149fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f26b00b0040 tx=0x7f26b0005a80 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 shutdown_connections 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26b815c5b0 0x7f26b8149fe0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f26b815c1d0 0x7f26b814b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2694071a70 0x7f2694073f30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 --2- 172.21.15.78:0/994635081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26b80d15d0 0x7f26b8155d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 >> 172.21.15.78:0/994635081 conn(0x7f26b80ceab0 msgr2=0x7f26b80b2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 shutdown_connections 2024-09-11T12:55:16.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.511+0000 7f26bfa00640 1 -- 172.21.15.78:0/994635081 wait complete. 2024-09-11T12:55:16.534 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:55:16.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:55:16.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- 172.21.15.78:0/2411767527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840aed20 msgr2=0x7f5b840cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 --2- 172.21.15.78:0/2411767527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840aed20 0x7f5b840cf280 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f5b74009c80 tx=0x7f5b7401cce0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- 172.21.15.78:0/2411767527 shutdown_connections 2024-09-11T12:55:16.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 --2- 172.21.15.78:0/2411767527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840aed20 0x7f5b840cf280 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- 172.21.15.78:0/2411767527 >> 172.21.15.78:0/2411767527 conn(0x7f5b8406ca80 msgr2=0x7f5b840b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:16.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- 172.21.15.78:0/2411767527 shutdown_connections 2024-09-11T12:55:16.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- 172.21.15.78:0/2411767527 wait complete. 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 Processor -- start 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- start start 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 0x7f5b84138f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b8415d1e0 0x7f5b84138340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840cd460 con 0x7f5b840cf7c0 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b840cd2e0 con 0x7f5b840aed20 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b8b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b840cd5e0 con 0x7f5b8415d1e0 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57836/0 (socket says 172.21.15.78:57836) 2024-09-11T12:55:16.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 -- 172.21.15.78:0/3693412016 learned_addr learned my addr 172.21.15.78:0/3693412016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83e00640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 0x7f5b84138f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b89200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b8415d1e0 0x7f5b84138340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 -- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b8415d1e0 msgr2=0x7f5b84138340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b8415d1e0 0x7f5b84138340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 -- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 msgr2=0x7f5b84138f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 0x7f5b84138f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 -- 172.21.15.78:0/3693412016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b840aaf60 con 0x7f5b840cf7c0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83e00640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 0x7f5b84138f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b83400640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c4b0 tx=0x7f5b7000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.675+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b700052d0 con 0x7f5b840cf7c0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b70005470 con 0x7f5b840cf7c0 2024-09-11T12:55:16.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b70012870 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b841363e0 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b84136680 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b840cdb20 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f5b70003600 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b7be00640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 0x7f5b64073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f5b70092100 con 0x7f5b840cf7c0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b83e00640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 0x7f5b64073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:16.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.679+0000 7f5b83e00640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 0x7f5b64073f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b740af000 tx=0x7f5b7401b520 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:16.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.683+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5b7005d9e0 con 0x7f5b840cf7c0 2024-09-11T12:55:16.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.847+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b8414de40 con 0x7f5b64071a70 2024-09-11T12:55:16.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.851+0000 7f5b7be00640 1 -- 172.21.15.78:0/3693412016 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f5b8414de40 con 0x7f5b64071a70 2024-09-11T12:55:16.858 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:16.859 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:55:16.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 msgr2=0x7f5b64073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 0x7f5b64073f30 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5b740af000 tx=0x7f5b7401b520 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 msgr2=0x7f5b840cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f5b7000c4b0 tx=0x7f5b7000c970 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 shutdown_connections 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b8415d1e0 0x7f5b84138340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b840cf7c0 0x7f5b840cbb20 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b64071a70 0x7f5b64073f30 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 --2- 172.21.15.78:0/3693412016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b840aed20 0x7f5b84138f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 >> 172.21.15.78:0/3693412016 conn(0x7f5b8406ca80 msgr2=0x7f5b8406fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 shutdown_connections 2024-09-11T12:55:16.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:16.855+0000 7f5b8b200640 1 -- 172.21.15.78:0/3693412016 wait complete. 2024-09-11T12:55:16.879 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-11T12:55:16.490021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261972,"kb_used_data":12452,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712748,"statfs":{"total":773094113280,"available":772825853952,"internally_reserved":0,"allocated":12750848,"data_stored":6218704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737657"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788585+0000","last_change":"2024-09-11T12:55:12.796965+0000","last_active":"2024-09-11T12:55:13.788585+0000","last_peered":"2024-09-11T12:55:13.788585+0000","last_clean":"2024-09-11T12:55:13.788585+0000","last_became_active":"2024-09-11T12:55:12.796665+0000","last_became_peered":"2024-09-11T12:55:12.796665+0000","last_unstale":"2024-09-11T12:55:13.788585+0000","last_undegraded":"2024-09-11T12:55:13.788585+0000","last_fullsized":"2024-09-11T12:55:13.788585+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:34.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804356+0000","last_change":"2024-09-11T12:55:12.803718+0000","last_active":"2024-09-11T12:55:14.804356+0000","last_peered":"2024-09-11T12:55:14.804356+0000","last_clean":"2024-09-11T12:55:14.804356+0000","last_became_active":"2024-09-11T12:55:12.803470+0000","last_became_peered":"2024-09-11T12:55:12.803470+0000","last_unstale":"2024-09-11T12:55:14.804356+0000","last_undegraded":"2024-09-11T12:55:14.804356+0000","last_fullsized":"2024-09-11T12:55:14.804356+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:59.307681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804019+0000","last_change":"2024-09-11T12:55:12.799312+0000","last_active":"2024-09-11T12:55:14.804019+0000","last_peered":"2024-09-11T12:55:14.804019+0000","last_clean":"2024-09-11T12:55:14.804019+0000","last_became_active":"2024-09-11T12:55:12.798868+0000","last_became_peered":"2024-09-11T12:55:12.798868+0000","last_unstale":"2024-09-11T12:55:14.804019+0000","last_undegraded":"2024-09-11T12:55:14.804019+0000","last_fullsized":"2024-09-11T12:55:14.804019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:36.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787998+0000","last_change":"2024-09-11T12:55:12.792663+0000","last_active":"2024-09-11T12:55:13.787998+0000","last_peered":"2024-09-11T12:55:13.787998+0000","last_clean":"2024-09-11T12:55:13.787998+0000","last_became_active":"2024-09-11T12:55:12.792265+0000","last_became_peered":"2024-09-11T12:55:12.792265+0000","last_unstale":"2024-09-11T12:55:13.787998+0000","last_undegraded":"2024-09-11T12:55:13.787998+0000","last_fullsized":"2024-09-11T12:55:13.787998+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:24:58.043614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794622+0000","last_change":"2024-09-11T12:55:12.801516+0000","last_active":"2024-09-11T12:55:13.794622+0000","last_peered":"2024-09-11T12:55:13.794622+0000","last_clean":"2024-09-11T12:55:13.794622+0000","last_became_active":"2024-09-11T12:55:12.801236+0000","last_became_peered":"2024-09-11T12:55:12.801236+0000","last_unstale":"2024-09-11T12:55:13.794622+0000","last_undegraded":"2024-09-11T12:55:13.794622+0000","last_fullsized":"2024-09-11T12:55:13.794622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:23:46.106790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790235+0000","last_change":"2024-09-11T12:55:12.797396+0000","last_active":"2024-09-11T12:55:13.790235+0000","last_peered":"2024-09-11T12:55:13.790235+0000","last_clean":"2024-09-11T12:55:13.790235+0000","last_became_active":"2024-09-11T12:55:12.793134+0000","last_became_peered":"2024-09-11T12:55:12.793134+0000","last_unstale":"2024-09-11T12:55:13.790235+0000","last_undegraded":"2024-09-11T12:55:13.790235+0000","last_fullsized":"2024-09-11T12:55:13.790235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:36.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789712+0000","last_change":"2024-09-11T12:55:12.789830+0000","last_active":"2024-09-11T12:55:13.789712+0000","last_peered":"2024-09-11T12:55:13.789712+0000","last_clean":"2024-09-11T12:55:13.789712+0000","last_became_active":"2024-09-11T12:55:12.789557+0000","last_became_peered":"2024-09-11T12:55:12.789557+0000","last_unstale":"2024-09-11T12:55:13.789712+0000","last_undegraded":"2024-09-11T12:55:13.789712+0000","last_fullsized":"2024-09-11T12:55:13.789712+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:51.095861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803827+0000","last_change":"2024-09-11T12:55:12.795350+0000","last_active":"2024-09-11T12:55:14.803827+0000","last_peered":"2024-09-11T12:55:14.803827+0000","last_clean":"2024-09-11T12:55:14.803827+0000","last_became_active":"2024-09-11T12:55:12.794743+0000","last_became_peered":"2024-09-11T12:55:12.794743+0000","last_unstale":"2024-09-11T12:55:14.803827+0000","last_undegraded":"2024-09-11T12:55:14.803827+0000","last_fullsized":"2024-09-11T12:55:14.803827+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:51.095861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804267+0000","last_change":"2024-09-11T12:55:12.806463+0000","last_active":"2024-09-11T12:55:14.804267+0000","last_peered":"2024-09-11T12:55:14.804267+0000","last_clean":"2024-09-11T12:55:14.804267+0000","last_became_active":"2024-09-11T12:55:12.806161+0000","last_became_peered":"2024-09-11T12:55:12.806161+0000","last_unstale":"2024-09-11T12:55:14.804267+0000","last_undegraded":"2024-09-11T12:55:14.804267+0000","last_fullsized":"2024-09-11T12:55:14.804267+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804533+0000","last_change":"2024-09-11T12:55:12.803570+0000","last_active":"2024-09-11T12:55:14.804533+0000","last_peered":"2024-09-11T12:55:14.804533+0000","last_clean":"2024-09-11T12:55:14.804533+0000","last_became_active":"2024-09-11T12:55:12.803273+0000","last_became_peered":"2024-09-11T12:55:12.803273+0000","last_unstale":"2024-09-11T12:55:14.804533+0000","last_undegraded":"2024-09-11T12:55:14.804533+0000","last_fullsized":"2024-09-11T12:55:14.804533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:57.011365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804042+0000","last_change":"2024-09-11T12:55:12.790631+0000","last_active":"2024-09-11T12:55:14.804042+0000","last_peered":"2024-09-11T12:55:14.804042+0000","last_clean":"2024-09-11T12:55:14.804042+0000","last_became_active":"2024-09-11T12:55:12.788939+0000","last_became_peered":"2024-09-11T12:55:12.788939+0000","last_unstale":"2024-09-11T12:55:14.804042+0000","last_undegraded":"2024-09-11T12:55:14.804042+0000","last_fullsized":"2024-09-11T12:55:14.804042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:48.102689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803894+0000","last_change":"2024-09-11T12:55:12.800517+0000","last_active":"2024-09-11T12:55:14.803894+0000","last_peered":"2024-09-11T12:55:14.803894+0000","last_clean":"2024-09-11T12:55:14.803894+0000","last_became_active":"2024-09-11T12:55:12.800179+0000","last_became_peered":"2024-09-11T12:55:12.800179+0000","last_unstale":"2024-09-11T12:55:14.803894+0000","last_undegraded":"2024-09-11T12:55:14.803894+0000","last_fullsized":"2024-09-11T12:55:14.803894+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:25:45.314239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787677+0000","last_change":"2024-09-11T12:55:12.783195+0000","last_active":"2024-09-11T12:55:13.787677+0000","last_peered":"2024-09-11T12:55:13.787677+0000","last_clean":"2024-09-11T12:55:13.787677+0000","last_became_active":"2024-09-11T12:55:12.782924+0000","last_became_peered":"2024-09-11T12:55:12.782924+0000","last_unstale":"2024-09-11T12:55:13.787677+0000","last_undegraded":"2024-09-11T12:55:13.787677+0000","last_fullsized":"2024-09-11T12:55:13.787677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:28:13.412999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804850+0000","last_change":"2024-09-11T12:55:12.800628+0000","last_active":"2024-09-11T12:55:14.804850+0000","last_peered":"2024-09-11T12:55:14.804850+0000","last_clean":"2024-09-11T12:55:14.804850+0000","last_became_active":"2024-09-11T12:55:12.800307+0000","last_became_peered":"2024-09-11T12:55:12.800307+0000","last_unstale":"2024-09-11T12:55:14.804850+0000","last_undegraded":"2024-09-11T12:55:14.804850+0000","last_fullsized":"2024-09-11T12:55:14.804850+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:53.098319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787621+0000","last_change":"2024-09-11T12:55:12.786548+0000","last_active":"2024-09-11T12:55:13.787621+0000","last_peered":"2024-09-11T12:55:13.787621+0000","last_clean":"2024-09-11T12:55:13.787621+0000","last_became_active":"2024-09-11T12:55:12.786272+0000","last_became_peered":"2024-09-11T12:55:12.786272+0000","last_unstale":"2024-09-11T12:55:13.787621+0000","last_undegraded":"2024-09-11T12:55:13.787621+0000","last_fullsized":"2024-09-11T12:55:13.787621+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:09.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804667+0000","last_change":"2024-09-11T12:55:12.801258+0000","last_active":"2024-09-11T12:55:14.804667+0000","last_peered":"2024-09-11T12:55:14.804667+0000","last_clean":"2024-09-11T12:55:14.804667+0000","last_became_active":"2024-09-11T12:55:12.800939+0000","last_became_peered":"2024-09-11T12:55:12.800939+0000","last_unstale":"2024-09-11T12:55:14.804667+0000","last_undegraded":"2024-09-11T12:55:14.804667+0000","last_fullsized":"2024-09-11T12:55:14.804667+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790418+0000","last_change":"2024-09-11T12:55:12.781745+0000","last_active":"2024-09-11T12:55:13.790418+0000","last_peered":"2024-09-11T12:55:13.790418+0000","last_clean":"2024-09-11T12:55:13.790418+0000","last_became_active":"2024-09-11T12:55:12.781393+0000","last_became_peered":"2024-09-11T12:55:12.781393+0000","last_unstale":"2024-09-11T12:55:13.790418+0000","last_undegraded":"2024-09-11T12:55:13.790418+0000","last_fullsized":"2024-09-11T12:55:13.790418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805286+0000","last_change":"2024-09-11T12:55:12.788781+0000","last_active":"2024-09-11T12:55:14.805286+0000","last_peered":"2024-09-11T12:55:14.805286+0000","last_clean":"2024-09-11T12:55:14.805286+0000","last_became_active":"2024-09-11T12:55:12.788004+0000","last_became_peered":"2024-09-11T12:55:12.788004+0000","last_unstale":"2024-09-11T12:55:14.805286+0000","last_undegraded":"2024-09-11T12:55:14.805286+0000","last_fullsized":"2024-09-11T12:55:14.805286+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:49.540650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804078+0000","last_change":"2024-09-11T12:55:12.790179+0000","last_active":"2024-09-11T12:55:14.804078+0000","last_peered":"2024-09-11T12:55:14.804078+0000","last_clean":"2024-09-11T12:55:14.804078+0000","last_became_active":"2024-09-11T12:55:12.789426+0000","last_became_peered":"2024-09-11T12:55:12.789426+0000","last_unstale":"2024-09-11T12:55:14.804078+0000","last_undegraded":"2024-09-11T12:55:14.804078+0000","last_fullsized":"2024-09-11T12:55:14.804078+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:12.619514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805108+0000","last_change":"2024-09-11T12:55:12.795384+0000","last_active":"2024-09-11T12:55:14.805108+0000","last_peered":"2024-09-11T12:55:14.805108+0000","last_clean":"2024-09-11T12:55:14.805108+0000","last_became_active":"2024-09-11T12:55:12.794112+0000","last_became_peered":"2024-09-11T12:55:12.794112+0000","last_unstale":"2024-09-11T12:55:14.805108+0000","last_undegraded":"2024-09-11T12:55:14.805108+0000","last_fullsized":"2024-09-11T12:55:14.805108+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:55.560900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794313+0000","last_change":"2024-09-11T12:55:12.801428+0000","last_active":"2024-09-11T12:55:13.794313+0000","last_peered":"2024-09-11T12:55:13.794313+0000","last_clean":"2024-09-11T12:55:13.794313+0000","last_became_active":"2024-09-11T12:55:12.800817+0000","last_became_peered":"2024-09-11T12:55:12.800817+0000","last_unstale":"2024-09-11T12:55:13.794313+0000","last_undegraded":"2024-09-11T12:55:13.794313+0000","last_fullsized":"2024-09-11T12:55:13.794313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:01.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803778+0000","last_change":"2024-09-11T12:55:12.796967+0000","last_active":"2024-09-11T12:55:14.803778+0000","last_peered":"2024-09-11T12:55:14.803778+0000","last_clean":"2024-09-11T12:55:14.803778+0000","last_became_active":"2024-09-11T12:55:12.796619+0000","last_became_peered":"2024-09-11T12:55:12.796619+0000","last_unstale":"2024-09-11T12:55:14.803778+0000","last_undegraded":"2024-09-11T12:55:14.803778+0000","last_fullsized":"2024-09-11T12:55:14.803778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:37.708319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803951+0000","last_change":"2024-09-11T12:55:12.799247+0000","last_active":"2024-09-11T12:55:14.803951+0000","last_peered":"2024-09-11T12:55:14.803951+0000","last_clean":"2024-09-11T12:55:14.803951+0000","last_became_active":"2024-09-11T12:55:12.799001+0000","last_became_peered":"2024-09-11T12:55:12.799001+0000","last_unstale":"2024-09-11T12:55:14.803951+0000","last_undegraded":"2024-09-11T12:55:14.803951+0000","last_fullsized":"2024-09-11T12:55:14.803951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:50.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803677+0000","last_change":"2024-09-11T12:55:12.790203+0000","last_active":"2024-09-11T12:55:14.803677+0000","last_peered":"2024-09-11T12:55:14.803677+0000","last_clean":"2024-09-11T12:55:14.803677+0000","last_became_active":"2024-09-11T12:55:12.789937+0000","last_became_peered":"2024-09-11T12:55:12.789937+0000","last_unstale":"2024-09-11T12:55:14.803677+0000","last_undegraded":"2024-09-11T12:55:14.803677+0000","last_fullsized":"2024-09-11T12:55:14.803677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:05.662510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804221+0000","last_change":"2024-09-11T12:55:12.796528+0000","last_active":"2024-09-11T12:55:14.804221+0000","last_peered":"2024-09-11T12:55:14.804221+0000","last_clean":"2024-09-11T12:55:14.804221+0000","last_became_active":"2024-09-11T12:55:12.796183+0000","last_became_peered":"2024-09-11T12:55:12.796183+0000","last_unstale":"2024-09-11T12:55:14.804221+0000","last_undegraded":"2024-09-11T12:55:14.804221+0000","last_fullsized":"2024-09-11T12:55:14.804221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:53.316378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805450+0000","last_change":"2024-09-11T12:55:12.788560+0000","last_active":"2024-09-11T12:55:14.805450+0000","last_peered":"2024-09-11T12:55:14.805450+0000","last_clean":"2024-09-11T12:55:14.805450+0000","last_became_active":"2024-09-11T12:55:12.787733+0000","last_became_peered":"2024-09-11T12:55:12.787733+0000","last_unstale":"2024-09-11T12:55:14.805450+0000","last_undegraded":"2024-09-11T12:55:14.805450+0000","last_fullsized":"2024-09-11T12:55:14.805450+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.076209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787909+0000","last_change":"2024-09-11T12:55:12.796145+0000","last_active":"2024-09-11T12:55:13.787909+0000","last_peered":"2024-09-11T12:55:13.787909+0000","last_clean":"2024-09-11T12:55:13.787909+0000","last_became_active":"2024-09-11T12:55:12.795836+0000","last_became_peered":"2024-09-11T12:55:12.795836+0000","last_unstale":"2024-09-11T12:55:13.787909+0000","last_undegraded":"2024-09-11T12:55:13.787909+0000","last_fullsized":"2024-09-11T12:55:13.787909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:53.249522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789313+0000","last_change":"2024-09-11T12:55:12.786899+0000","last_active":"2024-09-11T12:55:13.789313+0000","last_peered":"2024-09-11T12:55:13.789313+0000","last_clean":"2024-09-11T12:55:13.789313+0000","last_became_active":"2024-09-11T12:55:12.786620+0000","last_became_peered":"2024-09-11T12:55:12.786620+0000","last_unstale":"2024-09-11T12:55:13.789313+0000","last_undegraded":"2024-09-11T12:55:13.789313+0000","last_fullsized":"2024-09-11T12:55:13.789313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:16.809556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788405+0000","last_change":"2024-09-11T12:55:12.787777+0000","last_active":"2024-09-11T12:55:13.788405+0000","last_peered":"2024-09-11T12:55:13.788405+0000","last_clean":"2024-09-11T12:55:13.788405+0000","last_became_active":"2024-09-11T12:55:12.787442+0000","last_became_peered":"2024-09-11T12:55:12.787442+0000","last_unstale":"2024-09-11T12:55:13.788405+0000","last_undegraded":"2024-09-11T12:55:13.788405+0000","last_fullsized":"2024-09-11T12:55:13.788405+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:23:54.243869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787677+0000","last_change":"2024-09-11T12:55:12.788564+0000","last_active":"2024-09-11T12:55:13.787677+0000","last_peered":"2024-09-11T12:55:13.787677+0000","last_clean":"2024-09-11T12:55:13.787677+0000","last_became_active":"2024-09-11T12:55:12.788247+0000","last_became_peered":"2024-09-11T12:55:12.788247+0000","last_unstale":"2024-09-11T12:55:13.787677+0000","last_undegraded":"2024-09-11T12:55:13.787677+0000","last_fullsized":"2024-09-11T12:55:13.787677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:44.262619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794126+0000","last_change":"2024-09-11T12:55:12.801381+0000","last_active":"2024-09-11T12:55:13.794126+0000","last_peered":"2024-09-11T12:55:13.794126+0000","last_clean":"2024-09-11T12:55:13.794126+0000","last_became_active":"2024-09-11T12:55:12.800736+0000","last_became_peered":"2024-09-11T12:55:12.800736+0000","last_unstale":"2024-09-11T12:55:13.794126+0000","last_undegraded":"2024-09-11T12:55:13.794126+0000","last_fullsized":"2024-09-11T12:55:13.794126+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:00.009725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789837+0000","last_change":"2024-09-11T12:55:12.794456+0000","last_active":"2024-09-11T12:55:13.789837+0000","last_peered":"2024-09-11T12:55:13.789837+0000","last_clean":"2024-09-11T12:55:13.789837+0000","last_became_active":"2024-09-11T12:55:12.794158+0000","last_became_peered":"2024-09-11T12:55:12.794158+0000","last_unstale":"2024-09-11T12:55:13.789837+0000","last_undegraded":"2024-09-11T12:55:13.789837+0000","last_fullsized":"2024-09-11T12:55:13.789837+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:25:45.314239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805446+0000","last_change":"2024-09-11T12:55:12.788248+0000","last_active":"2024-09-11T12:55:14.805446+0000","last_peered":"2024-09-11T12:55:14.805446+0000","last_clean":"2024-09-11T12:55:14.805446+0000","last_became_active":"2024-09-11T12:55:12.787396+0000","last_became_peered":"2024-09-11T12:55:12.787396+0000","last_unstale":"2024-09-11T12:55:14.805446+0000","last_undegraded":"2024-09-11T12:55:14.805446+0000","last_fullsized":"2024-09-11T12:55:14.805446+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:47.522830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"115'4","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794705+0000","last_change":"2024-09-11T12:55:11.782536+0000","last_active":"2024-09-11T12:55:13.794705+0000","last_peered":"2024-09-11T12:55:13.794705+0000","last_clean":"2024-09-11T12:55:13.794705+0000","last_became_active":"2024-09-11T12:55:11.782176+0000","last_became_peered":"2024-09-11T12:55:11.782176+0000","last_unstale":"2024-09-11T12:55:13.794705+0000","last_undegraded":"2024-09-11T12:55:13.794705+0000","last_fullsized":"2024-09-11T12:55:13.794705+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:14.897024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804300+0000","last_change":"2024-09-11T12:55:12.801482+0000","last_active":"2024-09-11T12:55:14.804300+0000","last_peered":"2024-09-11T12:55:14.804300+0000","last_clean":"2024-09-11T12:55:14.804300+0000","last_became_active":"2024-09-11T12:55:12.801183+0000","last_became_peered":"2024-09-11T12:55:12.801183+0000","last_unstale":"2024-09-11T12:55:14.804300+0000","last_undegraded":"2024-09-11T12:55:14.804300+0000","last_fullsized":"2024-09-11T12:55:14.804300+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:38.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789160+0000","last_change":"2024-09-11T12:55:11.775261+0000","last_active":"2024-09-11T12:55:13.789160+0000","last_peered":"2024-09-11T12:55:13.789160+0000","last_clean":"2024-09-11T12:55:13.789160+0000","last_became_active":"2024-09-11T12:55:11.774990+0000","last_became_peered":"2024-09-11T12:55:11.774990+0000","last_unstale":"2024-09-11T12:55:13.789160+0000","last_undegraded":"2024-09-11T12:55:13.789160+0000","last_fullsized":"2024-09-11T12:55:13.789160+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:17.837536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794661+0000","last_change":"2024-09-11T12:55:12.802842+0000","last_active":"2024-09-11T12:55:13.794661+0000","last_peered":"2024-09-11T12:55:13.794661+0000","last_clean":"2024-09-11T12:55:13.794661+0000","last_became_active":"2024-09-11T12:55:12.802551+0000","last_became_peered":"2024-09-11T12:55:12.802551+0000","last_unstale":"2024-09-11T12:55:13.794661+0000","last_undegraded":"2024-09-11T12:55:13.794661+0000","last_fullsized":"2024-09-11T12:55:13.794661+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:53.513962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804141+0000","last_change":"2024-09-11T12:55:11.775364+0000","last_active":"2024-09-11T12:55:14.804141+0000","last_peered":"2024-09-11T12:55:14.804141+0000","last_clean":"2024-09-11T12:55:14.804141+0000","last_became_active":"2024-09-11T12:55:11.775083+0000","last_became_peered":"2024-09-11T12:55:11.775083+0000","last_unstale":"2024-09-11T12:55:14.804141+0000","last_undegraded":"2024-09-11T12:55:14.804141+0000","last_fullsized":"2024-09-11T12:55:14.804141+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:15.913964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794285+0000","last_change":"2024-09-11T12:55:12.796692+0000","last_active":"2024-09-11T12:55:13.794285+0000","last_peered":"2024-09-11T12:55:13.794285+0000","last_clean":"2024-09-11T12:55:13.794285+0000","last_became_active":"2024-09-11T12:55:12.796396+0000","last_became_peered":"2024-09-11T12:55:12.796396+0000","last_unstale":"2024-09-11T12:55:13.794285+0000","last_undegraded":"2024-09-11T12:55:13.794285+0000","last_fullsized":"2024-09-11T12:55:13.794285+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:56.151247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804422+0000","last_change":"2024-09-11T12:55:11.784018+0000","last_active":"2024-09-11T12:55:14.804422+0000","last_peered":"2024-09-11T12:55:14.804422+0000","last_clean":"2024-09-11T12:55:14.804422+0000","last_became_active":"2024-09-11T12:55:11.783332+0000","last_became_peered":"2024-09-11T12:55:11.783332+0000","last_unstale":"2024-09-11T12:55:14.804422+0000","last_undegraded":"2024-09-11T12:55:14.804422+0000","last_fullsized":"2024-09-11T12:55:14.804422+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:17.837536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804656+0000","last_change":"2024-09-11T12:55:12.789813+0000","last_active":"2024-09-11T12:55:14.804656+0000","last_peered":"2024-09-11T12:55:14.804656+0000","last_clean":"2024-09-11T12:55:14.804656+0000","last_became_active":"2024-09-11T12:55:12.789532+0000","last_became_peered":"2024-09-11T12:55:12.789532+0000","last_unstale":"2024-09-11T12:55:14.804656+0000","last_undegraded":"2024-09-11T12:55:14.804656+0000","last_fullsized":"2024-09-11T12:55:14.804656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804076+0000","last_change":"2024-09-11T12:55:11.782608+0000","last_active":"2024-09-11T12:55:14.804076+0000","last_peered":"2024-09-11T12:55:14.804076+0000","last_clean":"2024-09-11T12:55:14.804076+0000","last_became_active":"2024-09-11T12:55:11.782337+0000","last_became_peered":"2024-09-11T12:55:11.782337+0000","last_unstale":"2024-09-11T12:55:14.804076+0000","last_undegraded":"2024-09-11T12:55:14.804076+0000","last_fullsized":"2024-09-11T12:55:14.804076+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:51.901604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789711+0000","last_change":"2024-09-11T12:55:12.788083+0000","last_active":"2024-09-11T12:55:13.789711+0000","last_peered":"2024-09-11T12:55:13.789711+0000","last_clean":"2024-09-11T12:55:13.789711+0000","last_became_active":"2024-09-11T12:55:12.786243+0000","last_became_peered":"2024-09-11T12:55:12.786243+0000","last_unstale":"2024-09-11T12:55:13.789711+0000","last_undegraded":"2024-09-11T12:55:13.789711+0000","last_fullsized":"2024-09-11T12:55:13.789711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:15.903891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804780+0000","last_change":"2024-09-11T12:55:11.786588+0000","last_active":"2024-09-11T12:55:14.804780+0000","last_peered":"2024-09-11T12:55:14.804780+0000","last_clean":"2024-09-11T12:55:14.804780+0000","last_became_active":"2024-09-11T12:55:11.786301+0000","last_became_peered":"2024-09-11T12:55:11.786301+0000","last_unstale":"2024-09-11T12:55:14.804780+0000","last_undegraded":"2024-09-11T12:55:14.804780+0000","last_fullsized":"2024-09-11T12:55:14.804780+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:26.261895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804082+0000","last_change":"2024-09-11T12:55:12.795005+0000","last_active":"2024-09-11T12:55:14.804082+0000","last_peered":"2024-09-11T12:55:14.804082+0000","last_clean":"2024-09-11T12:55:14.804082+0000","last_became_active":"2024-09-11T12:55:12.794343+0000","last_became_peered":"2024-09-11T12:55:12.794343+0000","last_unstale":"2024-09-11T12:55:14.804082+0000","last_undegraded":"2024-09-11T12:55:14.804082+0000","last_fullsized":"2024-09-11T12:55:14.804082+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:49.781395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789519+0000","last_change":"2024-09-11T12:55:11.773630+0000","last_active":"2024-09-11T12:55:13.789519+0000","last_peered":"2024-09-11T12:55:13.789519+0000","last_clean":"2024-09-11T12:55:13.789519+0000","last_became_active":"2024-09-11T12:55:11.773220+0000","last_became_peered":"2024-09-11T12:55:11.773220+0000","last_unstale":"2024-09-11T12:55:13.789519+0000","last_undegraded":"2024-09-11T12:55:13.789519+0000","last_fullsized":"2024-09-11T12:55:13.789519+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:46.224193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803872+0000","last_change":"2024-09-11T12:55:12.798625+0000","last_active":"2024-09-11T12:55:14.803872+0000","last_peered":"2024-09-11T12:55:14.803872+0000","last_clean":"2024-09-11T12:55:14.803872+0000","last_became_active":"2024-09-11T12:55:12.798331+0000","last_became_peered":"2024-09-11T12:55:12.798331+0000","last_unstale":"2024-09-11T12:55:14.803872+0000","last_undegraded":"2024-09-11T12:55:14.803872+0000","last_fullsized":"2024-09-11T12:55:14.803872+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:56.301009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803829+0000","last_change":"2024-09-11T12:55:11.778942+0000","last_active":"2024-09-11T12:55:14.803829+0000","last_peered":"2024-09-11T12:55:14.803829+0000","last_clean":"2024-09-11T12:55:14.803829+0000","last_became_active":"2024-09-11T12:55:11.778674+0000","last_became_peered":"2024-09-11T12:55:11.778674+0000","last_unstale":"2024-09-11T12:55:14.803829+0000","last_undegraded":"2024-09-11T12:55:14.803829+0000","last_fullsized":"2024-09-11T12:55:14.803829+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:10.403675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788294+0000","last_change":"2024-09-11T12:55:12.783725+0000","last_active":"2024-09-11T12:55:13.788294+0000","last_peered":"2024-09-11T12:55:13.788294+0000","last_clean":"2024-09-11T12:55:13.788294+0000","last_became_active":"2024-09-11T12:55:12.782955+0000","last_became_peered":"2024-09-11T12:55:12.782955+0000","last_unstale":"2024-09-11T12:55:13.788294+0000","last_undegraded":"2024-09-11T12:55:13.788294+0000","last_fullsized":"2024-09-11T12:55:13.788294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:45:55.734191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790191+0000","last_change":"2024-09-11T12:55:11.770102+0000","last_active":"2024-09-11T12:55:13.790191+0000","last_peered":"2024-09-11T12:55:13.790191+0000","last_clean":"2024-09-11T12:55:13.790191+0000","last_became_active":"2024-09-11T12:55:11.769803+0000","last_became_peered":"2024-09-11T12:55:11.769803+0000","last_unstale":"2024-09-11T12:55:13.790191+0000","last_undegraded":"2024-09-11T12:55:13.790191+0000","last_fullsized":"2024-09-11T12:55:13.790191+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:51.901604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790068+0000","last_change":"2024-09-11T12:55:12.800640+0000","last_active":"2024-09-11T12:55:13.790068+0000","last_peered":"2024-09-11T12:55:13.790068+0000","last_clean":"2024-09-11T12:55:13.790068+0000","last_became_active":"2024-09-11T12:55:12.800342+0000","last_became_peered":"2024-09-11T12:55:12.800342+0000","last_unstale":"2024-09-11T12:55:13.790068+0000","last_undegraded":"2024-09-11T12:55:13.790068+0000","last_fullsized":"2024-09-11T12:55:13.790068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:24.238489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805526+0000","last_change":"2024-09-11T12:55:11.780460+0000","last_active":"2024-09-11T12:55:14.805526+0000","last_peered":"2024-09-11T12:55:14.805526+0000","last_clean":"2024-09-11T12:55:14.805526+0000","last_became_active":"2024-09-11T12:55:11.780159+0000","last_became_peered":"2024-09-11T12:55:11.780159+0000","last_unstale":"2024-09-11T12:55:14.805526+0000","last_undegraded":"2024-09-11T12:55:14.805526+0000","last_fullsized":"2024-09-11T12:55:14.805526+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:56.004498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794601+0000","last_change":"2024-09-11T12:55:12.806930+0000","last_active":"2024-09-11T12:55:13.794601+0000","last_peered":"2024-09-11T12:55:13.794601+0000","last_clean":"2024-09-11T12:55:13.794601+0000","last_became_active":"2024-09-11T12:55:12.806665+0000","last_became_peered":"2024-09-11T12:55:12.806665+0000","last_unstale":"2024-09-11T12:55:13.794601+0000","last_undegraded":"2024-09-11T12:55:13.794601+0000","last_fullsized":"2024-09-11T12:55:13.794601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:25.081626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"115'3","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790104+0000","last_change":"2024-09-11T12:55:11.770394+0000","last_active":"2024-09-11T12:55:13.790104+0000","last_peered":"2024-09-11T12:55:13.790104+0000","last_clean":"2024-09-11T12:55:13.790104+0000","last_became_active":"2024-09-11T12:55:11.769973+0000","last_became_peered":"2024-09-11T12:55:11.769973+0000","last_unstale":"2024-09-11T12:55:13.790104+0000","last_undegraded":"2024-09-11T12:55:13.790104+0000","last_fullsized":"2024-09-11T12:55:13.790104+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:09:31.545516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794517+0000","last_change":"2024-09-11T12:55:12.796980+0000","last_active":"2024-09-11T12:55:13.794517+0000","last_peered":"2024-09-11T12:55:13.794517+0000","last_clean":"2024-09-11T12:55:13.794517+0000","last_became_active":"2024-09-11T12:55:12.796697+0000","last_became_peered":"2024-09-11T12:55:12.796697+0000","last_unstale":"2024-09-11T12:55:13.794517+0000","last_undegraded":"2024-09-11T12:55:13.794517+0000","last_fullsized":"2024-09-11T12:55:13.794517+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:18.034475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803904+0000","last_change":"2024-09-11T12:55:11.776549+0000","last_active":"2024-09-11T12:55:14.803904+0000","last_peered":"2024-09-11T12:55:14.803904+0000","last_clean":"2024-09-11T12:55:14.803904+0000","last_became_active":"2024-09-11T12:55:11.776252+0000","last_became_peered":"2024-09-11T12:55:11.776252+0000","last_unstale":"2024-09-11T12:55:14.803904+0000","last_undegraded":"2024-09-11T12:55:14.803904+0000","last_fullsized":"2024-09-11T12:55:14.803904+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:46.224193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787832+0000","last_change":"2024-09-11T12:55:12.786332+0000","last_active":"2024-09-11T12:55:13.787832+0000","last_peered":"2024-09-11T12:55:13.787832+0000","last_clean":"2024-09-11T12:55:13.787832+0000","last_became_active":"2024-09-11T12:55:12.786098+0000","last_became_peered":"2024-09-11T12:55:12.786098+0000","last_unstale":"2024-09-11T12:55:13.787832+0000","last_undegraded":"2024-09-11T12:55:13.787832+0000","last_fullsized":"2024-09-11T12:55:13.787832+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:05.811750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787783+0000","last_change":"2024-09-11T12:55:11.766700+0000","last_active":"2024-09-11T12:55:13.787783+0000","last_peered":"2024-09-11T12:55:13.787783+0000","last_clean":"2024-09-11T12:55:13.787783+0000","last_became_active":"2024-09-11T12:55:11.766394+0000","last_became_peered":"2024-09-11T12:55:11.766394+0000","last_unstale":"2024-09-11T12:55:13.787783+0000","last_undegraded":"2024-09-11T12:55:13.787783+0000","last_fullsized":"2024-09-11T12:55:13.787783+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:30.851849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803695+0000","last_change":"2024-09-11T12:55:12.799285+0000","last_active":"2024-09-11T12:55:14.803695+0000","last_peered":"2024-09-11T12:55:14.803695+0000","last_clean":"2024-09-11T12:55:14.803695+0000","last_became_active":"2024-09-11T12:55:12.798866+0000","last_became_peered":"2024-09-11T12:55:12.798866+0000","last_unstale":"2024-09-11T12:55:14.803695+0000","last_undegraded":"2024-09-11T12:55:14.803695+0000","last_fullsized":"2024-09-11T12:55:14.803695+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:40.681972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803939+0000","last_change":"2024-09-11T12:55:11.782587+0000","last_active":"2024-09-11T12:55:14.803939+0000","last_peered":"2024-09-11T12:55:14.803939+0000","last_clean":"2024-09-11T12:55:14.803939+0000","last_became_active":"2024-09-11T12:55:11.782110+0000","last_became_peered":"2024-09-11T12:55:11.782110+0000","last_unstale":"2024-09-11T12:55:14.803939+0000","last_undegraded":"2024-09-11T12:55:14.803939+0000","last_fullsized":"2024-09-11T12:55:14.803939+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:09:31.545516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803832+0000","last_change":"2024-09-11T12:55:12.790950+0000","last_active":"2024-09-11T12:55:14.803832+0000","last_peered":"2024-09-11T12:55:14.803832+0000","last_clean":"2024-09-11T12:55:14.803832+0000","last_became_active":"2024-09-11T12:55:12.790674+0000","last_became_peered":"2024-09-11T12:55:12.790674+0000","last_unstale":"2024-09-11T12:55:14.803832+0000","last_undegraded":"2024-09-11T12:55:14.803832+0000","last_fullsized":"2024-09-11T12:55:14.803832+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:15.903891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"115'8","reported_seq":23,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803973+0000","last_change":"2024-09-11T12:55:11.780181+0000","last_active":"2024-09-11T12:55:14.803973+0000","last_peered":"2024-09-11T12:55:14.803973+0000","last_clean":"2024-09-11T12:55:14.803973+0000","last_became_active":"2024-09-11T12:55:11.779689+0000","last_became_peered":"2024-09-11T12:55:11.779689+0000","last_unstale":"2024-09-11T12:55:14.803973+0000","last_undegraded":"2024-09-11T12:55:14.803973+0000","last_fullsized":"2024-09-11T12:55:14.803973+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:55.294142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804156+0000","last_change":"2024-09-11T12:55:12.806469+0000","last_active":"2024-09-11T12:55:14.804156+0000","last_peered":"2024-09-11T12:55:14.804156+0000","last_clean":"2024-09-11T12:55:14.804156+0000","last_became_active":"2024-09-11T12:55:12.806061+0000","last_became_peered":"2024-09-11T12:55:12.806061+0000","last_unstale":"2024-09-11T12:55:14.804156+0000","last_undegraded":"2024-09-11T12:55:14.804156+0000","last_fullsized":"2024-09-11T12:55:14.804156+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790299+0000","last_change":"2024-09-11T12:55:11.778758+0000","last_active":"2024-09-11T12:55:13.790299+0000","last_peered":"2024-09-11T12:55:13.790299+0000","last_clean":"2024-09-11T12:55:13.790299+0000","last_became_active":"2024-09-11T12:55:11.778481+0000","last_became_peered":"2024-09-11T12:55:11.778481+0000","last_unstale":"2024-09-11T12:55:13.790299+0000","last_undegraded":"2024-09-11T12:55:13.790299+0000","last_fullsized":"2024-09-11T12:55:13.790299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:59.194460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790215+0000","last_change":"2024-09-11T12:55:12.780204+0000","last_active":"2024-09-11T12:55:13.790215+0000","last_peered":"2024-09-11T12:55:13.790215+0000","last_clean":"2024-09-11T12:55:13.790215+0000","last_became_active":"2024-09-11T12:55:12.779830+0000","last_became_peered":"2024-09-11T12:55:12.779830+0000","last_unstale":"2024-09-11T12:55:13.790215+0000","last_undegraded":"2024-09-11T12:55:13.790215+0000","last_fullsized":"2024-09-11T12:55:13.790215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:44.826592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803681+0000","last_change":"2024-09-11T12:55:11.779199+0000","last_active":"2024-09-11T12:55:14.803681+0000","last_peered":"2024-09-11T12:55:14.803681+0000","last_clean":"2024-09-11T12:55:14.803681+0000","last_became_active":"2024-09-11T12:55:11.778899+0000","last_became_peered":"2024-09-11T12:55:11.778899+0000","last_unstale":"2024-09-11T12:55:14.803681+0000","last_undegraded":"2024-09-11T12:55:14.803681+0000","last_fullsized":"2024-09-11T12:55:14.803681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:01.174201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805145+0000","last_change":"2024-09-11T12:55:12.795629+0000","last_active":"2024-09-11T12:55:14.805145+0000","last_peered":"2024-09-11T12:55:14.805145+0000","last_clean":"2024-09-11T12:55:14.805145+0000","last_became_active":"2024-09-11T12:55:12.794996+0000","last_became_peered":"2024-09-11T12:55:12.794996+0000","last_unstale":"2024-09-11T12:55:14.805145+0000","last_undegraded":"2024-09-11T12:55:14.805145+0000","last_fullsized":"2024-09-11T12:55:14.805145+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:25.749449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789886+0000","last_change":"2024-09-11T12:55:11.770251+0000","last_active":"2024-09-11T12:55:13.789886+0000","last_peered":"2024-09-11T12:55:13.789886+0000","last_clean":"2024-09-11T12:55:13.789886+0000","last_became_active":"2024-09-11T12:55:11.769972+0000","last_became_peered":"2024-09-11T12:55:11.769972+0000","last_unstale":"2024-09-11T12:55:13.789886+0000","last_undegraded":"2024-09-11T12:55:13.789886+0000","last_fullsized":"2024-09-11T12:55:13.789886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:45.579348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804384+0000","last_change":"2024-09-11T12:55:12.793032+0000","last_active":"2024-09-11T12:55:14.804384+0000","last_peered":"2024-09-11T12:55:14.804384+0000","last_clean":"2024-09-11T12:55:14.804384+0000","last_became_active":"2024-09-11T12:55:12.792759+0000","last_became_peered":"2024-09-11T12:55:12.792759+0000","last_unstale":"2024-09-11T12:55:14.804384+0000","last_undegraded":"2024-09-11T12:55:14.804384+0000","last_fullsized":"2024-09-11T12:55:14.804384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:31:30.666618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790013+0000","last_change":"2024-09-11T12:55:11.776566+0000","last_active":"2024-09-11T12:55:13.790013+0000","last_peered":"2024-09-11T12:55:13.790013+0000","last_clean":"2024-09-11T12:55:13.790013+0000","last_became_active":"2024-09-11T12:55:11.776246+0000","last_became_peered":"2024-09-11T12:55:11.776246+0000","last_unstale":"2024-09-11T12:55:13.790013+0000","last_undegraded":"2024-09-11T12:55:13.790013+0000","last_fullsized":"2024-09-11T12:55:13.790013+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:15.211842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787711+0000","last_change":"2024-09-11T12:55:12.780819+0000","last_active":"2024-09-11T12:55:13.787711+0000","last_peered":"2024-09-11T12:55:13.787711+0000","last_clean":"2024-09-11T12:55:13.787711+0000","last_became_active":"2024-09-11T12:55:12.780127+0000","last_became_peered":"2024-09-11T12:55:12.780127+0000","last_unstale":"2024-09-11T12:55:13.787711+0000","last_undegraded":"2024-09-11T12:55:13.787711+0000","last_fullsized":"2024-09-11T12:55:13.787711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:41.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789281+0000","last_change":"2024-09-11T12:55:11.763717+0000","last_active":"2024-09-11T12:55:13.789281+0000","last_peered":"2024-09-11T12:55:13.789281+0000","last_clean":"2024-09-11T12:55:13.789281+0000","last_became_active":"2024-09-11T12:55:11.763368+0000","last_became_peered":"2024-09-11T12:55:11.763368+0000","last_unstale":"2024-09-11T12:55:13.789281+0000","last_undegraded":"2024-09-11T12:55:13.789281+0000","last_fullsized":"2024-09-11T12:55:13.789281+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:10.403675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790331+0000","last_change":"2024-09-11T12:55:12.788210+0000","last_active":"2024-09-11T12:55:13.790331+0000","last_peered":"2024-09-11T12:55:13.790331+0000","last_clean":"2024-09-11T12:55:13.790331+0000","last_became_active":"2024-09-11T12:55:12.787915+0000","last_became_peered":"2024-09-11T12:55:12.787915+0000","last_unstale":"2024-09-11T12:55:13.790331+0000","last_undegraded":"2024-09-11T12:55:13.790331+0000","last_fullsized":"2024-09-11T12:55:13.790331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:49.781395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804871+0000","last_change":"2024-09-11T12:55:11.779857+0000","last_active":"2024-09-11T12:55:14.804871+0000","last_peered":"2024-09-11T12:55:14.804871+0000","last_clean":"2024-09-11T12:55:14.804871+0000","last_became_active":"2024-09-11T12:55:11.779533+0000","last_became_peered":"2024-09-11T12:55:11.779533+0000","last_unstale":"2024-09-11T12:55:14.804871+0000","last_undegraded":"2024-09-11T12:55:14.804871+0000","last_fullsized":"2024-09-11T12:55:14.804871+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:58.300814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803827+0000","last_change":"2024-09-11T12:55:12.786947+0000","last_active":"2024-09-11T12:55:14.803827+0000","last_peered":"2024-09-11T12:55:14.803827+0000","last_clean":"2024-09-11T12:55:14.803827+0000","last_became_active":"2024-09-11T12:55:12.786637+0000","last_became_peered":"2024-09-11T12:55:12.786637+0000","last_unstale":"2024-09-11T12:55:14.803827+0000","last_undegraded":"2024-09-11T12:55:14.803827+0000","last_fullsized":"2024-09-11T12:55:14.803827+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:46.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804746+0000","last_change":"2024-09-11T12:55:11.776937+0000","last_active":"2024-09-11T12:55:14.804746+0000","last_peered":"2024-09-11T12:55:14.804746+0000","last_clean":"2024-09-11T12:55:14.804746+0000","last_became_active":"2024-09-11T12:55:11.776621+0000","last_became_peered":"2024-09-11T12:55:11.776621+0000","last_unstale":"2024-09-11T12:55:14.804746+0000","last_undegraded":"2024-09-11T12:55:14.804746+0000","last_fullsized":"2024-09-11T12:55:14.804746+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:31:29.659751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794130+0000","last_change":"2024-09-11T12:55:12.801021+0000","last_active":"2024-09-11T12:55:13.794130+0000","last_peered":"2024-09-11T12:55:13.794130+0000","last_clean":"2024-09-11T12:55:13.794130+0000","last_became_active":"2024-09-11T12:55:12.800718+0000","last_became_peered":"2024-09-11T12:55:12.800718+0000","last_unstale":"2024-09-11T12:55:13.794130+0000","last_undegraded":"2024-09-11T12:55:13.794130+0000","last_fullsized":"2024-09-11T12:55:13.794130+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:47.522830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803325+0000","last_change":"2024-09-11T12:55:11.779532+0000","last_active":"2024-09-11T12:55:14.803325+0000","last_peered":"2024-09-11T12:55:14.803325+0000","last_clean":"2024-09-11T12:55:14.803325+0000","last_became_active":"2024-09-11T12:55:11.779274+0000","last_became_peered":"2024-09-11T12:55:11.779274+0000","last_unstale":"2024-09-11T12:55:14.803325+0000","last_undegraded":"2024-09-11T12:55:14.803325+0000","last_fullsized":"2024-09-11T12:55:14.803325+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:06.931611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790372+0000","last_change":"2024-09-11T12:55:12.778912+0000","last_active":"2024-09-11T12:55:13.790372+0000","last_peered":"2024-09-11T12:55:13.790372+0000","last_clean":"2024-09-11T12:55:13.790372+0000","last_became_active":"2024-09-11T12:55:12.778269+0000","last_became_peered":"2024-09-11T12:55:12.778269+0000","last_unstale":"2024-09-11T12:55:13.790372+0000","last_undegraded":"2024-09-11T12:55:13.790372+0000","last_fullsized":"2024-09-11T12:55:13.790372+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788657+0000","last_change":"2024-09-11T12:55:11.769333+0000","last_active":"2024-09-11T12:55:13.788657+0000","last_peered":"2024-09-11T12:55:13.788657+0000","last_clean":"2024-09-11T12:55:13.788657+0000","last_became_active":"2024-09-11T12:55:11.769005+0000","last_became_peered":"2024-09-11T12:55:11.769005+0000","last_unstale":"2024-09-11T12:55:13.788657+0000","last_undegraded":"2024-09-11T12:55:13.788657+0000","last_fullsized":"2024-09-11T12:55:13.788657+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:46.434600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788528+0000","last_change":"2024-09-11T12:55:12.791932+0000","last_active":"2024-09-11T12:55:13.788528+0000","last_peered":"2024-09-11T12:55:13.788528+0000","last_clean":"2024-09-11T12:55:13.788528+0000","last_became_active":"2024-09-11T12:55:12.791605+0000","last_became_peered":"2024-09-11T12:55:12.791605+0000","last_unstale":"2024-09-11T12:55:13.788528+0000","last_undegraded":"2024-09-11T12:55:13.788528+0000","last_fullsized":"2024-09-11T12:55:13.788528+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:46.610702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"116'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.796182+0000","last_change":"2024-09-11T12:55:11.781652+0000","last_active":"2024-09-11T12:55:13.796182+0000","last_peered":"2024-09-11T12:55:13.796182+0000","last_clean":"2024-09-11T12:55:13.796182+0000","last_became_active":"2024-09-11T12:55:11.781312+0000","last_became_peered":"2024-09-11T12:55:11.781312+0000","last_unstale":"2024-09-11T12:55:13.796182+0000","last_undegraded":"2024-09-11T12:55:13.796182+0000","last_fullsized":"2024-09-11T12:55:13.796182+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:50.088994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804306+0000","last_change":"2024-09-11T12:55:12.796791+0000","last_active":"2024-09-11T12:55:14.804306+0000","last_peered":"2024-09-11T12:55:14.804306+0000","last_clean":"2024-09-11T12:55:14.804306+0000","last_became_active":"2024-09-11T12:55:12.796529+0000","last_became_peered":"2024-09-11T12:55:12.796529+0000","last_unstale":"2024-09-11T12:55:14.804306+0000","last_undegraded":"2024-09-11T12:55:14.804306+0000","last_fullsized":"2024-09-11T12:55:14.804306+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:27.268762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804492+0000","last_change":"2024-09-11T12:55:11.783642+0000","last_active":"2024-09-11T12:55:14.804492+0000","last_peered":"2024-09-11T12:55:14.804492+0000","last_clean":"2024-09-11T12:55:14.804492+0000","last_became_active":"2024-09-11T12:55:11.781680+0000","last_became_peered":"2024-09-11T12:55:11.781680+0000","last_unstale":"2024-09-11T12:55:14.804492+0000","last_undegraded":"2024-09-11T12:55:14.804492+0000","last_fullsized":"2024-09-11T12:55:14.804492+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:59.194460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790033+0000","last_change":"2024-09-11T12:55:12.784793+0000","last_active":"2024-09-11T12:55:13.790033+0000","last_peered":"2024-09-11T12:55:13.790033+0000","last_clean":"2024-09-11T12:55:13.790033+0000","last_became_active":"2024-09-11T12:55:12.783097+0000","last_became_peered":"2024-09-11T12:55:12.783097+0000","last_unstale":"2024-09-11T12:55:13.790033+0000","last_undegraded":"2024-09-11T12:55:13.790033+0000","last_fullsized":"2024-09-11T12:55:13.790033+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:50.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803645+0000","last_change":"2024-09-11T12:55:11.785695+0000","last_active":"2024-09-11T12:55:14.803645+0000","last_peered":"2024-09-11T12:55:14.803645+0000","last_clean":"2024-09-11T12:55:14.803645+0000","last_became_active":"2024-09-11T12:55:11.785404+0000","last_became_peered":"2024-09-11T12:55:11.785404+0000","last_unstale":"2024-09-11T12:55:14.803645+0000","last_undegraded":"2024-09-11T12:55:14.803645+0000","last_fullsized":"2024-09-11T12:55:14.803645+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:24:55.687698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794114+0000","last_change":"2024-09-11T12:55:12.795081+0000","last_active":"2024-09-11T12:55:13.794114+0000","last_peered":"2024-09-11T12:55:13.794114+0000","last_clean":"2024-09-11T12:55:13.794114+0000","last_became_active":"2024-09-11T12:55:12.794808+0000","last_became_peered":"2024-09-11T12:55:12.794808+0000","last_unstale":"2024-09-11T12:55:13.794114+0000","last_undegraded":"2024-09-11T12:55:13.794114+0000","last_fullsized":"2024-09-11T12:55:13.794114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:54.598632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804645+0000","last_change":"2024-09-11T12:55:11.782108+0000","last_active":"2024-09-11T12:55:14.804645+0000","last_peered":"2024-09-11T12:55:14.804645+0000","last_clean":"2024-09-11T12:55:14.804645+0000","last_became_active":"2024-09-11T12:55:11.781759+0000","last_became_peered":"2024-09-11T12:55:11.781759+0000","last_unstale":"2024-09-11T12:55:14.804645+0000","last_undegraded":"2024-09-11T12:55:14.804645+0000","last_fullsized":"2024-09-11T12:55:14.804645+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:15.211842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789154+0000","last_change":"2024-09-11T12:55:12.777986+0000","last_active":"2024-09-11T12:55:13.789154+0000","last_peered":"2024-09-11T12:55:13.789154+0000","last_clean":"2024-09-11T12:55:13.789154+0000","last_became_active":"2024-09-11T12:55:12.777705+0000","last_became_peered":"2024-09-11T12:55:12.777705+0000","last_unstale":"2024-09-11T12:55:13.789154+0000","last_undegraded":"2024-09-11T12:55:13.789154+0000","last_fullsized":"2024-09-11T12:55:13.789154+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:52:09.078354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794290+0000","last_change":"2024-09-11T12:55:11.777911+0000","last_active":"2024-09-11T12:55:13.794290+0000","last_peered":"2024-09-11T12:55:13.794290+0000","last_clean":"2024-09-11T12:55:13.794290+0000","last_became_active":"2024-09-11T12:55:11.777379+0000","last_became_peered":"2024-09-11T12:55:11.777379+0000","last_unstale":"2024-09-11T12:55:13.794290+0000","last_undegraded":"2024-09-11T12:55:13.794290+0000","last_fullsized":"2024-09-11T12:55:13.794290+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:33.900412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805022+0000","last_change":"2024-09-11T12:55:12.794383+0000","last_active":"2024-09-11T12:55:14.805022+0000","last_peered":"2024-09-11T12:55:14.805022+0000","last_clean":"2024-09-11T12:55:14.805022+0000","last_became_active":"2024-09-11T12:55:12.793907+0000","last_became_peered":"2024-09-11T12:55:12.793907+0000","last_unstale":"2024-09-11T12:55:14.805022+0000","last_undegraded":"2024-09-11T12:55:14.805022+0000","last_fullsized":"2024-09-11T12:55:14.805022+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:38.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804780+0000","last_change":"2024-09-11T12:55:11.780050+0000","last_active":"2024-09-11T12:55:14.804780+0000","last_peered":"2024-09-11T12:55:14.804780+0000","last_clean":"2024-09-11T12:55:14.804780+0000","last_became_active":"2024-09-11T12:55:11.779772+0000","last_became_peered":"2024-09-11T12:55:11.779772+0000","last_unstale":"2024-09-11T12:55:14.804780+0000","last_undegraded":"2024-09-11T12:55:14.804780+0000","last_fullsized":"2024-09-11T12:55:14.804780+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:52.091452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803783+0000","last_change":"2024-09-11T12:55:12.796009+0000","last_active":"2024-09-11T12:55:14.803783+0000","last_peered":"2024-09-11T12:55:14.803783+0000","last_clean":"2024-09-11T12:55:14.803783+0000","last_became_active":"2024-09-11T12:55:12.793642+0000","last_became_peered":"2024-09-11T12:55:12.793642+0000","last_unstale":"2024-09-11T12:55:14.803783+0000","last_undegraded":"2024-09-11T12:55:14.803783+0000","last_fullsized":"2024-09-11T12:55:14.803783+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:47.231060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803452+0000","last_change":"2024-09-11T12:55:11.778484+0000","last_active":"2024-09-11T12:55:14.803452+0000","last_peered":"2024-09-11T12:55:14.803452+0000","last_clean":"2024-09-11T12:55:14.803452+0000","last_became_active":"2024-09-11T12:55:11.778209+0000","last_became_peered":"2024-09-11T12:55:11.778209+0000","last_unstale":"2024-09-11T12:55:14.803452+0000","last_undegraded":"2024-09-11T12:55:14.803452+0000","last_fullsized":"2024-09-11T12:55:14.803452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:45.904287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794198+0000","last_change":"2024-09-11T12:55:12.790694+0000","last_active":"2024-09-11T12:55:13.794198+0000","last_peered":"2024-09-11T12:55:13.794198+0000","last_clean":"2024-09-11T12:55:13.794198+0000","last_became_active":"2024-09-11T12:55:12.789799+0000","last_became_peered":"2024-09-11T12:55:12.789799+0000","last_unstale":"2024-09-11T12:55:13.794198+0000","last_undegraded":"2024-09-11T12:55:13.794198+0000","last_fullsized":"2024-09-11T12:55:13.794198+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.076209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794461+0000","last_change":"2024-09-11T12:55:11.782469+0000","last_active":"2024-09-11T12:55:13.794461+0000","last_peered":"2024-09-11T12:55:13.794461+0000","last_clean":"2024-09-11T12:55:13.794461+0000","last_became_active":"2024-09-11T12:55:11.782128+0000","last_became_peered":"2024-09-11T12:55:11.782128+0000","last_unstale":"2024-09-11T12:55:13.794461+0000","last_undegraded":"2024-09-11T12:55:13.794461+0000","last_fullsized":"2024-09-11T12:55:13.794461+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:23.457097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.793831+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:55:13.793831+0000","last_peered":"2024-09-11T12:55:13.793831+0000","last_clean":"2024-09-11T12:55:13.793831+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:55:13.793831+0000","last_undegraded":"2024-09-11T12:55:13.793831+0000","last_fullsized":"2024-09-11T12:55:13.793831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":1224124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1441792,"data_stored":626818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":631618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":631576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":631666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":632122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":625206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2023424,"data_stored":1215574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:55:16.886 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- 172.21.15.78:0/1748329291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7560157ae0 msgr2=0x7f7560070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- 172.21.15.78:0/1748329291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7560157ae0 0x7f7560070a00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7550009c80 tx=0x7f755001cd10 comp rx=0 tx=0).stop 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- 172.21.15.78:0/1748329291 shutdown_connections 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- 172.21.15.78:0/1748329291 >> v2:172.21.15.157:3300/0 conn(0x7f75600736d0 0x7f756015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- 172.21.15.78:0/1748329291 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7560157ae0 0x7f7560070a00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- 172.21.15.78:0/1748329291 >> 172.21.15.78:0/1748329291 conn(0x7f756006ca80 msgr2=0x7f75600b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- 172.21.15.78:0/1748329291 shutdown_connections 2024-09-11T12:55:17.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- 172.21.15.78:0/1748329291 wait complete. 2024-09-11T12:55:17.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 Processor -- start 2024-09-11T12:55:17.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- start start 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600736d0 0x7f75600b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75600b4db0 0x7f75601f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f75601691a0 con 0x7f7560157ae0 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7560169020 con 0x7f75600b4db0 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f7565000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7560169320 con 0x7f75600736d0 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57846/0 (socket says 172.21.15.78:57846) 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 -- 172.21.15.78:0/1921974619 learned_addr learned my addr 172.21.15.78:0/1921974619 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755de00640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600736d0 0x7f75600b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755d400640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75600b4db0 0x7f75601f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 -- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600736d0 msgr2=0x7f75600b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600736d0 0x7f75600b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 -- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75600b4db0 msgr2=0x7f75601f8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75600b4db0 0x7f75601f8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.059+0000 7f755e800640 1 -- 172.21.15.78:0/1921974619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75600af380 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f755e800640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f755800c400 tx=0x7f755800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558017070 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75580053c0 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75600b0a10 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558003860 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75600b0fd0 con 0x7f7560157ae0 2024-09-11T12:55:17.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f756015af70 con 0x7f7560157ae0 2024-09-11T12:55:17.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f755801a070 con 0x7f7560157ae0 2024-09-11T12:55:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 0x7f7540073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f75580917e0 con 0x7f7560157ae0 2024-09-11T12:55:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.063+0000 7f755de00640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 0x7f7540073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.067+0000 7f755de00640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 0x7f7540073f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75500af000 tx=0x7f755001b550 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.067+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f755805cab0 con 0x7f7560157ae0 2024-09-11T12:55:17.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.231+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7560156fb0 con 0x7f7540071a70 2024-09-11T12:55:17.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.235+0000 7f7556000640 1 -- 172.21.15.78:0/1921974619 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252474 (secure 0 0 0) 0x7f7560156fb0 con 0x7f7540071a70 2024-09-11T12:55:17.239 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:17.240 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:55:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 msgr2=0x7f7540073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 0x7f7540073f30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f75500af000 tx=0x7f755001b550 comp rx=0 tx=0).stop 2024-09-11T12:55:17.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 msgr2=0x7f75600aee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f755800c400 tx=0x7f755800c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 shutdown_connections 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7560157ae0 0x7f75600aee40 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7540071a70 0x7f7540073f30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f75600b4db0 0x7f75601f8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 --2- 172.21.15.78:0/1921974619 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f75600736d0 0x7f75600b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 >> 172.21.15.78:0/1921974619 conn(0x7f756006ca80 msgr2=0x7f75601554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 shutdown_connections 2024-09-11T12:55:17.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.239+0000 7f7565000640 1 -- 172.21.15.78:0/1921974619 wait complete. 2024-09-11T12:55:17.258 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":99,"stamp":"2024-09-11T12:55:16.490021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":261972,"kb_used_data":12452,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712748,"statfs":{"total":773094113280,"available":772825853952,"internally_reserved":0,"allocated":12750848,"data_stored":6218704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.737657"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788585+0000","last_change":"2024-09-11T12:55:12.796965+0000","last_active":"2024-09-11T12:55:13.788585+0000","last_peered":"2024-09-11T12:55:13.788585+0000","last_clean":"2024-09-11T12:55:13.788585+0000","last_became_active":"2024-09-11T12:55:12.796665+0000","last_became_peered":"2024-09-11T12:55:12.796665+0000","last_unstale":"2024-09-11T12:55:13.788585+0000","last_undegraded":"2024-09-11T12:55:13.788585+0000","last_fullsized":"2024-09-11T12:55:13.788585+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:34.611322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804356+0000","last_change":"2024-09-11T12:55:12.803718+0000","last_active":"2024-09-11T12:55:14.804356+0000","last_peered":"2024-09-11T12:55:14.804356+0000","last_clean":"2024-09-11T12:55:14.804356+0000","last_became_active":"2024-09-11T12:55:12.803470+0000","last_became_peered":"2024-09-11T12:55:12.803470+0000","last_unstale":"2024-09-11T12:55:14.804356+0000","last_undegraded":"2024-09-11T12:55:14.804356+0000","last_fullsized":"2024-09-11T12:55:14.804356+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:59.307681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804019+0000","last_change":"2024-09-11T12:55:12.799312+0000","last_active":"2024-09-11T12:55:14.804019+0000","last_peered":"2024-09-11T12:55:14.804019+0000","last_clean":"2024-09-11T12:55:14.804019+0000","last_became_active":"2024-09-11T12:55:12.798868+0000","last_became_peered":"2024-09-11T12:55:12.798868+0000","last_unstale":"2024-09-11T12:55:14.804019+0000","last_undegraded":"2024-09-11T12:55:14.804019+0000","last_fullsized":"2024-09-11T12:55:14.804019+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:36.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787998+0000","last_change":"2024-09-11T12:55:12.792663+0000","last_active":"2024-09-11T12:55:13.787998+0000","last_peered":"2024-09-11T12:55:13.787998+0000","last_clean":"2024-09-11T12:55:13.787998+0000","last_became_active":"2024-09-11T12:55:12.792265+0000","last_became_peered":"2024-09-11T12:55:12.792265+0000","last_unstale":"2024-09-11T12:55:13.787998+0000","last_undegraded":"2024-09-11T12:55:13.787998+0000","last_fullsized":"2024-09-11T12:55:13.787998+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:24:58.043614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794622+0000","last_change":"2024-09-11T12:55:12.801516+0000","last_active":"2024-09-11T12:55:13.794622+0000","last_peered":"2024-09-11T12:55:13.794622+0000","last_clean":"2024-09-11T12:55:13.794622+0000","last_became_active":"2024-09-11T12:55:12.801236+0000","last_became_peered":"2024-09-11T12:55:12.801236+0000","last_unstale":"2024-09-11T12:55:13.794622+0000","last_undegraded":"2024-09-11T12:55:13.794622+0000","last_fullsized":"2024-09-11T12:55:13.794622+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:23:46.106790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790235+0000","last_change":"2024-09-11T12:55:12.797396+0000","last_active":"2024-09-11T12:55:13.790235+0000","last_peered":"2024-09-11T12:55:13.790235+0000","last_clean":"2024-09-11T12:55:13.790235+0000","last_became_active":"2024-09-11T12:55:12.793134+0000","last_became_peered":"2024-09-11T12:55:12.793134+0000","last_unstale":"2024-09-11T12:55:13.790235+0000","last_undegraded":"2024-09-11T12:55:13.790235+0000","last_fullsized":"2024-09-11T12:55:13.790235+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:36.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789712+0000","last_change":"2024-09-11T12:55:12.789830+0000","last_active":"2024-09-11T12:55:13.789712+0000","last_peered":"2024-09-11T12:55:13.789712+0000","last_clean":"2024-09-11T12:55:13.789712+0000","last_became_active":"2024-09-11T12:55:12.789557+0000","last_became_peered":"2024-09-11T12:55:12.789557+0000","last_unstale":"2024-09-11T12:55:13.789712+0000","last_undegraded":"2024-09-11T12:55:13.789712+0000","last_fullsized":"2024-09-11T12:55:13.789712+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:51.095861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803827+0000","last_change":"2024-09-11T12:55:12.795350+0000","last_active":"2024-09-11T12:55:14.803827+0000","last_peered":"2024-09-11T12:55:14.803827+0000","last_clean":"2024-09-11T12:55:14.803827+0000","last_became_active":"2024-09-11T12:55:12.794743+0000","last_became_peered":"2024-09-11T12:55:12.794743+0000","last_unstale":"2024-09-11T12:55:14.803827+0000","last_undegraded":"2024-09-11T12:55:14.803827+0000","last_fullsized":"2024-09-11T12:55:14.803827+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:51.095861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804267+0000","last_change":"2024-09-11T12:55:12.806463+0000","last_active":"2024-09-11T12:55:14.804267+0000","last_peered":"2024-09-11T12:55:14.804267+0000","last_clean":"2024-09-11T12:55:14.804267+0000","last_became_active":"2024-09-11T12:55:12.806161+0000","last_became_peered":"2024-09-11T12:55:12.806161+0000","last_unstale":"2024-09-11T12:55:14.804267+0000","last_undegraded":"2024-09-11T12:55:14.804267+0000","last_fullsized":"2024-09-11T12:55:14.804267+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804533+0000","last_change":"2024-09-11T12:55:12.803570+0000","last_active":"2024-09-11T12:55:14.804533+0000","last_peered":"2024-09-11T12:55:14.804533+0000","last_clean":"2024-09-11T12:55:14.804533+0000","last_became_active":"2024-09-11T12:55:12.803273+0000","last_became_peered":"2024-09-11T12:55:12.803273+0000","last_unstale":"2024-09-11T12:55:14.804533+0000","last_undegraded":"2024-09-11T12:55:14.804533+0000","last_fullsized":"2024-09-11T12:55:14.804533+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:57.011365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804042+0000","last_change":"2024-09-11T12:55:12.790631+0000","last_active":"2024-09-11T12:55:14.804042+0000","last_peered":"2024-09-11T12:55:14.804042+0000","last_clean":"2024-09-11T12:55:14.804042+0000","last_became_active":"2024-09-11T12:55:12.788939+0000","last_became_peered":"2024-09-11T12:55:12.788939+0000","last_unstale":"2024-09-11T12:55:14.804042+0000","last_undegraded":"2024-09-11T12:55:14.804042+0000","last_fullsized":"2024-09-11T12:55:14.804042+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:48.102689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803894+0000","last_change":"2024-09-11T12:55:12.800517+0000","last_active":"2024-09-11T12:55:14.803894+0000","last_peered":"2024-09-11T12:55:14.803894+0000","last_clean":"2024-09-11T12:55:14.803894+0000","last_became_active":"2024-09-11T12:55:12.800179+0000","last_became_peered":"2024-09-11T12:55:12.800179+0000","last_unstale":"2024-09-11T12:55:14.803894+0000","last_undegraded":"2024-09-11T12:55:14.803894+0000","last_fullsized":"2024-09-11T12:55:14.803894+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:25:45.314239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787677+0000","last_change":"2024-09-11T12:55:12.783195+0000","last_active":"2024-09-11T12:55:13.787677+0000","last_peered":"2024-09-11T12:55:13.787677+0000","last_clean":"2024-09-11T12:55:13.787677+0000","last_became_active":"2024-09-11T12:55:12.782924+0000","last_became_peered":"2024-09-11T12:55:12.782924+0000","last_unstale":"2024-09-11T12:55:13.787677+0000","last_undegraded":"2024-09-11T12:55:13.787677+0000","last_fullsized":"2024-09-11T12:55:13.787677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:28:13.412999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804850+0000","last_change":"2024-09-11T12:55:12.800628+0000","last_active":"2024-09-11T12:55:14.804850+0000","last_peered":"2024-09-11T12:55:14.804850+0000","last_clean":"2024-09-11T12:55:14.804850+0000","last_became_active":"2024-09-11T12:55:12.800307+0000","last_became_peered":"2024-09-11T12:55:12.800307+0000","last_unstale":"2024-09-11T12:55:14.804850+0000","last_undegraded":"2024-09-11T12:55:14.804850+0000","last_fullsized":"2024-09-11T12:55:14.804850+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:53.098319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787621+0000","last_change":"2024-09-11T12:55:12.786548+0000","last_active":"2024-09-11T12:55:13.787621+0000","last_peered":"2024-09-11T12:55:13.787621+0000","last_clean":"2024-09-11T12:55:13.787621+0000","last_became_active":"2024-09-11T12:55:12.786272+0000","last_became_peered":"2024-09-11T12:55:12.786272+0000","last_unstale":"2024-09-11T12:55:13.787621+0000","last_undegraded":"2024-09-11T12:55:13.787621+0000","last_fullsized":"2024-09-11T12:55:13.787621+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:09:09.394176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804667+0000","last_change":"2024-09-11T12:55:12.801258+0000","last_active":"2024-09-11T12:55:14.804667+0000","last_peered":"2024-09-11T12:55:14.804667+0000","last_clean":"2024-09-11T12:55:14.804667+0000","last_became_active":"2024-09-11T12:55:12.800939+0000","last_became_peered":"2024-09-11T12:55:12.800939+0000","last_unstale":"2024-09-11T12:55:14.804667+0000","last_undegraded":"2024-09-11T12:55:14.804667+0000","last_fullsized":"2024-09-11T12:55:14.804667+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790418+0000","last_change":"2024-09-11T12:55:12.781745+0000","last_active":"2024-09-11T12:55:13.790418+0000","last_peered":"2024-09-11T12:55:13.790418+0000","last_clean":"2024-09-11T12:55:13.790418+0000","last_became_active":"2024-09-11T12:55:12.781393+0000","last_became_peered":"2024-09-11T12:55:12.781393+0000","last_unstale":"2024-09-11T12:55:13.790418+0000","last_undegraded":"2024-09-11T12:55:13.790418+0000","last_fullsized":"2024-09-11T12:55:13.790418+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:03:43.797511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805286+0000","last_change":"2024-09-11T12:55:12.788781+0000","last_active":"2024-09-11T12:55:14.805286+0000","last_peered":"2024-09-11T12:55:14.805286+0000","last_clean":"2024-09-11T12:55:14.805286+0000","last_became_active":"2024-09-11T12:55:12.788004+0000","last_became_peered":"2024-09-11T12:55:12.788004+0000","last_unstale":"2024-09-11T12:55:14.805286+0000","last_undegraded":"2024-09-11T12:55:14.805286+0000","last_fullsized":"2024-09-11T12:55:14.805286+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:39:49.540650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804078+0000","last_change":"2024-09-11T12:55:12.790179+0000","last_active":"2024-09-11T12:55:14.804078+0000","last_peered":"2024-09-11T12:55:14.804078+0000","last_clean":"2024-09-11T12:55:14.804078+0000","last_became_active":"2024-09-11T12:55:12.789426+0000","last_became_peered":"2024-09-11T12:55:12.789426+0000","last_unstale":"2024-09-11T12:55:14.804078+0000","last_undegraded":"2024-09-11T12:55:14.804078+0000","last_fullsized":"2024-09-11T12:55:14.804078+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:49:12.619514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805108+0000","last_change":"2024-09-11T12:55:12.795384+0000","last_active":"2024-09-11T12:55:14.805108+0000","last_peered":"2024-09-11T12:55:14.805108+0000","last_clean":"2024-09-11T12:55:14.805108+0000","last_became_active":"2024-09-11T12:55:12.794112+0000","last_became_peered":"2024-09-11T12:55:12.794112+0000","last_unstale":"2024-09-11T12:55:14.805108+0000","last_undegraded":"2024-09-11T12:55:14.805108+0000","last_fullsized":"2024-09-11T12:55:14.805108+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:10:55.560900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794313+0000","last_change":"2024-09-11T12:55:12.801428+0000","last_active":"2024-09-11T12:55:13.794313+0000","last_peered":"2024-09-11T12:55:13.794313+0000","last_clean":"2024-09-11T12:55:13.794313+0000","last_became_active":"2024-09-11T12:55:12.800817+0000","last_became_peered":"2024-09-11T12:55:12.800817+0000","last_unstale":"2024-09-11T12:55:13.794313+0000","last_undegraded":"2024-09-11T12:55:13.794313+0000","last_fullsized":"2024-09-11T12:55:13.794313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:01.557374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803778+0000","last_change":"2024-09-11T12:55:12.796967+0000","last_active":"2024-09-11T12:55:14.803778+0000","last_peered":"2024-09-11T12:55:14.803778+0000","last_clean":"2024-09-11T12:55:14.803778+0000","last_became_active":"2024-09-11T12:55:12.796619+0000","last_became_peered":"2024-09-11T12:55:12.796619+0000","last_unstale":"2024-09-11T12:55:14.803778+0000","last_undegraded":"2024-09-11T12:55:14.803778+0000","last_fullsized":"2024-09-11T12:55:14.803778+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:30:37.708319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803951+0000","last_change":"2024-09-11T12:55:12.799247+0000","last_active":"2024-09-11T12:55:14.803951+0000","last_peered":"2024-09-11T12:55:14.803951+0000","last_clean":"2024-09-11T12:55:14.803951+0000","last_became_active":"2024-09-11T12:55:12.799001+0000","last_became_peered":"2024-09-11T12:55:12.799001+0000","last_unstale":"2024-09-11T12:55:14.803951+0000","last_undegraded":"2024-09-11T12:55:14.803951+0000","last_fullsized":"2024-09-11T12:55:14.803951+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:50.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803677+0000","last_change":"2024-09-11T12:55:12.790203+0000","last_active":"2024-09-11T12:55:14.803677+0000","last_peered":"2024-09-11T12:55:14.803677+0000","last_clean":"2024-09-11T12:55:14.803677+0000","last_became_active":"2024-09-11T12:55:12.789937+0000","last_became_peered":"2024-09-11T12:55:12.789937+0000","last_unstale":"2024-09-11T12:55:14.803677+0000","last_undegraded":"2024-09-11T12:55:14.803677+0000","last_fullsized":"2024-09-11T12:55:14.803677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:05.662510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804221+0000","last_change":"2024-09-11T12:55:12.796528+0000","last_active":"2024-09-11T12:55:14.804221+0000","last_peered":"2024-09-11T12:55:14.804221+0000","last_clean":"2024-09-11T12:55:14.804221+0000","last_became_active":"2024-09-11T12:55:12.796183+0000","last_became_peered":"2024-09-11T12:55:12.796183+0000","last_unstale":"2024-09-11T12:55:14.804221+0000","last_undegraded":"2024-09-11T12:55:14.804221+0000","last_fullsized":"2024-09-11T12:55:14.804221+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:06:53.316378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805450+0000","last_change":"2024-09-11T12:55:12.788560+0000","last_active":"2024-09-11T12:55:14.805450+0000","last_peered":"2024-09-11T12:55:14.805450+0000","last_clean":"2024-09-11T12:55:14.805450+0000","last_became_active":"2024-09-11T12:55:12.787733+0000","last_became_peered":"2024-09-11T12:55:12.787733+0000","last_unstale":"2024-09-11T12:55:14.805450+0000","last_undegraded":"2024-09-11T12:55:14.805450+0000","last_fullsized":"2024-09-11T12:55:14.805450+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.076209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787909+0000","last_change":"2024-09-11T12:55:12.796145+0000","last_active":"2024-09-11T12:55:13.787909+0000","last_peered":"2024-09-11T12:55:13.787909+0000","last_clean":"2024-09-11T12:55:13.787909+0000","last_became_active":"2024-09-11T12:55:12.795836+0000","last_became_peered":"2024-09-11T12:55:12.795836+0000","last_unstale":"2024-09-11T12:55:13.787909+0000","last_undegraded":"2024-09-11T12:55:13.787909+0000","last_fullsized":"2024-09-11T12:55:13.787909+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:53.249522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789313+0000","last_change":"2024-09-11T12:55:12.786899+0000","last_active":"2024-09-11T12:55:13.789313+0000","last_peered":"2024-09-11T12:55:13.789313+0000","last_clean":"2024-09-11T12:55:13.789313+0000","last_became_active":"2024-09-11T12:55:12.786620+0000","last_became_peered":"2024-09-11T12:55:12.786620+0000","last_unstale":"2024-09-11T12:55:13.789313+0000","last_undegraded":"2024-09-11T12:55:13.789313+0000","last_fullsized":"2024-09-11T12:55:13.789313+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:04:16.809556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788405+0000","last_change":"2024-09-11T12:55:12.787777+0000","last_active":"2024-09-11T12:55:13.788405+0000","last_peered":"2024-09-11T12:55:13.788405+0000","last_clean":"2024-09-11T12:55:13.788405+0000","last_became_active":"2024-09-11T12:55:12.787442+0000","last_became_peered":"2024-09-11T12:55:12.787442+0000","last_unstale":"2024-09-11T12:55:13.788405+0000","last_undegraded":"2024-09-11T12:55:13.788405+0000","last_fullsized":"2024-09-11T12:55:13.788405+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:23:54.243869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787677+0000","last_change":"2024-09-11T12:55:12.788564+0000","last_active":"2024-09-11T12:55:13.787677+0000","last_peered":"2024-09-11T12:55:13.787677+0000","last_clean":"2024-09-11T12:55:13.787677+0000","last_became_active":"2024-09-11T12:55:12.788247+0000","last_became_peered":"2024-09-11T12:55:12.788247+0000","last_unstale":"2024-09-11T12:55:13.787677+0000","last_undegraded":"2024-09-11T12:55:13.787677+0000","last_fullsized":"2024-09-11T12:55:13.787677+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:44.262619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794126+0000","last_change":"2024-09-11T12:55:12.801381+0000","last_active":"2024-09-11T12:55:13.794126+0000","last_peered":"2024-09-11T12:55:13.794126+0000","last_clean":"2024-09-11T12:55:13.794126+0000","last_became_active":"2024-09-11T12:55:12.800736+0000","last_became_peered":"2024-09-11T12:55:12.800736+0000","last_unstale":"2024-09-11T12:55:13.794126+0000","last_undegraded":"2024-09-11T12:55:13.794126+0000","last_fullsized":"2024-09-11T12:55:13.794126+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:00.009725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789837+0000","last_change":"2024-09-11T12:55:12.794456+0000","last_active":"2024-09-11T12:55:13.789837+0000","last_peered":"2024-09-11T12:55:13.789837+0000","last_clean":"2024-09-11T12:55:13.789837+0000","last_became_active":"2024-09-11T12:55:12.794158+0000","last_became_peered":"2024-09-11T12:55:12.794158+0000","last_unstale":"2024-09-11T12:55:13.789837+0000","last_undegraded":"2024-09-11T12:55:13.789837+0000","last_fullsized":"2024-09-11T12:55:13.789837+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:25:45.314239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805446+0000","last_change":"2024-09-11T12:55:12.788248+0000","last_active":"2024-09-11T12:55:14.805446+0000","last_peered":"2024-09-11T12:55:14.805446+0000","last_clean":"2024-09-11T12:55:14.805446+0000","last_became_active":"2024-09-11T12:55:12.787396+0000","last_became_peered":"2024-09-11T12:55:12.787396+0000","last_unstale":"2024-09-11T12:55:14.805446+0000","last_undegraded":"2024-09-11T12:55:14.805446+0000","last_fullsized":"2024-09-11T12:55:14.805446+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:47.522830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"115'4","reported_seq":19,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794705+0000","last_change":"2024-09-11T12:55:11.782536+0000","last_active":"2024-09-11T12:55:13.794705+0000","last_peered":"2024-09-11T12:55:13.794705+0000","last_clean":"2024-09-11T12:55:13.794705+0000","last_became_active":"2024-09-11T12:55:11.782176+0000","last_became_peered":"2024-09-11T12:55:11.782176+0000","last_unstale":"2024-09-11T12:55:13.794705+0000","last_undegraded":"2024-09-11T12:55:13.794705+0000","last_fullsized":"2024-09-11T12:55:13.794705+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:14.897024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804300+0000","last_change":"2024-09-11T12:55:12.801482+0000","last_active":"2024-09-11T12:55:14.804300+0000","last_peered":"2024-09-11T12:55:14.804300+0000","last_clean":"2024-09-11T12:55:14.804300+0000","last_became_active":"2024-09-11T12:55:12.801183+0000","last_became_peered":"2024-09-11T12:55:12.801183+0000","last_unstale":"2024-09-11T12:55:14.804300+0000","last_undegraded":"2024-09-11T12:55:14.804300+0000","last_fullsized":"2024-09-11T12:55:14.804300+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:38.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789160+0000","last_change":"2024-09-11T12:55:11.775261+0000","last_active":"2024-09-11T12:55:13.789160+0000","last_peered":"2024-09-11T12:55:13.789160+0000","last_clean":"2024-09-11T12:55:13.789160+0000","last_became_active":"2024-09-11T12:55:11.774990+0000","last_became_peered":"2024-09-11T12:55:11.774990+0000","last_unstale":"2024-09-11T12:55:13.789160+0000","last_undegraded":"2024-09-11T12:55:13.789160+0000","last_fullsized":"2024-09-11T12:55:13.789160+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:17.837536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794661+0000","last_change":"2024-09-11T12:55:12.802842+0000","last_active":"2024-09-11T12:55:13.794661+0000","last_peered":"2024-09-11T12:55:13.794661+0000","last_clean":"2024-09-11T12:55:13.794661+0000","last_became_active":"2024-09-11T12:55:12.802551+0000","last_became_peered":"2024-09-11T12:55:12.802551+0000","last_unstale":"2024-09-11T12:55:13.794661+0000","last_undegraded":"2024-09-11T12:55:13.794661+0000","last_fullsized":"2024-09-11T12:55:13.794661+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:14:53.513962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804141+0000","last_change":"2024-09-11T12:55:11.775364+0000","last_active":"2024-09-11T12:55:14.804141+0000","last_peered":"2024-09-11T12:55:14.804141+0000","last_clean":"2024-09-11T12:55:14.804141+0000","last_became_active":"2024-09-11T12:55:11.775083+0000","last_became_peered":"2024-09-11T12:55:11.775083+0000","last_unstale":"2024-09-11T12:55:14.804141+0000","last_undegraded":"2024-09-11T12:55:14.804141+0000","last_fullsized":"2024-09-11T12:55:14.804141+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:08:15.913964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794285+0000","last_change":"2024-09-11T12:55:12.796692+0000","last_active":"2024-09-11T12:55:13.794285+0000","last_peered":"2024-09-11T12:55:13.794285+0000","last_clean":"2024-09-11T12:55:13.794285+0000","last_became_active":"2024-09-11T12:55:12.796396+0000","last_became_peered":"2024-09-11T12:55:12.796396+0000","last_unstale":"2024-09-11T12:55:13.794285+0000","last_undegraded":"2024-09-11T12:55:13.794285+0000","last_fullsized":"2024-09-11T12:55:13.794285+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:56.151247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804422+0000","last_change":"2024-09-11T12:55:11.784018+0000","last_active":"2024-09-11T12:55:14.804422+0000","last_peered":"2024-09-11T12:55:14.804422+0000","last_clean":"2024-09-11T12:55:14.804422+0000","last_became_active":"2024-09-11T12:55:11.783332+0000","last_became_peered":"2024-09-11T12:55:11.783332+0000","last_unstale":"2024-09-11T12:55:14.804422+0000","last_undegraded":"2024-09-11T12:55:14.804422+0000","last_fullsized":"2024-09-11T12:55:14.804422+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:17.837536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804656+0000","last_change":"2024-09-11T12:55:12.789813+0000","last_active":"2024-09-11T12:55:14.804656+0000","last_peered":"2024-09-11T12:55:14.804656+0000","last_clean":"2024-09-11T12:55:14.804656+0000","last_became_active":"2024-09-11T12:55:12.789532+0000","last_became_peered":"2024-09-11T12:55:12.789532+0000","last_unstale":"2024-09-11T12:55:14.804656+0000","last_undegraded":"2024-09-11T12:55:14.804656+0000","last_fullsized":"2024-09-11T12:55:14.804656+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804076+0000","last_change":"2024-09-11T12:55:11.782608+0000","last_active":"2024-09-11T12:55:14.804076+0000","last_peered":"2024-09-11T12:55:14.804076+0000","last_clean":"2024-09-11T12:55:14.804076+0000","last_became_active":"2024-09-11T12:55:11.782337+0000","last_became_peered":"2024-09-11T12:55:11.782337+0000","last_unstale":"2024-09-11T12:55:14.804076+0000","last_undegraded":"2024-09-11T12:55:14.804076+0000","last_fullsized":"2024-09-11T12:55:14.804076+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:51.901604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789711+0000","last_change":"2024-09-11T12:55:12.788083+0000","last_active":"2024-09-11T12:55:13.789711+0000","last_peered":"2024-09-11T12:55:13.789711+0000","last_clean":"2024-09-11T12:55:13.789711+0000","last_became_active":"2024-09-11T12:55:12.786243+0000","last_became_peered":"2024-09-11T12:55:12.786243+0000","last_unstale":"2024-09-11T12:55:13.789711+0000","last_undegraded":"2024-09-11T12:55:13.789711+0000","last_fullsized":"2024-09-11T12:55:13.789711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:15.903891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804780+0000","last_change":"2024-09-11T12:55:11.786588+0000","last_active":"2024-09-11T12:55:14.804780+0000","last_peered":"2024-09-11T12:55:14.804780+0000","last_clean":"2024-09-11T12:55:14.804780+0000","last_became_active":"2024-09-11T12:55:11.786301+0000","last_became_peered":"2024-09-11T12:55:11.786301+0000","last_unstale":"2024-09-11T12:55:14.804780+0000","last_undegraded":"2024-09-11T12:55:14.804780+0000","last_fullsized":"2024-09-11T12:55:14.804780+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:26.261895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804082+0000","last_change":"2024-09-11T12:55:12.795005+0000","last_active":"2024-09-11T12:55:14.804082+0000","last_peered":"2024-09-11T12:55:14.804082+0000","last_clean":"2024-09-11T12:55:14.804082+0000","last_became_active":"2024-09-11T12:55:12.794343+0000","last_became_peered":"2024-09-11T12:55:12.794343+0000","last_unstale":"2024-09-11T12:55:14.804082+0000","last_undegraded":"2024-09-11T12:55:14.804082+0000","last_fullsized":"2024-09-11T12:55:14.804082+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:49.781395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789519+0000","last_change":"2024-09-11T12:55:11.773630+0000","last_active":"2024-09-11T12:55:13.789519+0000","last_peered":"2024-09-11T12:55:13.789519+0000","last_clean":"2024-09-11T12:55:13.789519+0000","last_became_active":"2024-09-11T12:55:11.773220+0000","last_became_peered":"2024-09-11T12:55:11.773220+0000","last_unstale":"2024-09-11T12:55:13.789519+0000","last_undegraded":"2024-09-11T12:55:13.789519+0000","last_fullsized":"2024-09-11T12:55:13.789519+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:46.224193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803872+0000","last_change":"2024-09-11T12:55:12.798625+0000","last_active":"2024-09-11T12:55:14.803872+0000","last_peered":"2024-09-11T12:55:14.803872+0000","last_clean":"2024-09-11T12:55:14.803872+0000","last_became_active":"2024-09-11T12:55:12.798331+0000","last_became_peered":"2024-09-11T12:55:12.798331+0000","last_unstale":"2024-09-11T12:55:14.803872+0000","last_undegraded":"2024-09-11T12:55:14.803872+0000","last_fullsized":"2024-09-11T12:55:14.803872+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:56.301009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803829+0000","last_change":"2024-09-11T12:55:11.778942+0000","last_active":"2024-09-11T12:55:14.803829+0000","last_peered":"2024-09-11T12:55:14.803829+0000","last_clean":"2024-09-11T12:55:14.803829+0000","last_became_active":"2024-09-11T12:55:11.778674+0000","last_became_peered":"2024-09-11T12:55:11.778674+0000","last_unstale":"2024-09-11T12:55:14.803829+0000","last_undegraded":"2024-09-11T12:55:14.803829+0000","last_fullsized":"2024-09-11T12:55:14.803829+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:10.403675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788294+0000","last_change":"2024-09-11T12:55:12.783725+0000","last_active":"2024-09-11T12:55:13.788294+0000","last_peered":"2024-09-11T12:55:13.788294+0000","last_clean":"2024-09-11T12:55:13.788294+0000","last_became_active":"2024-09-11T12:55:12.782955+0000","last_became_peered":"2024-09-11T12:55:12.782955+0000","last_unstale":"2024-09-11T12:55:13.788294+0000","last_undegraded":"2024-09-11T12:55:13.788294+0000","last_fullsized":"2024-09-11T12:55:13.788294+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:45:55.734191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790191+0000","last_change":"2024-09-11T12:55:11.770102+0000","last_active":"2024-09-11T12:55:13.790191+0000","last_peered":"2024-09-11T12:55:13.790191+0000","last_clean":"2024-09-11T12:55:13.790191+0000","last_became_active":"2024-09-11T12:55:11.769803+0000","last_became_peered":"2024-09-11T12:55:11.769803+0000","last_unstale":"2024-09-11T12:55:13.790191+0000","last_undegraded":"2024-09-11T12:55:13.790191+0000","last_fullsized":"2024-09-11T12:55:13.790191+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:59:51.901604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790068+0000","last_change":"2024-09-11T12:55:12.800640+0000","last_active":"2024-09-11T12:55:13.790068+0000","last_peered":"2024-09-11T12:55:13.790068+0000","last_clean":"2024-09-11T12:55:13.790068+0000","last_became_active":"2024-09-11T12:55:12.800342+0000","last_became_peered":"2024-09-11T12:55:12.800342+0000","last_unstale":"2024-09-11T12:55:13.790068+0000","last_undegraded":"2024-09-11T12:55:13.790068+0000","last_fullsized":"2024-09-11T12:55:13.790068+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:24.238489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805526+0000","last_change":"2024-09-11T12:55:11.780460+0000","last_active":"2024-09-11T12:55:14.805526+0000","last_peered":"2024-09-11T12:55:14.805526+0000","last_clean":"2024-09-11T12:55:14.805526+0000","last_became_active":"2024-09-11T12:55:11.780159+0000","last_became_peered":"2024-09-11T12:55:11.780159+0000","last_unstale":"2024-09-11T12:55:14.805526+0000","last_undegraded":"2024-09-11T12:55:14.805526+0000","last_fullsized":"2024-09-11T12:55:14.805526+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:11:56.004498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794601+0000","last_change":"2024-09-11T12:55:12.806930+0000","last_active":"2024-09-11T12:55:13.794601+0000","last_peered":"2024-09-11T12:55:13.794601+0000","last_clean":"2024-09-11T12:55:13.794601+0000","last_became_active":"2024-09-11T12:55:12.806665+0000","last_became_peered":"2024-09-11T12:55:12.806665+0000","last_unstale":"2024-09-11T12:55:13.794601+0000","last_undegraded":"2024-09-11T12:55:13.794601+0000","last_fullsized":"2024-09-11T12:55:13.794601+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:47:25.081626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"115'3","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790104+0000","last_change":"2024-09-11T12:55:11.770394+0000","last_active":"2024-09-11T12:55:13.790104+0000","last_peered":"2024-09-11T12:55:13.790104+0000","last_clean":"2024-09-11T12:55:13.790104+0000","last_became_active":"2024-09-11T12:55:11.769973+0000","last_became_peered":"2024-09-11T12:55:11.769973+0000","last_unstale":"2024-09-11T12:55:13.790104+0000","last_undegraded":"2024-09-11T12:55:13.790104+0000","last_fullsized":"2024-09-11T12:55:13.790104+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:09:31.545516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794517+0000","last_change":"2024-09-11T12:55:12.796980+0000","last_active":"2024-09-11T12:55:13.794517+0000","last_peered":"2024-09-11T12:55:13.794517+0000","last_clean":"2024-09-11T12:55:13.794517+0000","last_became_active":"2024-09-11T12:55:12.796697+0000","last_became_peered":"2024-09-11T12:55:12.796697+0000","last_unstale":"2024-09-11T12:55:13.794517+0000","last_undegraded":"2024-09-11T12:55:13.794517+0000","last_fullsized":"2024-09-11T12:55:13.794517+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:14:18.034475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"115'6","reported_seq":21,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803904+0000","last_change":"2024-09-11T12:55:11.776549+0000","last_active":"2024-09-11T12:55:14.803904+0000","last_peered":"2024-09-11T12:55:14.803904+0000","last_clean":"2024-09-11T12:55:14.803904+0000","last_became_active":"2024-09-11T12:55:11.776252+0000","last_became_peered":"2024-09-11T12:55:11.776252+0000","last_unstale":"2024-09-11T12:55:14.803904+0000","last_undegraded":"2024-09-11T12:55:14.803904+0000","last_fullsized":"2024-09-11T12:55:14.803904+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:46.224193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787832+0000","last_change":"2024-09-11T12:55:12.786332+0000","last_active":"2024-09-11T12:55:13.787832+0000","last_peered":"2024-09-11T12:55:13.787832+0000","last_clean":"2024-09-11T12:55:13.787832+0000","last_became_active":"2024-09-11T12:55:12.786098+0000","last_became_peered":"2024-09-11T12:55:12.786098+0000","last_unstale":"2024-09-11T12:55:13.787832+0000","last_undegraded":"2024-09-11T12:55:13.787832+0000","last_fullsized":"2024-09-11T12:55:13.787832+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:05.811750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787783+0000","last_change":"2024-09-11T12:55:11.766700+0000","last_active":"2024-09-11T12:55:13.787783+0000","last_peered":"2024-09-11T12:55:13.787783+0000","last_clean":"2024-09-11T12:55:13.787783+0000","last_became_active":"2024-09-11T12:55:11.766394+0000","last_became_peered":"2024-09-11T12:55:11.766394+0000","last_unstale":"2024-09-11T12:55:13.787783+0000","last_undegraded":"2024-09-11T12:55:13.787783+0000","last_fullsized":"2024-09-11T12:55:13.787783+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:30.851849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803695+0000","last_change":"2024-09-11T12:55:12.799285+0000","last_active":"2024-09-11T12:55:14.803695+0000","last_peered":"2024-09-11T12:55:14.803695+0000","last_clean":"2024-09-11T12:55:14.803695+0000","last_became_active":"2024-09-11T12:55:12.798866+0000","last_became_peered":"2024-09-11T12:55:12.798866+0000","last_unstale":"2024-09-11T12:55:14.803695+0000","last_undegraded":"2024-09-11T12:55:14.803695+0000","last_fullsized":"2024-09-11T12:55:14.803695+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:21:40.681972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803939+0000","last_change":"2024-09-11T12:55:11.782587+0000","last_active":"2024-09-11T12:55:14.803939+0000","last_peered":"2024-09-11T12:55:14.803939+0000","last_clean":"2024-09-11T12:55:14.803939+0000","last_became_active":"2024-09-11T12:55:11.782110+0000","last_became_peered":"2024-09-11T12:55:11.782110+0000","last_unstale":"2024-09-11T12:55:14.803939+0000","last_undegraded":"2024-09-11T12:55:14.803939+0000","last_fullsized":"2024-09-11T12:55:14.803939+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:09:31.545516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803832+0000","last_change":"2024-09-11T12:55:12.790950+0000","last_active":"2024-09-11T12:55:14.803832+0000","last_peered":"2024-09-11T12:55:14.803832+0000","last_clean":"2024-09-11T12:55:14.803832+0000","last_became_active":"2024-09-11T12:55:12.790674+0000","last_became_peered":"2024-09-11T12:55:12.790674+0000","last_unstale":"2024-09-11T12:55:14.803832+0000","last_undegraded":"2024-09-11T12:55:14.803832+0000","last_fullsized":"2024-09-11T12:55:14.803832+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:15.903891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"115'8","reported_seq":23,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803973+0000","last_change":"2024-09-11T12:55:11.780181+0000","last_active":"2024-09-11T12:55:14.803973+0000","last_peered":"2024-09-11T12:55:14.803973+0000","last_clean":"2024-09-11T12:55:14.803973+0000","last_became_active":"2024-09-11T12:55:11.779689+0000","last_became_peered":"2024-09-11T12:55:11.779689+0000","last_unstale":"2024-09-11T12:55:14.803973+0000","last_undegraded":"2024-09-11T12:55:14.803973+0000","last_fullsized":"2024-09-11T12:55:14.803973+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:33:55.294142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804156+0000","last_change":"2024-09-11T12:55:12.806469+0000","last_active":"2024-09-11T12:55:14.804156+0000","last_peered":"2024-09-11T12:55:14.804156+0000","last_clean":"2024-09-11T12:55:14.804156+0000","last_became_active":"2024-09-11T12:55:12.806061+0000","last_became_peered":"2024-09-11T12:55:12.806061+0000","last_unstale":"2024-09-11T12:55:14.804156+0000","last_undegraded":"2024-09-11T12:55:14.804156+0000","last_fullsized":"2024-09-11T12:55:14.804156+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790299+0000","last_change":"2024-09-11T12:55:11.778758+0000","last_active":"2024-09-11T12:55:13.790299+0000","last_peered":"2024-09-11T12:55:13.790299+0000","last_clean":"2024-09-11T12:55:13.790299+0000","last_became_active":"2024-09-11T12:55:11.778481+0000","last_became_peered":"2024-09-11T12:55:11.778481+0000","last_unstale":"2024-09-11T12:55:13.790299+0000","last_undegraded":"2024-09-11T12:55:13.790299+0000","last_fullsized":"2024-09-11T12:55:13.790299+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:59.194460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790215+0000","last_change":"2024-09-11T12:55:12.780204+0000","last_active":"2024-09-11T12:55:13.790215+0000","last_peered":"2024-09-11T12:55:13.790215+0000","last_clean":"2024-09-11T12:55:13.790215+0000","last_became_active":"2024-09-11T12:55:12.779830+0000","last_became_peered":"2024-09-11T12:55:12.779830+0000","last_unstale":"2024-09-11T12:55:13.790215+0000","last_undegraded":"2024-09-11T12:55:13.790215+0000","last_fullsized":"2024-09-11T12:55:13.790215+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:44.826592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803681+0000","last_change":"2024-09-11T12:55:11.779199+0000","last_active":"2024-09-11T12:55:14.803681+0000","last_peered":"2024-09-11T12:55:14.803681+0000","last_clean":"2024-09-11T12:55:14.803681+0000","last_became_active":"2024-09-11T12:55:11.778899+0000","last_became_peered":"2024-09-11T12:55:11.778899+0000","last_unstale":"2024-09-11T12:55:14.803681+0000","last_undegraded":"2024-09-11T12:55:14.803681+0000","last_fullsized":"2024-09-11T12:55:14.803681+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:01.174201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805145+0000","last_change":"2024-09-11T12:55:12.795629+0000","last_active":"2024-09-11T12:55:14.805145+0000","last_peered":"2024-09-11T12:55:14.805145+0000","last_clean":"2024-09-11T12:55:14.805145+0000","last_became_active":"2024-09-11T12:55:12.794996+0000","last_became_peered":"2024-09-11T12:55:12.794996+0000","last_unstale":"2024-09-11T12:55:14.805145+0000","last_undegraded":"2024-09-11T12:55:14.805145+0000","last_fullsized":"2024-09-11T12:55:14.805145+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:25.749449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"115'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789886+0000","last_change":"2024-09-11T12:55:11.770251+0000","last_active":"2024-09-11T12:55:13.789886+0000","last_peered":"2024-09-11T12:55:13.789886+0000","last_clean":"2024-09-11T12:55:13.789886+0000","last_became_active":"2024-09-11T12:55:11.769972+0000","last_became_peered":"2024-09-11T12:55:11.769972+0000","last_unstale":"2024-09-11T12:55:13.789886+0000","last_undegraded":"2024-09-11T12:55:13.789886+0000","last_fullsized":"2024-09-11T12:55:13.789886+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:45.579348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804384+0000","last_change":"2024-09-11T12:55:12.793032+0000","last_active":"2024-09-11T12:55:14.804384+0000","last_peered":"2024-09-11T12:55:14.804384+0000","last_clean":"2024-09-11T12:55:14.804384+0000","last_became_active":"2024-09-11T12:55:12.792759+0000","last_became_peered":"2024-09-11T12:55:12.792759+0000","last_unstale":"2024-09-11T12:55:14.804384+0000","last_undegraded":"2024-09-11T12:55:14.804384+0000","last_fullsized":"2024-09-11T12:55:14.804384+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:31:30.666618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790013+0000","last_change":"2024-09-11T12:55:11.776566+0000","last_active":"2024-09-11T12:55:13.790013+0000","last_peered":"2024-09-11T12:55:13.790013+0000","last_clean":"2024-09-11T12:55:13.790013+0000","last_became_active":"2024-09-11T12:55:11.776246+0000","last_became_peered":"2024-09-11T12:55:11.776246+0000","last_unstale":"2024-09-11T12:55:13.790013+0000","last_undegraded":"2024-09-11T12:55:13.790013+0000","last_fullsized":"2024-09-11T12:55:13.790013+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:15.211842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.787711+0000","last_change":"2024-09-11T12:55:12.780819+0000","last_active":"2024-09-11T12:55:13.787711+0000","last_peered":"2024-09-11T12:55:13.787711+0000","last_clean":"2024-09-11T12:55:13.787711+0000","last_became_active":"2024-09-11T12:55:12.780127+0000","last_became_peered":"2024-09-11T12:55:12.780127+0000","last_unstale":"2024-09-11T12:55:13.787711+0000","last_undegraded":"2024-09-11T12:55:13.787711+0000","last_fullsized":"2024-09-11T12:55:13.787711+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:41.112456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789281+0000","last_change":"2024-09-11T12:55:11.763717+0000","last_active":"2024-09-11T12:55:13.789281+0000","last_peered":"2024-09-11T12:55:13.789281+0000","last_clean":"2024-09-11T12:55:13.789281+0000","last_became_active":"2024-09-11T12:55:11.763368+0000","last_became_peered":"2024-09-11T12:55:11.763368+0000","last_unstale":"2024-09-11T12:55:13.789281+0000","last_undegraded":"2024-09-11T12:55:13.789281+0000","last_fullsized":"2024-09-11T12:55:13.789281+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:57:10.403675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790331+0000","last_change":"2024-09-11T12:55:12.788210+0000","last_active":"2024-09-11T12:55:13.790331+0000","last_peered":"2024-09-11T12:55:13.790331+0000","last_clean":"2024-09-11T12:55:13.790331+0000","last_became_active":"2024-09-11T12:55:12.787915+0000","last_became_peered":"2024-09-11T12:55:12.787915+0000","last_unstale":"2024-09-11T12:55:13.790331+0000","last_undegraded":"2024-09-11T12:55:13.790331+0000","last_fullsized":"2024-09-11T12:55:13.790331+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:13:49.781395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804871+0000","last_change":"2024-09-11T12:55:11.779857+0000","last_active":"2024-09-11T12:55:14.804871+0000","last_peered":"2024-09-11T12:55:14.804871+0000","last_clean":"2024-09-11T12:55:14.804871+0000","last_became_active":"2024-09-11T12:55:11.779533+0000","last_became_peered":"2024-09-11T12:55:11.779533+0000","last_unstale":"2024-09-11T12:55:14.804871+0000","last_undegraded":"2024-09-11T12:55:14.804871+0000","last_fullsized":"2024-09-11T12:55:14.804871+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:52:58.300814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803827+0000","last_change":"2024-09-11T12:55:12.786947+0000","last_active":"2024-09-11T12:55:14.803827+0000","last_peered":"2024-09-11T12:55:14.803827+0000","last_clean":"2024-09-11T12:55:14.803827+0000","last_became_active":"2024-09-11T12:55:12.786637+0000","last_became_peered":"2024-09-11T12:55:12.786637+0000","last_unstale":"2024-09-11T12:55:14.803827+0000","last_undegraded":"2024-09-11T12:55:14.803827+0000","last_fullsized":"2024-09-11T12:55:14.803827+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:18:46.586215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804746+0000","last_change":"2024-09-11T12:55:11.776937+0000","last_active":"2024-09-11T12:55:14.804746+0000","last_peered":"2024-09-11T12:55:14.804746+0000","last_clean":"2024-09-11T12:55:14.804746+0000","last_became_active":"2024-09-11T12:55:11.776621+0000","last_became_peered":"2024-09-11T12:55:11.776621+0000","last_unstale":"2024-09-11T12:55:14.804746+0000","last_undegraded":"2024-09-11T12:55:14.804746+0000","last_fullsized":"2024-09-11T12:55:14.804746+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:31:29.659751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794130+0000","last_change":"2024-09-11T12:55:12.801021+0000","last_active":"2024-09-11T12:55:13.794130+0000","last_peered":"2024-09-11T12:55:13.794130+0000","last_clean":"2024-09-11T12:55:13.794130+0000","last_became_active":"2024-09-11T12:55:12.800718+0000","last_became_peered":"2024-09-11T12:55:12.800718+0000","last_unstale":"2024-09-11T12:55:13.794130+0000","last_undegraded":"2024-09-11T12:55:13.794130+0000","last_fullsized":"2024-09-11T12:55:13.794130+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:08:47.522830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803325+0000","last_change":"2024-09-11T12:55:11.779532+0000","last_active":"2024-09-11T12:55:14.803325+0000","last_peered":"2024-09-11T12:55:14.803325+0000","last_clean":"2024-09-11T12:55:14.803325+0000","last_became_active":"2024-09-11T12:55:11.779274+0000","last_became_peered":"2024-09-11T12:55:11.779274+0000","last_unstale":"2024-09-11T12:55:14.803325+0000","last_undegraded":"2024-09-11T12:55:14.803325+0000","last_fullsized":"2024-09-11T12:55:14.803325+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:21:06.931611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790372+0000","last_change":"2024-09-11T12:55:12.778912+0000","last_active":"2024-09-11T12:55:13.790372+0000","last_peered":"2024-09-11T12:55:13.790372+0000","last_clean":"2024-09-11T12:55:13.790372+0000","last_became_active":"2024-09-11T12:55:12.778269+0000","last_became_peered":"2024-09-11T12:55:12.778269+0000","last_unstale":"2024-09-11T12:55:13.790372+0000","last_undegraded":"2024-09-11T12:55:13.790372+0000","last_fullsized":"2024-09-11T12:55:13.790372+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:37:31.911707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788657+0000","last_change":"2024-09-11T12:55:11.769333+0000","last_active":"2024-09-11T12:55:13.788657+0000","last_peered":"2024-09-11T12:55:13.788657+0000","last_clean":"2024-09-11T12:55:13.788657+0000","last_became_active":"2024-09-11T12:55:11.769005+0000","last_became_peered":"2024-09-11T12:55:11.769005+0000","last_unstale":"2024-09-11T12:55:13.788657+0000","last_undegraded":"2024-09-11T12:55:13.788657+0000","last_fullsized":"2024-09-11T12:55:13.788657+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:46.434600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.788528+0000","last_change":"2024-09-11T12:55:12.791932+0000","last_active":"2024-09-11T12:55:13.788528+0000","last_peered":"2024-09-11T12:55:13.788528+0000","last_clean":"2024-09-11T12:55:13.788528+0000","last_became_active":"2024-09-11T12:55:12.791605+0000","last_became_peered":"2024-09-11T12:55:12.791605+0000","last_unstale":"2024-09-11T12:55:13.788528+0000","last_undegraded":"2024-09-11T12:55:13.788528+0000","last_fullsized":"2024-09-11T12:55:13.788528+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:46.610702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"116'1","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.796182+0000","last_change":"2024-09-11T12:55:11.781652+0000","last_active":"2024-09-11T12:55:13.796182+0000","last_peered":"2024-09-11T12:55:13.796182+0000","last_clean":"2024-09-11T12:55:13.796182+0000","last_became_active":"2024-09-11T12:55:11.781312+0000","last_became_peered":"2024-09-11T12:55:11.781312+0000","last_unstale":"2024-09-11T12:55:13.796182+0000","last_undegraded":"2024-09-11T12:55:13.796182+0000","last_fullsized":"2024-09-11T12:55:13.796182+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:30:50.088994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804306+0000","last_change":"2024-09-11T12:55:12.796791+0000","last_active":"2024-09-11T12:55:14.804306+0000","last_peered":"2024-09-11T12:55:14.804306+0000","last_clean":"2024-09-11T12:55:14.804306+0000","last_became_active":"2024-09-11T12:55:12.796529+0000","last_became_peered":"2024-09-11T12:55:12.796529+0000","last_unstale":"2024-09-11T12:55:14.804306+0000","last_undegraded":"2024-09-11T12:55:14.804306+0000","last_fullsized":"2024-09-11T12:55:14.804306+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:27.268762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804492+0000","last_change":"2024-09-11T12:55:11.783642+0000","last_active":"2024-09-11T12:55:14.804492+0000","last_peered":"2024-09-11T12:55:14.804492+0000","last_clean":"2024-09-11T12:55:14.804492+0000","last_became_active":"2024-09-11T12:55:11.781680+0000","last_became_peered":"2024-09-11T12:55:11.781680+0000","last_unstale":"2024-09-11T12:55:14.804492+0000","last_undegraded":"2024-09-11T12:55:14.804492+0000","last_fullsized":"2024-09-11T12:55:14.804492+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:33:59.194460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.790033+0000","last_change":"2024-09-11T12:55:12.784793+0000","last_active":"2024-09-11T12:55:13.790033+0000","last_peered":"2024-09-11T12:55:13.790033+0000","last_clean":"2024-09-11T12:55:13.790033+0000","last_became_active":"2024-09-11T12:55:12.783097+0000","last_became_peered":"2024-09-11T12:55:12.783097+0000","last_unstale":"2024-09-11T12:55:13.790033+0000","last_undegraded":"2024-09-11T12:55:13.790033+0000","last_fullsized":"2024-09-11T12:55:13.790033+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:50.754830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803645+0000","last_change":"2024-09-11T12:55:11.785695+0000","last_active":"2024-09-11T12:55:14.803645+0000","last_peered":"2024-09-11T12:55:14.803645+0000","last_clean":"2024-09-11T12:55:14.803645+0000","last_became_active":"2024-09-11T12:55:11.785404+0000","last_became_peered":"2024-09-11T12:55:11.785404+0000","last_unstale":"2024-09-11T12:55:14.803645+0000","last_undegraded":"2024-09-11T12:55:14.803645+0000","last_fullsized":"2024-09-11T12:55:14.803645+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:24:55.687698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794114+0000","last_change":"2024-09-11T12:55:12.795081+0000","last_active":"2024-09-11T12:55:13.794114+0000","last_peered":"2024-09-11T12:55:13.794114+0000","last_clean":"2024-09-11T12:55:13.794114+0000","last_became_active":"2024-09-11T12:55:12.794808+0000","last_became_peered":"2024-09-11T12:55:12.794808+0000","last_unstale":"2024-09-11T12:55:13.794114+0000","last_undegraded":"2024-09-11T12:55:13.794114+0000","last_fullsized":"2024-09-11T12:55:13.794114+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:54.598632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804645+0000","last_change":"2024-09-11T12:55:11.782108+0000","last_active":"2024-09-11T12:55:14.804645+0000","last_peered":"2024-09-11T12:55:14.804645+0000","last_clean":"2024-09-11T12:55:14.804645+0000","last_became_active":"2024-09-11T12:55:11.781759+0000","last_became_peered":"2024-09-11T12:55:11.781759+0000","last_unstale":"2024-09-11T12:55:14.804645+0000","last_undegraded":"2024-09-11T12:55:14.804645+0000","last_fullsized":"2024-09-11T12:55:14.804645+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:15.211842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.789154+0000","last_change":"2024-09-11T12:55:12.777986+0000","last_active":"2024-09-11T12:55:13.789154+0000","last_peered":"2024-09-11T12:55:13.789154+0000","last_clean":"2024-09-11T12:55:13.789154+0000","last_became_active":"2024-09-11T12:55:12.777705+0000","last_became_peered":"2024-09-11T12:55:12.777705+0000","last_unstale":"2024-09-11T12:55:13.789154+0000","last_undegraded":"2024-09-11T12:55:13.789154+0000","last_fullsized":"2024-09-11T12:55:13.789154+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:52:09.078354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794290+0000","last_change":"2024-09-11T12:55:11.777911+0000","last_active":"2024-09-11T12:55:13.794290+0000","last_peered":"2024-09-11T12:55:13.794290+0000","last_clean":"2024-09-11T12:55:13.794290+0000","last_became_active":"2024-09-11T12:55:11.777379+0000","last_became_peered":"2024-09-11T12:55:11.777379+0000","last_unstale":"2024-09-11T12:55:13.794290+0000","last_undegraded":"2024-09-11T12:55:13.794290+0000","last_fullsized":"2024-09-11T12:55:13.794290+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:34:33.900412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.805022+0000","last_change":"2024-09-11T12:55:12.794383+0000","last_active":"2024-09-11T12:55:14.805022+0000","last_peered":"2024-09-11T12:55:14.805022+0000","last_clean":"2024-09-11T12:55:14.805022+0000","last_became_active":"2024-09-11T12:55:12.793907+0000","last_became_peered":"2024-09-11T12:55:12.793907+0000","last_unstale":"2024-09-11T12:55:14.805022+0000","last_undegraded":"2024-09-11T12:55:14.805022+0000","last_fullsized":"2024-09-11T12:55:14.805022+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:52:38.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.804780+0000","last_change":"2024-09-11T12:55:11.780050+0000","last_active":"2024-09-11T12:55:14.804780+0000","last_peered":"2024-09-11T12:55:14.804780+0000","last_clean":"2024-09-11T12:55:14.804780+0000","last_became_active":"2024-09-11T12:55:11.779772+0000","last_became_peered":"2024-09-11T12:55:11.779772+0000","last_unstale":"2024-09-11T12:55:14.804780+0000","last_undegraded":"2024-09-11T12:55:14.804780+0000","last_fullsized":"2024-09-11T12:55:14.804780+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:52.091452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803783+0000","last_change":"2024-09-11T12:55:12.796009+0000","last_active":"2024-09-11T12:55:14.803783+0000","last_peered":"2024-09-11T12:55:14.803783+0000","last_clean":"2024-09-11T12:55:14.803783+0000","last_became_active":"2024-09-11T12:55:12.793642+0000","last_became_peered":"2024-09-11T12:55:12.793642+0000","last_unstale":"2024-09-11T12:55:14.803783+0000","last_undegraded":"2024-09-11T12:55:14.803783+0000","last_fullsized":"2024-09-11T12:55:14.803783+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:55:47.231060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:14.803452+0000","last_change":"2024-09-11T12:55:11.778484+0000","last_active":"2024-09-11T12:55:14.803452+0000","last_peered":"2024-09-11T12:55:14.803452+0000","last_clean":"2024-09-11T12:55:14.803452+0000","last_became_active":"2024-09-11T12:55:11.778209+0000","last_became_peered":"2024-09-11T12:55:11.778209+0000","last_unstale":"2024-09-11T12:55:14.803452+0000","last_undegraded":"2024-09-11T12:55:14.803452+0000","last_fullsized":"2024-09-11T12:55:14.803452+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:12:45.904287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794198+0000","last_change":"2024-09-11T12:55:12.790694+0000","last_active":"2024-09-11T12:55:13.794198+0000","last_peered":"2024-09-11T12:55:13.794198+0000","last_clean":"2024-09-11T12:55:13.794198+0000","last_became_active":"2024-09-11T12:55:12.789799+0000","last_became_peered":"2024-09-11T12:55:12.789799+0000","last_unstale":"2024-09-11T12:55:13.794198+0000","last_undegraded":"2024-09-11T12:55:13.794198+0000","last_fullsized":"2024-09-11T12:55:13.794198+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:11.751574+0000","last_clean_scrub_stamp":"2024-09-11T12:55:11.751574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.076209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"115'2","reported_seq":17,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.794461+0000","last_change":"2024-09-11T12:55:11.782469+0000","last_active":"2024-09-11T12:55:13.794461+0000","last_peered":"2024-09-11T12:55:13.794461+0000","last_clean":"2024-09-11T12:55:13.794461+0000","last_became_active":"2024-09-11T12:55:11.782128+0000","last_became_peered":"2024-09-11T12:55:11.782128+0000","last_unstale":"2024-09-11T12:55:13.794461+0000","last_undegraded":"2024-09-11T12:55:13.794461+0000","last_fullsized":"2024-09-11T12:55:13.794461+0000","mapping_epoch":113,"log_start":"0'0","ondisk_log_start":"0'0","created":113,"last_epoch_clean":114,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:10.744706+0000","last_clean_scrub_stamp":"2024-09-11T12:55:10.744706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:23.457097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":308,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-11T12:55:13.793831+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:55:13.793831+0000","last_peered":"2024-09-11T12:55:13.793831+0000","last_clean":"2024-09-11T12:55:13.793831+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:55:13.793831+0000","last_undegraded":"2024-09-11T12:55:13.793831+0000","last_fullsized":"2024-09-11T12:55:13.793831+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509512,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":1224124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1441792,"data_stored":626818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509512,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":631618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1445888,"data_stored":631576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509512,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":631666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":632122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32592,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339248,"statfs":{"total":96636764160,"available":96603389952,"internally_reserved":0,"allocated":1429504,"data_stored":625206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33164,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338676,"statfs":{"total":96636764160,"available":96602804224,"internally_reserved":0,"allocated":2023424,"data_stored":1215574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:55:17.262 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:55:17.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 -- 172.21.15.78:0/3712599553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680b3c50 msgr2=0x7f0768141290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 --2- 172.21.15.78:0/3712599553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680b3c50 0x7f0768141290 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f075c009870 tx=0x7f075c01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 -- 172.21.15.78:0/3712599553 shutdown_connections 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 --2- 172.21.15.78:0/3712599553 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680ce830 0x7f07680cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 --2- 172.21.15.78:0/3712599553 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680b3c50 0x7f0768141290 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 -- 172.21.15.78:0/3712599553 >> 172.21.15.78:0/3712599553 conn(0x7f076806ce00 msgr2=0x7f07680cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:17.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 -- 172.21.15.78:0/3712599553 shutdown_connections 2024-09-11T12:55:17.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.447+0000 7f076fe00640 1 -- 172.21.15.78:0/3712599553 wait complete. 2024-09-11T12:55:17.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 Processor -- start 2024-09-11T12:55:17.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- start start 2024-09-11T12:55:17.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9de0 0x7f0768074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ce830 0x7f07681fa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f076816ab70 con 0x7f07680b3c50 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0768143490 con 0x7f07680ce830 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f076816acf0 con 0x7f07680c9de0 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57854/0 (socket says 172.21.15.78:57854) 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 -- 172.21.15.78:0/817887974 learned_addr learned my addr 172.21.15.78:0/817887974 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076de00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ce830 0x7f07681fa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076ca00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9de0 0x7f0768074740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 -- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9de0 msgr2=0x7f0768074740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9de0 0x7f0768074740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 -- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ce830 msgr2=0x7f07681fa8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ce830 0x7f07681fa8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 -- 172.21.15.78:0/817887974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0768139630 con 0x7f07680b3c50 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076d400640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f075c01ccd0 tx=0x7f075c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0ba020 con 0x7f07680b3c50 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f075c004330 con 0x7f07680b3c50 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0768072d50 con 0x7f07680b3c50 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f075c0b2040 con 0x7f07680b3c50 2024-09-11T12:55:17.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.451+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0768147310 con 0x7f07680b3c50 2024-09-11T12:55:17.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0768141290 con 0x7f07680b3c50 2024-09-11T12:55:17.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f075c0bd070 con 0x7f07680b3c50 2024-09-11T12:55:17.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f0765600640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 0x7f0748073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f075c12b920 con 0x7f07680b3c50 2024-09-11T12:55:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f076ca00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 0x7f0748073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.455+0000 7f076ca00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 0x7f0748073f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07680b39c0 tx=0x7f0754001420 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.459+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f075c016410 con 0x7f07680b3c50 2024-09-11T12:55:17.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.683+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f0768156e40 con 0x7f07680b3c50 2024-09-11T12:55:17.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.687+0000 7f0765600640 1 -- 172.21.15.78:0/817887974 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7f075c0f6bb0 con 0x7f07680b3c50 2024-09-11T12:55:17.689 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:55:17.690 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:55:17.690 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:55:17.690 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:55:17.690 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:55:17.690 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:55:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.687+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 msgr2=0x7f0748073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.687+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 0x7f0748073f30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07680b39c0 tx=0x7f0754001420 comp rx=0 tx=0).stop 2024-09-11T12:55:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.687+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 msgr2=0x7f0768146dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.687+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f075c01ccd0 tx=0x7f075c005db0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 shutdown_connections 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0748071a70 0x7f0748073f30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f07680ce830 0x7f07681fa8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f07680c9de0 0x7f0768074740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 --2- 172.21.15.78:0/817887974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f07680b3c50 0x7f0768146dd0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 >> 172.21.15.78:0/817887974 conn(0x7f076806ce00 msgr2=0x7f07680cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 shutdown_connections 2024-09-11T12:55:17.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.691+0000 7f076fe00640 1 -- 172.21.15.78:0/817887974 wait complete. 2024-09-11T12:55:17.711 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- 172.21.15.78:0/584716500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c8070360 msgr2=0x7f14c815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 --2- 172.21.15.78:0/584716500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c8070360 0x7f14c815b350 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009c80 tx=0x7f14c001cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- 172.21.15.78:0/584716500 shutdown_connections 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 --2- 172.21.15.78:0/584716500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c8070360 0x7f14c815b350 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 --2- 172.21.15.78:0/584716500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c80741e0 0x7f14c8070910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- 172.21.15.78:0/584716500 >> 172.21.15.78:0/584716500 conn(0x7f14c806ca80 msgr2=0x7f14c80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- 172.21.15.78:0/584716500 shutdown_connections 2024-09-11T12:55:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- 172.21.15.78:0/584716500 wait complete. 2024-09-11T12:55:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 Processor -- start 2024-09-11T12:55:17.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.851+0000 7f14cf200640 1 -- start start 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14c8070360 0x7f14c81515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 0x7f14c8163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14c815be20 con 0x7f14c80741e0 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14c815bca0 con 0x7f14c8070360 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f14c8167eb0 con 0x7f14c8072230 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57880/0 (socket says 172.21.15.78:57880) 2024-09-11T12:55:17.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 -- 172.21.15.78:0/1759216299 learned_addr learned my addr 172.21.15.78:0/1759216299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14c7e00640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14c8070360 0x7f14c81515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14c7400640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 0x7f14c8163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 -- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 msgr2=0x7f14c8163b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 0x7f14c8163b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 -- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14c8070360 msgr2=0x7f14c81515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14c8070360 0x7f14c81515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 -- 172.21.15.78:0/1759216299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c81f85e0 con 0x7f14c80741e0 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14c7400640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 0x7f14c8163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cd200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009bd0 tx=0x7f14c00a59f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00bb070 con 0x7f14c80741e0 2024-09-11T12:55:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14c0004d60 con 0x7f14c80741e0 2024-09-11T12:55:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00b2040 con 0x7f14c80741e0 2024-09-11T12:55:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c81f88c0 con 0x7f14c80741e0 2024-09-11T12:55:17.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c8157fe0 con 0x7f14c80741e0 2024-09-11T12:55:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.855+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c8157af0 con 0x7f14c80741e0 2024-09-11T12:55:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.859+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f14c0004320 con 0x7f14c80741e0 2024-09-11T12:55:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.859+0000 7f14bfe00640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 0x7f14a8073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.859+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f14c012c4c0 con 0x7f14c80741e0 2024-09-11T12:55:17.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.859+0000 7f14c7e00640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 0x7f14a8073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:17.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.859+0000 7f14c7e00640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 0x7f14a8073f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14c8152db0 tx=0x7f14b800a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:17.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:17.863+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f14c0016990 con 0x7f14c80741e0 2024-09-11T12:55:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f14c815b350 con 0x7f14c80741e0 2024-09-11T12:55:18.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14bfe00640 1 -- 172.21.15.78:0/1759216299 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5231 (secure 0 0 0) 0x7f14c00f7790 con 0x7f14c80741e0 2024-09-11T12:55:18.065 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:18.065 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":81,"btime":"2024-09-11T12:55:15:790926+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:18.065 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 81 2024-09-11T12:55:18.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 msgr2=0x7f14a8073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 0x7f14a8073f30 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f14c8152db0 tx=0x7f14b800a250 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 msgr2=0x7f14c81f7a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009bd0 tx=0x7f14c00a59f0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 shutdown_connections 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14c80741e0 0x7f14c81f7a80 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14c8072230 0x7f14c8163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f14a8071a70 0x7f14a8073f30 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 --2- 172.21.15.78:0/1759216299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14c8070360 0x7f14c81515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 >> 172.21.15.78:0/1759216299 conn(0x7f14c806ca80 msgr2=0x7f14c806fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 shutdown_connections 2024-09-11T12:55:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.063+0000 7f14cf200640 1 -- 172.21.15.78:0/1759216299 wait complete. 2024-09-11T12:55:18.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 81, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:55:12.153435+0000', 'modified': '2024-09-11T12:55:15.177704+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7000}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7000': {'gid': 7000, 'name': 'b', 'rank': 0, 'incarnation': 77, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/2586938426', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6800', 'nonce': 2586938426}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 2586938426}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7000, 'qdb_cluster': [7000]} 2024-09-11T12:55:18.488 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:55:18.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 -- 172.21.15.78:0/475804645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15eda0 msgr2=0x7f09bc15cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 --2- 172.21.15.78:0/475804645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15eda0 0x7f09bc15cf00 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0cfe10 tx=0x7f09b001ccf0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 -- 172.21.15.78:0/475804645 shutdown_connections 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 --2- 172.21.15.78:0/475804645 >> v2:172.21.15.157:3300/0 conn(0x7f09bc0b4520 0x7f09bc0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 --2- 172.21.15.78:0/475804645 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15eda0 0x7f09bc15cf00 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.627+0000 7f09c4600640 1 -- 172.21.15.78:0/475804645 >> 172.21.15.78:0/475804645 conn(0x7f09bc06ca80 msgr2=0x7f09bc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- 172.21.15.78:0/475804645 shutdown_connections 2024-09-11T12:55:18.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- 172.21.15.78:0/475804645 wait complete. 2024-09-11T12:55:18.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 Processor -- start 2024-09-11T12:55:18.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- start start 2024-09-11T12:55:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc0b4520 0x7f09bc1484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc15eda0 0x7f09bc14e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc0cd380 con 0x7f09bc15bee0 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc0cd200 con 0x7f09bc15eda0 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c4600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09bc0cd500 con 0x7f09bc0b4520 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57894/0 (socket says 172.21.15.78:57894) 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 -- 172.21.15.78:0/586304563 learned_addr learned my addr 172.21.15.78:0/586304563 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c2600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc15eda0 0x7f09bc14e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1c00640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc0b4520 0x7f09bc1484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 -- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc0b4520 msgr2=0x7f09bc1484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc0b4520 0x7f09bc1484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 -- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc15eda0 msgr2=0x7f09bc14e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc15eda0 0x7f09bc14e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 -- 172.21.15.78:0/586304563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09bc1512c0 con 0x7f09bc15bee0 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09c1200640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d950 tx=0x7f09a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09b9600640 1 -- 172.21.15.78:0/586304563 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8017070 con 0x7f09bc15bee0 2024-09-11T12:55:18.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09b9600640 1 -- 172.21.15.78:0/586304563 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a8009d20 con 0x7f09bc15bee0 2024-09-11T12:55:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.631+0000 7f09b9600640 1 -- 172.21.15.78:0/586304563 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80106c0 con 0x7f09bc15bee0 2024-09-11T12:55:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09bc1f8410 con 0x7f09bc15bee0 2024-09-11T12:55:18.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09bc1f89d0 con 0x7f09bc15bee0 2024-09-11T12:55:18.646 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09b9600640 1 -- 172.21.15.78:0/586304563 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f09a8010860 con 0x7f09bc15bee0 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09b9600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 0x7f099c073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09b9600640 1 -- 172.21.15.78:0/586304563 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f09a80928c0 con 0x7f09bc15bee0 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09c79dc000 1 Processor -- start 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09c1c00640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 0x7f099c073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.635+0000 7f09c79dc000 1 -- start start 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 0x7f099c073f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09b00af000 tx=0x7f09b001b530 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be567353c0 0x55be566e7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566e8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be56545270 con 0x55be56734aa0 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55be56534770 con 0x55be566e8340 2024-09-11T12:55:18.647 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55be565387b0 con 0x55be567353c0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57898/0 (socket says 172.21.15.78:57898) 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 -- 172.21.15.78:0/851024338 learned_addr learned my addr 172.21.15.78:0/851024338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1200640 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be567353c0 0x55be566e7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c2600640 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566e8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 -- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be567353c0 msgr2=0x55be566e7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be567353c0 0x55be566e7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 -- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 msgr2=0x55be566e8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566e8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 -- 172.21.15.78:0/851024338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be564d3f90 con 0x55be56734aa0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c1c00640 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f09b001c760 tx=0x7f09b00a7050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527d166fdaa52796 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09ae000640 1 -- 172.21.15.78:0/851024338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b00b2040 con 0x55be56734aa0 2024-09-11T12:55:18.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09ae000640 1 -- 172.21.15.78:0/851024338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09b00c8070 con 0x55be56734aa0 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09ae000640 1 -- 172.21.15.78:0/851024338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b0004d40 con 0x55be56734aa0 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- 172.21.15.78:0/851024338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 msgr2=0x55be56734e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f09b001c760 tx=0x7f09b00a7050 comp rx=0 tx=0).stop 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- 172.21.15.78:0/851024338 shutdown_connections 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566e8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be567353c0 0x55be566e7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 --2- 172.21.15.78:0/851024338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56734aa0 0x55be56734e80 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- 172.21.15.78:0/851024338 >> 172.21.15.78:0/851024338 conn(0x55be56715b60 msgr2=0x55be56717f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- 172.21.15.78:0/851024338 shutdown_connections 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- 172.21.15.78:0/851024338 wait complete. 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 -- start start 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.639+0000 7f09c79dc000 1 Processor -- start 2024-09-11T12:55:18.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be56734aa0 0x55be566f6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be56604290 con 0x55be567353c0 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55be564e0ab0 con 0x55be566e8340 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55be564de170 con 0x55be56734aa0 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 10 client.0 _ll_register_callbacks cb 0x55be566d5830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57912/0 (socket says 172.21.15.78:57912) 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 -- 172.21.15.78:0/3385879340 learned_addr learned my addr 172.21.15.78:0/3385879340 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c1c00640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c1200640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be56734aa0 0x55be566f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 -- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be56734aa0 msgr2=0x55be566f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be56734aa0 0x55be566f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 -- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 msgr2=0x55be566f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 -- 172.21.15.78:0/3385879340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be565b96f0 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c2600640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f09b40122c0 tx=0x7f09b4012780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccef90d3d45fbbfc server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005320 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55be564cdf10 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f09b4007050 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f09b40054c0 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 10 client.7506 fetch_fsmap learned FSMap version 81 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55be56590c40 con 0x55be567353c0 2024-09-11T12:55:18.651 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55be564cdf10 con 0x55be567353c0 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f09b4007340 con 0x55be567353c0 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 10 client.7506 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 10 client.7506 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 20 client.7506 populate_metadata read hostname 'smithi078' 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be566f71e0 0x55be56746b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 4 client.7506 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 --> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be565471c0 con 0x55be566f71e0 2024-09-11T12:55:18.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c1c00640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be566f71e0 0x55be56746b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09c1c00640 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be566f71e0 0x55be56746b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=479264d199e9a912 server_cookie=e33ad236489620d3 in_seq=0 out_seq=0 2024-09-11T12:55:18.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.643+0000 7f09a2a00640 10 client.7506 ms_handle_connect on v2:172.21.15.157:6800/2586938426 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09a2a00640 1 -- 172.21.15.78:0/3385879340 <== mds.0 v2:172.21.15.157:6800/2586938426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be565471c0 con 0x55be566f71e0 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09a2a00640 10 client.7506 handle_command_reply: tid=0 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 client.7506 shutdown 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 msgr2=0x55be566f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f09b40122c0 tx=0x7f09b4012780 comp rx=0 tx=0).stop 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 shutdown_connections 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be566f71e0 0x55be56746b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be567353c0 0x55be566f67d0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be56734aa0 0x55be566f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 --2- 172.21.15.78:0/3385879340 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be566f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 >> 172.21.15.78:0/3385879340 conn(0x55be56715b60 msgr2=0x55be566e7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09a2a00640 0 client.7506 ms_handle_reset on v2:172.21.15.157:6800/2586938426 2024-09-11T12:55:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 shutdown_connections 2024-09-11T12:55:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 1 -- 172.21.15.78:0/3385879340 wait complete. 2024-09-11T12:55:18.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.647+0000 7f09c79dc000 20 client.7506 trim_cache size 0 max 16384 2024-09-11T12:55:18.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 Processor -- start 2024-09-11T12:55:18.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 -- start start 2024-09-11T12:55:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566f04b0 0x55be567aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be567aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc2009e0 con 0x55be56715b60 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09bc1ef1a0 con 0x55be566e8340 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.663+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09bc1f5b60 con 0x55be566f04b0 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57916/0 (socket says 172.21.15.78:57916) 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 -- 172.21.15.78:0/2558147526 learned_addr learned my addr 172.21.15.78:0/2558147526 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1c00640 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566f04b0 0x55be567aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c2600640 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be567aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 -- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566f04b0 msgr2=0x55be567aa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566f04b0 0x55be567aa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 -- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 msgr2=0x55be567aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be567aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 -- 172.21.15.78:0/2558147526 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be564e9e70 con 0x55be56715b60 2024-09-11T12:55:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f09a805d6b0 tx=0x7f09a801d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b69fe8749e357da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09ad600640 1 -- 172.21.15.78:0/2558147526 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a808f070 con 0x55be56715b60 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09ad600640 1 -- 172.21.15.78:0/2558147526 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a805d9b0 con 0x55be56715b60 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09ad600640 1 -- 172.21.15.78:0/2558147526 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8020040 con 0x55be56715b60 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- 172.21.15.78:0/2558147526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 msgr2=0x55be567aa5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f09a805d6b0 tx=0x7f09a801d040 comp rx=0 tx=0).stop 2024-09-11T12:55:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- 172.21.15.78:0/2558147526 shutdown_connections 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566e8340 0x55be567aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567aa5e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- 172.21.15.78:0/2558147526 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566f04b0 0x55be567aa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- 172.21.15.78:0/2558147526 >> 172.21.15.78:0/2558147526 conn(0x55be566e71f0 msgr2=0x55be56742020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- 172.21.15.78:0/2558147526 shutdown_connections 2024-09-11T12:55:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- 172.21.15.78:0/2558147526 wait complete. 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- start start 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 Processor -- start 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566e8340 0x55be567ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566f04b0 0x55be567adf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55be5653cde0 con 0x55be56715b60 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55be56537ac0 con 0x55be566f04b0 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55be56547ea0 con 0x55be566e8340 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c79dc000 10 client.0 _ll_register_callbacks cb 0x7f09b4001340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57918/0 (socket says 172.21.15.78:57918) 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c2600640 1 -- 172.21.15.78:0/679506541 learned_addr learned my addr 172.21.15.78:0/679506541 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1200640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566f04b0 0x55be567adf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.667+0000 7f09c1c00640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566e8340 0x55be567ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 -- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566e8340 msgr2=0x55be567ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566e8340 0x55be567ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 -- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566f04b0 msgr2=0x55be567adf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566f04b0 0x55be567adf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 -- 172.21.15.78:0/679506541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be564bebf0 con 0x55be56715b60 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c2600640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f09b400efd0 tx=0x7f09b4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367cbbc71425d7b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b402e060 con 0x55be56715b60 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f09b4021670 con 0x55be56715b60 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a805d9b0 con 0x55be56715b60 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f09b4034050 con 0x55be56715b60 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 10 client.7512 fetch_fsmap learned FSMap version 81 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=81,monmap=2+}) -- 0x55be564dac60 con 0x55be56715b60 2024-09-11T12:55:18.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a805d9b0 con 0x55be56715b60 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 81) ==== 3502+0+0 (secure 0 0 0) 0x7f09b4021870 con 0x55be56715b60 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 10 client.7512 fetch_fsmap finished waiting for FSMap version 81 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 10 client.7512 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 20 client.7512 populate_metadata read hostname 'smithi078' 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be56734aa0 0x55be56769910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 4 client.7512 mds_command: new command op to 7000 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 --> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] -- command(tid 0: {"prefix": "status"}) -- 0x55be56542c60 con 0x55be56734aa0 2024-09-11T12:55:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c1c00640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be56734aa0 0x55be56769910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c1c00640 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be56734aa0 0x55be56769910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e340def3e2eaae6 server_cookie=6656dd0b083a3766 in_seq=0 out_seq=0 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 10 client.7512 ms_handle_connect on v2:172.21.15.157:6800/2586938426 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 1 -- 172.21.15.78:0/679506541 <== mds.0 v2:172.21.15.157:6800/2586938426 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55be56542c60 con 0x55be56734aa0 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 10 client.7512 handle_command_reply: tid=0 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 client.7512 shutdown 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 msgr2=0x55be567ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f09b400efd0 tx=0x7f09b4003d50 comp rx=0 tx=0).stop 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 shutdown_connections 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:6800/2586938426,v1:172.21.15.157:6835/2586938426] conn(0x55be56734aa0 0x55be56769910 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55be56715b60 0x55be567ae470 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55be566f04b0 0x55be567adf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 --2- 172.21.15.78:0/679506541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55be566e8340 0x55be567ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09a2a00640 0 client.7512 ms_handle_reset on v2:172.21.15.157:6800/2586938426 2024-09-11T12:55:18.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 >> 172.21.15.78:0/679506541 conn(0x55be566e71f0 msgr2=0x55be56713e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 shutdown_connections 2024-09-11T12:55:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 1 -- 172.21.15.78:0/679506541 wait complete. 2024-09-11T12:55:18.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c79dc000 20 client.7512 trim_cache size 0 max 16384 2024-09-11T12:55:18.688 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7000, 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:55:18.689 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.8935695480000003, 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 81, 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 116, 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 116, 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.428478561 2024-09-11T12:55:18.690 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:55:18.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 msgr2=0x7f099c073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 0x7f099c073f30 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f09b00af000 tx=0x7f09b001b530 comp rx=0 tx=0).stop 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 msgr2=0x7f09bc1508e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f09a800d950 tx=0x7f09a800de10 comp rx=0 tx=0).stop 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 shutdown_connections 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f099c071a70 0x7f099c073f30 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09bc15eda0 0x7f09bc14e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09bc15bee0 0x7f09bc1508e0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 --2- 172.21.15.78:0/586304563 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09bc0b4520 0x7f09bc1484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 >> 172.21.15.78:0/586304563 conn(0x7f09bc06ca80 msgr2=0x7f09bc06ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 shutdown_connections 2024-09-11T12:55:18.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.671+0000 7f09c4600640 1 -- 172.21.15.78:0/586304563 wait complete. 2024-09-11T12:55:18.694 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 -- 172.21.15.78:0/3247083380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 msgr2=0x7f15d40a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 --2- 172.21.15.78:0/3247083380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d40a9d00 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f15c4009900 tx=0x7f15c401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 -- 172.21.15.78:0/3247083380 shutdown_connections 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 --2- 172.21.15.78:0/3247083380 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40d0bf0 0x7f15d40d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 --2- 172.21.15.78:0/3247083380 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d40a9d00 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 -- 172.21.15.78:0/3247083380 >> 172.21.15.78:0/3247083380 conn(0x7f15d4074b90 msgr2=0x7f15d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:18.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 -- 172.21.15.78:0/3247083380 shutdown_connections 2024-09-11T12:55:18.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.835+0000 7f15d9000640 1 -- 172.21.15.78:0/3247083380 wait complete. 2024-09-11T12:55:18.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 Processor -- start 2024-09-11T12:55:18.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- start start 2024-09-11T12:55:18.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d41fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40b14b0 0x7f15d414eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4168eb0 con 0x7f15d40d0bf0 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f15d4168d30 con 0x7f15d406fdf0 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f15d4169030 con 0x7f15d40b14b0 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57930/0 (socket says 172.21.15.78:57930) 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 -- 172.21.15.78:0/1061275542 learned_addr learned my addr 172.21.15.78:0/1061275542 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d1e00640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d41fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d1400640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40b14b0 0x7f15d414eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 -- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40b14b0 msgr2=0x7f15d414eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40b14b0 0x7f15d414eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 -- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 msgr2=0x7f15d41fb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d41fb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 -- 172.21.15.78:0/1061275542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15d40adf80 con 0x7f15d40d0bf0 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d2800640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ea80 tx=0x7f15cc00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc004b30 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15cc004cf0 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15d40ae260 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15cc005280 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15d41faf00 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.839+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15d40a9d00 con 0x7f15d40d0bf0 2024-09-11T12:55:18.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.843+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f15cc005420 con 0x7f15d40d0bf0 2024-09-11T12:55:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.843+0000 7f15ca000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 0x7f15b0073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.843+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f15cc093100 con 0x7f15d40d0bf0 2024-09-11T12:55:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.843+0000 7f15d1e00640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 0x7f15b0073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:18.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.843+0000 7f15d1e00640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 0x7f15b0073f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15bc007fd0 tx=0x7f15bc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:18.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:18.847+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f15cc05e1f0 con 0x7f15d40d0bf0 2024-09-11T12:55:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.047+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f15d40d1240 con 0x7f15d40d0bf0 2024-09-11T12:55:19.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.047+0000 7f15ca000640 1 -- 172.21.15.78:0/1061275542 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f15cc062740 con 0x7f15d40d0bf0 2024-09-11T12:55:19.051 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 msgr2=0x7f15b0073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 0x7f15b0073f30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f15bc007fd0 tx=0x7f15bc009040 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 msgr2=0x7f15d40ad610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f15cc00ea80 tx=0x7f15cc00ef40 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 shutdown_connections 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f15d40d0bf0 0x7f15d40ad610 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f15d40b14b0 0x7f15d414eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f15b0071a70 0x7f15b0073f30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 --2- 172.21.15.78:0/1061275542 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f15d406fdf0 0x7f15d41fb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 >> 172.21.15.78:0/1061275542 conn(0x7f15d4074b90 msgr2=0x7f15d40b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:19.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 shutdown_connections 2024-09-11T12:55:19.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.051+0000 7f15d9000640 1 -- 172.21.15.78:0/1061275542 wait complete. 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:19.073 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:55:19.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:55:19.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:55:19.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:19.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.075 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:55:19.083 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:55:19.084 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42115sec preferred_lft 42115sec 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:19.085 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:55:19.086 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:19.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.087 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:55:19.148 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:55:19.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.151 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:19.203 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:55:19.204 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:55:19.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:55:19.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:19.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.336 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:55:19.346 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:55:19.346 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:55:19.346 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:19.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:55:19.407 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:19.407 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:19.459 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:19.460 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:19.460 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:19.511 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:55:19.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.511 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:19.598 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:19.599 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:19.613 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:19.614 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:19.614 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:55:19.614 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:55:19.614 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:55:19.617 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:55:19.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:19.618 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:19.640 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:55:19.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:55:19.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:19.754 INFO:teuthology.orchestra.run.smithi078.stdout:client7138 2024-09-11T12:55:19.754 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:19.754 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:19.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:55:19.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 -- 172.21.15.78:0/1311254329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128003ab0 msgr2=0x7f0128003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:19.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 --2- 172.21.15.78:0/1311254329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128003ab0 0x7f0128003ed0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f01440cd5e0 tx=0x7f013401ce30 comp rx=0 tx=0).stop 2024-09-11T12:55:19.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 -- 172.21.15.78:0/1311254329 shutdown_connections 2024-09-11T12:55:19.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 --2- 172.21.15.78:0/1311254329 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0128003ab0 0x7f0128003ed0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 -- 172.21.15.78:0/1311254329 >> 172.21.15.78:0/1311254329 conn(0x7f014406ca80 msgr2=0x7f01440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:19.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 -- 172.21.15.78:0/1311254329 shutdown_connections 2024-09-11T12:55:19.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 -- 172.21.15.78:0/1311254329 wait complete. 2024-09-11T12:55:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.951+0000 7f0149000640 1 Processor -- start 2024-09-11T12:55:19.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- start start 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 0x7f01440ab9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01440aebd0 0x7f01440a9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0144160000 con 0x7f014415ed20 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f014415fe80 con 0x7f0128003ab0 2024-09-11T12:55:19.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0144160180 con 0x7f01440aebd0 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57948/0 (socket says 172.21.15.78:57948) 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 -- 172.21.15.78:0/975902145 learned_addr learned my addr 172.21.15.78:0/975902145 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141e00640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 0x7f01440ab9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0142800640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01440aebd0 0x7f01440a9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:19.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 -- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01440aebd0 msgr2=0x7f01440a9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01440aebd0 0x7f01440a9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 -- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 msgr2=0x7f01440ab9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 0x7f01440ab9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 -- 172.21.15.78:0/975902145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01440aaf00 con 0x7f014415ed20 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141e00640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 0x7f01440ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:19.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0141400640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0130004d50 tx=0x7f0130001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01300107d0 con 0x7f014415ed20 2024-09-11T12:55:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0130016070 con 0x7f014415ed20 2024-09-11T12:55:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01441f82b0 con 0x7f014415ed20 2024-09-11T12:55:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0130010d20 con 0x7f014415ed20 2024-09-11T12:55:19.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01440ab1a0 con 0x7f014415ed20 2024-09-11T12:55:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.955+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f014406ff50 con 0x7f014415ed20 2024-09-11T12:55:19.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.959+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f013001b460 con 0x7f014415ed20 2024-09-11T12:55:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.959+0000 7f013a000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 0x7f0124073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:19.966 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.959+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f0130092ce0 con 0x7f014415ed20 2024-09-11T12:55:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.959+0000 7f0141e00640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 0x7f0124073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.959+0000 7f0141e00640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 0x7f0124073f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01340ab000 tx=0x7f0134019c70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:19.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:19.963+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f013005e4b0 con 0x7f014415ed20 2024-09-11T12:55:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.163+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f014415a0b0 con 0x7f014415ed20 2024-09-11T12:55:20.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.163+0000 7f013a000640 1 -- 172.21.15.78:0/975902145 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v81) ==== 53+0+73 (secure 0 0 0) 0x7f013005ddb0 con 0x7f014415ed20 2024-09-11T12:55:20.167 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 msgr2=0x7f0124073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 0x7f0124073f60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f01340ab000 tx=0x7f0134019c70 comp rx=0 tx=0).stop 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 msgr2=0x7f01441f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0130004d50 tx=0x7f0130001640 comp rx=0 tx=0).stop 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 shutdown_connections 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01440aebd0 0x7f01440a9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f014415ed20 0x7f01441f7d70 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:20.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0124071aa0 0x7f0124073f60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 --2- 172.21.15.78:0/975902145 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0128003ab0 0x7f01440ab9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 >> 172.21.15.78:0/975902145 conn(0x7f014406ca80 msgr2=0x7f014415a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 shutdown_connections 2024-09-11T12:55:20.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:20.167+0000 7f0149000640 1 -- 172.21.15.78:0/975902145 wait complete. 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:55:20.189 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:20.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.190 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:55:20.199 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:55:20.200 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42124sec preferred_lft 42124sec 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:20.201 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:55:20.202 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:20.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.203 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:55:20.267 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:55:20.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.269 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:55:20.327 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:55:20.328 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:55:20.328 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:20.328 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:20.328 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:20.328 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:55:20.328 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:55:20.423 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:55:20.466 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:20.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.473 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:55:20.482 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:55:20.482 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:55:20.483 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:20.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.483 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:55:20.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:20.546 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:20.599 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:20.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:20.599 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:20.650 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:55:20.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.650 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:20.739 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:20.740 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:20.753 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:20.753 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:20.753 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:55:20.753 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:55:20.753 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:55:20.757 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:55:20.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:20.757 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:20.776 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:55:20.865 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:20 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:55:20.868 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:20.875 INFO:teuthology.orchestra.run.smithi157.stdout:client7524 2024-09-11T12:55:20.875 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:20.875 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:20.885 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.031+0000 7f2f04e00640 1 -- 172.21.15.78:0/1233955069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00151050 msgr2=0x7f2f00073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.031+0000 7f2f04e00640 1 --2- 172.21.15.78:0/1233955069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00151050 0x7f2f00073950 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009840 tx=0x7f2ef001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- 172.21.15.78:0/1233955069 shutdown_connections 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 --2- 172.21.15.78:0/1233955069 >> v2:172.21.15.157:3300/0 conn(0x7f2f00072520 0x7f2f0006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 --2- 172.21.15.78:0/1233955069 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00151050 0x7f2f00073950 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- 172.21.15.78:0/1233955069 >> 172.21.15.78:0/1233955069 conn(0x7f2f000b3050 msgr2=0x7f2f000b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:21.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- 172.21.15.78:0/1233955069 shutdown_connections 2024-09-11T12:55:21.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- 172.21.15.78:0/1233955069 wait complete. 2024-09-11T12:55:21.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 Processor -- start 2024-09-11T12:55:21.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- start start 2024-09-11T12:55:21.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00073e90 0x7f2f0014db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f00151050 0x7f2f0014e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00070340 con 0x7f2f00072520 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f000701c0 con 0x7f2f00151050 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f000704c0 con 0x7f2f00073e90 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57952/0 (socket says 172.21.15.78:57952) 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 -- 172.21.15.78:0/2569898415 learned_addr learned my addr 172.21.15.78:0/2569898415 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 -- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00073e90 msgr2=0x7f2f0014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00073e90 0x7f2f0014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 -- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f00151050 msgr2=0x7f2f0014e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f00151050 0x7f2f0014e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 -- 172.21.15.78:0/2569898415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0014b620 con 0x7f2f00072520 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2efde00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001ccd0 tx=0x7f2ef0005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef00b2040 con 0x7f2f00072520 2024-09-11T12:55:21.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef0004130 con 0x7f2f00072520 2024-09-11T12:55:21.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f0014b340 con 0x7f2f00072520 2024-09-11T12:55:21.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.035+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0003600 con 0x7f2f00072520 2024-09-11T12:55:21.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f0014fe30 con 0x7f2f00072520 2024-09-11T12:55:21.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f0006f5d0 con 0x7f2f00072520 2024-09-11T12:55:21.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f2ef00bd030 con 0x7f2f00072520 2024-09-11T12:55:21.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2ef6000640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 0x7f2ee0073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef012bf60 con 0x7f2f00072520 2024-09-11T12:55:21.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2efd400640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 0x7f2ee0073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.039+0000 7f2efd400640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 0x7f2ee0073f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0046c0 tx=0x7f2eec007910 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:21.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.043+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2ef00c4060 con 0x7f2f00072520 2024-09-11T12:55:21.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.247+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f2f00070ce0 con 0x7f2f00072520 2024-09-11T12:55:21.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.815+0000 7f2ef6000640 1 -- 172.21.15.78:0/2569898415 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v82) ==== 136+0+0 (secure 0 0 0) 0x7f2ef0016410 con 0x7f2f00072520 2024-09-11T12:55:21.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 msgr2=0x7f2ee0073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 0x7f2ee0073f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2eec0046c0 tx=0x7f2eec007910 comp rx=0 tx=0).stop 2024-09-11T12:55:21.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 msgr2=0x7f2f001fa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f2ef001ccd0 tx=0x7f2ef0005ca0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 shutdown_connections 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee0071a70 0x7f2ee0073f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f00151050 0x7f2f0014e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f00073e90 0x7f2f0014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 --2- 172.21.15.78:0/2569898415 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f00072520 0x7f2f001fa050 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 >> 172.21.15.78:0/2569898415 conn(0x7f2f000b3050 msgr2=0x7f2f000ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 shutdown_connections 2024-09-11T12:55:21.822 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.819+0000 7f2f04e00640 1 -- 172.21.15.78:0/2569898415 wait complete. 2024-09-11T12:55:21.843 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:55:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 -- 172.21.15.78:0/2317856619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 msgr2=0x7f96b406fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 --2- 172.21.15.78:0/2317856619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b406fa20 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f96b40cca90 tx=0x7f96ac01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 -- 172.21.15.78:0/2317856619 shutdown_connections 2024-09-11T12:55:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 --2- 172.21.15.78:0/2317856619 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b406fa20 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 --2- 172.21.15.78:0/2317856619 >> v2:172.21.15.157:3300/0 conn(0x7f96b406cd60 0x7f96b415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 -- 172.21.15.78:0/2317856619 >> 172.21.15.78:0/2317856619 conn(0x7f96b40ccd10 msgr2=0x7f96b40cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 -- 172.21.15.78:0/2317856619 shutdown_connections 2024-09-11T12:55:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.983+0000 7f96b9c00640 1 -- 172.21.15.78:0/2317856619 wait complete. 2024-09-11T12:55:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 Processor -- start 2024-09-11T12:55:21.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- start start 2024-09-11T12:55:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b406cd60 0x7f96b4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b41585f0 0x7f96b4157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4167840 con 0x7f96b4073080 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f96b4070be0 con 0x7f96b41585f0 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f96b41679c0 con 0x7f96b406cd60 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57968/0 (socket says 172.21.15.78:57968) 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 -- 172.21.15.78:0/1815336988 learned_addr learned my addr 172.21.15.78:0/1815336988 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b3400640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b41585f0 0x7f96b4157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2a00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b406cd60 0x7f96b4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 -- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b406cd60 msgr2=0x7f96b4151b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b406cd60 0x7f96b4151b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 -- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b41585f0 msgr2=0x7f96b4157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b41585f0 0x7f96b4157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:21.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 -- 172.21.15.78:0/1815336988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b40ab7a0 con 0x7f96b4073080 2024-09-11T12:55:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b2000640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d980 tx=0x7f96a000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0017070 con 0x7f96b4073080 2024-09-11T12:55:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a0009d60 con 0x7f96b4073080 2024-09-11T12:55:21.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a0005280 con 0x7f96b4073080 2024-09-11T12:55:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b40aba80 con 0x7f96b4073080 2024-09-11T12:55:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.987+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b40ac040 con 0x7f96b4073080 2024-09-11T12:55:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b415b950 con 0x7f96b4073080 2024-09-11T12:55:21.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f96a0005420 con 0x7f96b4073080 2024-09-11T12:55:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96aaa00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 0x7f9694073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f96a0092630 con 0x7f96b4073080 2024-09-11T12:55:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96b2a00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 0x7f9694073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:21.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.991+0000 7f96b2a00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 0x7f9694073f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a40012f0 tx=0x7f96a4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:22.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:21.995+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f96a005d7b0 con 0x7f96b4073080 2024-09-11T12:55:22.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.183+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f96b4072980 con 0x7f9694071a70 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.183+0000 7f96aaa00640 1 -- 172.21.15.78:0/1815336988 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f96b4072980 con 0x7f9694071a70 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:22.186 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:55:22.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 msgr2=0x7f9694073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 0x7f9694073f30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f96a40012f0 tx=0x7f96a4009040 comp rx=0 tx=0).stop 2024-09-11T12:55:22.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 msgr2=0x7f96b40af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f96a000d980 tx=0x7f96a000de40 comp rx=0 tx=0).stop 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 shutdown_connections 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9694071a70 0x7f9694073f30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f96b41585f0 0x7f96b4157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f96b4073080 0x7f96b40af250 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 --2- 172.21.15.78:0/1815336988 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f96b406cd60 0x7f96b4151b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 >> 172.21.15.78:0/1815336988 conn(0x7f96b40ccd10 msgr2=0x7f96b406c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 shutdown_connections 2024-09-11T12:55:22.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.187+0000 7f96b9c00640 1 -- 172.21.15.78:0/1815336988 wait complete. 2024-09-11T12:55:22.209 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:55:22.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- 172.21.15.78:0/924788103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 msgr2=0x7fb2b40cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 --2- 172.21.15.78:0/924788103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b40cdc10 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fb2a0009900 tx=0x7fb2a001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- 172.21.15.78:0/924788103 shutdown_connections 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 --2- 172.21.15.78:0/924788103 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b40cdc10 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 --2- 172.21.15.78:0/924788103 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b41518b0 0x7fb2b40b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- 172.21.15.78:0/924788103 >> 172.21.15.78:0/924788103 conn(0x7fb2b4071dd0 msgr2=0x7fb2b40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:22.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- 172.21.15.78:0/924788103 shutdown_connections 2024-09-11T12:55:22.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- 172.21.15.78:0/924788103 wait complete. 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 Processor -- start 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.351+0000 7fb2bb400640 1 -- start start 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b40ce150 0x7fb2b4145cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 0x7fb2b40b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:22.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b4167960 con 0x7fb2b40b1f50 2024-09-11T12:55:22.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b41677e0 con 0x7fb2b41518b0 2024-09-11T12:55:22.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b4167ae0 con 0x7fb2b40ce150 2024-09-11T12:55:22.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:22.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57978/0 (socket says 172.21.15.78:57978) 2024-09-11T12:55:22.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 -- 172.21.15.78:0/3664851338 learned_addr learned my addr 172.21.15.78:0/3664851338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:22.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b3e00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b40ce150 0x7fb2b4145cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:22.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b9400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 0x7fb2b40b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:22.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 -- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b40ce150 msgr2=0x7fb2b4145cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b40ce150 0x7fb2b4145cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 -- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 msgr2=0x7fb2b40b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 0x7fb2b40b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 -- 172.21.15.78:0/3664851338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41f7280 con 0x7fb2b40b1f50 2024-09-11T12:55:22.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b9400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 0x7fb2b40b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:22.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b8a00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400ebc0 tx=0x7fb2a400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:22.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4010630 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2a40108c0 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41f7560 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4010cf0 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41f7b20 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b40cdc10 con 0x7fb2b40b1f50 2024-09-11T12:55:22.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.355+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fb2a4018420 con 0x7fb2b40b1f50 2024-09-11T12:55:22.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.359+0000 7fb2b0c00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 0x7fb294073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:22.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.359+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fb2a4093270 con 0x7fb2b40b1f50 2024-09-11T12:55:22.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.359+0000 7fb2b3e00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 0x7fb294073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:22.367 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.359+0000 7fb2b3e00640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 0x7fb294073f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00af000 tx=0x7fb2a001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:22.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.363+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb2a405e360 con 0x7fb2b40b1f50 2024-09-11T12:55:22.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.527+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb2b4147d40 con 0x7fb2b40b1f50 2024-09-11T12:55:22.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.527+0000 7fb2b0c00640 1 -- 172.21.15.78:0/3664851338 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7fb2a40628b0 con 0x7fb2b40b1f50 2024-09-11T12:55:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 msgr2=0x7fb294073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 0x7fb294073f30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb2a00af000 tx=0x7fb2a001b120 comp rx=0 tx=0).stop 2024-09-11T12:55:22.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 msgr2=0x7fb2b415d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400ebc0 tx=0x7fb2a400c6a0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 shutdown_connections 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb294071a70 0x7fb294073f30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2b41518b0 0x7fb2b40b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2b40ce150 0x7fb2b4145cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 --2- 172.21.15.78:0/3664851338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2b40b1f50 0x7fb2b415d2c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 >> 172.21.15.78:0/3664851338 conn(0x7fb2b4071dd0 msgr2=0x7fb2b414e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 shutdown_connections 2024-09-11T12:55:22.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:22.531+0000 7fb2bb400640 1 -- 172.21.15.78:0/3664851338 wait complete. 2024-09-11T12:55:22.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.552 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol10650/43c255bf-2a7b-476d-82ad-34e906a0f03e) 2024-09-11T12:55:22.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.581 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol10650/43c255bf-2a7b-476d-82ad-34e906a0f03e/.snap/fake) 2024-09-11T12:55:22.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.652 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:55:22.709 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:55:22.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.709 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol10650/43c255bf-2a7b-476d-82ad-34e906a0f03e) 2024-09-11T12:55:22.779 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:22.779 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol10650/.meta 2024-09-11T12:55:22.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.844 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp44900/subvol34341/5d6a152b-c605-4425-934b-b945d5abbda1) 2024-09-11T12:55:22.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.916 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp44900/subvol34341/5d6a152b-c605-4425-934b-b945d5abbda1/.snap/fake) 2024-09-11T12:55:22.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:22.987 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:55:23.041 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:55:23.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:23.041 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp44900/subvol34341/5d6a152b-c605-4425-934b-b945d5abbda1) 2024-09-11T12:55:23.108 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:23.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp44900/subvol34341/.meta 2024-09-11T12:55:23.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:23.172 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol66422/208e9e59-07c8-4571-9366-35bf46eb72c7) 2024-09-11T12:55:23.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:23.238 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:55:23.289 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:55:23.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:23.290 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol66422/208e9e59-07c8-4571-9366-35bf46eb72c7) 2024-09-11T12:55:23.360 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:23.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol66422/.meta 2024-09-11T12:55:23.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol10650 2024-09-11T12:55:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.607+0000 7f6b76200640 1 -- 172.21.15.78:0/3668736157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b54003a90 msgr2=0x7f6b54003eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.607+0000 7f6b76200640 1 --2- 172.21.15.78:0/3668736157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b54003a90 0x7f6b54003eb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f6b60001dc0 tx=0x7f6b6001cb30 comp rx=0 tx=0).stop 2024-09-11T12:55:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6fe00640 1 -- 172.21.15.78:0/3668736157 reap_dead start 2024-09-11T12:55:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- 172.21.15.78:0/3668736157 shutdown_connections 2024-09-11T12:55:23.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- 172.21.15.78:0/3668736157 >> 172.21.15.78:0/3668736157 conn(0x7f6b70075610 msgr2=0x7f6b70075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- 172.21.15.78:0/3668736157 shutdown_connections 2024-09-11T12:55:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- 172.21.15.78:0/3668736157 wait complete. 2024-09-11T12:55:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 Processor -- start 2024-09-11T12:55:23.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- start start 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b54003a90 0x7f6b70156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b70152f70 0x7f6b701446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b701679f0 con 0x7f6b700b1ff0 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b70167870 con 0x7f6b54003a90 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b76200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6b70167b70 con 0x7f6b70152f70 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6f400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b54003a90 0x7f6b70156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:23.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:57994/0 (socket says 172.21.15.78:57994) 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 -- 172.21.15.78:0/1485493992 learned_addr learned my addr 172.21.15.78:0/1485493992 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6fe00640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b70152f70 0x7f6b701446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 -- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b70152f70 msgr2=0x7f6b701446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b70152f70 0x7f6b701446c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 -- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b54003a90 msgr2=0x7f6b70156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b54003a90 0x7f6b70156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 -- 172.21.15.78:0/1485493992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b701539b0 con 0x7f6b700b1ff0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.611+0000 7f6b6ea00640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c001410 tx=0x7f6b5c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c015070 con 0x7f6b700b1ff0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b5c011940 con 0x7f6b700b1ff0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c011d90 con 0x7f6b700b1ff0 2024-09-11T12:55:23.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b70153c90 con 0x7f6b700b1ff0 2024-09-11T12:55:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7015d8c0 con 0x7f6b700b1ff0 2024-09-11T12:55:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b700d04b0 con 0x7f6b700b1ff0 2024-09-11T12:55:23.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f6b5c018070 con 0x7f6b700b1ff0 2024-09-11T12:55:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.615+0000 7f6b67400640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 0x7f6b50073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.619+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f6b5c093150 con 0x7f6b700b1ff0 2024-09-11T12:55:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.619+0000 7f6b6f400640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 0x7f6b50073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:23.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.619+0000 7f6b6f400640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 0x7f6b50073f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b600aa000 tx=0x7f6b600b1040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:23.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.623+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6b5c05e310 con 0x7f6b700b1ff0 2024-09-11T12:55:23.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.811+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol10650", "target": ["mon-mgr", ""]}) -- 0x7f6b70153510 con 0x7f6b50071a70 2024-09-11T12:55:23.822 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:23.819+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:23.822 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:23.819+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:23.822 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:23.819+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:23.822 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:23.819+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:23.822 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:23.819+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:23.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b67400640 1 -- 172.21.15.78:0/1485493992 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f6b70153510 con 0x7f6b50071a70 2024-09-11T12:55:23.865 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol10650/43c255bf-2a7b-476d-82ad-34e906a0f03e 2024-09-11T12:55:23.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 msgr2=0x7f6b50073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 0x7f6b50073f30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6b600aa000 tx=0x7f6b600b1040 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 msgr2=0x7f6b7015d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c001410 tx=0x7f6b5c0039d0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 shutdown_connections 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b70152f70 0x7f6b701446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b700b1ff0 0x7f6b7015d380 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b50071a70 0x7f6b50073f30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 --2- 172.21.15.78:0/1485493992 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b54003a90 0x7f6b70156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 >> 172.21.15.78:0/1485493992 conn(0x7f6b70075610 msgr2=0x7f6b700aa660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 shutdown_connections 2024-09-11T12:55:23.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:23.863+0000 7f6b76200640 1 -- 172.21.15.78:0/1485493992 wait complete. 2024-09-11T12:55:23.887 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol34341 subvol_grp44900 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- 172.21.15.78:0/3265796919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc073e90 msgr2=0x7f62cc071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- 172.21.15.78:0/3265796919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc073e90 0x7f62cc071e10 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f62b800aa10 tx=0x7f62b801dc00 comp rx=0 tx=0).stop 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- 172.21.15.78:0/3265796919 shutdown_connections 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- 172.21.15.78:0/3265796919 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc073e90 0x7f62cc071e10 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- 172.21.15.78:0/3265796919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc151050 0x7f62cc073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- 172.21.15.78:0/3265796919 >> 172.21.15.78:0/3265796919 conn(0x7f62cc0b3050 msgr2=0x7f62cc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- 172.21.15.78:0/3265796919 shutdown_connections 2024-09-11T12:55:24.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- 172.21.15.78:0/3265796919 wait complete. 2024-09-11T12:55:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 Processor -- start 2024-09-11T12:55:24.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- start start 2024-09-11T12:55:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc072350 0x7f62cc14dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc073e90 0x7f62cc148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc15e970 con 0x7f62cc151050 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f62cc15e7f0 con 0x7f62cc073e90 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62d2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f62cc15eaf0 con 0x7f62cc072350 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46724/0 (socket says 172.21.15.78:46724) 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62cbe00640 1 -- 172.21.15.78:0/4161234615 learned_addr learned my addr 172.21.15.78:0/4161234615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62cb400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc072350 0x7f62cc14dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.031+0000 7f62caa00640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc073e90 0x7f62cc148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 -- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc072350 msgr2=0x7f62cc14dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc072350 0x7f62cc14dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 -- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc073e90 msgr2=0x7f62cc148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc073e90 0x7f62cc148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 -- 172.21.15.78:0/4161234615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62cc14cd10 con 0x7f62cc151050 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62cbe00640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7f0 tx=0x7f62c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c4003b80 con 0x7f62cc151050 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62c4003d20 con 0x7f62cc151050 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c400aa50 con 0x7f62cc151050 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62cc14cff0 con 0x7f62cc151050 2024-09-11T12:55:24.038 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62cc148c70 con 0x7f62cc151050 2024-09-11T12:55:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62cc071e10 con 0x7f62cc151050 2024-09-11T12:55:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f62c400abf0 con 0x7f62cc151050 2024-09-11T12:55:24.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 0x7f62ac073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.035+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f62c40921f0 con 0x7f62cc151050 2024-09-11T12:55:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.039+0000 7f62cb400640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 0x7f62ac073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.039+0000 7f62cb400640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 0x7f62ac073f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004630 tx=0x7f62bc00b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.039+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f62c405d480 con 0x7f62cc151050 2024-09-11T12:55:24.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.231+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol34341", "group_name": "subvol_grp44900", "target": ["mon-mgr", ""]}) -- 0x7f62cc070ce0 con 0x7f62ac071a70 2024-09-11T12:55:24.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62c3400640 1 -- 172.21.15.78:0/4161234615 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7f62cc070ce0 con 0x7f62ac071a70 2024-09-11T12:55:24.277 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp44900/subvol34341/5d6a152b-c605-4425-934b-b945d5abbda1 2024-09-11T12:55:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 msgr2=0x7f62ac073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 0x7f62ac073f30 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f62bc004630 tx=0x7f62bc00b2e0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 msgr2=0x7f62cc163880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f62c400c7f0 tx=0x7f62c400ccb0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 shutdown_connections 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f62cc151050 0x7f62cc163880 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f62ac071a70 0x7f62ac073f30 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f62cc073e90 0x7f62cc148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 --2- 172.21.15.78:0/4161234615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f62cc072350 0x7f62cc14dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 >> 172.21.15.78:0/4161234615 conn(0x7f62cc0b3050 msgr2=0x7f62cc158f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 shutdown_connections 2024-09-11T12:55:24.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.275+0000 7f62d2800640 1 -- 172.21.15.78:0/4161234615 wait complete. 2024-09-11T12:55:24.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol66422 2024-09-11T12:55:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 -- 172.21.15.78:0/3117899942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 msgr2=0x7f93340cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 --2- 172.21.15.78:0/3117899942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f93340cf6e0 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f93340b38b0 tx=0x7f931c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 -- 172.21.15.78:0/3117899942 shutdown_connections 2024-09-11T12:55:24.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 --2- 172.21.15.78:0/3117899942 >> v2:172.21.15.157:3300/0 conn(0x7f93340cd770 0x7f93340cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 --2- 172.21.15.78:0/3117899942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f93340cf6e0 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 -- 172.21.15.78:0/3117899942 >> 172.21.15.78:0/3117899942 conn(0x7f9334071dd0 msgr2=0x7f93340721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 -- 172.21.15.78:0/3117899942 shutdown_connections 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 -- 172.21.15.78:0/3117899942 wait complete. 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.439+0000 7f933b200640 1 Processor -- start 2024-09-11T12:55:24.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- start start 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 0x7f933413db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 0x7f933406fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f93340af160 con 0x7f93340b3420 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f93340aefe0 con 0x7f93340cd770 2024-09-11T12:55:24.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f93340af2e0 con 0x7f93340b3040 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46744/0 (socket says 172.21.15.78:46744) 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 -- 172.21.15.78:0/433668939 learned_addr learned my addr 172.21.15.78:0/433668939 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9333e00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 0x7f933413db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9339200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 0x7f933406fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 -- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 msgr2=0x7f933413db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 0x7f933413db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 -- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 msgr2=0x7f933406fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 0x7f933406fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 -- 172.21.15.78:0/433668939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93341f8850 con 0x7f93340b3420 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9339200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 0x7f933406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9333e00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 0x7f933413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:24.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f932be00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f931c01d160 tx=0x7f931c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0ba020 con 0x7f93340b3420 2024-09-11T12:55:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f931c005030 con 0x7f93340b3420 2024-09-11T12:55:24.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93341f8b30 con 0x7f93340b3420 2024-09-11T12:55:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f931c0b2040 con 0x7f93340b3420 2024-09-11T12:55:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f933413e0a0 con 0x7f93340b3420 2024-09-11T12:55:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.443+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93340cf6e0 con 0x7f93340b3420 2024-09-11T12:55:24.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.447+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f931c0046d0 con 0x7f93340b3420 2024-09-11T12:55:24.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.447+0000 7f9330c00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 0x7f9310073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.447+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f931c12c640 con 0x7f93340b3420 2024-09-11T12:55:24.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.447+0000 7f9333e00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 0x7f9310073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.447+0000 7f9333e00640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 0x7f9310073f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93240016e0 tx=0x7f9324009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.451+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f931c0164b0 con 0x7f93340b3420 2024-09-11T12:55:24.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.647+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol66422", "target": ["mon-mgr", ""]}) -- 0x7f93341f8600 con 0x7f9310071a70 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f9330c00640 1 -- 172.21.15.78:0/433668939 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f93341f8600 con 0x7f9310071a70 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "status": { 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "pending", 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "source": { 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "volume": "fake", 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "subvolume": "fake", 2024-09-11T12:55:24.685 INFO:teuthology.orchestra.run.smithi078.stdout: "snapshot": "fake" 2024-09-11T12:55:24.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:24.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:24.686 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 msgr2=0x7f9310073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 0x7f9310073f30 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f93240016e0 tx=0x7f9324009040 comp rx=0 tx=0).stop 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 msgr2=0x7f933406f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f931c01d160 tx=0x7f931c004040 comp rx=0 tx=0).stop 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 shutdown_connections 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9310071a70 0x7f9310073f30 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f93340cd770 0x7f933406fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f93340b3420 0x7f933406f6c0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 --2- 172.21.15.78:0/433668939 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93340b3040 0x7f933413db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 >> 172.21.15.78:0/433668939 conn(0x7f9334071dd0 msgr2=0x7f93340721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 shutdown_connections 2024-09-11T12:55:24.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.683+0000 7f933b200640 1 -- 172.21.15.78:0/433668939 wait complete. 2024-09-11T12:55:24.705 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol10650 fake 2024-09-11T12:55:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- 172.21.15.78:0/2541748541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 msgr2=0x7fa5fc0b30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 --2- 172.21.15.78:0/2541748541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc0b30d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009900 tx=0x7fa5e801c880 comp rx=0 tx=0).stop 2024-09-11T12:55:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- 172.21.15.78:0/2541748541 shutdown_connections 2024-09-11T12:55:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 --2- 172.21.15.78:0/2541748541 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc0b30d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 --2- 172.21.15.78:0/2541748541 >> v2:172.21.15.157:3300/0 conn(0x7fa5fc0cdb10 0x7fa5fc15c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- 172.21.15.78:0/2541748541 >> 172.21.15.78:0/2541748541 conn(0x7fa5fc0b1cb0 msgr2=0x7fa5fc070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- 172.21.15.78:0/2541748541 shutdown_connections 2024-09-11T12:55:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- 172.21.15.78:0/2541748541 wait complete. 2024-09-11T12:55:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 Processor -- start 2024-09-11T12:55:24.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.851+0000 7fa604400640 1 -- start start 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc15aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 0x7fa5fc150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc168eb0 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa5fc168d30 con 0x7fa5fc0ce8a0 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa5fc169030 con 0x7fa5fc0b4520 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46750/0 (socket says 172.21.15.78:46750) 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 -- 172.21.15.78:0/2135124151 learned_addr learned my addr 172.21.15.78:0/2135124151 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601a00640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc15aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 -- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 msgr2=0x7fa5fc15aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc15aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 -- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 msgr2=0x7fa5fc150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa602400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 0x7fa5fc150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 0x7fa5fc150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 -- 172.21.15.78:0/2135124151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5fc0ac800 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601a00640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa602400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 0x7fa5fc150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa601000640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009850 tx=0x7fa5e8005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80b2040 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5e80a5ca0 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5fc14b980 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5e8005280 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5fc14bc20 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.855+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5fc0b30d0 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.859+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fa5e8005420 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.859+0000 7fa5f9600640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 0x7fa5dc073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.859+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fa5e812bae0 con 0x7fa5fc0cdb10 2024-09-11T12:55:24.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.859+0000 7fa601a00640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 0x7fa5dc073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:24.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.859+0000 7fa601a00640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 0x7fa5dc073f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009b60 tx=0x7fa5f00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:24.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:24.863+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa5e80bd030 con 0x7fa5fc0cdb10 2024-09-11T12:55:25.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.055+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol10650", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7fa5fc14be60 con 0x7fa5dc071a70 2024-09-11T12:55:25.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa5f9600640 1 -- 172.21.15.78:0/2135124151 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa5fc14be60 con 0x7fa5dc071a70 2024-09-11T12:55:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 msgr2=0x7fa5dc073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 0x7fa5dc073f30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009b60 tx=0x7fa5f00091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 msgr2=0x7fa5fc0ac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa5e8009850 tx=0x7fa5e8005e70 comp rx=0 tx=0).stop 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 shutdown_connections 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa5dc071a70 0x7fa5dc073f30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa5fc0ce8a0 0x7fa5fc150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa5fc0cdb10 0x7fa5fc0ac2c0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 --2- 172.21.15.78:0/2135124151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa5fc0b4520 0x7fa5fc15aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 >> 172.21.15.78:0/2135124151 conn(0x7fa5fc0b1cb0 msgr2=0x7fa5fc0b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 shutdown_connections 2024-09-11T12:55:25.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.103+0000 7fa604400640 1 -- 172.21.15.78:0/2135124151 wait complete. 2024-09-11T12:55:25.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol34341 fake subvol_grp44900 2024-09-11T12:55:25.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 -- 172.21.15.78:0/1087370533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b6415dfc0 msgr2=0x7f4b6406faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 --2- 172.21.15.78:0/1087370533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b6415dfc0 0x7f4b6406faa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f4b58009c80 tx=0x7f4b5801cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 -- 172.21.15.78:0/1087370533 shutdown_connections 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 --2- 172.21.15.78:0/1087370533 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415af70 0x7f4b6406ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 --2- 172.21.15.78:0/1087370533 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b6415dfc0 0x7f4b6406faa0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 -- 172.21.15.78:0/1087370533 >> 172.21.15.78:0/1087370533 conn(0x7f4b640b2ec0 msgr2=0x7f4b640b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 -- 172.21.15.78:0/1087370533 shutdown_connections 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.271+0000 7f4b6c600640 1 -- 172.21.15.78:0/1087370533 wait complete. 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 Processor -- start 2024-09-11T12:55:25.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- start start 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415af70 0x7f4b641fb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 0x7f4b64158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64167a90 con 0x7f4b64072e70 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4b64167910 con 0x7f4b6415af70 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4b64167c10 con 0x7f4b6415dfc0 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46762/0 (socket says 172.21.15.78:46762) 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 -- 172.21.15.78:0/2383665673 learned_addr learned my addr 172.21.15.78:0/2383665673 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6a600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 0x7f4b64158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69200640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415af70 0x7f4b641fb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 -- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 msgr2=0x7f4b64158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 0x7f4b64158570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 -- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415af70 msgr2=0x7f4b641fb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415af70 0x7f4b641fb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 -- 172.21.15.78:0/2383665673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b641fb780 con 0x7f4b64072e70 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6a600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 0x7f4b64158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b69c00640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000f90 tx=0x7f4b580a5a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b580bb070 con 0x7f4b64072e70 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b58004e00 con 0x7f4b64072e70 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b580b2040 con 0x7f4b64072e70 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b641fba60 con 0x7f4b64072e70 2024-09-11T12:55:25.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b641f7ce0 con 0x7f4b64072e70 2024-09-11T12:55:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.275+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b6415e540 con 0x7f4b64072e70 2024-09-11T12:55:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.279+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7f4b580042c0 con 0x7f4b64072e70 2024-09-11T12:55:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.279+0000 7f4b61600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 0x7f4b44073f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.279+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f4b5812c300 con 0x7f4b64072e70 2024-09-11T12:55:25.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.279+0000 7f4b69200640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 0x7f4b44073f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.279+0000 7f4b69200640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 0x7f4b44073f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b6414cba0 tx=0x7f4b50009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:25.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.283+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4b58016990 con 0x7f4b64072e70 2024-09-11T12:55:25.485 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.483+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol34341", "snap_name": "fake", "group_name": "subvol_grp44900", "target": ["mon-mgr", ""]}) -- 0x7f4b64154080 con 0x7f4b44071aa0 2024-09-11T12:55:25.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.523+0000 7f4b61600640 1 -- 172.21.15.78:0/2383665673 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4b64154080 con 0x7f4b44071aa0 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 msgr2=0x7f4b44073f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 0x7f4b44073f60 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4b6414cba0 tx=0x7f4b50009040 comp rx=0 tx=0).stop 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 msgr2=0x7f4b6415a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000f90 tx=0x7f4b580a5a20 comp rx=0 tx=0).stop 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 shutdown_connections 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4b6415dfc0 0x7f4b64158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4b44071aa0 0x7f4b44073f60 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4b6415af70 0x7f4b641fb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 --2- 172.21.15.78:0/2383665673 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4b64072e70 0x7f4b6415a660 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 >> 172.21.15.78:0/2383665673 conn(0x7f4b640b2ec0 msgr2=0x7f4b64150880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:25.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 shutdown_connections 2024-09-11T12:55:25.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.527+0000 7f4b6c600640 1 -- 172.21.15.78:0/2383665673 wait complete. 2024-09-11T12:55:25.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:25.548 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol10650/.meta) 2024-09-11T12:55:25.569 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:55:25.570 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:55:25.570 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:55:25.570 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol10650/43c255bf-2a7b-476d-82ad-34e906a0f03e 2024-09-11T12:55:25.570 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:55:25.570 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:25.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:25.572 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp44900/subvol34341/.meta) 2024-09-11T12:55:25.641 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:55:25.642 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:55:25.642 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:55:25.642 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/subvol_grp44900/subvol34341/5d6a152b-c605-4425-934b-b945d5abbda1 2024-09-11T12:55:25.642 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:55:25.642 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:25.644 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol10650 2024-09-11T12:55:25.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 -- 172.21.15.78:0/944464063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3420 msgr2=0x7fb96415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 --2- 172.21.15.78:0/944464063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3420 0x7fb96415a700 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fb9640cee20 tx=0x7fb94c01c800 comp rx=0 tx=0).stop 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 -- 172.21.15.78:0/944464063 shutdown_connections 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 --2- 172.21.15.78:0/944464063 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3420 0x7fb96415a700 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 --2- 172.21.15.78:0/944464063 >> v2:172.21.15.157:3300/0 conn(0x7fb9640cd770 0x7fb9640cdc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 --2- 172.21.15.78:0/944464063 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3040 0x7fb9640cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 -- 172.21.15.78:0/944464063 >> 172.21.15.78:0/944464063 conn(0x7fb964071dd0 msgr2=0x7fb9640721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 -- 172.21.15.78:0/944464063 shutdown_connections 2024-09-11T12:55:25.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.827+0000 7fb969e00640 1 -- 172.21.15.78:0/944464063 wait complete. 2024-09-11T12:55:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 Processor -- start 2024-09-11T12:55:25.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 -- start start 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3420 0x7fb96415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 0x7fb964156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640aea90 con 0x7fb9640b3040 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9640ae910 con 0x7fb9640cd770 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb969e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb9640aec10 con 0x7fb9640b3420 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46768/0 (socket says 172.21.15.78:46768) 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 -- 172.21.15.78:0/243813518 learned_addr learned my addr 172.21.15.78:0/243813518 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb963400640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 0x7fb964156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962000640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3420 0x7fb96415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 -- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3420 msgr2=0x7fb96415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3420 0x7fb96415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 -- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 msgr2=0x7fb964156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 0x7fb964156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 -- 172.21.15.78:0/243813518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb964157950 con 0x7fb9640b3040 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb963400640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 0x7fb964156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:25.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.831+0000 7fb962a00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ccb0 tx=0x7fb95400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400ce60 con 0x7fb9640b3040 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb95400f740 con 0x7fb9640b3040 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400fb90 con 0x7fb9640b3040 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb964157c30 con 0x7fb9640b3040 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96415e170 con 0x7fb9640b3040 2024-09-11T12:55:25.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9640af130 con 0x7fb9640b3040 2024-09-11T12:55:25.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.835+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 18) ==== 95289+0+0 (secure 0 0 0) 0x7fb954018400 con 0x7fb9640b3040 2024-09-11T12:55:25.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.839+0000 7fb95b400640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 0x7fb940073f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:25.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.839+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fb954093130 con 0x7fb9640b3040 2024-09-11T12:55:25.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.839+0000 7fb962000640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 0x7fb940073f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:25.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.839+0000 7fb962000640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 0x7fb940073f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0af000 tx=0x7fb94c01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:25.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.843+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mgrmap(e 19) ==== 95340+0+0 (secure 0 0 0) 0x7fb95405a380 con 0x7fb9640b3040 2024-09-11T12:55:25.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:25.847+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb954014030 con 0x7fb9640b3040 2024-09-11T12:55:26.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.035+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol10650", "target": ["mon-mgr", ""]}) -- 0x7fb96415e3b0 con 0x7fb940071a70 2024-09-11T12:55:26.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb95b400640 1 -- 172.21.15.78:0/243813518 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb94c004380 con 0x7fb940071a70 2024-09-11T12:55:26.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 msgr2=0x7fb940073f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 0x7fb940073f30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb94c0af000 tx=0x7fb94c01b040 comp rx=0 tx=0).stop 2024-09-11T12:55:26.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 msgr2=0x7fb964153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fb95400ccb0 tx=0x7fb95400a620 comp rx=0 tx=0).stop 2024-09-11T12:55:26.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 shutdown_connections 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb940071a70 0x7fb940073f30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9640cd770 0x7fb964156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9640b3420 0x7fb96415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 --2- 172.21.15.78:0/243813518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9640b3040 0x7fb964153d20 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 >> 172.21.15.78:0/243813518 conn(0x7fb964071dd0 msgr2=0x7fb9640721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 shutdown_connections 2024-09-11T12:55:26.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.075+0000 7fb969e00640 1 -- 172.21.15.78:0/243813518 wait complete. 2024-09-11T12:55:26.083 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.079+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.083 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.079+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.083 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.079+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.084 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.079+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.084 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.079+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.100 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.095+0000 7f1ecea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.101 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.095+0000 7f1ecea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.101 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.095+0000 7f1ecea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.101 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.095+0000 7f1ecea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.101 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.095+0000 7f1ecea00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:26.108 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol34341 subvol_grp44900 2024-09-11T12:55:26.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- 172.21.15.78:0/1932597457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc40815c950 msgr2=0x7fc4080a8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- 172.21.15.78:0/1932597457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc40815c950 0x7fc4080a8540 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fc4080a71a0 tx=0x7fc40001d720 comp rx=0 tx=0).stop 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- 172.21.15.78:0/1932597457 shutdown_connections 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- 172.21.15.78:0/1932597457 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc40815c950 0x7fc4080a8540 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- 172.21.15.78:0/1932597457 >> v2:172.21.15.157:3300/0 conn(0x7fc4080b3590 0x7fc4080d0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- 172.21.15.78:0/1932597457 >> 172.21.15.78:0/1932597457 conn(0x7fc4080b1a60 msgr2=0x7fc408070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- 172.21.15.78:0/1932597457 shutdown_connections 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- 172.21.15.78:0/1932597457 wait complete. 2024-09-11T12:55:26.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 Processor -- start 2024-09-11T12:55:26.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- start start 2024-09-11T12:55:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4080b3590 0x7fc40806ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 0x7fc4081550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc408168630 con 0x7fc4080b10e0 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4080a9800 con 0x7fc40815c950 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc407200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4081687b0 con 0x7fc4080b3590 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46828/0 (socket says 172.21.15.78:46828) 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 -- 172.21.15.78:0/3036931758 learned_addr learned my addr 172.21.15.78:0/3036931758 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc406800640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 0x7fc4081550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc3fd400640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4080b3590 0x7fc40806ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 -- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4080b3590 msgr2=0x7fc40806ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4080b3590 0x7fc40806ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 -- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 msgr2=0x7fc4081550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 0x7fc4081550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc405e00640 1 -- 172.21.15.78:0/3036931758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc408162610 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.255+0000 7fc406800640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 0x7fc4081550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc405e00640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c500 tx=0x7fc3f800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8017070 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3f80053c0 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4081628f0 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8012890 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc408070510 con 0x7fc4080b10e0 2024-09-11T12:55:26.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4080d0cf0 con 0x7fc4080b10e0 2024-09-11T12:55:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 95340+0+0 (secure 0 0 0) 0x7fc3f801a070 con 0x7fc4080b10e0 2024-09-11T12:55:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 0x7fc3e4073fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fc3f80918e0 con 0x7fc4080b10e0 2024-09-11T12:55:26.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.259+0000 7fc3fd400640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 0x7fc3e4073fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.263+0000 7fc3fd400640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 0x7fc3e4073fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc408162010 tx=0x7fc3f000b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:26.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.263+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc3f805ca10 con 0x7fc4080b10e0 2024-09-11T12:55:26.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.455+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol34341", "group_name": "subvol_grp44900", "target": ["mon-mgr", ""]}) -- 0x7fc4080708d0 con 0x7fc3e4071b00 2024-09-11T12:55:26.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.491+0000 7fc3fea00640 1 -- 172.21.15.78:0/3036931758 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc3f0010c00 con 0x7fc3e4071b00 2024-09-11T12:55:26.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 msgr2=0x7fc3e4073fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 0x7fc3e4073fc0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc408162010 tx=0x7fc3f000b390 comp rx=0 tx=0).stop 2024-09-11T12:55:26.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 msgr2=0x7fc4081f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800c500 tx=0x7fc3f800c9c0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 shutdown_connections 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3e4071b00 0x7fc3e4073fc0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc40815c950 0x7fc4081550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4080b3590 0x7fc40806ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 --2- 172.21.15.78:0/3036931758 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4080b10e0 0x7fc4081f6c00 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 >> 172.21.15.78:0/3036931758 conn(0x7fc4080b1a60 msgr2=0x7fc40815d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:26.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 shutdown_connections 2024-09-11T12:55:26.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.495+0000 7fc407200640 1 -- 172.21.15.78:0/3036931758 wait complete. 2024-09-11T12:55:26.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol66422 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 -- 172.21.15.78:0/2117739217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840b1f50 msgr2=0x7fbb840cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 --2- 172.21.15.78:0/2117739217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840b1f50 0x7fbb840cdc10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fbb840b26f0 tx=0x7fbb7001c5e0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 -- 172.21.15.78:0/2117739217 shutdown_connections 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 --2- 172.21.15.78:0/2117739217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840b1f50 0x7fbb840cdc10 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 --2- 172.21.15.78:0/2117739217 >> v2:172.21.15.157:3300/0 conn(0x7fbb840ce150 0x7fbb8415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 --2- 172.21.15.78:0/2117739217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 0x7fbb840b1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 -- 172.21.15.78:0/2117739217 >> 172.21.15.78:0/2117739217 conn(0x7fbb84071dd0 msgr2=0x7fbb840721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 -- 172.21.15.78:0/2117739217 shutdown_connections 2024-09-11T12:55:26.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.659+0000 7fbb89c00640 1 -- 172.21.15.78:0/2117739217 wait complete. 2024-09-11T12:55:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 Processor -- start 2024-09-11T12:55:26.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- start start 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb840b1f50 0x7fbb84153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840ce150 0x7fbb841f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 0x7fbb840b0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b3380 con 0x7fbb840ce150 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbb840b3200 con 0x7fbb840b1f50 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbb840b3500 con 0x7fbb841518b0 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840ce150 0x7fbb841f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840ce150 0x7fbb841f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46848/0 (socket says 172.21.15.78:46848) 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 -- 172.21.15.78:0/3440695392 learned_addr learned my addr 172.21.15.78:0/3440695392 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:26.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82a00640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb840b1f50 0x7fbb84153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb83400640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 0x7fbb840b0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 -- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 msgr2=0x7fbb840b0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 0x7fbb840b0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 -- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb840b1f50 msgr2=0x7fbb84153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb840b1f50 0x7fbb84153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 -- 172.21.15.78:0/3440695392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb841f8100 con 0x7fbb840ce150 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb83400640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbb841518b0 0x7fbb840b0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82a00640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbb840b1f50 0x7fbb84153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb82000640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbb840ce150 0x7fbb841f7bc0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fbb7001ac90 tx=0x7fbb700a9460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb700ad040 con 0x7fbb840ce150 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb700aad40 con 0x7fbb840ce150 2024-09-11T12:55:26.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- 172.21.15.78:0/3440695392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb841fab20 con 0x7fbb840ce150 2024-09-11T12:55:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb70005280 con 0x7fbb840ce150 2024-09-11T12:55:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.663+0000 7fbb89c00640 1 -- 172.21.15.78:0/3440695392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb841fb0e0 con 0x7fbb840ce150 2024-09-11T12:55:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb89c00640 1 -- 172.21.15.78:0/3440695392 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb8415e4d0 con 0x7fbb840ce150 2024-09-11T12:55:26.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 95340+0+0 (secure 0 0 0) 0x7fbb700b0070 con 0x7fbb840ce150 2024-09-11T12:55:26.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb7aa00640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbb64071b30 0x7fbb64073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:26.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fbb7012cf40 con 0x7fbb840ce150 2024-09-11T12:55:26.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb82a00640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbb64071b30 0x7fbb64073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:26.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.667+0000 7fbb82a00640 1 --2- 172.21.15.78:0/3440695392 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbb64071b30 0x7fbb64073ff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbb74005fd0 tx=0x7fbb74005f00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:26.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.671+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbb70016490 con 0x7fbb840ce150 2024-09-11T12:55:26.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.863+0000 7fbb89c00640 1 -- 172.21.15.78:0/3440695392 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol66422", "target": ["mon-mgr", ""]}) -- 0x7fbb840709f0 con 0x7fbb64071b30 2024-09-11T12:55:26.885 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:26.883+0000 7f1eccc00640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol66422' is not ready for operation rm (use --force to override) 2024-09-11T12:55:26.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:26.883+0000 7fbb7aa00640 1 -- 172.21.15.78:0/3440695392 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol66422' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fbb840709f0 con 0x7fbb64071b30 2024-09-11T12:55:26.886 INFO:teuthology.orchestra.run.smithi078.stderr:Error EAGAIN: subvolume 'subvol66422' is not ready for operation rm (use --force to override) 2024-09-11T12:55:26.894 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-11T12:55:26.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:26.895 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol66422/.meta) 2024-09-11T12:55:26.917 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:55:26.917 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:type = clone 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol66422/208e9e59-07c8-4571-9366-35bf46eb72c7 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:state = pending 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:[source] 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:volume = fake 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:subvolume = fake 2024-09-11T12:55:26.918 INFO:teuthology.orchestra.run.smithi078.stdout:snapshot = fake 2024-09-11T12:55:26.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol66422 --force 2024-09-11T12:55:27.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- 172.21.15.78:0/3787381019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 msgr2=0x7f22dc06f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- 172.21.15.78:0/3787381019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc06f9c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0b4330 tx=0x7f22d001c5d0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- 172.21.15.78:0/3787381019 shutdown_connections 2024-09-11T12:55:27.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- 172.21.15.78:0/3787381019 >> v2:172.21.15.157:3300/0 conn(0x7f22dc0b5c80 0x7f22dc0b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- 172.21.15.78:0/3787381019 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc06ff00 0x7f22dc0b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- 172.21.15.78:0/3787381019 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc06f9c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- 172.21.15.78:0/3787381019 >> 172.21.15.78:0/3787381019 conn(0x7f22dc0d0dc0 msgr2=0x7f22dc0d11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:27.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- 172.21.15.78:0/3787381019 shutdown_connections 2024-09-11T12:55:27.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- 172.21.15.78:0/3787381019 wait complete. 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 Processor -- start 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- start start 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc06ff00 0x7f22dc140cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc0b5c80 0x7f22dc15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc072120 con 0x7f22dc15dca0 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc071fa0 con 0x7f22dc06ff00 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.107+0000 7f22e3600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f22dc0722a0 con 0x7f22dc0b5c80 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46868/0 (socket says 172.21.15.78:46868) 2024-09-11T12:55:27.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 -- 172.21.15.78:0/3967714423 learned_addr learned my addr 172.21.15.78:0/3967714423 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e0c00640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc06ff00 0x7f22dc140cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22dbe00640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc0b5c80 0x7f22dc15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 -- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc0b5c80 msgr2=0x7f22dc15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc0b5c80 0x7f22dc15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 -- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc06ff00 msgr2=0x7f22dc140cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc06ff00 0x7f22dc140cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 -- 172.21.15.78:0/3967714423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc15ae50 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e1600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f22d400daa0 tx=0x7f22d400df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4017070 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22d4004b30 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4005020 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc15b0e0 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc15b6a0 con 0x7f22dc15dca0 2024-09-11T12:55:27.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0b4520 con 0x7f22dc15dca0 2024-09-11T12:55:27.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.111+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 95340+0+0 (secure 0 0 0) 0x7f22d40051c0 con 0x7f22dc15dca0 2024-09-11T12:55:27.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.115+0000 7f22d8c00640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 0x7f22bc073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.115+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f22d4092940 con 0x7f22dc15dca0 2024-09-11T12:55:27.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.115+0000 7f22e0c00640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 0x7f22bc073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.115+0000 7f22e0c00640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 0x7f22bc073ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22d00ae000 tx=0x7f22d001ae10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:27.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.115+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f22d4020070 con 0x7f22dc15dca0 2024-09-11T12:55:27.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.307+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol66422", "force": true, "target": ["mon-mgr", ""]}) -- 0x7f22dc0d3930 con 0x7f22bc071b30 2024-09-11T12:55:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.335+0000 7f22d8c00640 1 -- 172.21.15.78:0/3967714423 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f22dc0d3930 con 0x7f22bc071b30 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 msgr2=0x7f22bc073ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 0x7f22bc073ff0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f22d00ae000 tx=0x7f22d001ae10 comp rx=0 tx=0).stop 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 msgr2=0x7f22dc15c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f22d400daa0 tx=0x7f22d400df60 comp rx=0 tx=0).stop 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 shutdown_connections 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f22dc15dca0 0x7f22dc15c6c0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f22dc0b5c80 0x7f22dc15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22bc071b30 0x7f22bc073ff0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 --2- 172.21.15.78:0/3967714423 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f22dc06ff00 0x7f22dc140cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 >> 172.21.15.78:0/3967714423 conn(0x7f22dc0d0dc0 msgr2=0x7f22dc073130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 shutdown_connections 2024-09-11T12:55:27.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.339+0000 7f22e3600640 1 -- 172.21.15.78:0/3967714423 wait complete. 2024-09-11T12:55:27.362 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-11T12:55:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- 172.21.15.78:0/3959874542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95900b2ec0 msgr2=0x7f95900b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 --2- 172.21.15.78:0/3959874542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95900b2ec0 0x7f95900b32f0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f95900b2c60 tx=0x7f958801cb80 comp rx=0 tx=0).stop 2024-09-11T12:55:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- 172.21.15.78:0/3959874542 shutdown_connections 2024-09-11T12:55:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 --2- 172.21.15.78:0/3959874542 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95900b2ec0 0x7f95900b32f0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 --2- 172.21.15.78:0/3959874542 >> v2:172.21.15.157:3300/0 conn(0x7f9590151010 0x7f959006fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- 172.21.15.78:0/3959874542 >> 172.21.15.78:0/3959874542 conn(0x7f9590075a10 msgr2=0x7f95900734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- 172.21.15.78:0/3959874542 shutdown_connections 2024-09-11T12:55:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- 172.21.15.78:0/3959874542 wait complete. 2024-09-11T12:55:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 Processor -- start 2024-09-11T12:55:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.503+0000 7f9594e00640 1 -- start start 2024-09-11T12:55:27.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95900b2ec0 0x7f959013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 0x7f95901577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f959015e100 con 0x7f9590151010 2024-09-11T12:55:27.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f959015df80 con 0x7f95900b4e30 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f959015e280 con 0x7f95900b2ec0 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46898/0 (socket says 172.21.15.78:46898) 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 -- 172.21.15.78:0/2298096311 learned_addr learned my addr 172.21.15.78:0/2298096311 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 -- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95900b2ec0 msgr2=0x7f959013be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95900b2ec0 0x7f959013be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 -- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 msgr2=0x7f95901577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958d400640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 0x7f95901577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 0x7f95901577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 -- 172.21.15.78:0/2298096311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959013c370 con 0x7f9590151010 2024-09-11T12:55:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958d400640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 0x7f95901577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f958e800640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f958801c5f0 tx=0x7f9588005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95880b2040 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95880bb070 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9588004330 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9590156790 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9590156d10 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.507+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9590071710 con 0x7f9590151010 2024-09-11T12:55:27.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.511+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 19) ==== 95340+0+0 (secure 0 0 0) 0x7f95880044d0 con 0x7f9590151010 2024-09-11T12:55:27.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.511+0000 7f9586000640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 0x7f9570073ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:27.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.511+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7f958812ced0 con 0x7f9590151010 2024-09-11T12:55:27.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.511+0000 7f958de00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 0x7f9570073ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:27.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.511+0000 7f958de00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 0x7f9570073ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95800058d0 tx=0x7f9580005860 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:27.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.515+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f959013c370 con 0x7f9590151010 2024-09-11T12:55:27.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.706+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f95900693d0 con 0x7f9570071b30 2024-09-11T12:55:27.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.710+0000 7f9586000640 1 -- 172.21.15.78:0/2298096311 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f95900693d0 con 0x7f9570071b30 2024-09-11T12:55:27.715 INFO:teuthology.orchestra.run.smithi078.stdout:[] 2024-09-11T12:55:27.717 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 msgr2=0x7f9570073ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 0x7f9570073ff0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f95800058d0 tx=0x7f9580005860 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 msgr2=0x7f959015d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f958801c5f0 tx=0x7f9588005a80 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 shutdown_connections 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9590151010 0x7f959015d070 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9570071b30 0x7f9570073ff0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95900b4e30 0x7f95901577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 --2- 172.21.15.78:0/2298096311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95900b2ec0 0x7f959013be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 >> 172.21.15.78:0/2298096311 conn(0x7f9590075a10 msgr2=0x7f95900ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 shutdown_connections 2024-09-11T12:55:27.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27.714+0000 7f9594e00640 1 -- 172.21.15.78:0/2298096311 wait complete. 2024-09-11T12:55:27.737 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:55:27.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:55:27.790 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:55:27.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:27.797 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:55:27.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:27.950 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:28.037 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:55:28.038 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:55:28.038 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:28.038 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:28.046 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:28.047 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:28.100 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- 172.21.15.78:0/158177366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 msgr2=0x7fcd380b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- 172.21.15.78:0/158177366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd380b4520 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009930 tx=0x7fcd2801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- 172.21.15.78:0/158177366 shutdown_connections 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- 172.21.15.78:0/158177366 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd380b4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- 172.21.15.78:0/158177366 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd380b4520 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- 172.21.15.78:0/158177366 >> 172.21.15.78:0/158177366 conn(0x7fcd3806ca80 msgr2=0x7fcd380b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- 172.21.15.78:0/158177366 shutdown_connections 2024-09-11T12:55:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- 172.21.15.78:0/158177366 wait complete. 2024-09-11T12:55:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 Processor -- start 2024-09-11T12:55:28.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- start start 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 0x7fcd381497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd381f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd38167b60 con 0x7fcd3815a940 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd381679e0 con 0x7fcd3806fc50 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.286+0000 7fcd3d000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd38167ce0 con 0x7fcd380cf580 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd35e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 0x7fcd381497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46912/0 (socket says 172.21.15.78:46912) 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 -- 172.21.15.78:0/2338998508 learned_addr learned my addr 172.21.15.78:0/2338998508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd35e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 0x7fcd381497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:34858/0 (socket says 172.21.15.78:34858) 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2fe00640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd381f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 -- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 msgr2=0x7fcd381f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd381f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 -- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 msgr2=0x7fcd381497e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 0x7fcd381497e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 -- 172.21.15.78:0/2338998508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381542c0 con 0x7fcd3815a940 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2fe00640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd381f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:28.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd36800640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000e9f0 tx=0x7fcd3000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000cd80 con 0x7fcd3815a940 2024-09-11T12:55:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd30004c20 con 0x7fcd3815a940 2024-09-11T12:55:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30005020 con 0x7fcd3815a940 2024-09-11T12:55:28.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd381545a0 con 0x7fcd3815a940 2024-09-11T12:55:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3814a640 con 0x7fcd3815a940 2024-09-11T12:55:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd38158440 con 0x7fcd3815a940 2024-09-11T12:55:28.295 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fcd300051c0 con 0x7fcd3815a940 2024-09-11T12:55:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.290+0000 7fcd2e000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 0x7fcd14074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.294+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7fcd300930e0 con 0x7fcd3815a940 2024-09-11T12:55:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.294+0000 7fcd35e00640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 0x7fcd14074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.296 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.294+0000 7fcd35e00640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 0x7fcd14074110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd280af000 tx=0x7fcd2801b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:28.299 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.298+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcd30016030 con 0x7fcd3815a940 2024-09-11T12:55:28.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd3814a840 con 0x7fcd3815a940 2024-09-11T12:55:28.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd2e000640 1 -- 172.21.15.78:0/2338998508 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7fcd3005dfd0 con 0x7fcd3815a940 2024-09-11T12:55:28.477 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 msgr2=0x7fcd14074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 0x7fcd14074110 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd280af000 tx=0x7fcd2801b0f0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 msgr2=0x7fcd381f8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000e9f0 tx=0x7fcd3000eeb0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 shutdown_connections 2024-09-11T12:55:28.479 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd3815a940 0x7fcd381f8f30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcd380cf580 0x7fcd381f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcd14071c50 0x7fcd14074110 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 --2- 172.21.15.78:0/2338998508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd3806fc50 0x7fcd381497e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 >> 172.21.15.78:0/2338998508 conn(0x7fcd3806ca80 msgr2=0x7fcd3814c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 shutdown_connections 2024-09-11T12:55:28.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.474+0000 7fcd3d000640 1 -- 172.21.15.78:0/2338998508 wait complete. 2024-09-11T12:55:28.500 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:55:28.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:28.500 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:28.547 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:55:28.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:28.547 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:55:28.600 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:55:28.600 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:28.600 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:28.609 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:28.609 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:28.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:55:28.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- 172.21.15.78:0/2884217199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11415fd90 msgr2=0x7ff1140cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 --2- 172.21.15.78:0/2884217199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11415fd90 0x7ff1140cebe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff0fc0098a0 tx=0x7ff0fc01c840 comp rx=0 tx=0).stop 2024-09-11T12:55:28.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- 172.21.15.78:0/2884217199 shutdown_connections 2024-09-11T12:55:28.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 --2- 172.21.15.78:0/2884217199 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff11415fd90 0x7ff1140cebe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- 172.21.15.78:0/2884217199 >> 172.21.15.78:0/2884217199 conn(0x7ff1140746c0 msgr2=0x7ff1140ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:28.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- 172.21.15.78:0/2884217199 shutdown_connections 2024-09-11T12:55:28.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- 172.21.15.78:0/2884217199 wait complete. 2024-09-11T12:55:28.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 Processor -- start 2024-09-11T12:55:28.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- start start 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 0x7ff114155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff114146890 0x7ff1140ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff11416a470 con 0x7ff11415fd90 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff11416a2f0 con 0x7ff11415f9b0 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff119200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff11416a5f0 con 0x7ff114146890 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff112a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 0x7ff114155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.806+0000 7ff113400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff113400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:34868/0 (socket says 172.21.15.78:34868) 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff113400640 1 -- 172.21.15.78:0/4050172824 learned_addr learned my addr 172.21.15.78:0/4050172824 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:28.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff113e00640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff114146890 0x7ff1140ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 -- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff114146890 msgr2=0x7ff1140ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff114146890 0x7ff1140ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 -- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 msgr2=0x7ff1141550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 -- 172.21.15.78:0/4050172824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11413d920 con 0x7ff11415fd90 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff113400640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff112a00640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 0x7ff114155f10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ac520 tx=0x7ff0fc0a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc004030 con 0x7ff11415fd90 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0fc0042c0 con 0x7ff11415fd90 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11413dc00 con 0x7ff11415fd90 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0fc005280 con 0x7ff11415fd90 2024-09-11T12:55:28.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff11413e1c0 con 0x7ff11415fd90 2024-09-11T12:55:28.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1140cebe0 con 0x7ff11415fd90 2024-09-11T12:55:28.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.810+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7ff0fc0c0020 con 0x7ff11415fd90 2024-09-11T12:55:28.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.814+0000 7ff10b400640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 0x7ff0f00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:28.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.814+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5709+0+0 (secure 0 0 0) 0x7ff0fc12cb00 con 0x7ff11415fd90 2024-09-11T12:55:28.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.814+0000 7ff113400640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 0x7ff0f00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:28.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.814+0000 7ff113400640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 0x7ff0f00740e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff104007b40 tx=0x7ff104007a90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:28.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:28.818+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff0fc0c7020 con 0x7ff11415fd90 2024-09-11T12:55:29.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff11406ee00 con 0x7ff11415fd90 2024-09-11T12:55:29.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff10b400640 1 -- 172.21.15.78:0/4050172824 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v117) ==== 80+0+0 (secure 0 0 0) 0x7ff0fc016430 con 0x7ff11415fd90 2024-09-11T12:55:29.009 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:55:29.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 msgr2=0x7ff0f00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 0x7ff0f00740e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff104007b40 tx=0x7ff104007a90 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 msgr2=0x7ff114155f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 0x7ff114155f10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7ff1140ac520 tx=0x7ff0fc0a6590 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 shutdown_connections 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff114146890 0x7ff1140ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff11415fd90 0x7ff114155f10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff0f0071c20 0x7ff0f00740e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 --2- 172.21.15.78:0/4050172824 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff11415f9b0 0x7ff1141550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 >> 172.21.15.78:0/4050172824 conn(0x7ff1140746c0 msgr2=0x7ff1140b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:29.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.006+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 shutdown_connections 2024-09-11T12:55:29.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.010+0000 7ff119200640 1 -- 172.21.15.78:0/4050172824 wait complete. 2024-09-11T12:55:29.031 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:55:29.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:29.032 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:29.070 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:55:29.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:29.070 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:55:29.119 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:55:29.119 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:29.119 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:29.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:55:29.128 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:55:29.129 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:29.129 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:29.171 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:55:29.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:29.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- 172.21.15.78:0/2838232152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 msgr2=0x7f136015c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 --2- 172.21.15.78:0/2838232152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136015c770 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f13600b1610 tx=0x7f135401c770 comp rx=0 tx=0).stop 2024-09-11T12:55:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- 172.21.15.78:0/2838232152 shutdown_connections 2024-09-11T12:55:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 --2- 172.21.15.78:0/2838232152 >> v2:172.21.15.157:3300/0 conn(0x7f13600b34d0 0x7f136015dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 --2- 172.21.15.78:0/2838232152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136015c770 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- 172.21.15.78:0/2838232152 >> 172.21.15.78:0/2838232152 conn(0x7f13600b19a0 msgr2=0x7f1360070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:29.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- 172.21.15.78:0/2838232152 shutdown_connections 2024-09-11T12:55:29.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- 172.21.15.78:0/2838232152 wait complete. 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 Processor -- start 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.318+0000 7f1366000640 1 -- start start 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13600b34d0 0x7f13601f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 0x7f1360145760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1360168960 con 0x7f136015c390 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13601687e0 con 0x7f13600b34d0 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1360168ae0 con 0x7f136015ccb0 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46942/0 (socket says 172.21.15.78:46942) 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 -- 172.21.15.78:0/2930464726 learned_addr learned my addr 172.21.15.78:0/2930464726 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1365600640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 0x7f1360145760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1364c00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13600b34d0 0x7f13601f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 -- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 msgr2=0x7f1360145760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 0x7f1360145760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 -- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13600b34d0 msgr2=0x7f13601f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13600b34d0 0x7f13601f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 -- 172.21.15.78:0/2930464726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13601f7f90 con 0x7f136015c390 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1365600640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 0x7f1360145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135fe00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c500 tx=0x7f134c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c018070 con 0x7f136015c390 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f134c003600 con 0x7f136015c390 2024-09-11T12:55:29.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f134c003ad0 con 0x7f136015c390 2024-09-11T12:55:29.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136014da30 con 0x7f136015c390 2024-09-11T12:55:29.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f136006e4f0 con 0x7f136015c390 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f136015dcc0 con 0x7f136015c390 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.322+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f134c01b070 con 0x7f136015c390 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.326+0000 7f135cc00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 0x7f1340074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.326+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f134c091b40 con 0x7f136015c390 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.326+0000 7f1364c00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 0x7f1340074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.326+0000 7f1364c00640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 0x7f1340074110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13540af000 tx=0x7f135401afb0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:29.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.330+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f134c003c70 con 0x7f136015c390 2024-09-11T12:55:29.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.526+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13600b0d00 con 0x7f136015c390 2024-09-11T12:55:29.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.526+0000 7f135cc00640 1 -- 172.21.15.78:0/2930464726 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5231 (secure 0 0 0) 0x7f134c065410 con 0x7f136015c390 2024-09-11T12:55:29.531 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:29.531 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"btime":"2024-09-11T12:55:21:818593+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:29.531 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 82 2024-09-11T12:55:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 msgr2=0x7f1340074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 0x7f1340074110 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f13540af000 tx=0x7f135401afb0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 msgr2=0x7f136014d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f134c00c500 tx=0x7f134c00c9c0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 shutdown_connections 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f136015ccb0 0x7f1360145760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f136015c390 0x7f136014d4f0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1340071c50 0x7f1340074110 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 --2- 172.21.15.78:0/2930464726 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13600b34d0 0x7f13601f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 >> 172.21.15.78:0/2930464726 conn(0x7f13600b19a0 msgr2=0x7f136014c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 shutdown_connections 2024-09-11T12:55:29.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.530+0000 7f1366000640 1 -- 172.21.15.78:0/2930464726 wait complete. 2024-09-11T12:55:29.553 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:29.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- 172.21.15.78:0/157575060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2441601d0 msgr2=0x7fd244141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 --2- 172.21.15.78:0/157575060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2441601d0 0x7fd244141460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009bc0 tx=0x7fd23c01cac0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- 172.21.15.78:0/157575060 shutdown_connections 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 --2- 172.21.15.78:0/157575060 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2441601d0 0x7fd244141460 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- 172.21.15.78:0/157575060 >> 172.21.15.78:0/157575060 conn(0x7fd2440746c0 msgr2=0x7fd2440ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- 172.21.15.78:0/157575060 shutdown_connections 2024-09-11T12:55:29.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- 172.21.15.78:0/157575060 wait complete. 2024-09-11T12:55:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 Processor -- start 2024-09-11T12:55:29.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.694+0000 7fd249a00640 1 -- start start 2024-09-11T12:55:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24415fd90 0x7fd24413a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24413e590 0x7fd24415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd244141db0 con 0x7fd24415f9b0 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd244141c30 con 0x7fd24415fd90 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd244141f30 con 0x7fd24413e590 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46952/0 (socket says 172.21.15.78:46952) 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 -- 172.21.15.78:0/2868441881 learned_addr learned my addr 172.21.15.78:0/2868441881 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 -- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24413e590 msgr2=0x7fd24415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24413e590 0x7fd24415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 -- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24415fd90 msgr2=0x7fd24413a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24415fd90 0x7fd24413a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 -- 172.21.15.78:0/2868441881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd244139560 con 0x7fd24415f9b0 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd243e00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c4b0 tx=0x7fd23400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:29.702 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2340052d0 con 0x7fd24415f9b0 2024-09-11T12:55:29.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd234005470 con 0x7fd24415f9b0 2024-09-11T12:55:29.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd244139840 con 0x7fd24415f9b0 2024-09-11T12:55:29.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd234012870 con 0x7fd24415f9b0 2024-09-11T12:55:29.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd244139e00 con 0x7fd24415f9b0 2024-09-11T12:55:29.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd244141460 con 0x7fd24415f9b0 2024-09-11T12:55:29.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.698+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fd234003600 con 0x7fd24415f9b0 2024-09-11T12:55:29.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.702+0000 7fd23be00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 0x7fd2240740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:29.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.702+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fd234092490 con 0x7fd24415f9b0 2024-09-11T12:55:29.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.702+0000 7fd243400640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 0x7fd2240740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:29.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.702+0000 7fd243400640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 0x7fd2240740e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd244138260 tx=0x7fd230001090 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:29.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.706+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd23405da10 con 0x7fd24415f9b0 2024-09-11T12:55:29.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.910+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd24406f320 con 0x7fd24415f9b0 2024-09-11T12:55:29.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.910+0000 7fd23be00640 1 -- 172.21.15.78:0/2868441881 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5231 (secure 0 0 0) 0x7fd23405d310 con 0x7fd24415f9b0 2024-09-11T12:55:29.914 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:29.914 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"btime":"2024-09-11T12:55:21:818593+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:29.915 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 82 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 msgr2=0x7fd2240740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 0x7fd2240740e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd244138260 tx=0x7fd230001090 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 msgr2=0x7fd24414e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c4b0 tx=0x7fd23400c970 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 shutdown_connections 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd24413e590 0x7fd24415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd224071c20 0x7fd2240740e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd24415fd90 0x7fd24413a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 --2- 172.21.15.78:0/2868441881 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd24415f9b0 0x7fd24414e9b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:29.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 >> 172.21.15.78:0/2868441881 conn(0x7fd2440746c0 msgr2=0x7fd24406f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:29.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 shutdown_connections 2024-09-11T12:55:29.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:29.914+0000 7fd249a00640 1 -- 172.21.15.78:0/2868441881 wait complete. 2024-09-11T12:55:29.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 -- 172.21.15.78:0/1010863894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 msgr2=0x7f4f8415d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1010863894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f8415d6b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4f74009c80 tx=0x7f4f7401cc30 comp rx=0 tx=0).stop 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 -- 172.21.15.78:0/1010863894 shutdown_connections 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1010863894 >> v2:172.21.15.157:3300/0 conn(0x7f4f840d0b30 0x7f4f840d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1010863894 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f8415d6b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 -- 172.21.15.78:0/1010863894 >> 172.21.15.78:0/1010863894 conn(0x7f4f84074b90 msgr2=0x7f4f840d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 -- 172.21.15.78:0/1010863894 shutdown_connections 2024-09-11T12:55:30.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.074+0000 7f4f8a200640 1 -- 172.21.15.78:0/1010863894 wait complete. 2024-09-11T12:55:30.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 Processor -- start 2024-09-11T12:55:30.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- start start 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f84158f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 0x7f4f841594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4f841414e0 con 0x7f4f840d0b30 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4f84141360 con 0x7f4f8415f7b0 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4f84141660 con 0x7f4f84140bd0 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46956/0 (socket says 172.21.15.78:46956) 2024-09-11T12:55:30.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 -- 172.21.15.78:0/1118297850 learned_addr learned my addr 172.21.15.78:0/1118297850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83e00640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 0x7f4f841594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f82a00640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f84158f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 -- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 msgr2=0x7f4f84158f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f84158f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 -- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 msgr2=0x7f4f841594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 0x7f4f841594d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 -- 172.21.15.78:0/1118297850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f841533f0 con 0x7f4f840d0b30 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83e00640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 0x7f4f841594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:30.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f83400640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000f90 tx=0x7f4f74004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740bb070 con 0x7f4f840d0b30 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f74003e70 con 0x7f4f840d0b30 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f740b2040 con 0x7f4f840d0b30 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f84153690 con 0x7f4f840d0b30 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.078+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f84153c50 con 0x7f4f840d0b30 2024-09-11T12:55:30.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f84141b10 con 0x7f4f840d0b30 2024-09-11T12:55:30.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f4f74004c90 con 0x7f4f840d0b30 2024-09-11T12:55:30.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f7b400640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 0x7f4f600740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f4f7412cac0 con 0x7f4f840d0b30 2024-09-11T12:55:30.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f82a00640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 0x7f4f600740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.082+0000 7f4f82a00640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 0x7f4f600740e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f84155e00 tx=0x7f4f6c00c040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.086+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4f740168b0 con 0x7f4f840d0b30 2024-09-11T12:55:30.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.254+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f8415d9d0 con 0x7f4f840d0b30 2024-09-11T12:55:30.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.254+0000 7f4f7b400640 1 -- 172.21.15.78:0/1118297850 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16850 (secure 0 0 0) 0x7f4f740f8080 con 0x7f4f840d0b30 2024-09-11T12:55:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:30.259 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":117,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:28.863501+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:09.848106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:11.124742+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 msgr2=0x7f4f600740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 0x7f4f600740e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4f84155e00 tx=0x7f4f6c00c040 comp rx=0 tx=0).stop 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 msgr2=0x7f4f8413cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f4f74000f90 tx=0x7f4f74004660 comp rx=0 tx=0).stop 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 shutdown_connections 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4f60071c20 0x7f4f600740e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4f8415f7b0 0x7f4f841594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4f84140bd0 0x7f4f84158f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 --2- 172.21.15.78:0/1118297850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4f840d0b30 0x7f4f8413cf40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 >> 172.21.15.78:0/1118297850 conn(0x7f4f84074b90 msgr2=0x7f4f8415dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:30.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 shutdown_connections 2024-09-11T12:55:30.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.258+0000 7f4f8a200640 1 -- 172.21.15.78:0/1118297850 wait complete. 2024-09-11T12:55:30.281 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:55:30.281 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:30.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 -- 172.21.15.78:0/751345593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e180b14b0 msgr2=0x7f0e1815c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 --2- 172.21.15.78:0/751345593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e180b14b0 0x7f0e1815c100 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009bc0 tx=0x7f0e0801cba0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 -- 172.21.15.78:0/751345593 shutdown_connections 2024-09-11T12:55:30.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 --2- 172.21.15.78:0/751345593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e1806fdf0 0x7f0e180a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 --2- 172.21.15.78:0/751345593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e180b14b0 0x7f0e1815c100 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 -- 172.21.15.78:0/751345593 >> 172.21.15.78:0/751345593 conn(0x7f0e18074b90 msgr2=0x7f0e180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 -- 172.21.15.78:0/751345593 shutdown_connections 2024-09-11T12:55:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.418+0000 7f0e1d800640 1 -- 172.21.15.78:0/751345593 wait complete. 2024-09-11T12:55:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 Processor -- start 2024-09-11T12:55:30.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- start start 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e180b14b0 0x7f0e1814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 0x7f0e1814f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0e18169330 con 0x7f0e180d0bf0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0e181691b0 con 0x7f0e1806fdf0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0e181694b0 con 0x7f0e180b14b0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e17400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 0x7f0e1814f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e180b14b0 0x7f0e1814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:34910/0 (socket says 172.21.15.78:34910) 2024-09-11T12:55:30.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 -- 172.21.15.78:0/3428985274 learned_addr learned my addr 172.21.15.78:0/3428985274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:30.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 -- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e180b14b0 msgr2=0x7f0e1814eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e180b14b0 0x7f0e1814eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 -- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 msgr2=0x7f0e1814f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 0x7f0e1814f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 -- 172.21.15.78:0/3428985274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1814e490 con 0x7f0e1806fdf0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e17400640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 0x7f0e1814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e16a00640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009b10 tx=0x7f0e080092a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e080b2040 con 0x7f0e1806fdf0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e08004d40 con 0x7f0e1806fdf0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08004240 con 0x7f0e1806fdf0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e181fc340 con 0x7f0e1806fdf0 2024-09-11T12:55:30.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.422+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e181fb580 con 0x7f0e1806fdf0 2024-09-11T12:55:30.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e180a9d00 con 0x7f0e1806fdf0 2024-09-11T12:55:30.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f0e080be060 con 0x7f0e1806fdf0 2024-09-11T12:55:30.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e0ea00640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 0x7f0df80740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f0e0812c630 con 0x7f0e1806fdf0 2024-09-11T12:55:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e16000640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 0x7f0df80740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.426+0000 7f0e16000640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 0x7f0df80740e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009600 tx=0x7f0e040091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.430+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0e080a5e20 con 0x7f0e1806fdf0 2024-09-11T12:55:30.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.634+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e18150230 con 0x7f0e1806fdf0 2024-09-11T12:55:30.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.634+0000 7f0e0ea00640 1 -- 172.21.15.78:0/3428985274 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5231 (secure 0 0 0) 0x7f0e08016790 con 0x7f0e1806fdf0 2024-09-11T12:55:30.638 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:30.638 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"btime":"2024-09-11T12:55:21:818593+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:30.638 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 82 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 msgr2=0x7f0df80740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 0x7f0df80740e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0e04009600 tx=0x7f0e040091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 msgr2=0x7f0e181426c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f0e08009b10 tx=0x7f0e080092a0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 shutdown_connections 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0e180d0bf0 0x7f0e1814f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0e180b14b0 0x7f0e1814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0df8071c20 0x7f0df80740e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 --2- 172.21.15.78:0/3428985274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0e1806fdf0 0x7f0e181426c0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 >> 172.21.15.78:0/3428985274 conn(0x7f0e18074b90 msgr2=0x7f0e180706e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 shutdown_connections 2024-09-11T12:55:30.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.638+0000 7f0e1d800640 1 -- 172.21.15.78:0/3428985274 wait complete. 2024-09-11T12:55:30.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:30.801 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- 172.21.15.78:0/3333371745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 msgr2=0x7fb9a806f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 --2- 172.21.15.78:0/3333371745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a806f000 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0009c80 tx=0x7fb9a001cd70 comp rx=0 tx=0).stop 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- 172.21.15.78:0/3333371745 shutdown_connections 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 --2- 172.21.15.78:0/3333371745 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a806f000 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 --2- 172.21.15.78:0/3333371745 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 0x7fb9a80727e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- 172.21.15.78:0/3333371745 >> 172.21.15.78:0/3333371745 conn(0x7fb9a80ccca0 msgr2=0x7fb9a80cd0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- 172.21.15.78:0/3333371745 shutdown_connections 2024-09-11T12:55:30.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- 172.21.15.78:0/3333371745 wait complete. 2024-09-11T12:55:30.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 Processor -- start 2024-09-11T12:55:30.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.798+0000 7fb9afe00640 1 -- start start 2024-09-11T12:55:30.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 0x7fb9a80b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 0x7fb9a80b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a806fde0 con 0x7fb9a80b0d40 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a806fc60 con 0x7fb9a815b3f0 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a806ff60 con 0x7fb9a8154800 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46996/0 (socket says 172.21.15.78:46996) 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 -- 172.21.15.78:0/139342903 learned_addr learned my addr 172.21.15.78:0/139342903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ade00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 0x7fb9a80b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9aca00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 0x7fb9a80b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 -- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 msgr2=0x7fb9a80b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 0x7fb9a80b4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 -- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 msgr2=0x7fb9a80b52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 0x7fb9a80b52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 -- 172.21.15.78:0/139342903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80cef20 con 0x7fb9a80b0d40 2024-09-11T12:55:30.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ade00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 0x7fb9a80b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9ad400640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00e970 tx=0x7fb99c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c00cd80 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb99c005020 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c005450 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a80cf200 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80cf7c0 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.802+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a806f000 con 0x7fb9a80b0d40 2024-09-11T12:55:30.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.806+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fb99c0047b0 con 0x7fb9a80b0d40 2024-09-11T12:55:30.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.806+0000 7fb9a5600640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 0x7fb9880740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:30.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.806+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7fb99c093430 con 0x7fb9a80b0d40 2024-09-11T12:55:30.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.806+0000 7fb9aca00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 0x7fb9880740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:30.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.806+0000 7fb9aca00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 0x7fb9880740e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9a814c470 tx=0x7fb994009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:30.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:30.810+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb99c05e8a0 con 0x7fb9a80b0d40 2024-09-11T12:55:31.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.010+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb9a81507a0 con 0x7fb9a80b0d40 2024-09-11T12:55:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.010+0000 7fb9a5600640 1 -- 172.21.15.78:0/139342903 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) ==== 94+0+5231 (secure 0 0 0) 0x7fb99c05e1a0 con 0x7fb9a80b0d40 2024-09-11T12:55:31.014 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:31.014 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":82,"btime":"2024-09-11T12:55:21:818593+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[{"mdsmap":{"epoch":81,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:12.153435+0000","modified":"2024-09-11T12:55:15.177704+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7000},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7000":{"gid":7000,"name":"b","rank":0,"incarnation":77,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/2586938426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2586938426},{"type":"v1","addr":"172.21.15.157:6835","nonce":2586938426}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7000,"qdb_cluster":[7000]},"id":9}]} 2024-09-11T12:55:31.014 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 82 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 msgr2=0x7fb9880740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 0x7fb9880740e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fb9a814c470 tx=0x7fb994009040 comp rx=0 tx=0).stop 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 msgr2=0x7fb9a813cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fb99c00e970 tx=0x7fb99c00ee30 comp rx=0 tx=0).stop 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 shutdown_connections 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb988071c20 0x7fb9880740e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb9a815b3f0 0x7fb9a80b52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb9a8154800 0x7fb9a80b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 --2- 172.21.15.78:0/139342903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb9a80b0d40 0x7fb9a813cc60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 >> 172.21.15.78:0/139342903 conn(0x7fb9a80ccca0 msgr2=0x7fb9a80b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 shutdown_connections 2024-09-11T12:55:31.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.014+0000 7fb9afe00640 1 -- 172.21.15.78:0/139342903 wait complete. 2024-09-11T12:55:31.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- 172.21.15.78:0/1288878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 msgr2=0x7f6fd8003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 --2- 172.21.15.78:0/1288878127 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6fd8003e80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009ee0 tx=0x7f6fe401d630 comp rx=0 tx=0).stop 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff8c00640 1 -- 172.21.15.78:0/1288878127 reap_dead start 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- 172.21.15.78:0/1288878127 shutdown_connections 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- 172.21.15.78:0/1288878127 >> 172.21.15.78:0/1288878127 conn(0x7f6ff406ca80 msgr2=0x7f6ff40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- 172.21.15.78:0/1288878127 shutdown_connections 2024-09-11T12:55:31.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- 172.21.15.78:0/1288878127 wait complete. 2024-09-11T12:55:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 Processor -- start 2024-09-11T12:55:31.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- start start 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 0x7f6ff41454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 0x7f6ff41459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff41667f0 con 0x7f6fd8003a60 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff415bec0 con 0x7f6ff415a190 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ffac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ff4166970 con 0x7f6ff415a5f0 2024-09-11T12:55:31.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47020/0 (socket says 172.21.15.78:47020) 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 -- 172.21.15.78:0/3482183520 learned_addr learned my addr 172.21.15.78:0/3482183520 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff8c00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 0x7f6ff41459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3400640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 0x7f6ff41454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 -- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 msgr2=0x7f6ff41459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 0x7f6ff41459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 -- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 msgr2=0x7f6ff41454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 0x7f6ff41454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3e00640 1 -- 172.21.15.78:0/3482183520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff4152170 con 0x7f6fd8003a60 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff8c00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 0x7f6ff41459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.174+0000 7f6ff3400640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 0x7f6ff41454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:31.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6ff3e00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009ee0 tx=0x7f6fe4009b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40c0070 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fe4003b80 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe40aa730 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff4152450 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff4152a10 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff4073f70 con 0x7f6fd8003a60 2024-09-11T12:55:31.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f6fe40aa8d0 con 0x7f6fd8003a60 2024-09-11T12:55:31.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.178+0000 7f6febe00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 0x7f6fd4074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.182+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f6fe412d6c0 con 0x7f6fd8003a60 2024-09-11T12:55:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.182+0000 7f6ff3400640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 0x7f6fd4074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.182+0000 7f6ff3400640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 0x7f6fd4074110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ff415b3a0 tx=0x7f6fe00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.186+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6fe40ae020 con 0x7f6fd8003a60 2024-09-11T12:55:31.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.350+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ff4152df0 con 0x7f6fd8003a60 2024-09-11T12:55:31.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6febe00640 1 -- 172.21.15.78:0/3482183520 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) ==== 74+0+16850 (secure 0 0 0) 0x7f6fe4017340 con 0x7f6fd8003a60 2024-09-11T12:55:31.356 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:31.356 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":117,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:28.863501+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:09.848106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"115","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:11.124742+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 msgr2=0x7f6fd4074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 0x7f6fd4074110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6ff415b3a0 tx=0x7f6fe00091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 msgr2=0x7f6ff4153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6fe4009ee0 tx=0x7f6fe4009b40 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 shutdown_connections 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ff415a5f0 0x7f6ff41459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6fd4071c50 0x7f6fd4074110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ff415a190 0x7f6ff41454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 --2- 172.21.15.78:0/3482183520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fd8003a60 0x7f6ff4153210 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 >> 172.21.15.78:0/3482183520 conn(0x7f6ff406ca80 msgr2=0x7f6ff415f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 shutdown_connections 2024-09-11T12:55:31.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.354+0000 7f6ffac00640 1 -- 172.21.15.78:0/3482183520 wait complete. 2024-09-11T12:55:31.379 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- 172.21.15.78:0/2113093125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb8004010 msgr2=0x7f4fb8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/2113093125 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fb8004010 0x7f4fb8004810 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f4fc00016f0 tx=0x7f4fc001c7b0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd3400640 1 -- 172.21.15.78:0/2113093125 reap_dead start 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- 172.21.15.78:0/2113093125 shutdown_connections 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- 172.21.15.78:0/2113093125 >> 172.21.15.78:0/2113093125 conn(0x7f4fd406ca80 msgr2=0x7f4fd40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- 172.21.15.78:0/2113093125 shutdown_connections 2024-09-11T12:55:31.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- 172.21.15.78:0/2113093125 wait complete. 2024-09-11T12:55:31.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 Processor -- start 2024-09-11T12:55:31.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.518+0000 7f4fd9c00640 1 -- start start 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb8004010 0x7f4fd41548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fd41f7b10 0x7f4fd4159440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406f280 con 0x7f4fd41f7f90 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd406f100 con 0x7f4fd41f7b10 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd406f400 con 0x7f4fb8004010 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47040/0 (socket says 172.21.15.78:47040) 2024-09-11T12:55:31.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 -- 172.21.15.78:0/69844714 learned_addr learned my addr 172.21.15.78:0/69844714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd2a00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb8004010 0x7f4fd41548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd2000640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fd41f7b10 0x7f4fd4159440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 -- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb8004010 msgr2=0x7f4fd41548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb8004010 0x7f4fd41548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 -- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fd41f7b10 msgr2=0x7f4fd4159440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fd41f7b10 0x7f4fd4159440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 -- 172.21.15.78:0/69844714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd41f8a90 con 0x7f4fd41f7f90 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd3400640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40b2c30 tx=0x7f4fcc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc004240 con 0x7f4fd41f7f90 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fcc0043e0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fcc00db00 con 0x7f4fd41f7f90 2024-09-11T12:55:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd41f84d0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41540c0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.522+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd40b4db0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.526+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f4fcc00dca0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.526+0000 7f4fcaa00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 0x7f4fb40740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.526+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5765+0+0 (secure 0 0 0) 0x7f4fcc092660 con 0x7f4fd41f7f90 2024-09-11T12:55:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.526+0000 7f4fd2a00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 0x7f4fb40740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.526+0000 7f4fd2a00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 0x7f4fb40740e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001bc0 tx=0x7f4fc4006b30 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.530+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4fcc05d6b0 con 0x7f4fd41f7f90 2024-09-11T12:55:31.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.730+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4fd4073000 con 0x7f4fd41f7f90 2024-09-11T12:55:31.734 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:55:31.730+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:55:31.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fcaa00640 1 -- 172.21.15.78:0/69844714 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) ==== 161+0+0 (secure 0 0 0) 0x7f4fcc061c00 con 0x7f4fd41f7f90 2024-09-11T12:55:31.736 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:55:31.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 msgr2=0x7f4fb40740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 0x7f4fb40740e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4001bc0 tx=0x7f4fc4006b30 comp rx=0 tx=0).stop 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 msgr2=0x7f4fd4159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f4fd40b2c30 tx=0x7f4fcc00ee80 comp rx=0 tx=0).stop 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 shutdown_connections 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4fd41f7f90 0x7f4fd4159980 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4fb4071c20 0x7f4fb40740e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4fd41f7b10 0x7f4fd4159440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 --2- 172.21.15.78:0/69844714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4fb8004010 0x7f4fd41548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 >> 172.21.15.78:0/69844714 conn(0x7f4fd406ca80 msgr2=0x7f4fd40cf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 shutdown_connections 2024-09-11T12:55:31.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.734+0000 7f4fd9c00640 1 -- 172.21.15.78:0/69844714 wait complete. 2024-09-11T12:55:31.759 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:55:31.808 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 -- 172.21.15.78:0/1883108902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d20003fe0 msgr2=0x7f8d20004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 --2- 172.21.15.78:0/1883108902 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d20003fe0 0x7f8d20004810 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f8d28009d30 tx=0x7f8d2801d450 comp rx=0 tx=0).stop 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d41000640 1 -- 172.21.15.78:0/1883108902 reap_dead start 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 -- 172.21.15.78:0/1883108902 shutdown_connections 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 -- 172.21.15.78:0/1883108902 >> 172.21.15.78:0/1883108902 conn(0x7f8d3c075a10 msgr2=0x7f8d3c0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 -- 172.21.15.78:0/1883108902 shutdown_connections 2024-09-11T12:55:31.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.902+0000 7f8d43000640 1 -- 172.21.15.78:0/1883108902 wait complete. 2024-09-11T12:55:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 Processor -- start 2024-09-11T12:55:31.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- start start 2024-09-11T12:55:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 0x7f8d3c154750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d3c14fd40 0x7f8d3c157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c06f910 con 0x7f8d3c1f5800 2024-09-11T12:55:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8d3c06f790 con 0x7f8d20003fe0 2024-09-11T12:55:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8d3c06fa90 con 0x7f8d3c14fd40 2024-09-11T12:55:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47050/0 (socket says 172.21.15.78:47050) 2024-09-11T12:55:31.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 -- 172.21.15.78:0/2818329000 learned_addr learned my addr 172.21.15.78:0/2818329000 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:31.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 -- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d3c14fd40 msgr2=0x7f8d3c157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d3c14fd40 0x7f8d3c157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 -- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 msgr2=0x7f8d3c154750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d3be00640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 0x7f8d3c154750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 0x7f8d3c154750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 -- 172.21.15.78:0/2818329000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d3c154c90 con 0x7f8d3c1f5800 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d3be00640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 0x7f8d3c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d41000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c072fa0 tx=0x7f8d34006410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d34003b80 con 0x7f8d3c1f5800 2024-09-11T12:55:31.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d34003d40 con 0x7f8d3c1f5800 2024-09-11T12:55:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d3c152080 con 0x7f8d3c1f5800 2024-09-11T12:55:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d340052d0 con 0x7f8d3c1f5800 2024-09-11T12:55:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.906+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d3c152640 con 0x7f8d3c1f5800 2024-09-11T12:55:31.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d3c0725d0 con 0x7f8d3c1f5800 2024-09-11T12:55:31.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f8d34005470 con 0x7f8d3c1f5800 2024-09-11T12:55:31.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d33e00640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 0x7f8d1c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:31.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5823+0+0 (secure 0 0 0) 0x7f8d34095de0 con 0x7f8d3c1f5800 2024-09-11T12:55:31.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d3be00640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 0x7f8d1c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:31.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.910+0000 7f8d3be00640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 0x7f8d1c0740e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005bd0 tx=0x7f8d2c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:31.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:31.914+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8d3409b020 con 0x7f8d3c1f5800 2024-09-11T12:55:32.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.114+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8d3c0cf560 con 0x7f8d3c1f5800 2024-09-11T12:55:32.738 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.734+0000 7f8d33e00640 1 -- 172.21.15.78:0/2818329000 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) ==== 104+0+0 (secure 0 0 0) 0x7f8d3c15b720 con 0x7f8d3c1f5800 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 msgr2=0x7f8d1c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 0x7f8d1c0740e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c005bd0 tx=0x7f8d2c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 msgr2=0x7f8d3c1f5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c072fa0 tx=0x7f8d34006410 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 shutdown_connections 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8d3c1f5800 0x7f8d3c1f5c50 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8d3c14fd40 0x7f8d3c157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8d1c071c20 0x7f8d1c0740e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 --2- 172.21.15.78:0/2818329000 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8d20003fe0 0x7f8d3c154750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 >> 172.21.15.78:0/2818329000 conn(0x7f8d3c075a10 msgr2=0x7f8d3c15c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:32.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 shutdown_connections 2024-09-11T12:55:32.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.738+0000 7f8d43000640 1 -- 172.21.15.78:0/2818329000 wait complete. 2024-09-11T12:55:32.763 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:55:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 -- 172.21.15.78:0/3346146236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac15e9c0 msgr2=0x7f6fac0cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/3346146236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac15e9c0 0x7f6fac0cc5e0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f6f98009900 tx=0x7f6f9801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 -- 172.21.15.78:0/3346146236 shutdown_connections 2024-09-11T12:55:32.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/3346146236 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac0ccb20 0x7f6fac13dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/3346146236 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac15e9c0 0x7f6fac0cc5e0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 -- 172.21.15.78:0/3346146236 >> 172.21.15.78:0/3346146236 conn(0x7f6fac0b2c70 msgr2=0x7f6fac0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 -- 172.21.15.78:0/3346146236 shutdown_connections 2024-09-11T12:55:32.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.910+0000 7f6fb0a00640 1 -- 172.21.15.78:0/3346146236 wait complete. 2024-09-11T12:55:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 Processor -- start 2024-09-11T12:55:32.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- start start 2024-09-11T12:55:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac13d4f0 0x7f6fac1fb130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac15e9c0 0x7f6fac1f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac1675c0 con 0x7f6fac0ccb20 2024-09-11T12:55:32.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6fac167440 con 0x7f6fac15e9c0 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6fac167740 con 0x7f6fac13d4f0 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47068/0 (socket says 172.21.15.78:47068) 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 -- 172.21.15.78:0/543886850 learned_addr learned my addr 172.21.15.78:0/543886850 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fab200640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac15e9c0 0x7f6fac1f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:32.919 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fa9e00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac13d4f0 0x7f6fac1fb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 -- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac13d4f0 msgr2=0x7f6fac1fb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac13d4f0 0x7f6fac1fb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 -- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac15e9c0 msgr2=0x7f6fac1f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:32.920 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac15e9c0 0x7f6fac1f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 -- 172.21.15.78:0/543886850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fac1fb670 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6faa800640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c400 tx=0x7f6f9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c017070 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f9c0053c0 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f9c003880 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fac1f7980 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.914+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fac1f7f40 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fac13dda0 con 0x7f6fac0ccb20 2024-09-11T12:55:32.921 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f6f9c01a070 con 0x7f6fac0ccb20 2024-09-11T12:55:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fa2a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 0x7f6f8c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5823+0+0 (secure 0 0 0) 0x7f6f9c091c20 con 0x7f6fac0ccb20 2024-09-11T12:55:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fa9e00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 0x7f6f8c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:32.923 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.918+0000 7f6fa9e00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 0x7f6f8c0740e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6f980af000 tx=0x7f6f9801b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:32.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:32.922+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6fac1fb670 con 0x7f6fac0ccb20 2024-09-11T12:55:33.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.090+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6fac156e20 con 0x7f6fac0ccb20 2024-09-11T12:55:33.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.890+0000 7f6fa2a00640 1 -- 172.21.15.78:0/543886850 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f6f9c001130 con 0x7f6fac0ccb20 2024-09-11T12:55:33.892 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.890+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 msgr2=0x7f6f8c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.890+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 0x7f6f8c0740e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6f980af000 tx=0x7f6f9801b0f0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.890+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 msgr2=0x7f6fac138ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.890+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f6f9c00c400 tx=0x7f6f9c00c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 shutdown_connections 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6f8c071c20 0x7f6f8c0740e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6fac15e9c0 0x7f6fac1f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6fac13d4f0 0x7f6fac1fb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 --2- 172.21.15.78:0/543886850 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6fac0ccb20 0x7f6fac138ba0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:33.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 >> 172.21.15.78:0/543886850 conn(0x7f6fac0b2c70 msgr2=0x7f6fac157b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:33.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 shutdown_connections 2024-09-11T12:55:33.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:33.894+0000 7f6fb0a00640 1 -- 172.21.15.78:0/543886850 wait complete. 2024-09-11T12:55:33.918 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 -- 172.21.15.78:0/4161697907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 msgr2=0x7f01d0073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 --2- 172.21.15.78:0/4161697907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d0073950 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f01d01581d0 tx=0x7f01c001cd50 comp rx=0 tx=0).stop 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 -- 172.21.15.78:0/4161697907 shutdown_connections 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 --2- 172.21.15.78:0/4161697907 >> v2:172.21.15.157:3300/0 conn(0x7f01d0072520 0x7f01d006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 --2- 172.21.15.78:0/4161697907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d0073950 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 -- 172.21.15.78:0/4161697907 >> 172.21.15.78:0/4161697907 conn(0x7f01d00b3050 msgr2=0x7f01d00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 -- 172.21.15.78:0/4161697907 shutdown_connections 2024-09-11T12:55:34.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.058+0000 7f01d4c00640 1 -- 172.21.15.78:0/4161697907 wait complete. 2024-09-11T12:55:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 Processor -- start 2024-09-11T12:55:34.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 -- start start 2024-09-11T12:55:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0072520 0x7f01d01fb580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 0x7f01d0149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:34.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f01d00703e0 con 0x7f01d0151050 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f01d0070260 con 0x7f01d0073e90 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f01d0070560 con 0x7f01d0072520 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59866/0 (socket says 172.21.15.78:59866) 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 -- 172.21.15.78:0/3356625597 learned_addr learned my addr 172.21.15.78:0/3356625597 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01cde00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0072520 0x7f01d01fb580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:34.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01cd400640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 0x7f01d0149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 -- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0072520 msgr2=0x7f01d01fb580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0072520 0x7f01d01fb580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 -- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 msgr2=0x7f01d0149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:34.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 0x7f01d0149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 -- 172.21.15.78:0/3356625597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01d00b0140 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01cd400640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 0x7f01d0149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01ce800640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f01c800dbc0 tx=0x7f01c800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8005020 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01c80052b0 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80047b0 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.062+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01d0149dd0 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01d00aece0 con 0x7f01d0151050 2024-09-11T12:55:34.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01d006f5d0 con 0x7f01d0151050 2024-09-11T12:55:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f01c8004950 con 0x7f01d0151050 2024-09-11T12:55:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01c6000640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 0x7f01b00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:34.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5357+0+0 (secure 0 0 0) 0x7f01c8092cc0 con 0x7f01d0151050 2024-09-11T12:55:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01cde00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 0x7f01b00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:34.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.066+0000 7f01cde00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 0x7f01b00740e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01c00af000 tx=0x7f01c001b590 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:34.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.070+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f01c805ddd0 con 0x7f01d0151050 2024-09-11T12:55:34.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.234+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01d0070ce0 con 0x7f01d0151050 2024-09-11T12:55:34.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.894+0000 7f01c6000640 1 -- 172.21.15.78:0/3356625597 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7f01c8062320 con 0x7f01d0151050 2024-09-11T12:55:34.900 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:55:34.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 msgr2=0x7f01b00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:34.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 0x7f01b00740e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f01c00af000 tx=0x7f01c001b590 comp rx=0 tx=0).stop 2024-09-11T12:55:34.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 msgr2=0x7f01d01564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:34.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f01c800dbc0 tx=0x7f01c800a620 comp rx=0 tx=0).stop 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 shutdown_connections 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01d0151050 0x7f01d01564e0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01b0071c20 0x7f01b00740e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f01d0073e90 0x7f01d0149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 --2- 172.21.15.78:0/3356625597 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01d0072520 0x7f01d01fb580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:34.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 >> 172.21.15.78:0/3356625597 conn(0x7f01d00b3050 msgr2=0x7f01d014ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:34.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 shutdown_connections 2024-09-11T12:55:34.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:34.898+0000 7f01d4c00640 1 -- 172.21.15.78:0/3356625597 wait complete. 2024-09-11T12:55:34.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.062+0000 7f2885600640 1 -- 172.21.15.78:0/263973122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f288015ba80 msgr2=0x7f288006fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.062+0000 7f2885600640 1 --2- 172.21.15.78:0/263973122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f288015ba80 0x7f288006fc50 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f288015b1b0 tx=0x7f286c01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- 172.21.15.78:0/263973122 shutdown_connections 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 --2- 172.21.15.78:0/263973122 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f288015ba80 0x7f288006fc50 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- 172.21.15.78:0/263973122 >> 172.21.15.78:0/263973122 conn(0x7f288006ca80 msgr2=0x7f28800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- 172.21.15.78:0/263973122 shutdown_connections 2024-09-11T12:55:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- 172.21.15.78:0/263973122 wait complete. 2024-09-11T12:55:35.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 Processor -- start 2024-09-11T12:55:35.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- start start 2024-09-11T12:55:35.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 0x7f288014bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 0x7f28801573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f288015ee40 con 0x7f2880074240 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f288015ecc0 con 0x7f2880070190 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2880168f60 con 0x7f288015ba80 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59894/0 (socket says 172.21.15.78:59894) 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 -- 172.21.15.78:0/2885328090 learned_addr learned my addr 172.21.15.78:0/2885328090 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287ea00640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 0x7f288014bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287f400640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 0x7f28801573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 -- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 msgr2=0x7f28801573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 0x7f28801573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 -- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 msgr2=0x7f288014bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 0x7f288014bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 -- 172.21.15.78:0/2885328090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28801f9710 con 0x7f2880074240 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287ea00640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 0x7f288014bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287f400640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 0x7f28801573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f287e000640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f286c000c10 tx=0x7f286c004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0bb070 con 0x7f2880074240 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f286c005120 con 0x7f2880074240 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f286c0b2040 con 0x7f2880074240 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.066+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28801f8950 con 0x7f2880074240 2024-09-11T12:55:35.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28801f8f10 con 0x7f2880074240 2024-09-11T12:55:35.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f288006fc50 con 0x7f2880074240 2024-09-11T12:55:35.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f286c0054d0 con 0x7f2880074240 2024-09-11T12:55:35.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f2876a00640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 0x7f28600740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f286c12c0c0 con 0x7f2880074240 2024-09-11T12:55:35.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f287ea00640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 0x7f28600740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.070+0000 7f287ea00640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 0x7f28600740e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2870007fd0 tx=0x7f2870009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:35.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.074+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f286c016410 con 0x7f2880074240 2024-09-11T12:55:35.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.238+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f288014dd10 con 0x7f2880074240 2024-09-11T12:55:35.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.242+0000 7f2876a00640 1 -- 172.21.15.78:0/2885328090 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f286c0f7980 con 0x7f2880074240 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 msgr2=0x7f28600740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 0x7f28600740e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2870007fd0 tx=0x7f2870009040 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 msgr2=0x7f2880156eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f286c000c10 tx=0x7f286c004220 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 shutdown_connections 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f288015ba80 0x7f28801573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2880074240 0x7f2880156eb0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2860071c20 0x7f28600740e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 --2- 172.21.15.78:0/2885328090 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2880070190 0x7f288014bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 >> 172.21.15.78:0/2885328090 conn(0x7f288006ca80 msgr2=0x7f28800ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:35.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 shutdown_connections 2024-09-11T12:55:35.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.246+0000 7f2885600640 1 -- 172.21.15.78:0/2885328090 wait complete. 2024-09-11T12:55:35.269 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 -- 172.21.15.78:0/986480407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c0d1400 msgr2=0x7f5d9c1520b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 --2- 172.21.15.78:0/986480407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c1520b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c0d1a30 tx=0x7f5d9401c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 -- 172.21.15.78:0/986480407 shutdown_connections 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 --2- 172.21.15.78:0/986480407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c1520b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 --2- 172.21.15.78:0/986480407 >> v2:172.21.15.157:3300/0 conn(0x7f5d9c06e860 0x7f5d9c0d0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 -- 172.21.15.78:0/986480407 >> 172.21.15.78:0/986480407 conn(0x7f5d9c0ceab0 msgr2=0x7f5d9c0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 -- 172.21.15.78:0/986480407 shutdown_connections 2024-09-11T12:55:35.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.406+0000 7f5da0a00640 1 -- 172.21.15.78:0/986480407 wait complete. 2024-09-11T12:55:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 Processor -- start 2024-09-11T12:55:35.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- start start 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c137770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 0x7f5d9c1fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c0aa2e0 con 0x7f5d9c06e860 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9c0aa160 con 0x7f5d9c0d1400 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5d9c0aa460 con 0x7f5d9c15c1d0 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59910/0 (socket says 172.21.15.78:59910) 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 -- 172.21.15.78:0/508289157 learned_addr learned my addr 172.21.15.78:0/508289157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99400640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c137770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d9a800640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 0x7f5d9c1fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 -- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 msgr2=0x7f5d9c1fb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 0x7f5d9c1fb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 -- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d9c0d1400 msgr2=0x7f5d9c137770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c137770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 -- 172.21.15.78:0/508289157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d9c155d10 con 0x7f5d9c06e860 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d9a800640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 0x7f5d9c1fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d99e00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c730 tx=0x7f5d8c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c003b80 con 0x7f5d9c06e860 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8c003d40 con 0x7f5d9c06e860 2024-09-11T12:55:35.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9c155ff0 con 0x7f5d9c06e860 2024-09-11T12:55:35.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00aab0 con 0x7f5d9c06e860 2024-09-11T12:55:35.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.410+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9c1fb4c0 con 0x7f5d9c06e860 2024-09-11T12:55:35.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d9c1520b0 con 0x7f5d9c06e860 2024-09-11T12:55:35.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f5d8c00ac50 con 0x7f5d9c06e860 2024-09-11T12:55:35.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5d92000640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 0x7f5d7c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:35.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f5d8c0922c0 con 0x7f5d9c06e860 2024-09-11T12:55:35.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5d99400640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 0x7f5d7c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:35.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.414+0000 7f5d99400640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 0x7f5d7c0740e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c154060 tx=0x7f5d880091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:35.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.418+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5d8c05dc50 con 0x7f5d9c06e860 2024-09-11T12:55:35.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.646+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f5d9c0b39c0 con 0x7f5d9c06e860 2024-09-11T12:55:35.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5d92000640 1 -- 172.21.15.78:0/508289157 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v330) ==== 148+0+0 (secure 0 0 0) 0x7f5d8c05d550 con 0x7f5d9c06e860 2024-09-11T12:55:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 msgr2=0x7f5d7c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 0x7f5d7c0740e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d9c154060 tx=0x7f5d880091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 msgr2=0x7f5d9c153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:35.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f5d8c00c730 tx=0x7f5d8c00cbf0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 shutdown_connections 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5d9c15c1d0 0x7f5d9c1fb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d7c071c20 0x7f5d7c0740e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5d9c0d1400 0x7f5d9c137770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 --2- 172.21.15.78:0/508289157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5d9c06e860 0x7f5d9c153b20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 >> 172.21.15.78:0/508289157 conn(0x7f5d9c0ceab0 msgr2=0x7f5d9c15d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 shutdown_connections 2024-09-11T12:55:35.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:35.898+0000 7f5da0a00640 1 -- 172.21.15.78:0/508289157 wait complete. 2024-09-11T12:55:35.923 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-11T12:55:35.923 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:55:35.923 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- 172.21.15.78:0/1003387896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 msgr2=0x7fc76c15e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 --2- 172.21.15.78:0/1003387896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c15e870 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fc760009900 tx=0x7fc76001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- 172.21.15.78:0/1003387896 shutdown_connections 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 --2- 172.21.15.78:0/1003387896 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c13d9e0 0x7fc76c070a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 --2- 172.21.15.78:0/1003387896 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c15e870 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- 172.21.15.78:0/1003387896 >> 172.21.15.78:0/1003387896 conn(0x7fc76c06d890 msgr2=0x7fc76c0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- 172.21.15.78:0/1003387896 shutdown_connections 2024-09-11T12:55:36.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- 172.21.15.78:0/1003387896 wait complete. 2024-09-11T12:55:36.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 Processor -- start 2024-09-11T12:55:36.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.062+0000 7fc773e00640 1 -- start start 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc76c13d9e0 0x7fc76c12a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 0x7fc76c0cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c168100 con 0x7fc76c15a870 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc76c070120 con 0x7fc76c13d9e0 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc76c168280 con 0x7fc76c154800 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59932/0 (socket says 172.21.15.78:59932) 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 -- 172.21.15.78:0/4005490759 learned_addr learned my addr 172.21.15.78:0/4005490759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771400640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc76c13d9e0 0x7fc76c12a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc770a00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 0x7fc76c0cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 -- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 msgr2=0x7fc76c0cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 0x7fc76c0cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 -- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc76c13d9e0 msgr2=0x7fc76c12a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc76c13d9e0 0x7fc76c12a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 -- 172.21.15.78:0/4005490759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76c12ae30 con 0x7fc76c15a870 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc770a00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 0x7fc76c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:36.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc771e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ea20 tx=0x7fc76400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76400cde0 con 0x7fc76c15a870 2024-09-11T12:55:36.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc764004c40 con 0x7fc76c15a870 2024-09-11T12:55:36.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc764005280 con 0x7fc76c15a870 2024-09-11T12:55:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76c1537f0 con 0x7fc76c15a870 2024-09-11T12:55:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76c153db0 con 0x7fc76c15a870 2024-09-11T12:55:36.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.066+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc76c070a90 con 0x7fc76c15a870 2024-09-11T12:55:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.070+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fc764005420 con 0x7fc76c15a870 2024-09-11T12:55:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.070+0000 7fc769600640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 0x7fc74c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.070+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7fc764092df0 con 0x7fc76c15a870 2024-09-11T12:55:36.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.070+0000 7fc771400640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 0x7fc74c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.070+0000 7fc771400640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 0x7fc74c0740e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7600af000 tx=0x7fc76001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.074+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc764016030 con 0x7fc76c15a870 2024-09-11T12:55:36.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.238+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc76c0cf4f0 con 0x7fc76c15a870 2024-09-11T12:55:36.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.238+0000 7fc769600640 1 -- 172.21.15.78:0/4005490759 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13044 (secure 0 0 0) 0x7fc76405e020 con 0x7fc76c15a870 2024-09-11T12:55:36.244 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:36.244 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":121,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:34.894819+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/2586938426":"2024-09-12T12:55:31.735911+0000","172.21.15.157:6800/2586938426":"2024-09-12T12:55:31.735911+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 msgr2=0x7fc74c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 0x7fc74c0740e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc7600af000 tx=0x7fc76001b120 comp rx=0 tx=0).stop 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 msgr2=0x7fc76c1597d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fc76400ea20 tx=0x7fc76400eee0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 shutdown_connections 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc76c15a870 0x7fc76c1597d0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc76c154800 0x7fc76c0cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc74c071c20 0x7fc74c0740e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 --2- 172.21.15.78:0/4005490759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc76c13d9e0 0x7fc76c12a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 >> 172.21.15.78:0/4005490759 conn(0x7fc76c06d890 msgr2=0x7fc76c15da20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:36.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 shutdown_connections 2024-09-11T12:55:36.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.242+0000 7fc773e00640 1 -- 172.21.15.78:0/4005490759 wait complete. 2024-09-11T12:55:36.265 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:55:36.265 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:55:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 -- 172.21.15.78:0/2387148660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240cd770 msgr2=0x7f24240cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 --2- 172.21.15.78:0/2387148660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240cd770 0x7f24240cdc20 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f241c009840 tx=0x7f241c01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 -- 172.21.15.78:0/2387148660 shutdown_connections 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 --2- 172.21.15.78:0/2387148660 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240cd770 0x7f24240cdc20 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 --2- 172.21.15.78:0/2387148660 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240b3420 0x7f24240cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 -- 172.21.15.78:0/2387148660 >> 172.21.15.78:0/2387148660 conn(0x7f2424071dd0 msgr2=0x7f24240721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 -- 172.21.15.78:0/2387148660 shutdown_connections 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.406+0000 7f242b400640 1 -- 172.21.15.78:0/2387148660 wait complete. 2024-09-11T12:55:36.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 Processor -- start 2024-09-11T12:55:36.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- start start 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24240b3040 0x7f24241fb470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240cd770 0x7f24240ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f24240af0c0 con 0x7f24240b3420 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f24240aef40 con 0x7f24240b3040 2024-09-11T12:55:36.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f24240af240 con 0x7f24240cd770 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59954/0 (socket says 172.21.15.78:59954) 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 -- 172.21.15.78:0/1692791185 learned_addr learned my addr 172.21.15.78:0/1692791185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2428a00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24240b3040 0x7f24241fb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2429400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240cd770 0x7f24240ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 -- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240cd770 msgr2=0x7f24240ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240cd770 0x7f24240ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 -- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24240b3040 msgr2=0x7f24241fb470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24240b3040 0x7f24241fb470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 -- 172.21.15.78:0/1692791185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24240b1470 con 0x7f24240b3420 2024-09-11T12:55:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2423e00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f241000d980 tx=0x7f241000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410017070 con 0x7f24240b3420 2024-09-11T12:55:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2410009d20 con 0x7f24240b3420 2024-09-11T12:55:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410004280 con 0x7f24240b3420 2024-09-11T12:55:36.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24240b1750 con 0x7f24240b3420 2024-09-11T12:55:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.410+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24241fac70 con 0x7f24240b3420 2024-09-11T12:55:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24240cf6e0 con 0x7f24240b3420 2024-09-11T12:55:36.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f2410003600 con 0x7f24240b3420 2024-09-11T12:55:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f2420c00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 0x7f24040740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f2410092670 con 0x7f24240b3420 2024-09-11T12:55:36.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f2428a00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 0x7f24040740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.414+0000 7f2428a00640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 0x7f24040740e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24140012f0 tx=0x7f2414009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.418+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f241005d7d0 con 0x7f24240b3420 2024-09-11T12:55:36.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.582+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2424140fc0 con 0x7f24240b3420 2024-09-11T12:55:36.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f2420c00640 1 -- 172.21.15.78:0/1692791185 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v121) ==== 93+0+10 (secure 0 0 0) 0x7f2410061d20 con 0x7f24240b3420 2024-09-11T12:55:36.588 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 msgr2=0x7f24040740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 0x7f24040740e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f24140012f0 tx=0x7f2414009040 comp rx=0 tx=0).stop 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 msgr2=0x7f242413c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f241000d980 tx=0x7f241000de40 comp rx=0 tx=0).stop 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 shutdown_connections 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f24240cd770 0x7f24240ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f24240b3420 0x7f242413c660 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2404071c20 0x7f24040740e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 --2- 172.21.15.78:0/1692791185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f24240b3040 0x7f24241fb470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 >> 172.21.15.78:0/1692791185 conn(0x7f2424071dd0 msgr2=0x7f24240701c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:36.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 shutdown_connections 2024-09-11T12:55:36.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.586+0000 7f242b400640 1 -- 172.21.15.78:0/1692791185 wait complete. 2024-09-11T12:55:36.609 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 -- 172.21.15.78:0/1831692608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc13d250 msgr2=0x7f55bc0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1831692608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc13d250 0x7f55bc0cc410 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f55bc06cf90 tx=0x7f55b401d7f0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 -- 172.21.15.78:0/1831692608 shutdown_connections 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1831692608 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc13d250 0x7f55bc0cc410 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1831692608 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc15a1c0 0x7f55bc0cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1831692608 >> v2:172.21.15.157:3300/0 conn(0x7f55bc0b4780 0x7f55bc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 -- 172.21.15.78:0/1831692608 >> 172.21.15.78:0/1831692608 conn(0x7f55bc06ca80 msgr2=0x7f55bc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.746+0000 7f55c0a00640 1 -- 172.21.15.78:0/1831692608 shutdown_connections 2024-09-11T12:55:36.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- 172.21.15.78:0/1831692608 wait complete. 2024-09-11T12:55:36.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 Processor -- start 2024-09-11T12:55:36.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- start start 2024-09-11T12:55:36.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55bc0b4780 0x7f55bc1651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc13d250 0x7f55bc0b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc06eed0 con 0x7f55bc15a1c0 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f55bc06ed50 con 0x7f55bc0b4780 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f55bc06f050 con 0x7f55bc13d250 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59976/0 (socket says 172.21.15.78:59976) 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 -- 172.21.15.78:0/1399356593 learned_addr learned my addr 172.21.15.78:0/1399356593 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55b9e00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55bc0b4780 0x7f55bc1651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55b9400640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc13d250 0x7f55bc0b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 -- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc13d250 msgr2=0x7f55bc0b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc13d250 0x7f55bc0b0220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 -- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55bc0b4780 msgr2=0x7f55bc1651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55bc0b4780 0x7f55bc1651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 -- 172.21.15.78:0/1399356593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55bc165730 con 0x7f55bc15a1c0 2024-09-11T12:55:36.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55ba800640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f55b401bea0 tx=0x7f55b40a66e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4007c60 con 0x7f55bc15a1c0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55b4003980 con 0x7f55bc15a1c0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55b4003d70 con 0x7f55bc15a1c0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.750+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55bc1f8b80 con 0x7f55bc15a1c0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55bc1f9140 con 0x7f55bc15a1c0 2024-09-11T12:55:36.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55bc15e1c0 con 0x7f55bc15a1c0 2024-09-11T12:55:36.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f55b40be060 con 0x7f55bc15a1c0 2024-09-11T12:55:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55b2000640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 0x7f559c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f55b412bf50 con 0x7f55bc15a1c0 2024-09-11T12:55:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.754+0000 7f55b9e00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 0x7f559c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:36.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.758+0000 7f55b9e00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 0x7f559c0740e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55bc141780 tx=0x7f55ac00a2d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:36.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.758+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f55b40174a0 con 0x7f55bc15a1c0 2024-09-11T12:55:36.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:36.990+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f55bc159d70 con 0x7f55bc15a1c0 2024-09-11T12:55:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.906+0000 7f55b2000640 1 -- 172.21.15.78:0/1399356593 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v332) ==== 154+0+0 (secure 0 0 0) 0x7f55b40f71c0 con 0x7f55bc15a1c0 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 msgr2=0x7f559c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 0x7f559c0740e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f55bc141780 tx=0x7f55ac00a2d0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 msgr2=0x7f55bc0b0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f55b401bea0 tx=0x7f55b40a66e0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 shutdown_connections 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f55bc15a1c0 0x7f55bc0b0760 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f55bc13d250 0x7f55bc0b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f559c071c20 0x7f559c0740e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 --2- 172.21.15.78:0/1399356593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f55bc0b4780 0x7f55bc1651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 >> 172.21.15.78:0/1399356593 conn(0x7f55bc06ca80 msgr2=0x7f55bc072bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 shutdown_connections 2024-09-11T12:55:37.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:37.910+0000 7f55c0a00640 1 -- 172.21.15.78:0/1399356593 wait complete. 2024-09-11T12:55:37.937 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:55:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- 172.21.15.78:0/986728499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 msgr2=0x7f81f00cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- 172.21.15.78:0/986728499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f00cbed0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f81f01512b0 tx=0x7f81dc01d470 comp rx=0 tx=0).stop 2024-09-11T12:55:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- 172.21.15.78:0/986728499 shutdown_connections 2024-09-11T12:55:38.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- 172.21.15.78:0/986728499 >> v2:172.21.15.157:3300/0 conn(0x7f81f013d250 0x7f81f00cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- 172.21.15.78:0/986728499 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f00cbed0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- 172.21.15.78:0/986728499 >> 172.21.15.78:0/986728499 conn(0x7f81f006ca80 msgr2=0x7f81f00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:38.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- 172.21.15.78:0/986728499 shutdown_connections 2024-09-11T12:55:38.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- 172.21.15.78:0/986728499 wait complete. 2024-09-11T12:55:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 Processor -- start 2024-09-11T12:55:38.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- start start 2024-09-11T12:55:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 0x7f81f013c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81f013d250 0x7f81f0142680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81f013ead0 con 0x7f81f015a1c0 2024-09-11T12:55:38.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81f013e950 con 0x7f81f00b4780 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f7c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81f013ec50 con 0x7f81f013d250 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59992/0 (socket says 172.21.15.78:59992) 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 -- 172.21.15.78:0/3399568271 learned_addr learned my addr 172.21.15.78:0/3399568271 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 -- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81f013d250 msgr2=0x7f81f0142680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81f013d250 0x7f81f0142680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 -- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 msgr2=0x7f81f013c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5200640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 0x7f81f013c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 0x7f81f013c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5c00640 1 -- 172.21.15.78:0/3399568271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f0155b20 con 0x7f81f015a1c0 2024-09-11T12:55:38.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.082+0000 7f81f5200640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 0x7f81f013c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81f5c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f81e800d750 tx=0x7f81e800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8017070 con 0x7f81f015a1c0 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81e8009d20 con 0x7f81f015a1c0 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e800de60 con 0x7f81f015a1c0 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f0142db0 con 0x7f81f015a1c0 2024-09-11T12:55:38.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f0143370 con 0x7f81f015a1c0 2024-09-11T12:55:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81f013f160 con 0x7f81f015a1c0 2024-09-11T12:55:38.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f81e8005280 con 0x7f81f015a1c0 2024-09-11T12:55:38.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 0x7f81d00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f81e80926e0 con 0x7f81f015a1c0 2024-09-11T12:55:38.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.086+0000 7f81f5200640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 0x7f81d00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.090+0000 7f81f5200640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 0x7f81d00740e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81e00012f0 tx=0x7f81e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:38.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.090+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f81e805de90 con 0x7f81f015a1c0 2024-09-11T12:55:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f81f01541d0 con 0x7f81f015a1c0 2024-09-11T12:55:38.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81ecc00640 1 -- 172.21.15.78:0/3399568271 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+1998 (secure 0 0 0) 0x7f81e80660d0 con 0x7f81f015a1c0 2024-09-11T12:55:38.264 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:55:38.264 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:55:38.265 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:55:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:55:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 msgr2=0x7f81d00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 0x7f81d00740e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f81e00012f0 tx=0x7f81e0009040 comp rx=0 tx=0).stop 2024-09-11T12:55:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 msgr2=0x7f81f0127d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f81e800d750 tx=0x7f81e800dc10 comp rx=0 tx=0).stop 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 shutdown_connections 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81f015a1c0 0x7f81f0127d80 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81f013d250 0x7f81f0142680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81d0071c20 0x7f81d00740e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 --2- 172.21.15.78:0/3399568271 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81f00b4780 0x7f81f013c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 >> 172.21.15.78:0/3399568271 conn(0x7f81f006ca80 msgr2=0x7f81f00cd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 shutdown_connections 2024-09-11T12:55:38.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.262+0000 7f81f7c00640 1 -- 172.21.15.78:0/3399568271 wait complete. 2024-09-11T12:55:38.285 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:55:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 -- 172.21.15.78:0/917956576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8004b00 msgr2=0x7f65c8004ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 --2- 172.21.15.78:0/917956576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65c8004b00 0x7f65c8004ee0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f65d4001e90 tx=0x7f65d401cb90 comp rx=0 tx=0).stop 2024-09-11T12:55:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e2800640 1 -- 172.21.15.78:0/917956576 reap_dead start 2024-09-11T12:55:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 -- 172.21.15.78:0/917956576 shutdown_connections 2024-09-11T12:55:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 -- 172.21.15.78:0/917956576 >> 172.21.15.78:0/917956576 conn(0x7f65e4071dd0 msgr2=0x7f65e40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 -- 172.21.15.78:0/917956576 shutdown_connections 2024-09-11T12:55:38.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.426+0000 7f65e8a00640 1 -- 172.21.15.78:0/917956576 wait complete. 2024-09-11T12:55:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 Processor -- start 2024-09-11T12:55:38.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- start start 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c8004b00 0x7f65e41423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e412e7c0 0x7f65e4140b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40aef00 con 0x7f65e4157a80 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f65e40aed80 con 0x7f65c8004b00 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f65e40af080 con 0x7f65e412e7c0 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60014/0 (socket says 172.21.15.78:60014) 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 -- 172.21.15.78:0/3990709368 learned_addr learned my addr 172.21.15.78:0/3990709368 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e1400640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e412e7c0 0x7f65e4140b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e1e00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c8004b00 0x7f65e41423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 -- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e412e7c0 msgr2=0x7f65e4140b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e412e7c0 0x7f65e4140b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 -- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c8004b00 msgr2=0x7f65e41423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c8004b00 0x7f65e41423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 -- 172.21.15.78:0/3990709368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e4138af0 con 0x7f65e4157a80 2024-09-11T12:55:38.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e2800640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f65e40cfc20 tx=0x7f65dc011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00f920 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65dc00fac0 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e4141090 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc003600 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.430+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e4141610 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65e415aec0 con 0x7f65e4157a80 2024-09-11T12:55:38.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f65dc0037a0 con 0x7f65e4157a80 2024-09-11T12:55:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65da000640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 0x7f65c40740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f65dc093090 con 0x7f65e4157a80 2024-09-11T12:55:38.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65e1e00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 0x7f65c40740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:38.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.434+0000 7f65e1e00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 0x7f65c40740e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65d4001e60 tx=0x7f65d40b1040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:38.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.438+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f65dc05e230 con 0x7f65e4157a80 2024-09-11T12:55:38.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.606+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f65e40701d0 con 0x7f65e4157a80 2024-09-11T12:55:38.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65da000640 1 -- 172.21.15.78:0/3990709368 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f65dc062780 con 0x7f65e4157a80 2024-09-11T12:55:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 msgr2=0x7f65c40740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 0x7f65c40740e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f65d4001e60 tx=0x7f65d40b1040 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 msgr2=0x7f65e40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f65e40cfc20 tx=0x7f65dc011b70 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 shutdown_connections 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f65e4157a80 0x7f65e40ad870 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f65e412e7c0 0x7f65e4140b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f65c4071c20 0x7f65c40740e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 --2- 172.21.15.78:0/3990709368 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f65c8004b00 0x7f65e41423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 >> 172.21.15.78:0/3990709368 conn(0x7f65e4071dd0 msgr2=0x7f65e41589c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 shutdown_connections 2024-09-11T12:55:38.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:38.610+0000 7f65e8a00640 1 -- 172.21.15.78:0/3990709368 wait complete. 2024-09-11T12:55:38.635 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:55:38.635 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:38.742 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:55:38.834 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:55:38.834 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:38.834 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:38.842 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:55:38.842 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:38.843 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:38.852 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:55:38.852 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:38.852 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:38.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:55:38.895 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:38.895 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:38.907 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:55:38.907 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:55:38.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 -- 172.21.15.78:0/1341269028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 msgr2=0x7f7b9400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1341269028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7b9400b9d0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f7ba400bb10 tx=0x7f7ba401cde0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb2200640 1 -- 172.21.15.78:0/1341269028 reap_dead start 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 -- 172.21.15.78:0/1341269028 shutdown_connections 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 -- 172.21.15.78:0/1341269028 >> 172.21.15.78:0/1341269028 conn(0x7f7bac06ca80 msgr2=0x7f7bac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 -- 172.21.15.78:0/1341269028 shutdown_connections 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.078+0000 7f7bb4200640 1 -- 172.21.15.78:0/1341269028 wait complete. 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 Processor -- start 2024-09-11T12:55:39.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- start start 2024-09-11T12:55:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bac1e8010 0x7f7bac159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bac1e8490 0x7f7bac15a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac074300 con 0x7f7b940051d0 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7bac074180 con 0x7f7bac1e8490 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bac074480 con 0x7f7bac1e8010 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60030/0 (socket says 172.21.15.78:60030) 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 -- 172.21.15.78:0/1161677854 learned_addr learned my addr 172.21.15.78:0/1161677854 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb0e00640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bac1e8010 0x7f7bac159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb2200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bac1e8490 0x7f7bac15a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 -- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bac1e8010 msgr2=0x7f7bac159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bac1e8010 0x7f7bac159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 -- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bac1e8490 msgr2=0x7f7bac15a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bac1e8490 0x7f7bac15a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 -- 172.21.15.78:0/1161677854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bac1e8a60 con 0x7f7b940051d0 2024-09-11T12:55:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb1800640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0010bf0 tx=0x7f7ba000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0003980 con 0x7f7b940051d0 2024-09-11T12:55:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ba0003b40 con 0x7f7b940051d0 2024-09-11T12:55:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ba0004030 con 0x7f7b940051d0 2024-09-11T12:55:39.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bac1e8d40 con 0x7f7b940051d0 2024-09-11T12:55:39.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bac1e9300 con 0x7f7b940051d0 2024-09-11T12:55:39.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.082+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bac06ecf0 con 0x7f7b940051d0 2024-09-11T12:55:39.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.086+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f7ba00041d0 con 0x7f7b940051d0 2024-09-11T12:55:39.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.086+0000 7f7ba9600640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 0x7f7b8c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.086+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f7ba00934e0 con 0x7f7b940051d0 2024-09-11T12:55:39.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.086+0000 7f7bb0e00640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 0x7f7b8c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.086+0000 7f7bb0e00640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 0x7f7b8c0740e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7bac1e7ed0 tx=0x7f7b98006040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:39.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.090+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7ba005e790 con 0x7f7b940051d0 2024-09-11T12:55:39.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.290+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bac070490 con 0x7f7b940051d0 2024-09-11T12:55:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.290+0000 7f7ba9600640 1 -- 172.21.15.78:0/1161677854 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) ==== 94+0+3917 (secure 0 0 0) 0x7f7ba0062ce0 con 0x7f7b940051d0 2024-09-11T12:55:39.294 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:39.294 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":84,"btime":"2024-09-11T12:55:32:738152+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6317,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/3646465795","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3646465795},{"type":"v1","addr":"172.21.15.78:6834","nonce":3646465795}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76},{"gid":6521,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/3795162381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":3795162381},{"type":"v1","addr":"172.21.15.157:6835","nonce":3795162381}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6991,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2762903553","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":2762903553},{"type":"v1","addr":"172.21.15.78:6835","nonce":2762903553}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":6997,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6834/3289712123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6801","nonce":3289712123},{"type":"v1","addr":"172.21.15.157:6834","nonce":3289712123}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":76}],"filesystems":[]} 2024-09-11T12:55:39.294 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 84 2024-09-11T12:55:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 msgr2=0x7f7b8c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 0x7f7b8c0740e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7bac1e7ed0 tx=0x7f7b98006040 comp rx=0 tx=0).stop 2024-09-11T12:55:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 msgr2=0x7f7bac1423b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f7ba0010bf0 tx=0x7f7ba000e720 comp rx=0 tx=0).stop 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 shutdown_connections 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7b8c071c20 0x7f7b8c0740e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bac1e8490 0x7f7bac15a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bac1e8010 0x7f7bac159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 --2- 172.21.15.78:0/1161677854 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7b940051d0 0x7f7bac1423b0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 >> 172.21.15.78:0/1161677854 conn(0x7f7bac06ca80 msgr2=0x7f7bac0b1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 shutdown_connections 2024-09-11T12:55:39.298 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.294+0000 7f7bb4200640 1 -- 172.21.15.78:0/1161677854 wait complete. 2024-09-11T12:55:39.316 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:55:39.316 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:55:39.317 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:55:39.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:39.317 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:55:39.317 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:55:39.317 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:55:39.317 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:39.318 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:55:39.318 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:55:39.318 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:55:39.318 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:39.318 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:55:39.319 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:55:39.319 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:55:39.319 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:55:39.329 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:55:39.330 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:55:39.331 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:55:39.331 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:55:39.333 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:55:39.333 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:55:39.334 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:55:39.334 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:55:39.374 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:55:39.374 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:39.375 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:55:39.376 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:39.382 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:55:39.382 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:39.383 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:55:39.383 DEBUG:teuthology.parallel:result is None 2024-09-11T12:55:39.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:55:39.459 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:55:39.460 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:55:39.475 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:55:39.476 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:55:39.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- 172.21.15.78:0/2314487091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a0003a60 msgr2=0x7f14a0003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 --2- 172.21.15.78:0/2314487091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a0003a60 0x7f14a0003e80 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f14ac00c640 tx=0x7f14ac01d2c0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- 172.21.15.78:0/2314487091 shutdown_connections 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 --2- 172.21.15.78:0/2314487091 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14a0003a60 0x7f14a0003e80 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- 172.21.15.78:0/2314487091 >> 172.21.15.78:0/2314487091 conn(0x7f14bc06ca80 msgr2=0x7f14bc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- 172.21.15.78:0/2314487091 shutdown_connections 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- 172.21.15.78:0/2314487091 wait complete. 2024-09-11T12:55:39.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 Processor -- start 2024-09-11T12:55:39.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- start start 2024-09-11T12:55:39.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 0x7f14bc14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 0x7f14bc1f8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc0b5310 con 0x7f14bc0b0d00 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc0b5190 con 0x7f14a0003a60 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14c2000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f14bc0b5490 con 0x7f14bc1f7e70 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14bb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 0x7f14bc14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60064/0 (socket says 172.21.15.78:60064) 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14bb400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 0x7f14bc14b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:32858/0 (socket says 172.21.15.78:32858) 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14baa00640 1 -- 172.21.15.78:0/1240533146 learned_addr learned my addr 172.21.15.78:0/1240533146 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.534+0000 7f14bbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 0x7f14bc1f8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 -- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 msgr2=0x7f14bc14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 0x7f14bc14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 -- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 msgr2=0x7f14bc0b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 -- 172.21.15.78:0/1240533146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc1f8ba0 con 0x7f14bc1f7e70 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14baa00640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:39.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bbe00640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 0x7f14bc1f8320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0748f0 tx=0x7f14b4001640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4011430 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14b40115d0 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc15a740 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4011a00 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc15ad00 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc06f200 con 0x7f14bc1f7e70 2024-09-11T12:55:39.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f14b4011ba0 con 0x7f14bc1f7e70 2024-09-11T12:55:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 0x7f149c074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(121..121 src has 1..121) ==== 4943+0+0 (secure 0 0 0) 0x7f14b4092990 con 0x7f14bc1f7e70 2024-09-11T12:55:39.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.538+0000 7f14bb400640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 0x7f149c074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:39.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.542+0000 7f14bb400640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 0x7f149c074110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0ac000 tx=0x7f14ac01a100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:39.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.546+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f14b405dd60 con 0x7f14bc1f7e70 2024-09-11T12:55:39.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.726+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f14bc159060 con 0x7f14bc1f7e70 2024-09-11T12:55:39.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14b3400640 1 -- 172.21.15.78:0/1240533146 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) ==== 97+0+0 (secure 0 0 0) 0x7f14b40622b0 con 0x7f14bc1f7e70 2024-09-11T12:55:39.936 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 msgr2=0x7f149c074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 0x7f149c074110 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0ac000 tx=0x7f14ac01a100 comp rx=0 tx=0).stop 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 msgr2=0x7f14bc1f8320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 0x7f14bc1f8320 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f14bc0748f0 tx=0x7f14b4001640 comp rx=0 tx=0).stop 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 shutdown_connections 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f14bc1f7e70 0x7f14bc1f8320 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f14bc0b0d00 0x7f14bc0b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f149c071c50 0x7f149c074110 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 --2- 172.21.15.78:0/1240533146 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f14a0003a60 0x7f14bc14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 >> 172.21.15.78:0/1240533146 conn(0x7f14bc06ca80 msgr2=0x7f14bc0ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 shutdown_connections 2024-09-11T12:55:39.940 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:39.934+0000 7f14c2000640 1 -- 172.21.15.78:0/1240533146 wait complete. 2024-09-11T12:55:39.959 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:55:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- 172.21.15.78:0/3206723661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 msgr2=0x7f90740ce8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 --2- 172.21.15.78:0/3206723661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90740ce8c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f906c009c80 tx=0x7f906c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- 172.21.15.78:0/3206723661 shutdown_connections 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 --2- 172.21.15.78:0/3206723661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90740ce8c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 --2- 172.21.15.78:0/3206723661 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907406fc50 0x7f9074070090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- 172.21.15.78:0/3206723661 >> 172.21.15.78:0/3206723661 conn(0x7f907406ca80 msgr2=0x7f90740b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- 172.21.15.78:0/3206723661 shutdown_connections 2024-09-11T12:55:40.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- 172.21.15.78:0/3206723661 wait complete. 2024-09-11T12:55:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 Processor -- start 2024-09-11T12:55:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.098+0000 7f907c200640 1 -- start start 2024-09-11T12:55:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 0x7f907415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907413d840 0x7f9074150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9074167ab0 con 0x7f90740ce450 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9074167930 con 0x7f907406fc50 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9074167c30 con 0x7f907413d840 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60082/0 (socket says 172.21.15.78:60082) 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 -- 172.21.15.78:0/1054267702 learned_addr learned my addr 172.21.15.78:0/1054267702 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907a200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907413d840 0x7f9074150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9079800640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 0x7f907415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 -- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907413d840 msgr2=0x7f9074150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907413d840 0x7f9074150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 -- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 msgr2=0x7f907415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 0x7f907415bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 -- 172.21.15.78:0/1054267702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9074073c20 con 0x7f90740ce450 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9079800640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 0x7f907415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9078e00640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f906000e940 tx=0x7f906000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f906000cd80 con 0x7f90740ce450 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9060005020 con 0x7f90740ce450 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9060005450 con 0x7f90740ce450 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9074068b40 con 0x7f90740ce450 2024-09-11T12:55:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9074151320 con 0x7f90740ce450 2024-09-11T12:55:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907415c5c0 con 0x7f90740ce450 2024-09-11T12:55:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.102+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f90600047b0 con 0x7f90740ce450 2024-09-11T12:55:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.106+0000 7f9071600640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 0x7f90540740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.106+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4857+0+0 (secure 0 0 0) 0x7f9060093010 con 0x7f90740ce450 2024-09-11T12:55:40.108 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.106+0000 7f9079800640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 0x7f90540740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.109 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.106+0000 7f9079800640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 0x7f90540740e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f907406cfb0 tx=0x7f9068009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.110+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f906005e810 con 0x7f90740ce450 2024-09-11T12:55:40.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.334+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f907406f880 con 0x7f90740ce450 2024-09-11T12:55:40.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.334+0000 7f9071600640 1 -- 172.21.15.78:0/1054267702 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) ==== 80+0+5304 (secure 0 0 0) 0x7f906005e110 con 0x7f90740ce450 2024-09-11T12:55:40.340 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:40.340 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:55:40.340 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.341 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:40.342 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:40.342 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:40.343 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:40.344 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.344 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.344 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:55:40.345 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:55:40.345 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:40.345 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:55:40.346 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.348 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:55:40.348 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:55:40.348 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.350 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:55:40.350 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:55:40.350 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.350 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.351 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.351 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.352 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.353 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:55:40.353 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:55:40.353 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.355 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:55:40.355 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:55:40.355 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.355 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.356 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.356 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.357 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.358 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:55:40.358 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:55:40.358 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.360 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:55:40.360 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:55:40.360 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.360 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:55:40.360 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:55:40.361 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:55:40.362 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.362 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.362 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:55:40.362 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:55:40.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.363 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.364 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:40.364 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:40.364 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:40.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.365 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:40.366 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:40.366 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:40.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:55:40.367 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:55:40.368 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:55:40.368 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.368 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:55:40.369 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.370 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.370 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:55:40.370 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:55:40.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.372 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:55:40.372 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:55:40.372 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:55:40.372 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:55:40.373 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:55:40.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 msgr2=0x7f90540740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 0x7f90540740e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f907406cfb0 tx=0x7f9068009040 comp rx=0 tx=0).stop 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 msgr2=0x7f90741f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f906000e940 tx=0x7f906000ee00 comp rx=0 tx=0).stop 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 shutdown_connections 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f907413d840 0x7f9074150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f90740ce450 0x7f90741f81a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9054071c20 0x7f90540740e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 --2- 172.21.15.78:0/1054267702 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f907406fc50 0x7f907415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 >> 172.21.15.78:0/1054267702 conn(0x7f907406ca80 msgr2=0x7f9074072ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 shutdown_connections 2024-09-11T12:55:40.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.338+0000 7f907c200640 1 -- 172.21.15.78:0/1054267702 wait complete. 2024-09-11T12:55:40.379 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 -- 172.21.15.78:0/8841837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454073ed0 msgr2=0x7ff45415db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 --2- 172.21.15.78:0/8841837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454073ed0 0x7ff45415db60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff440009840 tx=0x7ff44001c790 comp rx=0 tx=0).stop 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 -- 172.21.15.78:0/8841837 shutdown_connections 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 --2- 172.21.15.78:0/8841837 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff454071d60 0x7ff4540721f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 --2- 172.21.15.78:0/8841837 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454073ed0 0x7ff45415db60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 -- 172.21.15.78:0/8841837 >> 172.21.15.78:0/8841837 conn(0x7ff45406ca80 msgr2=0x7ff4540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 -- 172.21.15.78:0/8841837 shutdown_connections 2024-09-11T12:55:40.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.522+0000 7ff45b000640 1 -- 172.21.15.78:0/8841837 wait complete. 2024-09-11T12:55:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 Processor -- start 2024-09-11T12:55:40.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- start start 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454071d60 0x7ff454150300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff454073ed0 0x7ff454150840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540b51a0 con 0x7ff4540707f0 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff4540b5020 con 0x7ff454071d60 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff4540b5320 con 0x7ff454073ed0 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60098/0 (socket says 172.21.15.78:60098) 2024-09-11T12:55:40.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 -- 172.21.15.78:0/3353401027 learned_addr learned my addr 172.21.15.78:0/3353401027 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:40.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 -- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff454073ed0 msgr2=0x7ff454150840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff454073ed0 0x7ff454150840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 -- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454071d60 msgr2=0x7ff454150300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454071d60 0x7ff454150300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 -- 172.21.15.78:0/3353401027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45414e010 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff453e00640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff44400cb10 tx=0x7ff44400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff444016020 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff44400ce50 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45414e2f0 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff44400ae00 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.526+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff45414e8b0 con 0x7ff4540707f0 2024-09-11T12:55:40.532 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4540721f0 con 0x7ff4540707f0 2024-09-11T12:55:40.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7ff444019020 con 0x7ff4540707f0 2024-09-11T12:55:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff44be00640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 0x7ff4340740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4857+0+0 (secure 0 0 0) 0x7ff4440928c0 con 0x7ff4540707f0 2024-09-11T12:55:40.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff453400640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 0x7ff4340740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.530+0000 7ff453400640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 0x7ff4340740e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4400af000 tx=0x7ff440019590 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.534+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff444020020 con 0x7ff4540707f0 2024-09-11T12:55:40.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7ff45414f630 con 0x7ff4540707f0 2024-09-11T12:55:40.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff44be00640 1 -- 172.21.15.78:0/3353401027 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v125) ==== 84+0+65 (secure 0 0 0) 0x7ff44405e150 con 0x7ff4540707f0 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:40.709 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:55:40.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 msgr2=0x7ff4340740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 0x7ff4340740e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff4400af000 tx=0x7ff440019590 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 msgr2=0x7ff454145470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7ff44400cb10 tx=0x7ff44400cc00 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 shutdown_connections 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff454073ed0 0x7ff454150840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff434071c20 0x7ff4340740e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff454071d60 0x7ff454150300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 --2- 172.21.15.78:0/3353401027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff4540707f0 0x7ff454145470 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 >> 172.21.15.78:0/3353401027 conn(0x7ff45406ca80 msgr2=0x7ff45414fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 shutdown_connections 2024-09-11T12:55:40.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.706+0000 7ff45b000640 1 -- 172.21.15.78:0/3353401027 wait complete. 2024-09-11T12:55:40.730 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:55:40.730 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:55:40.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- 172.21.15.78:0/1453864026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 msgr2=0x7f092415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 --2- 172.21.15.78:0/1453864026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f092415d060 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0918009840 tx=0x7f091801c820 comp rx=0 tx=0).stop 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- 172.21.15.78:0/1453864026 shutdown_connections 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 --2- 172.21.15.78:0/1453864026 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f092415d060 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- 172.21.15.78:0/1453864026 >> 172.21.15.78:0/1453864026 conn(0x7f09240b3050 msgr2=0x7f09240b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- 172.21.15.78:0/1453864026 shutdown_connections 2024-09-11T12:55:40.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- 172.21.15.78:0/1453864026 wait complete. 2024-09-11T12:55:40.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 Processor -- start 2024-09-11T12:55:40.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- start start 2024-09-11T12:55:40.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 0x7f09240af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f09241429f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f092416a360 con 0x7f09240cf520 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f092416a1e0 con 0x7f0924152990 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f092416a4e0 con 0x7f09240b5c40 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f0928c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f0928c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60102/0 (socket says 172.21.15.78:60102) 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f0928c00640 1 -- 172.21.15.78:0/2705542136 learned_addr learned my addr 172.21.15.78:0/2705542136 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f0929600640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 0x7f09240af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.874+0000 7f092a000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f09241429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 -- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 msgr2=0x7f09240af6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 0x7f09240af6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 -- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 msgr2=0x7f09241429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f09241429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:40.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 -- 172.21.15.78:0/2705542136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09241483c0 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0929600640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 0x7f09240af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0928c00640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f091000c490 tx=0x7f091000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100052d0 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0910005490 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0910012870 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09241486a0 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0924071fb0 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0924146b50 con 0x7f09240cf520 2024-09-11T12:55:40.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f0910012a10 con 0x7f09240cf520 2024-09-11T12:55:40.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 0x7f0904074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:40.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.878+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(125..125 src has 1..125) ==== 4857+0+0 (secure 0 0 0) 0x7f0910092010 con 0x7f09240cf520 2024-09-11T12:55:40.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.882+0000 7f0929600640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 0x7f0904074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:40.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.882+0000 7f0929600640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 0x7f0904074110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f09180af000 tx=0x7f091801b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:40.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:40.882+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f091005d2d0 con 0x7f09240cf520 2024-09-11T12:55:41.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.054+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f09240b52d0 con 0x7f09240cf520 2024-09-11T12:55:41.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.942+0000 7f0921600640 1 -- 172.21.15.78:0/2705542136 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v126) ==== 138+0+0 (secure 0 0 0) 0x7f0910061820 con 0x7f09240cf520 2024-09-11T12:55:41.945 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 msgr2=0x7f0904074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 0x7f0904074110 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f09180af000 tx=0x7f091801b060 comp rx=0 tx=0).stop 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 msgr2=0x7f09241424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f091000c490 tx=0x7f091000c950 comp rx=0 tx=0).stop 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 shutdown_connections 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0904071c50 0x7f0904074110 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:41.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0924152990 0x7f09241429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09240cf520 0x7f09241424b0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 --2- 172.21.15.78:0/2705542136 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09240b5c40 0x7f09240af6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 >> 172.21.15.78:0/2705542136 conn(0x7f09240b3050 msgr2=0x7f0924159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 shutdown_connections 2024-09-11T12:55:41.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:41.946+0000 7f092c000640 1 -- 172.21.15.78:0/2705542136 wait complete. 2024-09-11T12:55:41.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:55:42.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- 172.21.15.78:0/3234716224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f712806fd50 msgr2=0x7f712815b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:42.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 --2- 172.21.15.78:0/3234716224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f712806fd50 0x7f712815b8d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f712000a950 tx=0x7f712001dab0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- 172.21.15.78:0/3234716224 shutdown_connections 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 --2- 172.21.15.78:0/3234716224 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f712806fd50 0x7f712815b8d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 --2- 172.21.15.78:0/3234716224 >> v2:172.21.15.157:3300/0 conn(0x7f7128143810 0x7f712806f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- 172.21.15.78:0/3234716224 >> 172.21.15.78:0/3234716224 conn(0x7f71280b1f30 msgr2=0x7f71280b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- 172.21.15.78:0/3234716224 shutdown_connections 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- 172.21.15.78:0/3234716224 wait complete. 2024-09-11T12:55:42.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 Processor -- start 2024-09-11T12:55:42.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.126+0000 7f712ec00640 1 -- start start 2024-09-11T12:55:42.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7128143430 0x7f7128073640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7128143810 0x7f7128151630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f712815c4b0 con 0x7f712806fd50 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f712815c330 con 0x7f7128143430 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f712815c630 con 0x7f7128143810 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60116/0 (socket says 172.21.15.78:60116) 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 -- 172.21.15.78:0/30198412 learned_addr learned my addr 172.21.15.78:0/30198412 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ce00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7128143430 0x7f7128073640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712e200640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7128143810 0x7f7128151630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 -- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7128143810 msgr2=0x7f7128151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7128143810 0x7f7128151630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 -- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7128143430 msgr2=0x7f7128073640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:42.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7128143430 0x7f7128073640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 -- 172.21.15.78:0/30198412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7128141850 con 0x7f712806fd50 2024-09-11T12:55:42.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712d800640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f711c00cbd0 tx=0x7f711c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:42.134 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c016020 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f711c00aa50 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7128162380 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c012420 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7128162940 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.130+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f712815cb50 con 0x7f712806fd50 2024-09-11T12:55:42.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.134+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f711c019020 con 0x7f712806fd50 2024-09-11T12:55:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.134+0000 7f7125600640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 0x7f71080740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.134+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 5227+0+0 (secure 0 0 0) 0x7f711c092ce0 con 0x7f712806fd50 2024-09-11T12:55:42.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.134+0000 7f712ce00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 0x7f71080740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:42.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.134+0000 7f712ce00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 0x7f71080740e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7128151180 tx=0x7f71140091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:42.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.138+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f711c020020 con 0x7f712806fd50 2024-09-11T12:55:42.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.306+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f71280cc7a0 con 0x7f712806fd50 2024-09-11T12:55:42.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:55:42.621 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:55:42.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.950+0000 7f7125600640 1 -- 172.21.15.78:0/30198412 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v127) ==== 170+0+0 (secure 0 0 0) 0x7f7128141850 con 0x7f712806fd50 2024-09-11T12:55:42.979 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 msgr2=0x7f71080740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 0x7f71080740e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7128151180 tx=0x7f71140091e0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 msgr2=0x7f71281409e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f711c00cbd0 tx=0x7f711c00ccc0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 shutdown_connections 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7128143810 0x7f7128151630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7108071c20 0x7f71080740e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7128143430 0x7f7128073640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 --2- 172.21.15.78:0/30198412 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f712806fd50 0x7f71281409e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:42.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 >> 172.21.15.78:0/30198412 conn(0x7f71280b1f30 msgr2=0x7f712806f050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:42.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 shutdown_connections 2024-09-11T12:55:42.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:42.954+0000 7f712ec00640 1 -- 172.21.15.78:0/30198412 wait complete. 2024-09-11T12:55:42.996 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:55:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- 172.21.15.78:0/1310777012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140460 msgr2=0x7f03840b37a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 --2- 172.21.15.78:0/1310777012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140460 0x7f03840b37a0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f038415a510 tx=0x7f037001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- 172.21.15.78:0/1310777012 shutdown_connections 2024-09-11T12:55:43.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 --2- 172.21.15.78:0/1310777012 >> v2:172.21.15.157:3300/0 conn(0x7f038415b100 0x7f03840731a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 --2- 172.21.15.78:0/1310777012 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140460 0x7f03840b37a0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- 172.21.15.78:0/1310777012 >> 172.21.15.78:0/1310777012 conn(0x7f0384075c10 msgr2=0x7f03840b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- 172.21.15.78:0/1310777012 shutdown_connections 2024-09-11T12:55:43.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- 172.21.15.78:0/1310777012 wait complete. 2024-09-11T12:55:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 Processor -- start 2024-09-11T12:55:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.138+0000 7f0389c00640 1 -- start start 2024-09-11T12:55:43.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:43.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0384140460 0x7f038414f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:43.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 0x7f038414ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:43.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f038415c0e0 con 0x7f0384140080 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f038415bf60 con 0x7f038415b100 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f038415c260 con 0x7f0384140460 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60132/0 (socket says 172.21.15.78:60132) 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 -- 172.21.15.78:0/4127570333 learned_addr learned my addr 172.21.15.78:0/4127570333 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:43.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389200640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 0x7f038414ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383400640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0384140460 0x7f038414f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 -- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0384140460 msgr2=0x7f038414f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0384140460 0x7f038414f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 -- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 msgr2=0x7f038414ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 0x7f038414ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.150 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 -- 172.21.15.78:0/4127570333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f038414e590 con 0x7f0384140080 2024-09-11T12:55:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389200640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 0x7f038414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0383e00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f037400c870 tx=0x7f037400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0374003b80 con 0x7f0384140080 2024-09-11T12:55:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0374003d20 con 0x7f0384140080 2024-09-11T12:55:43.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f037400aab0 con 0x7f0384140080 2024-09-11T12:55:43.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03841504b0 con 0x7f0384140080 2024-09-11T12:55:43.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f038414edf0 con 0x7f0384140080 2024-09-11T12:55:43.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f038414e940 con 0x7f0384140080 2024-09-11T12:55:43.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.142+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f037400ac50 con 0x7f0384140080 2024-09-11T12:55:43.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.146+0000 7f037be00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 0x7f03640740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:43.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.146+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 5609+0+0 (secure 0 0 0) 0x7f03740931a0 con 0x7f0384140080 2024-09-11T12:55:43.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.146+0000 7f0383400640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 0x7f03640740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:43.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.146+0000 7f0383400640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 0x7f03640740e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f037001b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:43.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.150+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f037405e840 con 0x7f0384140080 2024-09-11T12:55:43.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.346+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f03840731a0 con 0x7f0384140080 2024-09-11T12:55:43.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.966+0000 7f037be00640 1 -- 172.21.15.78:0/4127570333 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v87) ==== 174+0+0 (secure 0 0 0) 0x7f037405e140 con 0x7f0384140080 2024-09-11T12:55:43.974 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 20 and data pool 21 2024-09-11T12:55:43.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 msgr2=0x7f03640740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:43.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 0x7f03640740e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f03700af000 tx=0x7f037001b100 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 msgr2=0x7f0384150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f037400c870 tx=0x7f037400cd30 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 shutdown_connections 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0364071c20 0x7f03640740e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f038415b100 0x7f038414ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0384140460 0x7f038414f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 --2- 172.21.15.78:0/4127570333 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0384140080 0x7f0384150d70 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:43.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.974+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 >> 172.21.15.78:0/4127570333 conn(0x7f0384075c10 msgr2=0x7f03840a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:43.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.978+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 shutdown_connections 2024-09-11T12:55:43.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:43.982+0000 7f0389c00640 1 -- 172.21.15.78:0/4127570333 wait complete. 2024-09-11T12:55:44.020 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 -- 172.21.15.78:0/629741332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c15c5b0 msgr2=0x7f536c15ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 --2- 172.21.15.78:0/629741332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c15c5b0 0x7f536c15ca00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f536c06fc60 tx=0x7f535401c890 comp rx=0 tx=0).stop 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 -- 172.21.15.78:0/629741332 shutdown_connections 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 --2- 172.21.15.78:0/629741332 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c15c5b0 0x7f536c15ca00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 --2- 172.21.15.78:0/629741332 >> v2:172.21.15.157:3300/0 conn(0x7f536c15c1d0 0x7f536c06e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 -- 172.21.15.78:0/629741332 >> 172.21.15.78:0/629741332 conn(0x7f536c0ceab0 msgr2=0x7f536c0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 -- 172.21.15.78:0/629741332 shutdown_connections 2024-09-11T12:55:44.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.158+0000 7f5373800640 1 -- 172.21.15.78:0/629741332 wait complete. 2024-09-11T12:55:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 Processor -- start 2024-09-11T12:55:44.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- start start 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f536c15c1d0 0x7f536c06df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536c15c5b0 0x7f536c1fb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0aa480 con 0x7f536c0d0ec0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f536c0aa300 con 0x7f536c15c5b0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f536c0aa600 con 0x7f536c15c1d0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55396/0 (socket says 172.21.15.78:55396) 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 -- 172.21.15.78:0/3814257675 learned_addr learned my addr 172.21.15.78:0/3814257675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5363e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f536c15c1d0 0x7f536c06df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5371800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536c15c5b0 0x7f536c1fb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 -- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f536c15c1d0 msgr2=0x7f536c06df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f536c15c1d0 0x7f536c06df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 -- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536c15c5b0 msgr2=0x7f536c1fb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536c15c5b0 0x7f536c1fb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 -- 172.21.15.78:0/3814257675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c06e480 con 0x7f536c0d0ec0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5370e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e940 tx=0x7f535c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c00cd80 con 0x7f536c0d0ec0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f535c005020 con 0x7f536c0d0ec0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c1fbfa0 con 0x7f536c0d0ec0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c005450 con 0x7f536c0d0ec0 2024-09-11T12:55:44.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.162+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c1f7f00 con 0x7f536c0d0ec0 2024-09-11T12:55:44.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536c0aaad0 con 0x7f536c0d0ec0 2024-09-11T12:55:44.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f535c0047b0 con 0x7f536c0d0ec0 2024-09-11T12:55:44.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5369600640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 0x7f53480740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7f535c093210 con 0x7f536c0d0ec0 2024-09-11T12:55:44.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5363e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 0x7f53480740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.166+0000 7f5363e00640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 0x7f53480740e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53540af000 tx=0x7f53540a7040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.170+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f535c05e0b0 con 0x7f536c0d0ec0 2024-09-11T12:55:44.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.338+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f536c06ff00 con 0x7f536c0d0ec0 2024-09-11T12:55:44.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5369600640 1 -- 172.21.15.78:0/3814257675 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16752 (secure 0 0 0) 0x7f535c062600 con 0x7f536c0d0ec0 2024-09-11T12:55:44.344 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:44.344 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":128,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:43.353362+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:41.069903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:42.319816+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 msgr2=0x7f53480740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 0x7f53480740e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f53540af000 tx=0x7f53540a7040 comp rx=0 tx=0).stop 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 msgr2=0x7f536c1f89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f535c00e940 tx=0x7f535c00ee00 comp rx=0 tx=0).stop 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 shutdown_connections 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5348071c20 0x7f53480740e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536c15c5b0 0x7f536c1fb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.346 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f536c15c1d0 0x7f536c06df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 --2- 172.21.15.78:0/3814257675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f536c0d0ec0 0x7f536c1f89e0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 >> 172.21.15.78:0/3814257675 conn(0x7f536c0ceab0 msgr2=0x7f536c15e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 shutdown_connections 2024-09-11T12:55:44.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.342+0000 7f5373800640 1 -- 172.21.15.78:0/3814257675 wait complete. 2024-09-11T12:55:44.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:44.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 -- 172.21.15.78:0/1537417114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd3815df20 msgr2=0x7fbd38073190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 --2- 172.21.15.78:0/1537417114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd3815df20 0x7fbd38073190 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009840 tx=0x7fbd2401c820 comp rx=0 tx=0).stop 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 -- 172.21.15.78:0/1537417114 shutdown_connections 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 --2- 172.21.15.78:0/1537417114 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd380af170 0x7fbd380736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 --2- 172.21.15.78:0/1537417114 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd3815df20 0x7fbd38073190 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 -- 172.21.15.78:0/1537417114 >> 172.21.15.78:0/1537417114 conn(0x7fbd3806d870 msgr2=0x7fbd3806dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 -- 172.21.15.78:0/1537417114 shutdown_connections 2024-09-11T12:55:44.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.502+0000 7fbd3e400640 1 -- 172.21.15.78:0/1537417114 wait complete. 2024-09-11T12:55:44.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 Processor -- start 2024-09-11T12:55:44.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- start start 2024-09-11T12:55:44.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd380af170 0x7fbd3815c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd3815df20 0x7fbd3813de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3816aa30 con 0x7fbd381466a0 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd3816a8b0 con 0x7fbd3815df20 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd3816abb0 con 0x7fbd380af170 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55418/0 (socket says 172.21.15.78:55418) 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 -- 172.21.15.78:0/3136334562 learned_addr learned my addr 172.21.15.78:0/3136334562 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 -- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd380af170 msgr2=0x7fbd3815c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd380af170 0x7fbd3815c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 -- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd3815df20 msgr2=0x7fbd3813de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd3815df20 0x7fbd3813de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 -- 172.21.15.78:0/3136334562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd3815cef0 con 0x7fbd381466a0 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd37e00640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009790 tx=0x7fbd240a5670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240bb070 con 0x7fbd381466a0 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd24004470 con 0x7fbd381466a0 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd240b2040 con 0x7fbd381466a0 2024-09-11T12:55:44.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd3815d1c0 con 0x7fbd381466a0 2024-09-11T12:55:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd3815d780 con 0x7fbd381466a0 2024-09-11T12:55:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.506+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd38146a80 con 0x7fbd381466a0 2024-09-11T12:55:44.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.510+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fbd240be070 con 0x7fbd381466a0 2024-09-11T12:55:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.510+0000 7fbd34c00640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 0x7fbd180740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.510+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fbd2412c120 con 0x7fbd381466a0 2024-09-11T12:55:44.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.510+0000 7fbd3d000640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 0x7fbd180740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.510+0000 7fbd3d000640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 0x7fbd180740e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd280015d0 tx=0x7fbd28007a40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.514+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbd24016410 con 0x7fbd381466a0 2024-09-11T12:55:44.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.682+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbd3815fa00 con 0x7fbd381466a0 2024-09-11T12:55:44.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.682+0000 7fbd34c00640 1 -- 172.21.15.78:0/3136334562 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) ==== 74+0+16752 (secure 0 0 0) 0x7fbd240f76f0 con 0x7fbd381466a0 2024-09-11T12:55:44.687 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:44.687 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":128,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:43.353362+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:41.069903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:42.319816+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 msgr2=0x7fbd180740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 0x7fbd180740e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbd280015d0 tx=0x7fbd28007a40 comp rx=0 tx=0).stop 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 msgr2=0x7fbd380b1770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009790 tx=0x7fbd240a5670 comp rx=0 tx=0).stop 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 shutdown_connections 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd18071c20 0x7fbd180740e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd3815df20 0x7fbd3813de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd381466a0 0x7fbd380b1770 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 --2- 172.21.15.78:0/3136334562 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd380af170 0x7fbd3815c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 >> 172.21.15.78:0/3136334562 conn(0x7fbd3806d870 msgr2=0x7fbd380733b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:44.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 shutdown_connections 2024-09-11T12:55:44.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.686+0000 7fbd3e400640 1 -- 172.21.15.78:0/3136334562 wait complete. 2024-09-11T12:55:44.708 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:55:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- 172.21.15.78:0/3519172735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc5d0 msgr2=0x7fe48c0b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- 172.21.15.78:0/3519172735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc5d0 0x7fe48c0b2de0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0b3e90 tx=0x7fe47801c820 comp rx=0 tx=0).stop 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- 172.21.15.78:0/3519172735 shutdown_connections 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- 172.21.15.78:0/3519172735 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe48c15ca30 0x7fe48c1416e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- 172.21.15.78:0/3519172735 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc5d0 0x7fe48c0b2de0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- 172.21.15.78:0/3519172735 >> 172.21.15.78:0/3519172735 conn(0x7fe48c071dd0 msgr2=0x7fe48c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- 172.21.15.78:0/3519172735 shutdown_connections 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- 172.21.15.78:0/3519172735 wait complete. 2024-09-11T12:55:44.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 Processor -- start 2024-09-11T12:55:44.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- start start 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe48c0cc5d0 0x7fe48c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe48c15ca30 0x7fe48c1f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c168e40 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe48c168cc0 con 0x7fe48c15ca30 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.850+0000 7fe492800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe48c168fc0 con 0x7fe48c0cc5d0 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55450/0 (socket says 172.21.15.78:55450) 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 -- 172.21.15.78:0/4199557730 learned_addr learned my addr 172.21.15.78:0/4199557730 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 -- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe48c0cc5d0 msgr2=0x7fe48c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe48c0cc5d0 0x7fe48c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 -- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe48c15ca30 msgr2=0x7fe48c1f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe48c15ca30 0x7fe48c1f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 -- 172.21.15.78:0/4199557730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe48c13cb70 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe48b400640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c7f0 tx=0x7fe47c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c017070 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe47c005070 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe48c1f6770 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0054c0 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe48c1366b0 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe48c0b2de0 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.854+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fe47c01a070 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.858+0000 7fe483400640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 0x7fe46c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.858+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 5709+0+0 (secure 0 0 0) 0x7fe47c091fb0 con 0x7fe48c0cc1f0 2024-09-11T12:55:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.858+0000 7fe48aa00640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 0x7fe46c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:44.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.858+0000 7fe48aa00640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 0x7fe46c0740e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4780af000 tx=0x7fe4780195c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:44.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:44.862+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe48c1f6770 con 0x7fe48c0cc1f0 2024-09-11T12:55:45.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.066+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe48c0cece0 con 0x7fe48c0cc1f0 2024-09-11T12:55:45.941 INFO:tasks.ceph.mds.a.smithi078.stderr:2024-09-11T12:55:45.938+0000 7ff2b5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:55:45.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.974+0000 7fe483400640 1 -- 172.21.15.78:0/4199557730 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v90) ==== 118+0+0 (secure 0 0 0) 0x7fe47c05d600 con 0x7fe48c0cc1f0 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 msgr2=0x7fe46c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 0x7fe46c0740e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe4780af000 tx=0x7fe4780195c0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 msgr2=0x7fe48c13c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fe47c00c7f0 tx=0x7fe47c00ccb0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 shutdown_connections 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe46c071c20 0x7fe46c0740e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe48c15ca30 0x7fe48c1f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe48c0cc5d0 0x7fe48c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 --2- 172.21.15.78:0/4199557730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe48c0cc1f0 0x7fe48c13c270 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 >> 172.21.15.78:0/4199557730 conn(0x7fe48c071dd0 msgr2=0x7fe48c155bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 shutdown_connections 2024-09-11T12:55:45.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:45.978+0000 7fe492800640 1 -- 172.21.15.78:0/4199557730 wait complete. 2024-09-11T12:55:46.002 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:55:46.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 -- 172.21.15.78:0/1770495265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40003a10 msgr2=0x7f8e40003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 --2- 172.21.15.78:0/1770495265 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e40003a10 0x7f8e40003e30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00c2e0 tx=0x7f8e4c01cf60 comp rx=0 tx=0).stop 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e5e400640 1 -- 172.21.15.78:0/1770495265 reap_dead start 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 -- 172.21.15.78:0/1770495265 shutdown_connections 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 -- 172.21.15.78:0/1770495265 >> 172.21.15.78:0/1770495265 conn(0x7f8e58071dd0 msgr2=0x7f8e580721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:46.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 -- 172.21.15.78:0/1770495265 shutdown_connections 2024-09-11T12:55:46.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.146+0000 7f8e60400640 1 -- 172.21.15.78:0/1770495265 wait complete. 2024-09-11T12:55:46.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 Processor -- start 2024-09-11T12:55:46.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- start start 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40003a10 0x7f8e581f7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e5815d7e0 0x7f8e580b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8e58165740 con 0x7f8e58144ee0 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8e581655c0 con 0x7f8e40003a10 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8e581658c0 con 0x7f8e5815d7e0 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55470/0 (socket says 172.21.15.78:55470) 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 -- 172.21.15.78:0/3433939137 learned_addr learned my addr 172.21.15.78:0/3433939137 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5da00640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40003a10 0x7f8e581f7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:46.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5e400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e5815d7e0 0x7f8e580b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 -- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e5815d7e0 msgr2=0x7f8e580b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e5815d7e0 0x7f8e580b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 -- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40003a10 msgr2=0x7f8e581f7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40003a10 0x7f8e581f7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 -- 172.21.15.78:0/3433939137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5815ce00 con 0x7f8e58144ee0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e5d000640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003f60 tx=0x7f8e44003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44005220 con 0x7f8e58144ee0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e440053c0 con 0x7f8e58144ee0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e440047b0 con 0x7f8e58144ee0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5815bfa0 con 0x7f8e58144ee0 2024-09-11T12:55:46.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.150+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e5815c560 con 0x7f8e58144ee0 2024-09-11T12:55:46.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e581446a0 con 0x7f8e58144ee0 2024-09-11T12:55:46.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f8e44004950 con 0x7f8e58144ee0 2024-09-11T12:55:46.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e55600640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 0x7f8e38074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:46.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f8e440930f0 con 0x7f8e58144ee0 2024-09-11T12:55:46.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e5da00640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 0x7f8e38074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:46.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.154+0000 7f8e5da00640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 0x7f8e38074110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ac000 tx=0x7f8e4c019da0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:46.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.158+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8e4405e0a0 con 0x7f8e58144ee0 2024-09-11T12:55:46.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.362+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f8e581f7f80 con 0x7f8e58144ee0 2024-09-11T12:55:46.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.982+0000 7f8e55600640 1 -- 172.21.15.78:0/3433939137 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v91) ==== 122+0+0 (secure 0 0 0) 0x7f8e440625f0 con 0x7f8e58144ee0 2024-09-11T12:55:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 msgr2=0x7f8e38074110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 0x7f8e38074110 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ac000 tx=0x7f8e4c019da0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 msgr2=0x7f8e5815d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f8e44003f60 tx=0x7f8e44003f90 comp rx=0 tx=0).stop 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 shutdown_connections 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8e5815d7e0 0x7f8e580b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8e58144ee0 0x7f8e5815d2a0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8e38071c50 0x7f8e38074110 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 --2- 172.21.15.78:0/3433939137 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8e40003a10 0x7f8e581f7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 >> 172.21.15.78:0/3433939137 conn(0x7f8e58071dd0 msgr2=0x7f8e580af990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 shutdown_connections 2024-09-11T12:55:46.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:46.986+0000 7f8e60400640 1 -- 172.21.15.78:0/3433939137 wait complete. 2024-09-11T12:55:47.010 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:55:47.010 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:47.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- 172.21.15.78:0/3116770814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c15cad0 msgr2=0x7f6e3c15cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 --2- 172.21.15.78:0/3116770814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c15cf60 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c0b58e0 tx=0x7f6e2801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- 172.21.15.78:0/3116770814 shutdown_connections 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 --2- 172.21.15.78:0/3116770814 >> v2:172.21.15.157:3300/0 conn(0x7f6e3c0b4520 0x7f6e3c0b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 --2- 172.21.15.78:0/3116770814 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c15cf60 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- 172.21.15.78:0/3116770814 >> 172.21.15.78:0/3116770814 conn(0x7f6e3c06ca80 msgr2=0x7f6e3c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- 172.21.15.78:0/3116770814 shutdown_connections 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- 172.21.15.78:0/3116770814 wait complete. 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 Processor -- start 2024-09-11T12:55:47.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.150+0000 7f6e44400640 1 -- start start 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e3c0b4520 0x7f6e3c15b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c1407e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd250 con 0x7f6e3c0aed20 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd0d0 con 0x7f6e3c15cad0 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6e3c0cd3d0 con 0x7f6e3c0b4520 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55488/0 (socket says 172.21.15.78:55488) 2024-09-11T12:55:47.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 -- 172.21.15.78:0/1308537666 learned_addr learned my addr 172.21.15.78:0/1308537666 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e42400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c1407e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41000640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e3c0b4520 0x7f6e3c15b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 -- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e3c0b4520 msgr2=0x7f6e3c15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e3c0b4520 0x7f6e3c15b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 -- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 msgr2=0x7f6e3c1407e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c1407e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 -- 172.21.15.78:0/1308537666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3c140d20 con 0x7f6e3c0aed20 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e42400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c1407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e41a00640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c800 tx=0x7f6e3000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e30017070 con 0x7f6e3c0aed20 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e30005070 con 0x7f6e3c0aed20 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e300054c0 con 0x7f6e3c0aed20 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e3c0704f0 con 0x7f6e3c0aed20 2024-09-11T12:55:47.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e3c1567a0 con 0x7f6e3c0aed20 2024-09-11T12:55:47.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e3c0b4930 con 0x7f6e3c0aed20 2024-09-11T12:55:47.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.154+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f6e3001a070 con 0x7f6e3c0aed20 2024-09-11T12:55:47.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.158+0000 7f6e39600640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 0x7f6e1c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.158+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f6e30092000 con 0x7f6e3c0aed20 2024-09-11T12:55:47.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.158+0000 7f6e41000640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 0x7f6e1c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.158+0000 7f6e41000640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 0x7f6e1c0740e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e280af000 tx=0x7f6e2801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.162+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6e3c0b4e70 con 0x7f6e3c0aed20 2024-09-11T12:55:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.362+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e3c0709e0 con 0x7f6e3c0aed20 2024-09-11T12:55:47.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.362+0000 7f6e39600640 1 -- 172.21.15.78:0/1308537666 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5227 (secure 0 0 0) 0x7f6e3005d5c0 con 0x7f6e3c0aed20 2024-09-11T12:55:47.366 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:47.366 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"btime":"2024-09-11T12:55:46:980902+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:55:47.366 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2024-09-11T12:55:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 msgr2=0x7f6e1c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 0x7f6e1c0740e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6e280af000 tx=0x7f6e2801b100 comp rx=0 tx=0).stop 2024-09-11T12:55:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 msgr2=0x7f6e3c06fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f6e3000c800 tx=0x7f6e3000ccc0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.368 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 shutdown_connections 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6e1c071c20 0x7f6e1c0740e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6e3c15cad0 0x7f6e3c1407e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6e3c0b4520 0x7f6e3c15b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 --2- 172.21.15.78:0/1308537666 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6e3c0aed20 0x7f6e3c06fec0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 >> 172.21.15.78:0/1308537666 conn(0x7f6e3c06ca80 msgr2=0x7f6e3c0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 shutdown_connections 2024-09-11T12:55:47.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.366+0000 7f6e44400640 1 -- 172.21.15.78:0/1308537666 wait complete. 2024-09-11T12:55:47.388 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- 172.21.15.78:0/2013139588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a81585f0 msgr2=0x7fd5a815b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 --2- 172.21.15.78:0/2013139588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b950 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fd598009900 tx=0x7fd59801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- 172.21.15.78:0/2013139588 shutdown_connections 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 --2- 172.21.15.78:0/2013139588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 0x7fd5a806c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 --2- 172.21.15.78:0/2013139588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b950 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- 172.21.15.78:0/2013139588 >> 172.21.15.78:0/2013139588 conn(0x7fd5a80ccd10 msgr2=0x7fd5a80cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- 172.21.15.78:0/2013139588 shutdown_connections 2024-09-11T12:55:47.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- 172.21.15.78:0/2013139588 wait complete. 2024-09-11T12:55:47.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 Processor -- start 2024-09-11T12:55:47.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.526+0000 7fd5ae200640 1 -- start start 2024-09-11T12:55:47.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 0x7fd5a8152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a81679c0 con 0x7fd5a806cb90 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8167840 con 0x7fd5a81585f0 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a8167b40 con 0x7fd5a8073080 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55502/0 (socket says 172.21.15.78:55502) 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 -- 172.21.15.78:0/1080411498 learned_addr learned my addr 172.21.15.78:0/1080411498 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7e00640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd59fe00640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 0x7fd5a8152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 -- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 msgr2=0x7fd5a8152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 0x7fd5a8152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 -- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5a81585f0 msgr2=0x7fd5a815b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 -- 172.21.15.78:0/1080411498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a815dd30 con 0x7fd5a806cb90 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5a7400640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd598009850 tx=0x7fd5980a66b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5980bd070 con 0x7fd5a806cb90 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5980043b0 con 0x7fd5a806cb90 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a81fafc0 con 0x7fd5a806cb90 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd598005290 con 0x7fd5a806cb90 2024-09-11T12:55:47.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a81fb580 con 0x7fd5a806cb90 2024-09-11T12:55:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.530+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5a806c650 con 0x7fd5a806cb90 2024-09-11T12:55:47.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.534+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fd5980c0070 con 0x7fd5a806cb90 2024-09-11T12:55:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.534+0000 7fd59f400640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 0x7fd5840740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.534+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fd59812c640 con 0x7fd5a806cb90 2024-09-11T12:55:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.534+0000 7fd59fe00640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 0x7fd5840740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.534+0000 7fd59fe00640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 0x7fd5840740e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5a814ed30 tx=0x7fd59000b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.538+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd5980164d0 con 0x7fd5a806cb90 2024-09-11T12:55:47.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.702+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5a8072b70 con 0x7fd5a806cb90 2024-09-11T12:55:47.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd59f400640 1 -- 172.21.15.78:0/1080411498 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16739 (secure 0 0 0) 0x7fd5980f7530 con 0x7fd5a806cb90 2024-09-11T12:55:47.709 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:47.709 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:44.968878+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:41.069903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:42.319816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 msgr2=0x7fd5840740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 0x7fd5840740e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fd5a814ed30 tx=0x7fd59000b390 comp rx=0 tx=0).stop 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 msgr2=0x7fd5a815d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fd598009850 tx=0x7fd5980a66b0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 shutdown_connections 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd584071c20 0x7fd5840740e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd5a81585f0 0x7fd5a815b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5a8073080 0x7fd5a8152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 --2- 172.21.15.78:0/1080411498 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5a806cb90 0x7fd5a815d7f0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 >> 172.21.15.78:0/1080411498 conn(0x7fd5a80ccd10 msgr2=0x7fd5a8071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:47.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 shutdown_connections 2024-09-11T12:55:47.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.706+0000 7fd5ae200640 1 -- 172.21.15.78:0/1080411498 wait complete. 2024-09-11T12:55:47.731 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:55:47.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:55:47.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- 172.21.15.78:0/1236875203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 msgr2=0x7f60c8004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 --2- 172.21.15.78:0/1236875203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60c8004020 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f60d4001860 tx=0x7f60d401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e5e00640 1 -- 172.21.15.78:0/1236875203 reap_dead start 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- 172.21.15.78:0/1236875203 shutdown_connections 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- 172.21.15.78:0/1236875203 >> 172.21.15.78:0/1236875203 conn(0x7f60e006ca80 msgr2=0x7f60e00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- 172.21.15.78:0/1236875203 shutdown_connections 2024-09-11T12:55:47.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- 172.21.15.78:0/1236875203 wait complete. 2024-09-11T12:55:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 Processor -- start 2024-09-11T12:55:47.875 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.870+0000 7f60e7e00640 1 -- start start 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 0x7f60e0151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60e0153760 0x7f60e014c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0151190 con 0x7f60c8003c20 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60e0151010 con 0x7f60e01fa960 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60e0151310 con 0x7f60e0153760 2024-09-11T12:55:47.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55526/0 (socket says 172.21.15.78:55526) 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 -- 172.21.15.78:0/3697556481 learned_addr learned my addr 172.21.15.78:0/3697556481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60e0153760 0x7f60e014c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e4a00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 0x7f60e0151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 -- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60e0153760 msgr2=0x7f60e014c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60e0153760 0x7f60e014c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 -- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 msgr2=0x7f60e0151d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 0x7f60e0151d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 -- 172.21.15.78:0/3697556481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e01f7c70 con 0x7f60c8003c20 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e4a00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 0x7f60e0151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e5400640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000c10 tx=0x7f60d4003c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d4001480 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d40b4030 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d40068d0 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e01f7f50 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e0152f60 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e01518a0 con 0x7f60c8003c20 2024-09-11T12:55:47.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.874+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f60d40a5b60 con 0x7f60c8003c20 2024-09-11T12:55:47.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.878+0000 7f60dd600640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 0x7f60c00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:47.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.878+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f60d412c6d0 con 0x7f60c8003c20 2024-09-11T12:55:47.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.878+0000 7f60e4a00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 0x7f60c00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:47.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.878+0000 7f60e4a00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 0x7f60c00740e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60cc001ec0 tx=0x7f60cc0011d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:47.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:47.882+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f60d4016690 con 0x7f60c8003c20 2024-09-11T12:55:48.049 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.046+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60e00695e0 con 0x7f60c0071c20 2024-09-11T12:55:48.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.050+0000 7f60dd600640 1 -- 172.21.15.78:0/3697556481 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7f60e00695e0 con 0x7f60c0071c20 2024-09-11T12:55:48.055 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:48.057 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 msgr2=0x7f60c00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 0x7f60c00740e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f60cc001ec0 tx=0x7f60cc0011d0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 msgr2=0x7f60e01f7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f60d4000c10 tx=0x7f60d4003c40 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 shutdown_connections 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60e0153760 0x7f60e014c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60c0071c20 0x7f60c00740e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60e01fa960 0x7f60e0151d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 --2- 172.21.15.78:0/3697556481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60c8003c20 0x7f60e01f7370 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 >> 172.21.15.78:0/3697556481 conn(0x7f60e006ca80 msgr2=0x7f60e014ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:48.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 shutdown_connections 2024-09-11T12:55:48.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.054+0000 7f60e7e00640 1 -- 172.21.15.78:0/3697556481 wait complete. 2024-09-11T12:55:48.077 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-11T12:55:46.495934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267420,"kb_used_data":13740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707300,"statfs":{"total":773094113280,"available":772820275200,"internally_reserved":0,"allocated":14069760,"data_stored":6811408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548023"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976813+0000","last_change":"2024-09-11T12:55:43.977565+0000","last_active":"2024-09-11T12:55:44.976813+0000","last_peered":"2024-09-11T12:55:44.976813+0000","last_clean":"2024-09-11T12:55:44.976813+0000","last_became_active":"2024-09-11T12:55:43.977269+0000","last_became_peered":"2024-09-11T12:55:43.977269+0000","last_unstale":"2024-09-11T12:55:44.976813+0000","last_undegraded":"2024-09-11T12:55:44.976813+0000","last_fullsized":"2024-09-11T12:55:44.976813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223561+0000","last_change":"2024-09-11T12:55:43.985671+0000","last_active":"2024-09-11T12:55:45.223561+0000","last_peered":"2024-09-11T12:55:45.223561+0000","last_clean":"2024-09-11T12:55:45.223561+0000","last_became_active":"2024-09-11T12:55:43.985355+0000","last_became_peered":"2024-09-11T12:55:43.985355+0000","last_unstale":"2024-09-11T12:55:45.223561+0000","last_undegraded":"2024-09-11T12:55:45.223561+0000","last_fullsized":"2024-09-11T12:55:45.223561+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:18.584208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224497+0000","last_change":"2024-09-11T12:55:43.991058+0000","last_active":"2024-09-11T12:55:45.224497+0000","last_peered":"2024-09-11T12:55:45.224497+0000","last_clean":"2024-09-11T12:55:45.224497+0000","last_became_active":"2024-09-11T12:55:43.990201+0000","last_became_peered":"2024-09-11T12:55:43.990201+0000","last_unstale":"2024-09-11T12:55:45.224497+0000","last_undegraded":"2024-09-11T12:55:45.224497+0000","last_fullsized":"2024-09-11T12:55:45.224497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:52.644639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977603+0000","last_change":"2024-09-11T12:55:43.984007+0000","last_active":"2024-09-11T12:55:44.977603+0000","last_peered":"2024-09-11T12:55:44.977603+0000","last_clean":"2024-09-11T12:55:44.977603+0000","last_became_active":"2024-09-11T12:55:43.983648+0000","last_became_peered":"2024-09-11T12:55:43.983648+0000","last_unstale":"2024-09-11T12:55:44.977603+0000","last_undegraded":"2024-09-11T12:55:44.977603+0000","last_fullsized":"2024-09-11T12:55:44.977603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:41.368659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978050+0000","last_change":"2024-09-11T12:55:43.975377+0000","last_active":"2024-09-11T12:55:44.978050+0000","last_peered":"2024-09-11T12:55:44.978050+0000","last_clean":"2024-09-11T12:55:44.978050+0000","last_became_active":"2024-09-11T12:55:43.975122+0000","last_became_peered":"2024-09-11T12:55:43.975122+0000","last_unstale":"2024-09-11T12:55:44.978050+0000","last_undegraded":"2024-09-11T12:55:44.978050+0000","last_fullsized":"2024-09-11T12:55:44.978050+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:18.719014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975867+0000","last_change":"2024-09-11T12:55:43.992153+0000","last_active":"2024-09-11T12:55:44.975867+0000","last_peered":"2024-09-11T12:55:44.975867+0000","last_clean":"2024-09-11T12:55:44.975867+0000","last_became_active":"2024-09-11T12:55:43.991579+0000","last_became_peered":"2024-09-11T12:55:43.991579+0000","last_unstale":"2024-09-11T12:55:44.975867+0000","last_undegraded":"2024-09-11T12:55:44.975867+0000","last_fullsized":"2024-09-11T12:55:44.975867+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:31.646495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223314+0000","last_change":"2024-09-11T12:55:43.980050+0000","last_active":"2024-09-11T12:55:45.223314+0000","last_peered":"2024-09-11T12:55:45.223314+0000","last_clean":"2024-09-11T12:55:45.223314+0000","last_became_active":"2024-09-11T12:55:43.979602+0000","last_became_peered":"2024-09-11T12:55:43.979602+0000","last_unstale":"2024-09-11T12:55:45.223314+0000","last_undegraded":"2024-09-11T12:55:45.223314+0000","last_fullsized":"2024-09-11T12:55:45.223314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:05.455335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975691+0000","last_change":"2024-09-11T12:55:43.971160+0000","last_active":"2024-09-11T12:55:44.975691+0000","last_peered":"2024-09-11T12:55:44.975691+0000","last_clean":"2024-09-11T12:55:44.975691+0000","last_became_active":"2024-09-11T12:55:43.970897+0000","last_became_peered":"2024-09-11T12:55:43.970897+0000","last_unstale":"2024-09-11T12:55:44.975691+0000","last_undegraded":"2024-09-11T12:55:44.975691+0000","last_fullsized":"2024-09-11T12:55:44.975691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:38.522983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223551+0000","last_change":"2024-09-11T12:55:43.980848+0000","last_active":"2024-09-11T12:55:45.223551+0000","last_peered":"2024-09-11T12:55:45.223551+0000","last_clean":"2024-09-11T12:55:45.223551+0000","last_became_active":"2024-09-11T12:55:43.980326+0000","last_became_peered":"2024-09-11T12:55:43.980326+0000","last_unstale":"2024-09-11T12:55:45.223551+0000","last_undegraded":"2024-09-11T12:55:45.223551+0000","last_fullsized":"2024-09-11T12:55:45.223551+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:27.432044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986258+0000","last_change":"2024-09-11T12:55:43.992497+0000","last_active":"2024-09-11T12:55:44.986258+0000","last_peered":"2024-09-11T12:55:44.986258+0000","last_clean":"2024-09-11T12:55:44.986258+0000","last_became_active":"2024-09-11T12:55:43.988062+0000","last_became_peered":"2024-09-11T12:55:43.988062+0000","last_unstale":"2024-09-11T12:55:44.986258+0000","last_undegraded":"2024-09-11T12:55:44.986258+0000","last_fullsized":"2024-09-11T12:55:44.986258+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:21.789507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976429+0000","last_change":"2024-09-11T12:55:43.970481+0000","last_active":"2024-09-11T12:55:44.976429+0000","last_peered":"2024-09-11T12:55:44.976429+0000","last_clean":"2024-09-11T12:55:44.976429+0000","last_became_active":"2024-09-11T12:55:43.970224+0000","last_became_peered":"2024-09-11T12:55:43.970224+0000","last_unstale":"2024-09-11T12:55:44.976429+0000","last_undegraded":"2024-09-11T12:55:44.976429+0000","last_fullsized":"2024-09-11T12:55:44.976429+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:45.644370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986208+0000","last_change":"2024-09-11T12:55:43.986600+0000","last_active":"2024-09-11T12:55:44.986208+0000","last_peered":"2024-09-11T12:55:44.986208+0000","last_clean":"2024-09-11T12:55:44.986208+0000","last_became_active":"2024-09-11T12:55:43.986244+0000","last_became_peered":"2024-09-11T12:55:43.986244+0000","last_unstale":"2024-09-11T12:55:44.986208+0000","last_undegraded":"2024-09-11T12:55:44.986208+0000","last_fullsized":"2024-09-11T12:55:44.986208+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:25:49.562408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978232+0000","last_change":"2024-09-11T12:55:43.981300+0000","last_active":"2024-09-11T12:55:44.978232+0000","last_peered":"2024-09-11T12:55:44.978232+0000","last_clean":"2024-09-11T12:55:44.978232+0000","last_became_active":"2024-09-11T12:55:43.974143+0000","last_became_peered":"2024-09-11T12:55:43.974143+0000","last_unstale":"2024-09-11T12:55:44.978232+0000","last_undegraded":"2024-09-11T12:55:44.978232+0000","last_fullsized":"2024-09-11T12:55:44.978232+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:25.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977233+0000","last_change":"2024-09-11T12:55:43.980851+0000","last_active":"2024-09-11T12:55:44.977233+0000","last_peered":"2024-09-11T12:55:44.977233+0000","last_clean":"2024-09-11T12:55:44.977233+0000","last_became_active":"2024-09-11T12:55:43.980577+0000","last_became_peered":"2024-09-11T12:55:43.980577+0000","last_unstale":"2024-09-11T12:55:44.977233+0000","last_undegraded":"2024-09-11T12:55:44.977233+0000","last_fullsized":"2024-09-11T12:55:44.977233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985858+0000","last_change":"2024-09-11T12:55:43.981336+0000","last_active":"2024-09-11T12:55:44.985858+0000","last_peered":"2024-09-11T12:55:44.985858+0000","last_clean":"2024-09-11T12:55:44.985858+0000","last_became_active":"2024-09-11T12:55:43.981003+0000","last_became_peered":"2024-09-11T12:55:43.981003+0000","last_unstale":"2024-09-11T12:55:44.985858+0000","last_undegraded":"2024-09-11T12:55:44.985858+0000","last_fullsized":"2024-09-11T12:55:44.985858+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:33:46.865796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223842+0000","last_change":"2024-09-11T12:55:43.976193+0000","last_active":"2024-09-11T12:55:45.223842+0000","last_peered":"2024-09-11T12:55:45.223842+0000","last_clean":"2024-09-11T12:55:45.223842+0000","last_became_active":"2024-09-11T12:55:43.975904+0000","last_became_peered":"2024-09-11T12:55:43.975904+0000","last_unstale":"2024-09-11T12:55:45.223842+0000","last_undegraded":"2024-09-11T12:55:45.223842+0000","last_fullsized":"2024-09-11T12:55:45.223842+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:49.906520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986487+0000","last_change":"2024-09-11T12:55:43.980864+0000","last_active":"2024-09-11T12:55:44.986487+0000","last_peered":"2024-09-11T12:55:44.986487+0000","last_clean":"2024-09-11T12:55:44.986487+0000","last_became_active":"2024-09-11T12:55:43.980595+0000","last_became_peered":"2024-09-11T12:55:43.980595+0000","last_unstale":"2024-09-11T12:55:44.986487+0000","last_undegraded":"2024-09-11T12:55:44.986487+0000","last_fullsized":"2024-09-11T12:55:44.986487+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:02.004537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224489+0000","last_change":"2024-09-11T12:55:44.003554+0000","last_active":"2024-09-11T12:55:45.224489+0000","last_peered":"2024-09-11T12:55:45.224489+0000","last_clean":"2024-09-11T12:55:45.224489+0000","last_became_active":"2024-09-11T12:55:44.003219+0000","last_became_peered":"2024-09-11T12:55:44.003219+0000","last_unstale":"2024-09-11T12:55:45.224489+0000","last_undegraded":"2024-09-11T12:55:45.224489+0000","last_fullsized":"2024-09-11T12:55:45.224489+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:41.368659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976599+0000","last_change":"2024-09-11T12:55:43.973759+0000","last_active":"2024-09-11T12:55:44.976599+0000","last_peered":"2024-09-11T12:55:44.976599+0000","last_clean":"2024-09-11T12:55:44.976599+0000","last_became_active":"2024-09-11T12:55:43.973454+0000","last_became_peered":"2024-09-11T12:55:43.973454+0000","last_unstale":"2024-09-11T12:55:44.976599+0000","last_undegraded":"2024-09-11T12:55:44.976599+0000","last_fullsized":"2024-09-11T12:55:44.976599+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224562+0000","last_change":"2024-09-11T12:55:43.988236+0000","last_active":"2024-09-11T12:55:45.224562+0000","last_peered":"2024-09-11T12:55:45.224562+0000","last_clean":"2024-09-11T12:55:45.224562+0000","last_became_active":"2024-09-11T12:55:43.986022+0000","last_became_peered":"2024-09-11T12:55:43.986022+0000","last_unstale":"2024-09-11T12:55:45.224562+0000","last_undegraded":"2024-09-11T12:55:45.224562+0000","last_fullsized":"2024-09-11T12:55:45.224562+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:27.497194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986275+0000","last_change":"2024-09-11T12:55:43.993802+0000","last_active":"2024-09-11T12:55:44.986275+0000","last_peered":"2024-09-11T12:55:44.986275+0000","last_clean":"2024-09-11T12:55:44.986275+0000","last_became_active":"2024-09-11T12:55:43.992280+0000","last_became_peered":"2024-09-11T12:55:43.992280+0000","last_unstale":"2024-09-11T12:55:44.986275+0000","last_undegraded":"2024-09-11T12:55:44.986275+0000","last_fullsized":"2024-09-11T12:55:44.986275+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:57.594268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977415+0000","last_change":"2024-09-11T12:55:43.979347+0000","last_active":"2024-09-11T12:55:44.977415+0000","last_peered":"2024-09-11T12:55:44.977415+0000","last_clean":"2024-09-11T12:55:44.977415+0000","last_became_active":"2024-09-11T12:55:43.978597+0000","last_became_peered":"2024-09-11T12:55:43.978597+0000","last_unstale":"2024-09-11T12:55:44.977415+0000","last_undegraded":"2024-09-11T12:55:44.977415+0000","last_fullsized":"2024-09-11T12:55:44.977415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976871+0000","last_change":"2024-09-11T12:55:43.977059+0000","last_active":"2024-09-11T12:55:44.976871+0000","last_peered":"2024-09-11T12:55:44.976871+0000","last_clean":"2024-09-11T12:55:44.976871+0000","last_became_active":"2024-09-11T12:55:43.976747+0000","last_became_peered":"2024-09-11T12:55:43.976747+0000","last_unstale":"2024-09-11T12:55:44.976871+0000","last_undegraded":"2024-09-11T12:55:44.976871+0000","last_fullsized":"2024-09-11T12:55:44.976871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:58.103566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985794+0000","last_change":"2024-09-11T12:55:43.992321+0000","last_active":"2024-09-11T12:55:44.985794+0000","last_peered":"2024-09-11T12:55:44.985794+0000","last_clean":"2024-09-11T12:55:44.985794+0000","last_became_active":"2024-09-11T12:55:43.988806+0000","last_became_peered":"2024-09-11T12:55:43.988806+0000","last_unstale":"2024-09-11T12:55:44.985794+0000","last_undegraded":"2024-09-11T12:55:44.985794+0000","last_fullsized":"2024-09-11T12:55:44.985794+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:03.053908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224640+0000","last_change":"2024-09-11T12:55:43.972802+0000","last_active":"2024-09-11T12:55:45.224640+0000","last_peered":"2024-09-11T12:55:45.224640+0000","last_clean":"2024-09-11T12:55:45.224640+0000","last_became_active":"2024-09-11T12:55:43.972403+0000","last_became_peered":"2024-09-11T12:55:43.972403+0000","last_unstale":"2024-09-11T12:55:45.224640+0000","last_undegraded":"2024-09-11T12:55:45.224640+0000","last_fullsized":"2024-09-11T12:55:45.224640+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:17.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976462+0000","last_change":"2024-09-11T12:55:43.977525+0000","last_active":"2024-09-11T12:55:44.976462+0000","last_peered":"2024-09-11T12:55:44.976462+0000","last_clean":"2024-09-11T12:55:44.976462+0000","last_became_active":"2024-09-11T12:55:43.976643+0000","last_became_peered":"2024-09-11T12:55:43.976643+0000","last_unstale":"2024-09-11T12:55:44.976462+0000","last_undegraded":"2024-09-11T12:55:44.976462+0000","last_fullsized":"2024-09-11T12:55:44.976462+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:58.103566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224615+0000","last_change":"2024-09-11T12:55:43.988985+0000","last_active":"2024-09-11T12:55:45.224615+0000","last_peered":"2024-09-11T12:55:45.224615+0000","last_clean":"2024-09-11T12:55:45.224615+0000","last_became_active":"2024-09-11T12:55:43.988706+0000","last_became_peered":"2024-09-11T12:55:43.988706+0000","last_unstale":"2024-09-11T12:55:45.224615+0000","last_undegraded":"2024-09-11T12:55:45.224615+0000","last_fullsized":"2024-09-11T12:55:45.224615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:56:18.427244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223690+0000","last_change":"2024-09-11T12:55:43.984842+0000","last_active":"2024-09-11T12:55:45.223690+0000","last_peered":"2024-09-11T12:55:45.223690+0000","last_clean":"2024-09-11T12:55:45.223690+0000","last_became_active":"2024-09-11T12:55:43.984535+0000","last_became_peered":"2024-09-11T12:55:43.984535+0000","last_unstale":"2024-09-11T12:55:45.223690+0000","last_undegraded":"2024-09-11T12:55:45.223690+0000","last_fullsized":"2024-09-11T12:55:45.223690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:27:30.900809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976058+0000","last_change":"2024-09-11T12:55:43.991912+0000","last_active":"2024-09-11T12:55:44.976058+0000","last_peered":"2024-09-11T12:55:44.976058+0000","last_clean":"2024-09-11T12:55:44.976058+0000","last_became_active":"2024-09-11T12:55:43.991558+0000","last_became_peered":"2024-09-11T12:55:43.991558+0000","last_unstale":"2024-09-11T12:55:44.976058+0000","last_undegraded":"2024-09-11T12:55:44.976058+0000","last_fullsized":"2024-09-11T12:55:44.976058+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:26.714872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977627+0000","last_change":"2024-09-11T12:55:43.979127+0000","last_active":"2024-09-11T12:55:44.977627+0000","last_peered":"2024-09-11T12:55:44.977627+0000","last_clean":"2024-09-11T12:55:44.977627+0000","last_became_active":"2024-09-11T12:55:43.978198+0000","last_became_peered":"2024-09-11T12:55:43.978198+0000","last_unstale":"2024-09-11T12:55:44.977627+0000","last_undegraded":"2024-09-11T12:55:44.977627+0000","last_fullsized":"2024-09-11T12:55:44.977627+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:37.156786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223345+0000","last_change":"2024-09-11T12:55:43.999683+0000","last_active":"2024-09-11T12:55:45.223345+0000","last_peered":"2024-09-11T12:55:45.223345+0000","last_clean":"2024-09-11T12:55:45.223345+0000","last_became_active":"2024-09-11T12:55:43.999482+0000","last_became_peered":"2024-09-11T12:55:43.999482+0000","last_unstale":"2024-09-11T12:55:45.223345+0000","last_undegraded":"2024-09-11T12:55:45.223345+0000","last_fullsized":"2024-09-11T12:55:45.223345+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:34.407007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224001+0000","last_change":"2024-09-11T12:55:43.987557+0000","last_active":"2024-09-11T12:55:45.224001+0000","last_peered":"2024-09-11T12:55:45.224001+0000","last_clean":"2024-09-11T12:55:45.224001+0000","last_became_active":"2024-09-11T12:55:43.987212+0000","last_became_peered":"2024-09-11T12:55:43.987212+0000","last_unstale":"2024-09-11T12:55:45.224001+0000","last_undegraded":"2024-09-11T12:55:45.224001+0000","last_fullsized":"2024-09-11T12:55:45.224001+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:53:00.972508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986331+0000","last_change":"2024-09-11T12:55:43.992744+0000","last_active":"2024-09-11T12:55:44.986331+0000","last_peered":"2024-09-11T12:55:44.986331+0000","last_clean":"2024-09-11T12:55:44.986331+0000","last_became_active":"2024-09-11T12:55:43.992136+0000","last_became_peered":"2024-09-11T12:55:43.992136+0000","last_unstale":"2024-09-11T12:55:44.986331+0000","last_undegraded":"2024-09-11T12:55:44.986331+0000","last_fullsized":"2024-09-11T12:55:44.986331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:51:03.583636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986246+0000","last_change":"2024-09-11T12:55:42.989411+0000","last_active":"2024-09-11T12:55:44.986246+0000","last_peered":"2024-09-11T12:55:44.986246+0000","last_clean":"2024-09-11T12:55:44.986246+0000","last_became_active":"2024-09-11T12:55:42.989163+0000","last_became_peered":"2024-09-11T12:55:42.989163+0000","last_unstale":"2024-09-11T12:55:44.986246+0000","last_undegraded":"2024-09-11T12:55:44.986246+0000","last_fullsized":"2024-09-11T12:55:44.986246+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:41.860425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223719+0000","last_change":"2024-09-11T12:55:43.980618+0000","last_active":"2024-09-11T12:55:45.223719+0000","last_peered":"2024-09-11T12:55:45.223719+0000","last_clean":"2024-09-11T12:55:45.223719+0000","last_became_active":"2024-09-11T12:55:43.980080+0000","last_became_peered":"2024-09-11T12:55:43.980080+0000","last_unstale":"2024-09-11T12:55:45.223719+0000","last_undegraded":"2024-09-11T12:55:45.223719+0000","last_fullsized":"2024-09-11T12:55:45.223719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.267109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223632+0000","last_change":"2024-09-11T12:55:42.979871+0000","last_active":"2024-09-11T12:55:45.223632+0000","last_peered":"2024-09-11T12:55:45.223632+0000","last_clean":"2024-09-11T12:55:45.223632+0000","last_became_active":"2024-09-11T12:55:42.979625+0000","last_became_peered":"2024-09-11T12:55:42.979625+0000","last_unstale":"2024-09-11T12:55:45.223632+0000","last_undegraded":"2024-09-11T12:55:45.223632+0000","last_fullsized":"2024-09-11T12:55:45.223632+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.329329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976265+0000","last_change":"2024-09-11T12:55:43.978570+0000","last_active":"2024-09-11T12:55:44.976265+0000","last_peered":"2024-09-11T12:55:44.976265+0000","last_clean":"2024-09-11T12:55:44.976265+0000","last_became_active":"2024-09-11T12:55:43.978265+0000","last_became_peered":"2024-09-11T12:55:43.978265+0000","last_unstale":"2024-09-11T12:55:44.976265+0000","last_undegraded":"2024-09-11T12:55:44.976265+0000","last_fullsized":"2024-09-11T12:55:44.976265+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:22.332310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977378+0000","last_change":"2024-09-11T12:55:42.977572+0000","last_active":"2024-09-11T12:55:44.977378+0000","last_peered":"2024-09-11T12:55:44.977378+0000","last_clean":"2024-09-11T12:55:44.977378+0000","last_became_active":"2024-09-11T12:55:42.976340+0000","last_became_peered":"2024-09-11T12:55:42.976340+0000","last_unstale":"2024-09-11T12:55:44.977378+0000","last_undegraded":"2024-09-11T12:55:44.977378+0000","last_fullsized":"2024-09-11T12:55:44.977378+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:21.202496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986055+0000","last_change":"2024-09-11T12:55:43.986598+0000","last_active":"2024-09-11T12:55:44.986055+0000","last_peered":"2024-09-11T12:55:44.986055+0000","last_clean":"2024-09-11T12:55:44.986055+0000","last_became_active":"2024-09-11T12:55:43.986274+0000","last_became_peered":"2024-09-11T12:55:43.986274+0000","last_unstale":"2024-09-11T12:55:44.986055+0000","last_undegraded":"2024-09-11T12:55:44.986055+0000","last_fullsized":"2024-09-11T12:55:44.986055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:42.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977844+0000","last_change":"2024-09-11T12:55:42.977792+0000","last_active":"2024-09-11T12:55:44.977844+0000","last_peered":"2024-09-11T12:55:44.977844+0000","last_clean":"2024-09-11T12:55:44.977844+0000","last_became_active":"2024-09-11T12:55:42.976635+0000","last_became_peered":"2024-09-11T12:55:42.976635+0000","last_unstale":"2024-09-11T12:55:44.977844+0000","last_undegraded":"2024-09-11T12:55:44.977844+0000","last_fullsized":"2024-09-11T12:55:44.977844+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:33.644510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223504+0000","last_change":"2024-09-11T12:55:44.001190+0000","last_active":"2024-09-11T12:55:45.223504+0000","last_peered":"2024-09-11T12:55:45.223504+0000","last_clean":"2024-09-11T12:55:45.223504+0000","last_became_active":"2024-09-11T12:55:44.000897+0000","last_became_peered":"2024-09-11T12:55:44.000897+0000","last_unstale":"2024-09-11T12:55:45.223504+0000","last_undegraded":"2024-09-11T12:55:45.223504+0000","last_fullsized":"2024-09-11T12:55:45.223504+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:42.863406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224326+0000","last_change":"2024-09-11T12:55:42.980956+0000","last_active":"2024-09-11T12:55:45.224326+0000","last_peered":"2024-09-11T12:55:45.224326+0000","last_clean":"2024-09-11T12:55:45.224326+0000","last_became_active":"2024-09-11T12:55:42.980215+0000","last_became_peered":"2024-09-11T12:55:42.980215+0000","last_unstale":"2024-09-11T12:55:45.224326+0000","last_undegraded":"2024-09-11T12:55:45.224326+0000","last_fullsized":"2024-09-11T12:55:45.224326+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:49.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977162+0000","last_change":"2024-09-11T12:55:43.994480+0000","last_active":"2024-09-11T12:55:44.977162+0000","last_peered":"2024-09-11T12:55:44.977162+0000","last_clean":"2024-09-11T12:55:44.977162+0000","last_became_active":"2024-09-11T12:55:43.993950+0000","last_became_peered":"2024-09-11T12:55:43.993950+0000","last_unstale":"2024-09-11T12:55:44.977162+0000","last_undegraded":"2024-09-11T12:55:44.977162+0000","last_fullsized":"2024-09-11T12:55:44.977162+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977492+0000","last_change":"2024-09-11T12:55:42.985294+0000","last_active":"2024-09-11T12:55:44.977492+0000","last_peered":"2024-09-11T12:55:44.977492+0000","last_clean":"2024-09-11T12:55:44.977492+0000","last_became_active":"2024-09-11T12:55:42.985010+0000","last_became_peered":"2024-09-11T12:55:42.985010+0000","last_unstale":"2024-09-11T12:55:44.977492+0000","last_undegraded":"2024-09-11T12:55:44.977492+0000","last_fullsized":"2024-09-11T12:55:44.977492+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.657167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223253+0000","last_change":"2024-09-11T12:55:43.993699+0000","last_active":"2024-09-11T12:55:45.223253+0000","last_peered":"2024-09-11T12:55:45.223253+0000","last_clean":"2024-09-11T12:55:45.223253+0000","last_became_active":"2024-09-11T12:55:43.993400+0000","last_became_peered":"2024-09-11T12:55:43.993400+0000","last_unstale":"2024-09-11T12:55:45.223253+0000","last_undegraded":"2024-09-11T12:55:45.223253+0000","last_fullsized":"2024-09-11T12:55:45.223253+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:38:54.698712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224334+0000","last_change":"2024-09-11T12:55:42.973097+0000","last_active":"2024-09-11T12:55:45.224334+0000","last_peered":"2024-09-11T12:55:45.224334+0000","last_clean":"2024-09-11T12:55:45.224334+0000","last_became_active":"2024-09-11T12:55:42.972799+0000","last_became_peered":"2024-09-11T12:55:42.972799+0000","last_unstale":"2024-09-11T12:55:45.224334+0000","last_undegraded":"2024-09-11T12:55:45.224334+0000","last_fullsized":"2024-09-11T12:55:45.224334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:30.394530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977055+0000","last_change":"2024-09-11T12:55:43.968260+0000","last_active":"2024-09-11T12:55:44.977055+0000","last_peered":"2024-09-11T12:55:44.977055+0000","last_clean":"2024-09-11T12:55:44.977055+0000","last_became_active":"2024-09-11T12:55:43.967320+0000","last_became_peered":"2024-09-11T12:55:43.967320+0000","last_unstale":"2024-09-11T12:55:44.977055+0000","last_undegraded":"2024-09-11T12:55:44.977055+0000","last_fullsized":"2024-09-11T12:55:44.977055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:26.757084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976838+0000","last_change":"2024-09-11T12:55:42.980180+0000","last_active":"2024-09-11T12:55:44.976838+0000","last_peered":"2024-09-11T12:55:44.976838+0000","last_clean":"2024-09-11T12:55:44.976838+0000","last_became_active":"2024-09-11T12:55:42.979883+0000","last_became_peered":"2024-09-11T12:55:42.979883+0000","last_unstale":"2024-09-11T12:55:44.976838+0000","last_undegraded":"2024-09-11T12:55:44.976838+0000","last_fullsized":"2024-09-11T12:55:44.976838+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:38.356612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977990+0000","last_change":"2024-09-11T12:55:43.982364+0000","last_active":"2024-09-11T12:55:44.977990+0000","last_peered":"2024-09-11T12:55:44.977990+0000","last_clean":"2024-09-11T12:55:44.977990+0000","last_became_active":"2024-09-11T12:55:43.982083+0000","last_became_peered":"2024-09-11T12:55:43.982083+0000","last_unstale":"2024-09-11T12:55:44.977990+0000","last_undegraded":"2024-09-11T12:55:44.977990+0000","last_fullsized":"2024-09-11T12:55:44.977990+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:40:20.736834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223814+0000","last_change":"2024-09-11T12:55:42.977280+0000","last_active":"2024-09-11T12:55:45.223814+0000","last_peered":"2024-09-11T12:55:45.223814+0000","last_clean":"2024-09-11T12:55:45.223814+0000","last_became_active":"2024-09-11T12:55:42.976993+0000","last_became_peered":"2024-09-11T12:55:42.976993+0000","last_unstale":"2024-09-11T12:55:45.223814+0000","last_undegraded":"2024-09-11T12:55:45.223814+0000","last_fullsized":"2024-09-11T12:55:45.223814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:30.643514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986480+0000","last_change":"2024-09-11T12:55:43.983164+0000","last_active":"2024-09-11T12:55:44.986480+0000","last_peered":"2024-09-11T12:55:44.986480+0000","last_clean":"2024-09-11T12:55:44.986480+0000","last_became_active":"2024-09-11T12:55:43.980929+0000","last_became_peered":"2024-09-11T12:55:43.980929+0000","last_unstale":"2024-09-11T12:55:44.986480+0000","last_undegraded":"2024-09-11T12:55:44.986480+0000","last_fullsized":"2024-09-11T12:55:44.986480+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:01.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986498+0000","last_change":"2024-09-11T12:55:42.987541+0000","last_active":"2024-09-11T12:55:44.986498+0000","last_peered":"2024-09-11T12:55:44.986498+0000","last_clean":"2024-09-11T12:55:44.986498+0000","last_became_active":"2024-09-11T12:55:42.987263+0000","last_became_peered":"2024-09-11T12:55:42.987263+0000","last_unstale":"2024-09-11T12:55:44.986498+0000","last_undegraded":"2024-09-11T12:55:44.986498+0000","last_fullsized":"2024-09-11T12:55:44.986498+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:26.429063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224726+0000","last_change":"2024-09-11T12:55:43.972865+0000","last_active":"2024-09-11T12:55:45.224726+0000","last_peered":"2024-09-11T12:55:45.224726+0000","last_clean":"2024-09-11T12:55:45.224726+0000","last_became_active":"2024-09-11T12:55:43.972546+0000","last_became_peered":"2024-09-11T12:55:43.972546+0000","last_unstale":"2024-09-11T12:55:45.224726+0000","last_undegraded":"2024-09-11T12:55:45.224726+0000","last_fullsized":"2024-09-11T12:55:45.224726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:58.464947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"128'6","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223787+0000","last_change":"2024-09-11T12:55:42.974745+0000","last_active":"2024-09-11T12:55:45.223787+0000","last_peered":"2024-09-11T12:55:45.223787+0000","last_clean":"2024-09-11T12:55:45.223787+0000","last_became_active":"2024-09-11T12:55:42.974463+0000","last_became_peered":"2024-09-11T12:55:42.974463+0000","last_unstale":"2024-09-11T12:55:45.223787+0000","last_undegraded":"2024-09-11T12:55:45.223787+0000","last_fullsized":"2024-09-11T12:55:45.223787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:55.658763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224151+0000","last_change":"2024-09-11T12:55:43.984920+0000","last_active":"2024-09-11T12:55:45.224151+0000","last_peered":"2024-09-11T12:55:45.224151+0000","last_clean":"2024-09-11T12:55:45.224151+0000","last_became_active":"2024-09-11T12:55:43.984116+0000","last_became_peered":"2024-09-11T12:55:43.984116+0000","last_unstale":"2024-09-11T12:55:45.224151+0000","last_undegraded":"2024-09-11T12:55:45.224151+0000","last_fullsized":"2024-09-11T12:55:45.224151+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224172+0000","last_change":"2024-09-11T12:55:42.977504+0000","last_active":"2024-09-11T12:55:45.224172+0000","last_peered":"2024-09-11T12:55:45.224172+0000","last_clean":"2024-09-11T12:55:45.224172+0000","last_became_active":"2024-09-11T12:55:42.976950+0000","last_became_peered":"2024-09-11T12:55:42.976950+0000","last_unstale":"2024-09-11T12:55:45.224172+0000","last_undegraded":"2024-09-11T12:55:45.224172+0000","last_fullsized":"2024-09-11T12:55:45.224172+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:44.641389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223339+0000","last_change":"2024-09-11T12:55:43.980728+0000","last_active":"2024-09-11T12:55:45.223339+0000","last_peered":"2024-09-11T12:55:45.223339+0000","last_clean":"2024-09-11T12:55:45.223339+0000","last_became_active":"2024-09-11T12:55:43.980427+0000","last_became_peered":"2024-09-11T12:55:43.980427+0000","last_unstale":"2024-09-11T12:55:45.223339+0000","last_undegraded":"2024-09-11T12:55:45.223339+0000","last_fullsized":"2024-09-11T12:55:45.223339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:11.512023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224082+0000","last_change":"2024-09-11T12:55:42.982353+0000","last_active":"2024-09-11T12:55:45.224082+0000","last_peered":"2024-09-11T12:55:45.224082+0000","last_clean":"2024-09-11T12:55:45.224082+0000","last_became_active":"2024-09-11T12:55:42.982017+0000","last_became_peered":"2024-09-11T12:55:42.982017+0000","last_unstale":"2024-09-11T12:55:45.224082+0000","last_undegraded":"2024-09-11T12:55:45.224082+0000","last_fullsized":"2024-09-11T12:55:45.224082+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.329329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976861+0000","last_change":"2024-09-11T12:55:43.978821+0000","last_active":"2024-09-11T12:55:44.976861+0000","last_peered":"2024-09-11T12:55:44.976861+0000","last_clean":"2024-09-11T12:55:44.976861+0000","last_became_active":"2024-09-11T12:55:43.974856+0000","last_became_peered":"2024-09-11T12:55:43.974856+0000","last_unstale":"2024-09-11T12:55:44.976861+0000","last_undegraded":"2024-09-11T12:55:44.976861+0000","last_fullsized":"2024-09-11T12:55:44.976861+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:09.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975665+0000","last_change":"2024-09-11T12:55:42.975364+0000","last_active":"2024-09-11T12:55:44.975665+0000","last_peered":"2024-09-11T12:55:44.975665+0000","last_clean":"2024-09-11T12:55:44.975665+0000","last_became_active":"2024-09-11T12:55:42.974899+0000","last_became_peered":"2024-09-11T12:55:42.974899+0000","last_unstale":"2024-09-11T12:55:44.975665+0000","last_undegraded":"2024-09-11T12:55:44.975665+0000","last_fullsized":"2024-09-11T12:55:44.975665+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:23.707165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223974+0000","last_change":"2024-09-11T12:55:43.982628+0000","last_active":"2024-09-11T12:55:45.223974+0000","last_peered":"2024-09-11T12:55:45.223974+0000","last_clean":"2024-09-11T12:55:45.223974+0000","last_became_active":"2024-09-11T12:55:43.981674+0000","last_became_peered":"2024-09-11T12:55:43.981674+0000","last_unstale":"2024-09-11T12:55:45.223974+0000","last_undegraded":"2024-09-11T12:55:45.223974+0000","last_fullsized":"2024-09-11T12:55:45.223974+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"128'8","reported_seq":23,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224106+0000","last_change":"2024-09-11T12:55:42.970949+0000","last_active":"2024-09-11T12:55:45.224106+0000","last_peered":"2024-09-11T12:55:45.224106+0000","last_clean":"2024-09-11T12:55:45.224106+0000","last_became_active":"2024-09-11T12:55:42.970665+0000","last_became_peered":"2024-09-11T12:55:42.970665+0000","last_unstale":"2024-09-11T12:55:45.224106+0000","last_undegraded":"2024-09-11T12:55:45.224106+0000","last_fullsized":"2024-09-11T12:55:45.224106+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:46.411913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986405+0000","last_change":"2024-09-11T12:55:43.984443+0000","last_active":"2024-09-11T12:55:44.986405+0000","last_peered":"2024-09-11T12:55:44.986405+0000","last_clean":"2024-09-11T12:55:44.986405+0000","last_became_active":"2024-09-11T12:55:43.984175+0000","last_became_peered":"2024-09-11T12:55:43.984175+0000","last_unstale":"2024-09-11T12:55:44.986405+0000","last_undegraded":"2024-09-11T12:55:44.986405+0000","last_fullsized":"2024-09-11T12:55:44.986405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:26.192735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977870+0000","last_change":"2024-09-11T12:55:42.975686+0000","last_active":"2024-09-11T12:55:44.977870+0000","last_peered":"2024-09-11T12:55:44.977870+0000","last_clean":"2024-09-11T12:55:44.977870+0000","last_became_active":"2024-09-11T12:55:42.975420+0000","last_became_peered":"2024-09-11T12:55:42.975420+0000","last_unstale":"2024-09-11T12:55:44.977870+0000","last_undegraded":"2024-09-11T12:55:44.977870+0000","last_fullsized":"2024-09-11T12:55:44.977870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:35:05.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224663+0000","last_change":"2024-09-11T12:55:43.975522+0000","last_active":"2024-09-11T12:55:45.224663+0000","last_peered":"2024-09-11T12:55:45.224663+0000","last_clean":"2024-09-11T12:55:45.224663+0000","last_became_active":"2024-09-11T12:55:43.975093+0000","last_became_peered":"2024-09-11T12:55:43.975093+0000","last_unstale":"2024-09-11T12:55:45.224663+0000","last_undegraded":"2024-09-11T12:55:45.224663+0000","last_fullsized":"2024-09-11T12:55:45.224663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:24.512562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223663+0000","last_change":"2024-09-11T12:55:42.978418+0000","last_active":"2024-09-11T12:55:45.223663+0000","last_peered":"2024-09-11T12:55:45.223663+0000","last_clean":"2024-09-11T12:55:45.223663+0000","last_became_active":"2024-09-11T12:55:42.978176+0000","last_became_peered":"2024-09-11T12:55:42.978176+0000","last_unstale":"2024-09-11T12:55:45.223663+0000","last_undegraded":"2024-09-11T12:55:45.223663+0000","last_fullsized":"2024-09-11T12:55:45.223663+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:44.641389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224314+0000","last_change":"2024-09-11T12:55:43.984556+0000","last_active":"2024-09-11T12:55:45.224314+0000","last_peered":"2024-09-11T12:55:45.224314+0000","last_clean":"2024-09-11T12:55:45.224314+0000","last_became_active":"2024-09-11T12:55:43.984195+0000","last_became_peered":"2024-09-11T12:55:43.984195+0000","last_unstale":"2024-09-11T12:55:45.224314+0000","last_undegraded":"2024-09-11T12:55:45.224314+0000","last_fullsized":"2024-09-11T12:55:45.224314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224783+0000","last_change":"2024-09-11T12:55:42.984591+0000","last_active":"2024-09-11T12:55:45.224783+0000","last_peered":"2024-09-11T12:55:45.224783+0000","last_clean":"2024-09-11T12:55:45.224783+0000","last_became_active":"2024-09-11T12:55:42.984250+0000","last_became_peered":"2024-09-11T12:55:42.984250+0000","last_unstale":"2024-09-11T12:55:45.224783+0000","last_undegraded":"2024-09-11T12:55:45.224783+0000","last_fullsized":"2024-09-11T12:55:45.224783+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:23.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976453+0000","last_change":"2024-09-11T12:55:43.992153+0000","last_active":"2024-09-11T12:55:44.976453+0000","last_peered":"2024-09-11T12:55:44.976453+0000","last_clean":"2024-09-11T12:55:44.976453+0000","last_became_active":"2024-09-11T12:55:43.991825+0000","last_became_peered":"2024-09-11T12:55:43.991825+0000","last_unstale":"2024-09-11T12:55:44.976453+0000","last_undegraded":"2024-09-11T12:55:44.976453+0000","last_fullsized":"2024-09-11T12:55:44.976453+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:04.503928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223665+0000","last_change":"2024-09-11T12:55:42.977968+0000","last_active":"2024-09-11T12:55:45.223665+0000","last_peered":"2024-09-11T12:55:45.223665+0000","last_clean":"2024-09-11T12:55:45.223665+0000","last_became_active":"2024-09-11T12:55:42.977654+0000","last_became_peered":"2024-09-11T12:55:42.977654+0000","last_unstale":"2024-09-11T12:55:45.223665+0000","last_undegraded":"2024-09-11T12:55:45.223665+0000","last_fullsized":"2024-09-11T12:55:45.223665+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:37.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977788+0000","last_change":"2024-09-11T12:55:43.974832+0000","last_active":"2024-09-11T12:55:44.977788+0000","last_peered":"2024-09-11T12:55:44.977788+0000","last_clean":"2024-09-11T12:55:44.977788+0000","last_became_active":"2024-09-11T12:55:43.974552+0000","last_became_peered":"2024-09-11T12:55:43.974552+0000","last_unstale":"2024-09-11T12:55:44.977788+0000","last_undegraded":"2024-09-11T12:55:44.977788+0000","last_fullsized":"2024-09-11T12:55:44.977788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:52.644639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223838+0000","last_change":"2024-09-11T12:55:42.972270+0000","last_active":"2024-09-11T12:55:45.223838+0000","last_peered":"2024-09-11T12:55:45.223838+0000","last_clean":"2024-09-11T12:55:45.223838+0000","last_became_active":"2024-09-11T12:55:42.971991+0000","last_became_peered":"2024-09-11T12:55:42.971991+0000","last_unstale":"2024-09-11T12:55:45.223838+0000","last_undegraded":"2024-09-11T12:55:45.223838+0000","last_fullsized":"2024-09-11T12:55:45.223838+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:27.433191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223741+0000","last_change":"2024-09-11T12:55:43.991023+0000","last_active":"2024-09-11T12:55:45.223741+0000","last_peered":"2024-09-11T12:55:45.223741+0000","last_clean":"2024-09-11T12:55:45.223741+0000","last_became_active":"2024-09-11T12:55:43.990722+0000","last_became_peered":"2024-09-11T12:55:43.990722+0000","last_unstale":"2024-09-11T12:55:45.223741+0000","last_undegraded":"2024-09-11T12:55:45.223741+0000","last_fullsized":"2024-09-11T12:55:45.223741+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:25.910972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985897+0000","last_change":"2024-09-11T12:55:42.984294+0000","last_active":"2024-09-11T12:55:44.985897+0000","last_peered":"2024-09-11T12:55:44.985897+0000","last_clean":"2024-09-11T12:55:44.985897+0000","last_became_active":"2024-09-11T12:55:42.984003+0000","last_became_peered":"2024-09-11T12:55:42.984003+0000","last_unstale":"2024-09-11T12:55:44.985897+0000","last_undegraded":"2024-09-11T12:55:44.985897+0000","last_fullsized":"2024-09-11T12:55:44.985897+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:10.509042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976708+0000","last_change":"2024-09-11T12:55:43.978798+0000","last_active":"2024-09-11T12:55:44.976708+0000","last_peered":"2024-09-11T12:55:44.976708+0000","last_clean":"2024-09-11T12:55:44.976708+0000","last_became_active":"2024-09-11T12:55:43.977850+0000","last_became_peered":"2024-09-11T12:55:43.977850+0000","last_unstale":"2024-09-11T12:55:44.976708+0000","last_undegraded":"2024-09-11T12:55:44.976708+0000","last_fullsized":"2024-09-11T12:55:44.976708+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:04.503928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985782+0000","last_change":"2024-09-11T12:55:42.987260+0000","last_active":"2024-09-11T12:55:44.985782+0000","last_peered":"2024-09-11T12:55:44.985782+0000","last_clean":"2024-09-11T12:55:44.985782+0000","last_became_active":"2024-09-11T12:55:42.986948+0000","last_became_peered":"2024-09-11T12:55:42.986948+0000","last_unstale":"2024-09-11T12:55:44.985782+0000","last_undegraded":"2024-09-11T12:55:44.985782+0000","last_fullsized":"2024-09-11T12:55:44.985782+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:38:53.695731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223291+0000","last_change":"2024-09-11T12:55:43.984987+0000","last_active":"2024-09-11T12:55:45.223291+0000","last_peered":"2024-09-11T12:55:45.223291+0000","last_clean":"2024-09-11T12:55:45.223291+0000","last_became_active":"2024-09-11T12:55:43.984435+0000","last_became_peered":"2024-09-11T12:55:43.984435+0000","last_unstale":"2024-09-11T12:55:45.223291+0000","last_undegraded":"2024-09-11T12:55:45.223291+0000","last_fullsized":"2024-09-11T12:55:45.223291+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:32.000015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223884+0000","last_change":"2024-09-11T12:55:42.979048+0000","last_active":"2024-09-11T12:55:45.223884+0000","last_peered":"2024-09-11T12:55:45.223884+0000","last_clean":"2024-09-11T12:55:45.223884+0000","last_became_active":"2024-09-11T12:55:42.978773+0000","last_became_peered":"2024-09-11T12:55:42.978773+0000","last_unstale":"2024-09-11T12:55:45.223884+0000","last_undegraded":"2024-09-11T12:55:45.223884+0000","last_fullsized":"2024-09-11T12:55:45.223884+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:59.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978189+0000","last_change":"2024-09-11T12:55:43.981120+0000","last_active":"2024-09-11T12:55:44.978189+0000","last_peered":"2024-09-11T12:55:44.978189+0000","last_clean":"2024-09-11T12:55:44.978189+0000","last_became_active":"2024-09-11T12:55:43.980784+0000","last_became_peered":"2024-09-11T12:55:43.980784+0000","last_unstale":"2024-09-11T12:55:44.978189+0000","last_undegraded":"2024-09-11T12:55:44.978189+0000","last_fullsized":"2024-09-11T12:55:44.978189+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:54.272393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986568+0000","last_change":"2024-09-11T12:55:42.986972+0000","last_active":"2024-09-11T12:55:44.986568+0000","last_peered":"2024-09-11T12:55:44.986568+0000","last_clean":"2024-09-11T12:55:44.986568+0000","last_became_active":"2024-09-11T12:55:42.986650+0000","last_became_peered":"2024-09-11T12:55:42.986650+0000","last_unstale":"2024-09-11T12:55:44.986568+0000","last_undegraded":"2024-09-11T12:55:44.986568+0000","last_fullsized":"2024-09-11T12:55:44.986568+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:33.404026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224490+0000","last_change":"2024-09-11T12:55:43.988925+0000","last_active":"2024-09-11T12:55:45.224490+0000","last_peered":"2024-09-11T12:55:45.224490+0000","last_clean":"2024-09-11T12:55:45.224490+0000","last_became_active":"2024-09-11T12:55:43.988602+0000","last_became_peered":"2024-09-11T12:55:43.988602+0000","last_unstale":"2024-09-11T12:55:45.224490+0000","last_undegraded":"2024-09-11T12:55:45.224490+0000","last_fullsized":"2024-09-11T12:55:45.224490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:37.156786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.984621+0000","last_change":"2024-09-11T12:55:42.979120+0000","last_active":"2024-09-11T12:55:44.984621+0000","last_peered":"2024-09-11T12:55:44.984621+0000","last_clean":"2024-09-11T12:55:44.984621+0000","last_became_active":"2024-09-11T12:55:42.978842+0000","last_became_peered":"2024-09-11T12:55:42.978842+0000","last_unstale":"2024-09-11T12:55:44.984621+0000","last_undegraded":"2024-09-11T12:55:44.984621+0000","last_fullsized":"2024-09-11T12:55:44.984621+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:35:57.864818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223557+0000","last_change":"2024-09-11T12:55:43.981332+0000","last_active":"2024-09-11T12:55:45.223557+0000","last_peered":"2024-09-11T12:55:45.223557+0000","last_clean":"2024-09-11T12:55:45.223557+0000","last_became_active":"2024-09-11T12:55:43.981031+0000","last_became_peered":"2024-09-11T12:55:43.981031+0000","last_unstale":"2024-09-11T12:55:45.223557+0000","last_undegraded":"2024-09-11T12:55:45.223557+0000","last_fullsized":"2024-09-11T12:55:45.223557+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:04:31.686584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976670+0000","last_change":"2024-09-11T12:55:42.978896+0000","last_active":"2024-09-11T12:55:44.976670+0000","last_peered":"2024-09-11T12:55:44.976670+0000","last_clean":"2024-09-11T12:55:44.976670+0000","last_became_active":"2024-09-11T12:55:42.977806+0000","last_became_peered":"2024-09-11T12:55:42.977806+0000","last_unstale":"2024-09-11T12:55:44.976670+0000","last_undegraded":"2024-09-11T12:55:44.976670+0000","last_fullsized":"2024-09-11T12:55:44.976670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:39.224334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976591+0000","last_change":"2024-09-11T12:55:43.978722+0000","last_active":"2024-09-11T12:55:44.976591+0000","last_peered":"2024-09-11T12:55:44.976591+0000","last_clean":"2024-09-11T12:55:44.976591+0000","last_became_active":"2024-09-11T12:55:43.978429+0000","last_became_peered":"2024-09-11T12:55:43.978429+0000","last_unstale":"2024-09-11T12:55:44.976591+0000","last_undegraded":"2024-09-11T12:55:44.976591+0000","last_fullsized":"2024-09-11T12:55:44.976591+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976295+0000","last_change":"2024-09-11T12:55:42.977938+0000","last_active":"2024-09-11T12:55:44.976295+0000","last_peered":"2024-09-11T12:55:44.976295+0000","last_clean":"2024-09-11T12:55:44.976295+0000","last_became_active":"2024-09-11T12:55:42.977668+0000","last_became_peered":"2024-09-11T12:55:42.977668+0000","last_unstale":"2024-09-11T12:55:44.976295+0000","last_undegraded":"2024-09-11T12:55:44.976295+0000","last_fullsized":"2024-09-11T12:55:44.976295+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:30.109491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224363+0000","last_change":"2024-09-11T12:55:43.978822+0000","last_active":"2024-09-11T12:55:45.224363+0000","last_peered":"2024-09-11T12:55:45.224363+0000","last_clean":"2024-09-11T12:55:45.224363+0000","last_became_active":"2024-09-11T12:55:43.978513+0000","last_became_peered":"2024-09-11T12:55:43.978513+0000","last_unstale":"2024-09-11T12:55:45.224363+0000","last_undegraded":"2024-09-11T12:55:45.224363+0000","last_fullsized":"2024-09-11T12:55:45.224363+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:22:11.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223930+0000","last_change":"2024-09-11T12:55:42.976548+0000","last_active":"2024-09-11T12:55:45.223930+0000","last_peered":"2024-09-11T12:55:45.223930+0000","last_clean":"2024-09-11T12:55:45.223930+0000","last_became_active":"2024-09-11T12:55:42.976269+0000","last_became_peered":"2024-09-11T12:55:42.976269+0000","last_unstale":"2024-09-11T12:55:45.223930+0000","last_undegraded":"2024-09-11T12:55:45.223930+0000","last_fullsized":"2024-09-11T12:55:45.223930+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:35.655434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978275+0000","last_change":"2024-09-11T12:55:43.984946+0000","last_active":"2024-09-11T12:55:44.978275+0000","last_peered":"2024-09-11T12:55:44.978275+0000","last_clean":"2024-09-11T12:55:44.978275+0000","last_became_active":"2024-09-11T12:55:43.984622+0000","last_became_peered":"2024-09-11T12:55:43.984622+0000","last_unstale":"2024-09-11T12:55:44.978275+0000","last_undegraded":"2024-09-11T12:55:44.978275+0000","last_fullsized":"2024-09-11T12:55:44.978275+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:31.205909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986039+0000","last_change":"2024-09-11T12:55:42.984135+0000","last_active":"2024-09-11T12:55:44.986039+0000","last_peered":"2024-09-11T12:55:44.986039+0000","last_clean":"2024-09-11T12:55:44.986039+0000","last_became_active":"2024-09-11T12:55:42.983707+0000","last_became_peered":"2024-09-11T12:55:42.983707+0000","last_unstale":"2024-09-11T12:55:44.986039+0000","last_undegraded":"2024-09-11T12:55:44.986039+0000","last_fullsized":"2024-09-11T12:55:44.986039+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:04:30.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985940+0000","last_change":"2024-09-11T12:55:43.979854+0000","last_active":"2024-09-11T12:55:44.985940+0000","last_peered":"2024-09-11T12:55:44.985940+0000","last_clean":"2024-09-11T12:55:44.985940+0000","last_became_active":"2024-09-11T12:55:43.979280+0000","last_became_peered":"2024-09-11T12:55:43.979280+0000","last_unstale":"2024-09-11T12:55:44.985940+0000","last_undegraded":"2024-09-11T12:55:44.985940+0000","last_fullsized":"2024-09-11T12:55:44.985940+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:00.233357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224001+0000","last_change":"2024-09-11T12:55:42.977219+0000","last_active":"2024-09-11T12:55:45.224001+0000","last_peered":"2024-09-11T12:55:45.224001+0000","last_clean":"2024-09-11T12:55:45.224001+0000","last_became_active":"2024-09-11T12:55:42.976893+0000","last_became_peered":"2024-09-11T12:55:42.976893+0000","last_unstale":"2024-09-11T12:55:45.224001+0000","last_undegraded":"2024-09-11T12:55:45.224001+0000","last_fullsized":"2024-09-11T12:55:45.224001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:30.643514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977277+0000","last_change":"2024-09-11T12:55:43.974761+0000","last_active":"2024-09-11T12:55:44.977277+0000","last_peered":"2024-09-11T12:55:44.977277+0000","last_clean":"2024-09-11T12:55:44.977277+0000","last_became_active":"2024-09-11T12:55:43.974452+0000","last_became_peered":"2024-09-11T12:55:43.974452+0000","last_unstale":"2024-09-11T12:55:44.977277+0000","last_undegraded":"2024-09-11T12:55:44.977277+0000","last_fullsized":"2024-09-11T12:55:44.977277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:56.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223230+0000","last_change":"2024-09-11T12:55:42.979667+0000","last_active":"2024-09-11T12:55:45.223230+0000","last_peered":"2024-09-11T12:55:45.223230+0000","last_clean":"2024-09-11T12:55:45.223230+0000","last_became_active":"2024-09-11T12:55:42.979403+0000","last_became_peered":"2024-09-11T12:55:42.979403+0000","last_unstale":"2024-09-11T12:55:45.223230+0000","last_undegraded":"2024-09-11T12:55:45.223230+0000","last_fullsized":"2024-09-11T12:55:45.223230+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:25.711891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223866+0000","last_change":"2024-09-11T12:55:43.983947+0000","last_active":"2024-09-11T12:55:45.223866+0000","last_peered":"2024-09-11T12:55:45.223866+0000","last_clean":"2024-09-11T12:55:45.223866+0000","last_became_active":"2024-09-11T12:55:43.982576+0000","last_became_peered":"2024-09-11T12:55:43.982576+0000","last_unstale":"2024-09-11T12:55:45.223866+0000","last_undegraded":"2024-09-11T12:55:45.223866+0000","last_fullsized":"2024-09-11T12:55:45.223866+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:31.412081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"128'3","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224404+0000","last_change":"2024-09-11T12:55:42.979029+0000","last_active":"2024-09-11T12:55:45.224404+0000","last_peered":"2024-09-11T12:55:45.224404+0000","last_clean":"2024-09-11T12:55:45.224404+0000","last_became_active":"2024-09-11T12:55:42.978741+0000","last_became_peered":"2024-09-11T12:55:42.978741+0000","last_unstale":"2024-09-11T12:55:45.224404+0000","last_undegraded":"2024-09-11T12:55:45.224404+0000","last_fullsized":"2024-09-11T12:55:45.224404+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:25.711891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":332,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986189+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:55:44.986189+0000","last_peered":"2024-09-11T12:55:44.986189+0000","last_clean":"2024-09-11T12:55:44.986189+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:55:44.986189+0000","last_undegraded":"2024-09-11T12:55:44.986189+0000","last_fullsized":"2024-09-11T12:55:44.986189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2199552,"data_stored":1291889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1622016,"data_stored":707315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":700891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1609728,"data_stored":707169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":707259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":702551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":701451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":2207744,"data_stored":1292883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:55:48.081 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 -- 172.21.15.78:0/3686146120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 msgr2=0x7ffa140736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 --2- 172.21.15.78:0/3686146120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa140736d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009840 tx=0x7ffa0001c6b0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 -- 172.21.15.78:0/3686146120 shutdown_connections 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 --2- 172.21.15.78:0/3686146120 >> v2:172.21.15.157:3300/0 conn(0x7ffa140b4db0 0x7ffa140b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 --2- 172.21.15.78:0/3686146120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa140736d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.258+0000 7ffa1a400640 1 -- 172.21.15.78:0/3686146120 >> 172.21.15.78:0/3686146120 conn(0x7ffa1406ca80 msgr2=0x7ffa140b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- 172.21.15.78:0/3686146120 shutdown_connections 2024-09-11T12:55:48.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- 172.21.15.78:0/3686146120 wait complete. 2024-09-11T12:55:48.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 Processor -- start 2024-09-11T12:55:48.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- start start 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa141fb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa14157ae0 0x7ffa140afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1406f350 con 0x7ffa140b4db0 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1406f1d0 con 0x7ffa14157ae0 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa1a400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1406f4d0 con 0x7ffa14070a00 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55540/0 (socket says 172.21.15.78:55540) 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 -- 172.21.15.78:0/2848619422 learned_addr learned my addr 172.21.15.78:0/2848619422 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa13400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa141fb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa13e00640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa14157ae0 0x7ffa140afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 -- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 msgr2=0x7ffa141fb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa141fb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 -- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa14157ae0 msgr2=0x7ffa140afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa14157ae0 0x7ffa140afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 -- 172.21.15.78:0/2848619422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa141f7620 con 0x7ffa140b4db0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.262+0000 7ffa12a00640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ffa0001c4e0 tx=0x7ffa00005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000c0070 con 0x7ffa140b4db0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa000aa880 con 0x7ffa140b4db0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa141fba20 con 0x7ffa140b4db0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa000ad050 con 0x7ffa140b4db0 2024-09-11T12:55:48.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa14156d80 con 0x7ffa140b4db0 2024-09-11T12:55:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa140736d0 con 0x7ffa140b4db0 2024-09-11T12:55:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7ffa000c3070 con 0x7ffa140b4db0 2024-09-11T12:55:48.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 0x7ff9f40740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7ffa0012ca10 con 0x7ffa140b4db0 2024-09-11T12:55:48.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa13400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 0x7ff9f40740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.266+0000 7ffa13400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 0x7ff9f40740e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001840 tx=0x7ffa04009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:48.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.270+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ffa00016330 con 0x7ffa140b4db0 2024-09-11T12:55:48.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.434+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ffa140ce2f0 con 0x7ff9f4071c20 2024-09-11T12:55:48.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.438+0000 7ffa0b400640 1 -- 172.21.15.78:0/2848619422 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7ffa140ce2f0 con 0x7ff9f4071c20 2024-09-11T12:55:48.443 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:48.444 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:55:48.445 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 msgr2=0x7ff9f40740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 0x7ff9f40740e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ffa04001840 tx=0x7ffa04009040 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 msgr2=0x7ffa141f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7ffa0001c4e0 tx=0x7ffa00005d90 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 shutdown_connections 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff9f4071c20 0x7ff9f40740e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ffa14157ae0 0x7ffa140afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ffa140b4db0 0x7ffa141f7db0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 --2- 172.21.15.78:0/2848619422 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ffa14070a00 0x7ffa141fb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 >> 172.21.15.78:0/2848619422 conn(0x7ffa1406ca80 msgr2=0x7ffa14153560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 shutdown_connections 2024-09-11T12:55:48.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.442+0000 7ffa1a400640 1 -- 172.21.15.78:0/2848619422 wait complete. 2024-09-11T12:55:48.464 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-09-11T12:55:46.495934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":78,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267420,"kb_used_data":13740,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754707300,"statfs":{"total":773094113280,"available":772820275200,"internally_reserved":0,"allocated":14069760,"data_stored":6811408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.548023"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976813+0000","last_change":"2024-09-11T12:55:43.977565+0000","last_active":"2024-09-11T12:55:44.976813+0000","last_peered":"2024-09-11T12:55:44.976813+0000","last_clean":"2024-09-11T12:55:44.976813+0000","last_became_active":"2024-09-11T12:55:43.977269+0000","last_became_peered":"2024-09-11T12:55:43.977269+0000","last_unstale":"2024-09-11T12:55:44.976813+0000","last_undegraded":"2024-09-11T12:55:44.976813+0000","last_fullsized":"2024-09-11T12:55:44.976813+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223561+0000","last_change":"2024-09-11T12:55:43.985671+0000","last_active":"2024-09-11T12:55:45.223561+0000","last_peered":"2024-09-11T12:55:45.223561+0000","last_clean":"2024-09-11T12:55:45.223561+0000","last_became_active":"2024-09-11T12:55:43.985355+0000","last_became_peered":"2024-09-11T12:55:43.985355+0000","last_unstale":"2024-09-11T12:55:45.223561+0000","last_undegraded":"2024-09-11T12:55:45.223561+0000","last_fullsized":"2024-09-11T12:55:45.223561+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:18.584208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224497+0000","last_change":"2024-09-11T12:55:43.991058+0000","last_active":"2024-09-11T12:55:45.224497+0000","last_peered":"2024-09-11T12:55:45.224497+0000","last_clean":"2024-09-11T12:55:45.224497+0000","last_became_active":"2024-09-11T12:55:43.990201+0000","last_became_peered":"2024-09-11T12:55:43.990201+0000","last_unstale":"2024-09-11T12:55:45.224497+0000","last_undegraded":"2024-09-11T12:55:45.224497+0000","last_fullsized":"2024-09-11T12:55:45.224497+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:52.644639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977603+0000","last_change":"2024-09-11T12:55:43.984007+0000","last_active":"2024-09-11T12:55:44.977603+0000","last_peered":"2024-09-11T12:55:44.977603+0000","last_clean":"2024-09-11T12:55:44.977603+0000","last_became_active":"2024-09-11T12:55:43.983648+0000","last_became_peered":"2024-09-11T12:55:43.983648+0000","last_unstale":"2024-09-11T12:55:44.977603+0000","last_undegraded":"2024-09-11T12:55:44.977603+0000","last_fullsized":"2024-09-11T12:55:44.977603+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:41.368659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978050+0000","last_change":"2024-09-11T12:55:43.975377+0000","last_active":"2024-09-11T12:55:44.978050+0000","last_peered":"2024-09-11T12:55:44.978050+0000","last_clean":"2024-09-11T12:55:44.978050+0000","last_became_active":"2024-09-11T12:55:43.975122+0000","last_became_peered":"2024-09-11T12:55:43.975122+0000","last_unstale":"2024-09-11T12:55:44.978050+0000","last_undegraded":"2024-09-11T12:55:44.978050+0000","last_fullsized":"2024-09-11T12:55:44.978050+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:18.719014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975867+0000","last_change":"2024-09-11T12:55:43.992153+0000","last_active":"2024-09-11T12:55:44.975867+0000","last_peered":"2024-09-11T12:55:44.975867+0000","last_clean":"2024-09-11T12:55:44.975867+0000","last_became_active":"2024-09-11T12:55:43.991579+0000","last_became_peered":"2024-09-11T12:55:43.991579+0000","last_unstale":"2024-09-11T12:55:44.975867+0000","last_undegraded":"2024-09-11T12:55:44.975867+0000","last_fullsized":"2024-09-11T12:55:44.975867+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:31.646495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223314+0000","last_change":"2024-09-11T12:55:43.980050+0000","last_active":"2024-09-11T12:55:45.223314+0000","last_peered":"2024-09-11T12:55:45.223314+0000","last_clean":"2024-09-11T12:55:45.223314+0000","last_became_active":"2024-09-11T12:55:43.979602+0000","last_became_peered":"2024-09-11T12:55:43.979602+0000","last_unstale":"2024-09-11T12:55:45.223314+0000","last_undegraded":"2024-09-11T12:55:45.223314+0000","last_fullsized":"2024-09-11T12:55:45.223314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:05.455335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975691+0000","last_change":"2024-09-11T12:55:43.971160+0000","last_active":"2024-09-11T12:55:44.975691+0000","last_peered":"2024-09-11T12:55:44.975691+0000","last_clean":"2024-09-11T12:55:44.975691+0000","last_became_active":"2024-09-11T12:55:43.970897+0000","last_became_peered":"2024-09-11T12:55:43.970897+0000","last_unstale":"2024-09-11T12:55:44.975691+0000","last_undegraded":"2024-09-11T12:55:44.975691+0000","last_fullsized":"2024-09-11T12:55:44.975691+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:38.522983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223551+0000","last_change":"2024-09-11T12:55:43.980848+0000","last_active":"2024-09-11T12:55:45.223551+0000","last_peered":"2024-09-11T12:55:45.223551+0000","last_clean":"2024-09-11T12:55:45.223551+0000","last_became_active":"2024-09-11T12:55:43.980326+0000","last_became_peered":"2024-09-11T12:55:43.980326+0000","last_unstale":"2024-09-11T12:55:45.223551+0000","last_undegraded":"2024-09-11T12:55:45.223551+0000","last_fullsized":"2024-09-11T12:55:45.223551+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:27.432044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986258+0000","last_change":"2024-09-11T12:55:43.992497+0000","last_active":"2024-09-11T12:55:44.986258+0000","last_peered":"2024-09-11T12:55:44.986258+0000","last_clean":"2024-09-11T12:55:44.986258+0000","last_became_active":"2024-09-11T12:55:43.988062+0000","last_became_peered":"2024-09-11T12:55:43.988062+0000","last_unstale":"2024-09-11T12:55:44.986258+0000","last_undegraded":"2024-09-11T12:55:44.986258+0000","last_fullsized":"2024-09-11T12:55:44.986258+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:21.789507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976429+0000","last_change":"2024-09-11T12:55:43.970481+0000","last_active":"2024-09-11T12:55:44.976429+0000","last_peered":"2024-09-11T12:55:44.976429+0000","last_clean":"2024-09-11T12:55:44.976429+0000","last_became_active":"2024-09-11T12:55:43.970224+0000","last_became_peered":"2024-09-11T12:55:43.970224+0000","last_unstale":"2024-09-11T12:55:44.976429+0000","last_undegraded":"2024-09-11T12:55:44.976429+0000","last_fullsized":"2024-09-11T12:55:44.976429+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:45.644370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986208+0000","last_change":"2024-09-11T12:55:43.986600+0000","last_active":"2024-09-11T12:55:44.986208+0000","last_peered":"2024-09-11T12:55:44.986208+0000","last_clean":"2024-09-11T12:55:44.986208+0000","last_became_active":"2024-09-11T12:55:43.986244+0000","last_became_peered":"2024-09-11T12:55:43.986244+0000","last_unstale":"2024-09-11T12:55:44.986208+0000","last_undegraded":"2024-09-11T12:55:44.986208+0000","last_fullsized":"2024-09-11T12:55:44.986208+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:25:49.562408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978232+0000","last_change":"2024-09-11T12:55:43.981300+0000","last_active":"2024-09-11T12:55:44.978232+0000","last_peered":"2024-09-11T12:55:44.978232+0000","last_clean":"2024-09-11T12:55:44.978232+0000","last_became_active":"2024-09-11T12:55:43.974143+0000","last_became_peered":"2024-09-11T12:55:43.974143+0000","last_unstale":"2024-09-11T12:55:44.978232+0000","last_undegraded":"2024-09-11T12:55:44.978232+0000","last_fullsized":"2024-09-11T12:55:44.978232+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:25.794816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977233+0000","last_change":"2024-09-11T12:55:43.980851+0000","last_active":"2024-09-11T12:55:44.977233+0000","last_peered":"2024-09-11T12:55:44.977233+0000","last_clean":"2024-09-11T12:55:44.977233+0000","last_became_active":"2024-09-11T12:55:43.980577+0000","last_became_peered":"2024-09-11T12:55:43.980577+0000","last_unstale":"2024-09-11T12:55:44.977233+0000","last_undegraded":"2024-09-11T12:55:44.977233+0000","last_fullsized":"2024-09-11T12:55:44.977233+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985858+0000","last_change":"2024-09-11T12:55:43.981336+0000","last_active":"2024-09-11T12:55:44.985858+0000","last_peered":"2024-09-11T12:55:44.985858+0000","last_clean":"2024-09-11T12:55:44.985858+0000","last_became_active":"2024-09-11T12:55:43.981003+0000","last_became_peered":"2024-09-11T12:55:43.981003+0000","last_unstale":"2024-09-11T12:55:44.985858+0000","last_undegraded":"2024-09-11T12:55:44.985858+0000","last_fullsized":"2024-09-11T12:55:44.985858+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:33:46.865796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223842+0000","last_change":"2024-09-11T12:55:43.976193+0000","last_active":"2024-09-11T12:55:45.223842+0000","last_peered":"2024-09-11T12:55:45.223842+0000","last_clean":"2024-09-11T12:55:45.223842+0000","last_became_active":"2024-09-11T12:55:43.975904+0000","last_became_peered":"2024-09-11T12:55:43.975904+0000","last_unstale":"2024-09-11T12:55:45.223842+0000","last_undegraded":"2024-09-11T12:55:45.223842+0000","last_fullsized":"2024-09-11T12:55:45.223842+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:51:49.906520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986487+0000","last_change":"2024-09-11T12:55:43.980864+0000","last_active":"2024-09-11T12:55:44.986487+0000","last_peered":"2024-09-11T12:55:44.986487+0000","last_clean":"2024-09-11T12:55:44.986487+0000","last_became_active":"2024-09-11T12:55:43.980595+0000","last_became_peered":"2024-09-11T12:55:43.980595+0000","last_unstale":"2024-09-11T12:55:44.986487+0000","last_undegraded":"2024-09-11T12:55:44.986487+0000","last_fullsized":"2024-09-11T12:55:44.986487+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:02.004537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224489+0000","last_change":"2024-09-11T12:55:44.003554+0000","last_active":"2024-09-11T12:55:45.224489+0000","last_peered":"2024-09-11T12:55:45.224489+0000","last_clean":"2024-09-11T12:55:45.224489+0000","last_became_active":"2024-09-11T12:55:44.003219+0000","last_became_peered":"2024-09-11T12:55:44.003219+0000","last_unstale":"2024-09-11T12:55:45.224489+0000","last_undegraded":"2024-09-11T12:55:45.224489+0000","last_fullsized":"2024-09-11T12:55:45.224489+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:41.368659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976599+0000","last_change":"2024-09-11T12:55:43.973759+0000","last_active":"2024-09-11T12:55:44.976599+0000","last_peered":"2024-09-11T12:55:44.976599+0000","last_clean":"2024-09-11T12:55:44.976599+0000","last_became_active":"2024-09-11T12:55:43.973454+0000","last_became_peered":"2024-09-11T12:55:43.973454+0000","last_unstale":"2024-09-11T12:55:44.976599+0000","last_undegraded":"2024-09-11T12:55:44.976599+0000","last_fullsized":"2024-09-11T12:55:44.976599+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224562+0000","last_change":"2024-09-11T12:55:43.988236+0000","last_active":"2024-09-11T12:55:45.224562+0000","last_peered":"2024-09-11T12:55:45.224562+0000","last_clean":"2024-09-11T12:55:45.224562+0000","last_became_active":"2024-09-11T12:55:43.986022+0000","last_became_peered":"2024-09-11T12:55:43.986022+0000","last_unstale":"2024-09-11T12:55:45.224562+0000","last_undegraded":"2024-09-11T12:55:45.224562+0000","last_fullsized":"2024-09-11T12:55:45.224562+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:34:27.497194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986275+0000","last_change":"2024-09-11T12:55:43.993802+0000","last_active":"2024-09-11T12:55:44.986275+0000","last_peered":"2024-09-11T12:55:44.986275+0000","last_clean":"2024-09-11T12:55:44.986275+0000","last_became_active":"2024-09-11T12:55:43.992280+0000","last_became_peered":"2024-09-11T12:55:43.992280+0000","last_unstale":"2024-09-11T12:55:44.986275+0000","last_undegraded":"2024-09-11T12:55:44.986275+0000","last_fullsized":"2024-09-11T12:55:44.986275+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:30:57.594268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977415+0000","last_change":"2024-09-11T12:55:43.979347+0000","last_active":"2024-09-11T12:55:44.977415+0000","last_peered":"2024-09-11T12:55:44.977415+0000","last_clean":"2024-09-11T12:55:44.977415+0000","last_became_active":"2024-09-11T12:55:43.978597+0000","last_became_peered":"2024-09-11T12:55:43.978597+0000","last_unstale":"2024-09-11T12:55:44.977415+0000","last_undegraded":"2024-09-11T12:55:44.977415+0000","last_fullsized":"2024-09-11T12:55:44.977415+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976871+0000","last_change":"2024-09-11T12:55:43.977059+0000","last_active":"2024-09-11T12:55:44.976871+0000","last_peered":"2024-09-11T12:55:44.976871+0000","last_clean":"2024-09-11T12:55:44.976871+0000","last_became_active":"2024-09-11T12:55:43.976747+0000","last_became_peered":"2024-09-11T12:55:43.976747+0000","last_unstale":"2024-09-11T12:55:44.976871+0000","last_undegraded":"2024-09-11T12:55:44.976871+0000","last_fullsized":"2024-09-11T12:55:44.976871+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:58.103566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985794+0000","last_change":"2024-09-11T12:55:43.992321+0000","last_active":"2024-09-11T12:55:44.985794+0000","last_peered":"2024-09-11T12:55:44.985794+0000","last_clean":"2024-09-11T12:55:44.985794+0000","last_became_active":"2024-09-11T12:55:43.988806+0000","last_became_peered":"2024-09-11T12:55:43.988806+0000","last_unstale":"2024-09-11T12:55:44.985794+0000","last_undegraded":"2024-09-11T12:55:44.985794+0000","last_fullsized":"2024-09-11T12:55:44.985794+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:03.053908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224640+0000","last_change":"2024-09-11T12:55:43.972802+0000","last_active":"2024-09-11T12:55:45.224640+0000","last_peered":"2024-09-11T12:55:45.224640+0000","last_clean":"2024-09-11T12:55:45.224640+0000","last_became_active":"2024-09-11T12:55:43.972403+0000","last_became_peered":"2024-09-11T12:55:43.972403+0000","last_unstale":"2024-09-11T12:55:45.224640+0000","last_undegraded":"2024-09-11T12:55:45.224640+0000","last_fullsized":"2024-09-11T12:55:45.224640+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:19:17.782400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976462+0000","last_change":"2024-09-11T12:55:43.977525+0000","last_active":"2024-09-11T12:55:44.976462+0000","last_peered":"2024-09-11T12:55:44.976462+0000","last_clean":"2024-09-11T12:55:44.976462+0000","last_became_active":"2024-09-11T12:55:43.976643+0000","last_became_peered":"2024-09-11T12:55:43.976643+0000","last_unstale":"2024-09-11T12:55:44.976462+0000","last_undegraded":"2024-09-11T12:55:44.976462+0000","last_fullsized":"2024-09-11T12:55:44.976462+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:40:58.103566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224615+0000","last_change":"2024-09-11T12:55:43.988985+0000","last_active":"2024-09-11T12:55:45.224615+0000","last_peered":"2024-09-11T12:55:45.224615+0000","last_clean":"2024-09-11T12:55:45.224615+0000","last_became_active":"2024-09-11T12:55:43.988706+0000","last_became_peered":"2024-09-11T12:55:43.988706+0000","last_unstale":"2024-09-11T12:55:45.224615+0000","last_undegraded":"2024-09-11T12:55:45.224615+0000","last_fullsized":"2024-09-11T12:55:45.224615+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:56:18.427244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223690+0000","last_change":"2024-09-11T12:55:43.984842+0000","last_active":"2024-09-11T12:55:45.223690+0000","last_peered":"2024-09-11T12:55:45.223690+0000","last_clean":"2024-09-11T12:55:45.223690+0000","last_became_active":"2024-09-11T12:55:43.984535+0000","last_became_peered":"2024-09-11T12:55:43.984535+0000","last_unstale":"2024-09-11T12:55:45.223690+0000","last_undegraded":"2024-09-11T12:55:45.223690+0000","last_fullsized":"2024-09-11T12:55:45.223690+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:27:30.900809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976058+0000","last_change":"2024-09-11T12:55:43.991912+0000","last_active":"2024-09-11T12:55:44.976058+0000","last_peered":"2024-09-11T12:55:44.976058+0000","last_clean":"2024-09-11T12:55:44.976058+0000","last_became_active":"2024-09-11T12:55:43.991558+0000","last_became_peered":"2024-09-11T12:55:43.991558+0000","last_unstale":"2024-09-11T12:55:44.976058+0000","last_undegraded":"2024-09-11T12:55:44.976058+0000","last_fullsized":"2024-09-11T12:55:44.976058+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:26.714872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977627+0000","last_change":"2024-09-11T12:55:43.979127+0000","last_active":"2024-09-11T12:55:44.977627+0000","last_peered":"2024-09-11T12:55:44.977627+0000","last_clean":"2024-09-11T12:55:44.977627+0000","last_became_active":"2024-09-11T12:55:43.978198+0000","last_became_peered":"2024-09-11T12:55:43.978198+0000","last_unstale":"2024-09-11T12:55:44.977627+0000","last_undegraded":"2024-09-11T12:55:44.977627+0000","last_fullsized":"2024-09-11T12:55:44.977627+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:37.156786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223345+0000","last_change":"2024-09-11T12:55:43.999683+0000","last_active":"2024-09-11T12:55:45.223345+0000","last_peered":"2024-09-11T12:55:45.223345+0000","last_clean":"2024-09-11T12:55:45.223345+0000","last_became_active":"2024-09-11T12:55:43.999482+0000","last_became_peered":"2024-09-11T12:55:43.999482+0000","last_unstale":"2024-09-11T12:55:45.223345+0000","last_undegraded":"2024-09-11T12:55:45.223345+0000","last_fullsized":"2024-09-11T12:55:45.223345+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:34.407007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224001+0000","last_change":"2024-09-11T12:55:43.987557+0000","last_active":"2024-09-11T12:55:45.224001+0000","last_peered":"2024-09-11T12:55:45.224001+0000","last_clean":"2024-09-11T12:55:45.224001+0000","last_became_active":"2024-09-11T12:55:43.987212+0000","last_became_peered":"2024-09-11T12:55:43.987212+0000","last_unstale":"2024-09-11T12:55:45.224001+0000","last_undegraded":"2024-09-11T12:55:45.224001+0000","last_fullsized":"2024-09-11T12:55:45.224001+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:53:00.972508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986331+0000","last_change":"2024-09-11T12:55:43.992744+0000","last_active":"2024-09-11T12:55:44.986331+0000","last_peered":"2024-09-11T12:55:44.986331+0000","last_clean":"2024-09-11T12:55:44.986331+0000","last_became_active":"2024-09-11T12:55:43.992136+0000","last_became_peered":"2024-09-11T12:55:43.992136+0000","last_unstale":"2024-09-11T12:55:44.986331+0000","last_undegraded":"2024-09-11T12:55:44.986331+0000","last_fullsized":"2024-09-11T12:55:44.986331+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:51:03.583636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"128'4","reported_seq":19,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986246+0000","last_change":"2024-09-11T12:55:42.989411+0000","last_active":"2024-09-11T12:55:44.986246+0000","last_peered":"2024-09-11T12:55:44.986246+0000","last_clean":"2024-09-11T12:55:44.986246+0000","last_became_active":"2024-09-11T12:55:42.989163+0000","last_became_peered":"2024-09-11T12:55:42.989163+0000","last_unstale":"2024-09-11T12:55:44.986246+0000","last_undegraded":"2024-09-11T12:55:44.986246+0000","last_fullsized":"2024-09-11T12:55:44.986246+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:41.860425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223719+0000","last_change":"2024-09-11T12:55:43.980618+0000","last_active":"2024-09-11T12:55:45.223719+0000","last_peered":"2024-09-11T12:55:45.223719+0000","last_clean":"2024-09-11T12:55:45.223719+0000","last_became_active":"2024-09-11T12:55:43.980080+0000","last_became_peered":"2024-09-11T12:55:43.980080+0000","last_unstale":"2024-09-11T12:55:45.223719+0000","last_undegraded":"2024-09-11T12:55:45.223719+0000","last_fullsized":"2024-09-11T12:55:45.223719+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:02.267109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223632+0000","last_change":"2024-09-11T12:55:42.979871+0000","last_active":"2024-09-11T12:55:45.223632+0000","last_peered":"2024-09-11T12:55:45.223632+0000","last_clean":"2024-09-11T12:55:45.223632+0000","last_became_active":"2024-09-11T12:55:42.979625+0000","last_became_peered":"2024-09-11T12:55:42.979625+0000","last_unstale":"2024-09-11T12:55:45.223632+0000","last_undegraded":"2024-09-11T12:55:45.223632+0000","last_fullsized":"2024-09-11T12:55:45.223632+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.329329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976265+0000","last_change":"2024-09-11T12:55:43.978570+0000","last_active":"2024-09-11T12:55:44.976265+0000","last_peered":"2024-09-11T12:55:44.976265+0000","last_clean":"2024-09-11T12:55:44.976265+0000","last_became_active":"2024-09-11T12:55:43.978265+0000","last_became_peered":"2024-09-11T12:55:43.978265+0000","last_unstale":"2024-09-11T12:55:44.976265+0000","last_undegraded":"2024-09-11T12:55:44.976265+0000","last_fullsized":"2024-09-11T12:55:44.976265+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:22.332310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977378+0000","last_change":"2024-09-11T12:55:42.977572+0000","last_active":"2024-09-11T12:55:44.977378+0000","last_peered":"2024-09-11T12:55:44.977378+0000","last_clean":"2024-09-11T12:55:44.977378+0000","last_became_active":"2024-09-11T12:55:42.976340+0000","last_became_peered":"2024-09-11T12:55:42.976340+0000","last_unstale":"2024-09-11T12:55:44.977378+0000","last_undegraded":"2024-09-11T12:55:44.977378+0000","last_fullsized":"2024-09-11T12:55:44.977378+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:45:21.202496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986055+0000","last_change":"2024-09-11T12:55:43.986598+0000","last_active":"2024-09-11T12:55:44.986055+0000","last_peered":"2024-09-11T12:55:44.986055+0000","last_clean":"2024-09-11T12:55:44.986055+0000","last_became_active":"2024-09-11T12:55:43.986274+0000","last_became_peered":"2024-09-11T12:55:43.986274+0000","last_unstale":"2024-09-11T12:55:44.986055+0000","last_undegraded":"2024-09-11T12:55:44.986055+0000","last_fullsized":"2024-09-11T12:55:44.986055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:45:42.438460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977844+0000","last_change":"2024-09-11T12:55:42.977792+0000","last_active":"2024-09-11T12:55:44.977844+0000","last_peered":"2024-09-11T12:55:44.977844+0000","last_clean":"2024-09-11T12:55:44.977844+0000","last_became_active":"2024-09-11T12:55:42.976635+0000","last_became_peered":"2024-09-11T12:55:42.976635+0000","last_unstale":"2024-09-11T12:55:44.977844+0000","last_undegraded":"2024-09-11T12:55:44.977844+0000","last_fullsized":"2024-09-11T12:55:44.977844+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:38:33.644510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223504+0000","last_change":"2024-09-11T12:55:44.001190+0000","last_active":"2024-09-11T12:55:45.223504+0000","last_peered":"2024-09-11T12:55:45.223504+0000","last_clean":"2024-09-11T12:55:45.223504+0000","last_became_active":"2024-09-11T12:55:44.000897+0000","last_became_peered":"2024-09-11T12:55:44.000897+0000","last_unstale":"2024-09-11T12:55:45.223504+0000","last_undegraded":"2024-09-11T12:55:45.223504+0000","last_fullsized":"2024-09-11T12:55:45.223504+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:54:42.863406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224326+0000","last_change":"2024-09-11T12:55:42.980956+0000","last_active":"2024-09-11T12:55:45.224326+0000","last_peered":"2024-09-11T12:55:45.224326+0000","last_clean":"2024-09-11T12:55:45.224326+0000","last_became_active":"2024-09-11T12:55:42.980215+0000","last_became_peered":"2024-09-11T12:55:42.980215+0000","last_unstale":"2024-09-11T12:55:45.224326+0000","last_undegraded":"2024-09-11T12:55:45.224326+0000","last_fullsized":"2024-09-11T12:55:45.224326+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:49.037607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977162+0000","last_change":"2024-09-11T12:55:43.994480+0000","last_active":"2024-09-11T12:55:44.977162+0000","last_peered":"2024-09-11T12:55:44.977162+0000","last_clean":"2024-09-11T12:55:44.977162+0000","last_became_active":"2024-09-11T12:55:43.993950+0000","last_became_peered":"2024-09-11T12:55:43.993950+0000","last_unstale":"2024-09-11T12:55:44.977162+0000","last_undegraded":"2024-09-11T12:55:44.977162+0000","last_fullsized":"2024-09-11T12:55:44.977162+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977492+0000","last_change":"2024-09-11T12:55:42.985294+0000","last_active":"2024-09-11T12:55:44.977492+0000","last_peered":"2024-09-11T12:55:44.977492+0000","last_clean":"2024-09-11T12:55:44.977492+0000","last_became_active":"2024-09-11T12:55:42.985010+0000","last_became_peered":"2024-09-11T12:55:42.985010+0000","last_unstale":"2024-09-11T12:55:44.977492+0000","last_undegraded":"2024-09-11T12:55:44.977492+0000","last_fullsized":"2024-09-11T12:55:44.977492+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.657167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223253+0000","last_change":"2024-09-11T12:55:43.993699+0000","last_active":"2024-09-11T12:55:45.223253+0000","last_peered":"2024-09-11T12:55:45.223253+0000","last_clean":"2024-09-11T12:55:45.223253+0000","last_became_active":"2024-09-11T12:55:43.993400+0000","last_became_peered":"2024-09-11T12:55:43.993400+0000","last_unstale":"2024-09-11T12:55:45.223253+0000","last_undegraded":"2024-09-11T12:55:45.223253+0000","last_fullsized":"2024-09-11T12:55:45.223253+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:38:54.698712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224334+0000","last_change":"2024-09-11T12:55:42.973097+0000","last_active":"2024-09-11T12:55:45.224334+0000","last_peered":"2024-09-11T12:55:45.224334+0000","last_clean":"2024-09-11T12:55:45.224334+0000","last_became_active":"2024-09-11T12:55:42.972799+0000","last_became_peered":"2024-09-11T12:55:42.972799+0000","last_unstale":"2024-09-11T12:55:45.224334+0000","last_undegraded":"2024-09-11T12:55:45.224334+0000","last_fullsized":"2024-09-11T12:55:45.224334+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:34:30.394530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977055+0000","last_change":"2024-09-11T12:55:43.968260+0000","last_active":"2024-09-11T12:55:44.977055+0000","last_peered":"2024-09-11T12:55:44.977055+0000","last_clean":"2024-09-11T12:55:44.977055+0000","last_became_active":"2024-09-11T12:55:43.967320+0000","last_became_peered":"2024-09-11T12:55:43.967320+0000","last_unstale":"2024-09-11T12:55:44.977055+0000","last_undegraded":"2024-09-11T12:55:44.977055+0000","last_fullsized":"2024-09-11T12:55:44.977055+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:26.757084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976838+0000","last_change":"2024-09-11T12:55:42.980180+0000","last_active":"2024-09-11T12:55:44.976838+0000","last_peered":"2024-09-11T12:55:44.976838+0000","last_clean":"2024-09-11T12:55:44.976838+0000","last_became_active":"2024-09-11T12:55:42.979883+0000","last_became_peered":"2024-09-11T12:55:42.979883+0000","last_unstale":"2024-09-11T12:55:44.976838+0000","last_undegraded":"2024-09-11T12:55:44.976838+0000","last_fullsized":"2024-09-11T12:55:44.976838+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:51:38.356612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977990+0000","last_change":"2024-09-11T12:55:43.982364+0000","last_active":"2024-09-11T12:55:44.977990+0000","last_peered":"2024-09-11T12:55:44.977990+0000","last_clean":"2024-09-11T12:55:44.977990+0000","last_became_active":"2024-09-11T12:55:43.982083+0000","last_became_peered":"2024-09-11T12:55:43.982083+0000","last_unstale":"2024-09-11T12:55:44.977990+0000","last_undegraded":"2024-09-11T12:55:44.977990+0000","last_fullsized":"2024-09-11T12:55:44.977990+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:40:20.736834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223814+0000","last_change":"2024-09-11T12:55:42.977280+0000","last_active":"2024-09-11T12:55:45.223814+0000","last_peered":"2024-09-11T12:55:45.223814+0000","last_clean":"2024-09-11T12:55:45.223814+0000","last_became_active":"2024-09-11T12:55:42.976993+0000","last_became_peered":"2024-09-11T12:55:42.976993+0000","last_unstale":"2024-09-11T12:55:45.223814+0000","last_undegraded":"2024-09-11T12:55:45.223814+0000","last_fullsized":"2024-09-11T12:55:45.223814+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:30.643514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986480+0000","last_change":"2024-09-11T12:55:43.983164+0000","last_active":"2024-09-11T12:55:44.986480+0000","last_peered":"2024-09-11T12:55:44.986480+0000","last_clean":"2024-09-11T12:55:44.986480+0000","last_became_active":"2024-09-11T12:55:43.980929+0000","last_became_peered":"2024-09-11T12:55:43.980929+0000","last_unstale":"2024-09-11T12:55:44.986480+0000","last_undegraded":"2024-09-11T12:55:44.986480+0000","last_fullsized":"2024-09-11T12:55:44.986480+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:01.008208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986498+0000","last_change":"2024-09-11T12:55:42.987541+0000","last_active":"2024-09-11T12:55:44.986498+0000","last_peered":"2024-09-11T12:55:44.986498+0000","last_clean":"2024-09-11T12:55:44.986498+0000","last_became_active":"2024-09-11T12:55:42.987263+0000","last_became_peered":"2024-09-11T12:55:42.987263+0000","last_unstale":"2024-09-11T12:55:44.986498+0000","last_undegraded":"2024-09-11T12:55:44.986498+0000","last_fullsized":"2024-09-11T12:55:44.986498+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:38:26.429063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224726+0000","last_change":"2024-09-11T12:55:43.972865+0000","last_active":"2024-09-11T12:55:45.224726+0000","last_peered":"2024-09-11T12:55:45.224726+0000","last_clean":"2024-09-11T12:55:45.224726+0000","last_became_active":"2024-09-11T12:55:43.972546+0000","last_became_peered":"2024-09-11T12:55:43.972546+0000","last_unstale":"2024-09-11T12:55:45.224726+0000","last_undegraded":"2024-09-11T12:55:45.224726+0000","last_fullsized":"2024-09-11T12:55:45.224726+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:58.464947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"128'6","reported_seq":21,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223787+0000","last_change":"2024-09-11T12:55:42.974745+0000","last_active":"2024-09-11T12:55:45.223787+0000","last_peered":"2024-09-11T12:55:45.223787+0000","last_clean":"2024-09-11T12:55:45.223787+0000","last_became_active":"2024-09-11T12:55:42.974463+0000","last_became_peered":"2024-09-11T12:55:42.974463+0000","last_unstale":"2024-09-11T12:55:45.223787+0000","last_undegraded":"2024-09-11T12:55:45.223787+0000","last_fullsized":"2024-09-11T12:55:45.223787+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:55.658763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224151+0000","last_change":"2024-09-11T12:55:43.984920+0000","last_active":"2024-09-11T12:55:45.224151+0000","last_peered":"2024-09-11T12:55:45.224151+0000","last_clean":"2024-09-11T12:55:45.224151+0000","last_became_active":"2024-09-11T12:55:43.984116+0000","last_became_peered":"2024-09-11T12:55:43.984116+0000","last_unstale":"2024-09-11T12:55:45.224151+0000","last_undegraded":"2024-09-11T12:55:45.224151+0000","last_fullsized":"2024-09-11T12:55:45.224151+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:47.350986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224172+0000","last_change":"2024-09-11T12:55:42.977504+0000","last_active":"2024-09-11T12:55:45.224172+0000","last_peered":"2024-09-11T12:55:45.224172+0000","last_clean":"2024-09-11T12:55:45.224172+0000","last_became_active":"2024-09-11T12:55:42.976950+0000","last_became_peered":"2024-09-11T12:55:42.976950+0000","last_unstale":"2024-09-11T12:55:45.224172+0000","last_undegraded":"2024-09-11T12:55:45.224172+0000","last_fullsized":"2024-09-11T12:55:45.224172+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:44.641389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223339+0000","last_change":"2024-09-11T12:55:43.980728+0000","last_active":"2024-09-11T12:55:45.223339+0000","last_peered":"2024-09-11T12:55:45.223339+0000","last_clean":"2024-09-11T12:55:45.223339+0000","last_became_active":"2024-09-11T12:55:43.980427+0000","last_became_peered":"2024-09-11T12:55:43.980427+0000","last_unstale":"2024-09-11T12:55:45.223339+0000","last_undegraded":"2024-09-11T12:55:45.223339+0000","last_fullsized":"2024-09-11T12:55:45.223339+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:11.512023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224082+0000","last_change":"2024-09-11T12:55:42.982353+0000","last_active":"2024-09-11T12:55:45.224082+0000","last_peered":"2024-09-11T12:55:45.224082+0000","last_clean":"2024-09-11T12:55:45.224082+0000","last_became_active":"2024-09-11T12:55:42.982017+0000","last_became_peered":"2024-09-11T12:55:42.982017+0000","last_unstale":"2024-09-11T12:55:45.224082+0000","last_undegraded":"2024-09-11T12:55:45.224082+0000","last_fullsized":"2024-09-11T12:55:45.224082+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:21.329329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976861+0000","last_change":"2024-09-11T12:55:43.978821+0000","last_active":"2024-09-11T12:55:44.976861+0000","last_peered":"2024-09-11T12:55:44.976861+0000","last_clean":"2024-09-11T12:55:44.976861+0000","last_became_active":"2024-09-11T12:55:43.974856+0000","last_became_peered":"2024-09-11T12:55:43.974856+0000","last_unstale":"2024-09-11T12:55:44.976861+0000","last_undegraded":"2024-09-11T12:55:44.976861+0000","last_fullsized":"2024-09-11T12:55:44.976861+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:53:09.427332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.975665+0000","last_change":"2024-09-11T12:55:42.975364+0000","last_active":"2024-09-11T12:55:44.975665+0000","last_peered":"2024-09-11T12:55:44.975665+0000","last_clean":"2024-09-11T12:55:44.975665+0000","last_became_active":"2024-09-11T12:55:42.974899+0000","last_became_peered":"2024-09-11T12:55:42.974899+0000","last_unstale":"2024-09-11T12:55:44.975665+0000","last_undegraded":"2024-09-11T12:55:44.975665+0000","last_fullsized":"2024-09-11T12:55:44.975665+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:23.707165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223974+0000","last_change":"2024-09-11T12:55:43.982628+0000","last_active":"2024-09-11T12:55:45.223974+0000","last_peered":"2024-09-11T12:55:45.223974+0000","last_clean":"2024-09-11T12:55:45.223974+0000","last_became_active":"2024-09-11T12:55:43.981674+0000","last_became_peered":"2024-09-11T12:55:43.981674+0000","last_unstale":"2024-09-11T12:55:45.223974+0000","last_undegraded":"2024-09-11T12:55:45.223974+0000","last_fullsized":"2024-09-11T12:55:45.223974+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"128'8","reported_seq":23,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224106+0000","last_change":"2024-09-11T12:55:42.970949+0000","last_active":"2024-09-11T12:55:45.224106+0000","last_peered":"2024-09-11T12:55:45.224106+0000","last_clean":"2024-09-11T12:55:45.224106+0000","last_became_active":"2024-09-11T12:55:42.970665+0000","last_became_peered":"2024-09-11T12:55:42.970665+0000","last_unstale":"2024-09-11T12:55:45.224106+0000","last_undegraded":"2024-09-11T12:55:45.224106+0000","last_fullsized":"2024-09-11T12:55:45.224106+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:43:46.411913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986405+0000","last_change":"2024-09-11T12:55:43.984443+0000","last_active":"2024-09-11T12:55:44.986405+0000","last_peered":"2024-09-11T12:55:44.986405+0000","last_clean":"2024-09-11T12:55:44.986405+0000","last_became_active":"2024-09-11T12:55:43.984175+0000","last_became_peered":"2024-09-11T12:55:43.984175+0000","last_unstale":"2024-09-11T12:55:44.986405+0000","last_undegraded":"2024-09-11T12:55:44.986405+0000","last_fullsized":"2024-09-11T12:55:44.986405+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:41:26.192735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977870+0000","last_change":"2024-09-11T12:55:42.975686+0000","last_active":"2024-09-11T12:55:44.977870+0000","last_peered":"2024-09-11T12:55:44.977870+0000","last_clean":"2024-09-11T12:55:44.977870+0000","last_became_active":"2024-09-11T12:55:42.975420+0000","last_became_peered":"2024-09-11T12:55:42.975420+0000","last_unstale":"2024-09-11T12:55:44.977870+0000","last_undegraded":"2024-09-11T12:55:44.977870+0000","last_fullsized":"2024-09-11T12:55:44.977870+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:35:05.100483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224663+0000","last_change":"2024-09-11T12:55:43.975522+0000","last_active":"2024-09-11T12:55:45.224663+0000","last_peered":"2024-09-11T12:55:45.224663+0000","last_clean":"2024-09-11T12:55:45.224663+0000","last_became_active":"2024-09-11T12:55:43.975093+0000","last_became_peered":"2024-09-11T12:55:43.975093+0000","last_unstale":"2024-09-11T12:55:45.224663+0000","last_undegraded":"2024-09-11T12:55:45.224663+0000","last_fullsized":"2024-09-11T12:55:45.224663+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:07:24.512562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223663+0000","last_change":"2024-09-11T12:55:42.978418+0000","last_active":"2024-09-11T12:55:45.223663+0000","last_peered":"2024-09-11T12:55:45.223663+0000","last_clean":"2024-09-11T12:55:45.223663+0000","last_became_active":"2024-09-11T12:55:42.978176+0000","last_became_peered":"2024-09-11T12:55:42.978176+0000","last_unstale":"2024-09-11T12:55:45.223663+0000","last_undegraded":"2024-09-11T12:55:45.223663+0000","last_fullsized":"2024-09-11T12:55:45.223663+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:44.641389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224314+0000","last_change":"2024-09-11T12:55:43.984556+0000","last_active":"2024-09-11T12:55:45.224314+0000","last_peered":"2024-09-11T12:55:45.224314+0000","last_clean":"2024-09-11T12:55:45.224314+0000","last_became_active":"2024-09-11T12:55:43.984195+0000","last_became_peered":"2024-09-11T12:55:43.984195+0000","last_unstale":"2024-09-11T12:55:45.224314+0000","last_undegraded":"2024-09-11T12:55:45.224314+0000","last_fullsized":"2024-09-11T12:55:45.224314+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:02:46.361717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"128'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224783+0000","last_change":"2024-09-11T12:55:42.984591+0000","last_active":"2024-09-11T12:55:45.224783+0000","last_peered":"2024-09-11T12:55:45.224783+0000","last_clean":"2024-09-11T12:55:45.224783+0000","last_became_active":"2024-09-11T12:55:42.984250+0000","last_became_peered":"2024-09-11T12:55:42.984250+0000","last_unstale":"2024-09-11T12:55:45.224783+0000","last_undegraded":"2024-09-11T12:55:45.224783+0000","last_fullsized":"2024-09-11T12:55:45.224783+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:00:23.101674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976453+0000","last_change":"2024-09-11T12:55:43.992153+0000","last_active":"2024-09-11T12:55:44.976453+0000","last_peered":"2024-09-11T12:55:44.976453+0000","last_clean":"2024-09-11T12:55:44.976453+0000","last_became_active":"2024-09-11T12:55:43.991825+0000","last_became_peered":"2024-09-11T12:55:43.991825+0000","last_unstale":"2024-09-11T12:55:44.976453+0000","last_undegraded":"2024-09-11T12:55:44.976453+0000","last_fullsized":"2024-09-11T12:55:44.976453+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:04.503928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223665+0000","last_change":"2024-09-11T12:55:42.977968+0000","last_active":"2024-09-11T12:55:45.223665+0000","last_peered":"2024-09-11T12:55:45.223665+0000","last_clean":"2024-09-11T12:55:45.223665+0000","last_became_active":"2024-09-11T12:55:42.977654+0000","last_became_peered":"2024-09-11T12:55:42.977654+0000","last_unstale":"2024-09-11T12:55:45.223665+0000","last_undegraded":"2024-09-11T12:55:45.223665+0000","last_fullsized":"2024-09-11T12:55:45.223665+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:11:37.520002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977788+0000","last_change":"2024-09-11T12:55:43.974832+0000","last_active":"2024-09-11T12:55:44.977788+0000","last_peered":"2024-09-11T12:55:44.977788+0000","last_clean":"2024-09-11T12:55:44.977788+0000","last_became_active":"2024-09-11T12:55:43.974552+0000","last_became_peered":"2024-09-11T12:55:43.974552+0000","last_unstale":"2024-09-11T12:55:44.977788+0000","last_undegraded":"2024-09-11T12:55:44.977788+0000","last_fullsized":"2024-09-11T12:55:44.977788+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:21:52.644639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223838+0000","last_change":"2024-09-11T12:55:42.972270+0000","last_active":"2024-09-11T12:55:45.223838+0000","last_peered":"2024-09-11T12:55:45.223838+0000","last_clean":"2024-09-11T12:55:45.223838+0000","last_became_active":"2024-09-11T12:55:42.971991+0000","last_became_peered":"2024-09-11T12:55:42.971991+0000","last_unstale":"2024-09-11T12:55:45.223838+0000","last_undegraded":"2024-09-11T12:55:45.223838+0000","last_fullsized":"2024-09-11T12:55:45.223838+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:57:27.433191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223741+0000","last_change":"2024-09-11T12:55:43.991023+0000","last_active":"2024-09-11T12:55:45.223741+0000","last_peered":"2024-09-11T12:55:45.223741+0000","last_clean":"2024-09-11T12:55:45.223741+0000","last_became_active":"2024-09-11T12:55:43.990722+0000","last_became_peered":"2024-09-11T12:55:43.990722+0000","last_unstale":"2024-09-11T12:55:45.223741+0000","last_undegraded":"2024-09-11T12:55:45.223741+0000","last_fullsized":"2024-09-11T12:55:45.223741+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:25.910972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985897+0000","last_change":"2024-09-11T12:55:42.984294+0000","last_active":"2024-09-11T12:55:44.985897+0000","last_peered":"2024-09-11T12:55:44.985897+0000","last_clean":"2024-09-11T12:55:44.985897+0000","last_became_active":"2024-09-11T12:55:42.984003+0000","last_became_peered":"2024-09-11T12:55:42.984003+0000","last_unstale":"2024-09-11T12:55:44.985897+0000","last_undegraded":"2024-09-11T12:55:44.985897+0000","last_fullsized":"2024-09-11T12:55:44.985897+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:10.509042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976708+0000","last_change":"2024-09-11T12:55:43.978798+0000","last_active":"2024-09-11T12:55:44.976708+0000","last_peered":"2024-09-11T12:55:44.976708+0000","last_clean":"2024-09-11T12:55:44.976708+0000","last_became_active":"2024-09-11T12:55:43.977850+0000","last_became_peered":"2024-09-11T12:55:43.977850+0000","last_unstale":"2024-09-11T12:55:44.976708+0000","last_undegraded":"2024-09-11T12:55:44.976708+0000","last_fullsized":"2024-09-11T12:55:44.976708+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:46:04.503928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985782+0000","last_change":"2024-09-11T12:55:42.987260+0000","last_active":"2024-09-11T12:55:44.985782+0000","last_peered":"2024-09-11T12:55:44.985782+0000","last_clean":"2024-09-11T12:55:44.985782+0000","last_became_active":"2024-09-11T12:55:42.986948+0000","last_became_peered":"2024-09-11T12:55:42.986948+0000","last_unstale":"2024-09-11T12:55:44.985782+0000","last_undegraded":"2024-09-11T12:55:44.985782+0000","last_fullsized":"2024-09-11T12:55:44.985782+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:38:53.695731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223291+0000","last_change":"2024-09-11T12:55:43.984987+0000","last_active":"2024-09-11T12:55:45.223291+0000","last_peered":"2024-09-11T12:55:45.223291+0000","last_clean":"2024-09-11T12:55:45.223291+0000","last_became_active":"2024-09-11T12:55:43.984435+0000","last_became_peered":"2024-09-11T12:55:43.984435+0000","last_unstale":"2024-09-11T12:55:45.223291+0000","last_undegraded":"2024-09-11T12:55:45.223291+0000","last_fullsized":"2024-09-11T12:55:45.223291+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:53:32.000015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223884+0000","last_change":"2024-09-11T12:55:42.979048+0000","last_active":"2024-09-11T12:55:45.223884+0000","last_peered":"2024-09-11T12:55:45.223884+0000","last_clean":"2024-09-11T12:55:45.223884+0000","last_became_active":"2024-09-11T12:55:42.978773+0000","last_became_peered":"2024-09-11T12:55:42.978773+0000","last_unstale":"2024-09-11T12:55:45.223884+0000","last_undegraded":"2024-09-11T12:55:45.223884+0000","last_fullsized":"2024-09-11T12:55:45.223884+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:03:59.232317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978189+0000","last_change":"2024-09-11T12:55:43.981120+0000","last_active":"2024-09-11T12:55:44.978189+0000","last_peered":"2024-09-11T12:55:44.978189+0000","last_clean":"2024-09-11T12:55:44.978189+0000","last_became_active":"2024-09-11T12:55:43.980784+0000","last_became_peered":"2024-09-11T12:55:43.980784+0000","last_unstale":"2024-09-11T12:55:44.978189+0000","last_undegraded":"2024-09-11T12:55:44.978189+0000","last_fullsized":"2024-09-11T12:55:44.978189+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:54.272393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"128'2","reported_seq":17,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986568+0000","last_change":"2024-09-11T12:55:42.986972+0000","last_active":"2024-09-11T12:55:44.986568+0000","last_peered":"2024-09-11T12:55:44.986568+0000","last_clean":"2024-09-11T12:55:44.986568+0000","last_became_active":"2024-09-11T12:55:42.986650+0000","last_became_peered":"2024-09-11T12:55:42.986650+0000","last_unstale":"2024-09-11T12:55:44.986568+0000","last_undegraded":"2024-09-11T12:55:44.986568+0000","last_fullsized":"2024-09-11T12:55:44.986568+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:33.404026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224490+0000","last_change":"2024-09-11T12:55:43.988925+0000","last_active":"2024-09-11T12:55:45.224490+0000","last_peered":"2024-09-11T12:55:45.224490+0000","last_clean":"2024-09-11T12:55:45.224490+0000","last_became_active":"2024-09-11T12:55:43.988602+0000","last_became_peered":"2024-09-11T12:55:43.988602+0000","last_unstale":"2024-09-11T12:55:45.224490+0000","last_undegraded":"2024-09-11T12:55:45.224490+0000","last_fullsized":"2024-09-11T12:55:45.224490+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:37.156786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"129'1","reported_seq":16,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.984621+0000","last_change":"2024-09-11T12:55:42.979120+0000","last_active":"2024-09-11T12:55:44.984621+0000","last_peered":"2024-09-11T12:55:44.984621+0000","last_clean":"2024-09-11T12:55:44.984621+0000","last_became_active":"2024-09-11T12:55:42.978842+0000","last_became_peered":"2024-09-11T12:55:42.978842+0000","last_unstale":"2024-09-11T12:55:44.984621+0000","last_undegraded":"2024-09-11T12:55:44.984621+0000","last_fullsized":"2024-09-11T12:55:44.984621+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:35:57.864818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223557+0000","last_change":"2024-09-11T12:55:43.981332+0000","last_active":"2024-09-11T12:55:45.223557+0000","last_peered":"2024-09-11T12:55:45.223557+0000","last_clean":"2024-09-11T12:55:45.223557+0000","last_became_active":"2024-09-11T12:55:43.981031+0000","last_became_peered":"2024-09-11T12:55:43.981031+0000","last_unstale":"2024-09-11T12:55:45.223557+0000","last_undegraded":"2024-09-11T12:55:45.223557+0000","last_fullsized":"2024-09-11T12:55:45.223557+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:04:31.686584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976670+0000","last_change":"2024-09-11T12:55:42.978896+0000","last_active":"2024-09-11T12:55:44.976670+0000","last_peered":"2024-09-11T12:55:44.976670+0000","last_clean":"2024-09-11T12:55:44.976670+0000","last_became_active":"2024-09-11T12:55:42.977806+0000","last_became_peered":"2024-09-11T12:55:42.977806+0000","last_unstale":"2024-09-11T12:55:44.976670+0000","last_undegraded":"2024-09-11T12:55:44.976670+0000","last_fullsized":"2024-09-11T12:55:44.976670+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:18:39.224334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976591+0000","last_change":"2024-09-11T12:55:43.978722+0000","last_active":"2024-09-11T12:55:44.976591+0000","last_peered":"2024-09-11T12:55:44.976591+0000","last_clean":"2024-09-11T12:55:44.976591+0000","last_became_active":"2024-09-11T12:55:43.978429+0000","last_became_peered":"2024-09-11T12:55:43.978429+0000","last_unstale":"2024-09-11T12:55:44.976591+0000","last_undegraded":"2024-09-11T12:55:44.976591+0000","last_fullsized":"2024-09-11T12:55:44.976591+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:20:56.968985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.976295+0000","last_change":"2024-09-11T12:55:42.977938+0000","last_active":"2024-09-11T12:55:44.976295+0000","last_peered":"2024-09-11T12:55:44.976295+0000","last_clean":"2024-09-11T12:55:44.976295+0000","last_became_active":"2024-09-11T12:55:42.977668+0000","last_became_peered":"2024-09-11T12:55:42.977668+0000","last_unstale":"2024-09-11T12:55:44.976295+0000","last_undegraded":"2024-09-11T12:55:44.976295+0000","last_fullsized":"2024-09-11T12:55:44.976295+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:11:30.109491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224363+0000","last_change":"2024-09-11T12:55:43.978822+0000","last_active":"2024-09-11T12:55:45.224363+0000","last_peered":"2024-09-11T12:55:45.224363+0000","last_clean":"2024-09-11T12:55:45.224363+0000","last_became_active":"2024-09-11T12:55:43.978513+0000","last_became_peered":"2024-09-11T12:55:43.978513+0000","last_unstale":"2024-09-11T12:55:45.224363+0000","last_undegraded":"2024-09-11T12:55:45.224363+0000","last_fullsized":"2024-09-11T12:55:45.224363+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:22:11.878156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223930+0000","last_change":"2024-09-11T12:55:42.976548+0000","last_active":"2024-09-11T12:55:45.223930+0000","last_peered":"2024-09-11T12:55:45.223930+0000","last_clean":"2024-09-11T12:55:45.223930+0000","last_became_active":"2024-09-11T12:55:42.976269+0000","last_became_peered":"2024-09-11T12:55:42.976269+0000","last_unstale":"2024-09-11T12:55:45.223930+0000","last_undegraded":"2024-09-11T12:55:45.223930+0000","last_fullsized":"2024-09-11T12:55:45.223930+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:47:35.655434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.978275+0000","last_change":"2024-09-11T12:55:43.984946+0000","last_active":"2024-09-11T12:55:44.978275+0000","last_peered":"2024-09-11T12:55:44.978275+0000","last_clean":"2024-09-11T12:55:44.978275+0000","last_became_active":"2024-09-11T12:55:43.984622+0000","last_became_peered":"2024-09-11T12:55:43.984622+0000","last_unstale":"2024-09-11T12:55:44.978275+0000","last_undegraded":"2024-09-11T12:55:44.978275+0000","last_fullsized":"2024-09-11T12:55:44.978275+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:17:31.205909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986039+0000","last_change":"2024-09-11T12:55:42.984135+0000","last_active":"2024-09-11T12:55:44.986039+0000","last_peered":"2024-09-11T12:55:44.986039+0000","last_clean":"2024-09-11T12:55:44.986039+0000","last_became_active":"2024-09-11T12:55:42.983707+0000","last_became_peered":"2024-09-11T12:55:42.983707+0000","last_unstale":"2024-09-11T12:55:44.986039+0000","last_undegraded":"2024-09-11T12:55:44.986039+0000","last_fullsized":"2024-09-11T12:55:44.986039+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:04:30.683603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.985940+0000","last_change":"2024-09-11T12:55:43.979854+0000","last_active":"2024-09-11T12:55:44.985940+0000","last_peered":"2024-09-11T12:55:44.985940+0000","last_clean":"2024-09-11T12:55:44.985940+0000","last_became_active":"2024-09-11T12:55:43.979280+0000","last_became_peered":"2024-09-11T12:55:43.979280+0000","last_unstale":"2024-09-11T12:55:44.985940+0000","last_undegraded":"2024-09-11T12:55:44.985940+0000","last_fullsized":"2024-09-11T12:55:44.985940+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:00.233357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224001+0000","last_change":"2024-09-11T12:55:42.977219+0000","last_active":"2024-09-11T12:55:45.224001+0000","last_peered":"2024-09-11T12:55:45.224001+0000","last_clean":"2024-09-11T12:55:45.224001+0000","last_became_active":"2024-09-11T12:55:42.976893+0000","last_became_peered":"2024-09-11T12:55:42.976893+0000","last_unstale":"2024-09-11T12:55:45.224001+0000","last_undegraded":"2024-09-11T12:55:45.224001+0000","last_fullsized":"2024-09-11T12:55:45.224001+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:30.643514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.977277+0000","last_change":"2024-09-11T12:55:43.974761+0000","last_active":"2024-09-11T12:55:44.977277+0000","last_peered":"2024-09-11T12:55:44.977277+0000","last_clean":"2024-09-11T12:55:44.977277+0000","last_became_active":"2024-09-11T12:55:43.974452+0000","last_became_peered":"2024-09-11T12:55:43.974452+0000","last_unstale":"2024-09-11T12:55:44.977277+0000","last_undegraded":"2024-09-11T12:55:44.977277+0000","last_fullsized":"2024-09-11T12:55:44.977277+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:31:56.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223230+0000","last_change":"2024-09-11T12:55:42.979667+0000","last_active":"2024-09-11T12:55:45.223230+0000","last_peered":"2024-09-11T12:55:45.223230+0000","last_clean":"2024-09-11T12:55:45.223230+0000","last_became_active":"2024-09-11T12:55:42.979403+0000","last_became_peered":"2024-09-11T12:55:42.979403+0000","last_unstale":"2024-09-11T12:55:45.223230+0000","last_undegraded":"2024-09-11T12:55:45.223230+0000","last_fullsized":"2024-09-11T12:55:45.223230+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:25.711891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.223866+0000","last_change":"2024-09-11T12:55:43.983947+0000","last_active":"2024-09-11T12:55:45.223866+0000","last_peered":"2024-09-11T12:55:45.223866+0000","last_clean":"2024-09-11T12:55:45.223866+0000","last_became_active":"2024-09-11T12:55:43.982576+0000","last_became_peered":"2024-09-11T12:55:43.982576+0000","last_unstale":"2024-09-11T12:55:45.223866+0000","last_undegraded":"2024-09-11T12:55:45.223866+0000","last_fullsized":"2024-09-11T12:55:45.223866+0000","mapping_epoch":127,"log_start":"0'0","ondisk_log_start":"0'0","created":127,"last_epoch_clean":128,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:42.947758+0000","last_clean_scrub_stamp":"2024-09-11T12:55:42.947758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:50:31.412081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"128'3","reported_seq":18,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:45.224404+0000","last_change":"2024-09-11T12:55:42.979029+0000","last_active":"2024-09-11T12:55:45.224404+0000","last_peered":"2024-09-11T12:55:45.224404+0000","last_clean":"2024-09-11T12:55:45.224404+0000","last_became_active":"2024-09-11T12:55:42.978741+0000","last_became_peered":"2024-09-11T12:55:42.978741+0000","last_unstale":"2024-09-11T12:55:45.224404+0000","last_undegraded":"2024-09-11T12:55:45.224404+0000","last_fullsized":"2024-09-11T12:55:45.224404+0000","mapping_epoch":126,"log_start":"0'0","ondisk_log_start":"0'0","created":126,"last_epoch_clean":127,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:55:41.944777+0000","last_clean_scrub_stamp":"2024-09-11T12:55:41.944777+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:25.711891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":332,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-09-11T12:55:44.986189+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:55:44.986189+0000","last_peered":"2024-09-11T12:55:44.986189+0000","last_clean":"2024-09-11T12:55:44.986189+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:55:44.986189+0000","last_undegraded":"2024-09-11T12:55:44.986189+0000","last_fullsized":"2024-09-11T12:55:44.986189+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509518,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33344,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338496,"statfs":{"total":96636764160,"available":96602619904,"internally_reserved":0,"allocated":2199552,"data_stored":1291889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32772,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339068,"statfs":{"total":96636764160,"available":96603205632,"internally_reserved":0,"allocated":1622016,"data_stored":707315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":700891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36924,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334916,"statfs":{"total":96636764160,"available":96598953984,"internally_reserved":0,"allocated":1609728,"data_stored":707169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":707259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32764,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339076,"statfs":{"total":96636764160,"available":96603213824,"internally_reserved":0,"allocated":1613824,"data_stored":702551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509518,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":701451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33348,"kb_used_data":2156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338492,"statfs":{"total":96636764160,"available":96602615808,"internally_reserved":0,"allocated":2207744,"data_stored":1292883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:55:48.468 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:55:48.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- 172.21.15.78:0/2303182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64070a40 msgr2=0x7f6d6415e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2303182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64070a40 0x7f6d6415e6f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009bc0 tx=0x7f6d5c01cba0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- 172.21.15.78:0/2303182294 shutdown_connections 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2303182294 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64070a40 0x7f6d6415e6f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- 172.21.15.78:0/2303182294 >> 172.21.15.78:0/2303182294 conn(0x7f6d640b2ec0 msgr2=0x7f6d640b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- 172.21.15.78:0/2303182294 shutdown_connections 2024-09-11T12:55:48.657 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- 172.21.15.78:0/2303182294 wait complete. 2024-09-11T12:55:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 Processor -- start 2024-09-11T12:55:48.658 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- start start 2024-09-11T12:55:48.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d64070a40 0x7f6d6412e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64140400 0x7f6d640aef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64073b60 con 0x7f6d640cf790 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6d640739e0 con 0x7f6d64140400 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d68a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6d64167e40 con 0x7f6d64070a40 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d61400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d61400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55546/0 (socket says 172.21.15.78:55546) 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d61400640 1 -- 172.21.15.78:0/2238812891 learned_addr learned my addr 172.21.15.78:0/2238812891 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d61e00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d64070a40 0x7f6d6412e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.654+0000 7f6d62800640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64140400 0x7f6d640aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 -- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d64070a40 msgr2=0x7f6d6412e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d64070a40 0x7f6d6412e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 -- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64140400 msgr2=0x7f6d640aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64140400 0x7f6d640aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 -- 172.21.15.78:0/2238812891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6412c320 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d61400640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c500 tx=0x7f6d5000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d500052d0 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d50005490 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50012850 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6412ecb0 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d6412f270 con 0x7f6d640cf790 2024-09-11T12:55:48.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d6415e6f0 con 0x7f6d640cf790 2024-09-11T12:55:48.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.658+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f6d500129f0 con 0x7f6d640cf790 2024-09-11T12:55:48.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.662+0000 7f6d5a000640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 0x7f6d440740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:48.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.662+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f6d50092330 con 0x7f6d640cf790 2024-09-11T12:55:48.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.662+0000 7f6d61e00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 0x7f6d440740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:48.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.662+0000 7f6d61e00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 0x7f6d440740e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d54001720 tx=0x7f6d54007c70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:48.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.666+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6d5005d330 con 0x7f6d640cf790 2024-09-11T12:55:48.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.890+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6d6415d7a0 con 0x7f6d640cf790 2024-09-11T12:55:48.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d5a000640 1 -- 172.21.15.78:0/2238812891 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) ==== 190+0+147 (secure 0 0 0) 0x7f6d50061880 con 0x7f6d640cf790 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:55:48.896 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:55:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 msgr2=0x7f6d440740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 0x7f6d440740e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f6d54001720 tx=0x7f6d54007c70 comp rx=0 tx=0).stop 2024-09-11T12:55:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 msgr2=0x7f6d640ae9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000c500 tx=0x7f6d5000c9c0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 shutdown_connections 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6d44071c20 0x7f6d440740e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6d64140400 0x7f6d640aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6d640cf790 0x7f6d640ae9f0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 --2- 172.21.15.78:0/2238812891 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6d64070a40 0x7f6d6412e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 >> 172.21.15.78:0/2238812891 conn(0x7f6d640b2ec0 msgr2=0x7f6d64073e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 shutdown_connections 2024-09-11T12:55:48.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:48.894+0000 7f6d68a00640 1 -- 172.21.15.78:0/2238812891 wait complete. 2024-09-11T12:55:48.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- 172.21.15.78:0/824122249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a0153f60 msgr2=0x7f80a015a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 --2- 172.21.15.78:0/824122249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a0153f60 0x7f80a015a170 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f80a00ccff0 tx=0x7f809001d080 comp rx=0 tx=0).stop 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- 172.21.15.78:0/824122249 shutdown_connections 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 --2- 172.21.15.78:0/824122249 >> v2:172.21.15.157:3300/0 conn(0x7f80a0154340 0x7f80a00b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 --2- 172.21.15.78:0/824122249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a0153f60 0x7f80a015a170 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- 172.21.15.78:0/824122249 >> 172.21.15.78:0/824122249 conn(0x7f80a0075610 msgr2=0x7f80a0075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- 172.21.15.78:0/824122249 shutdown_connections 2024-09-11T12:55:49.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- 172.21.15.78:0/824122249 wait complete. 2024-09-11T12:55:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 Processor -- start 2024-09-11T12:55:49.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.058+0000 7f80a5000640 1 -- start start 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a0153f60 0x7f80a01407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a0154340 0x7f80a015e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0166c80 con 0x7f80a015a6b0 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0166b00 con 0x7f80a0153f60 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f80a0166e00 con 0x7f80a0154340 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55572/0 (socket says 172.21.15.78:55572) 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 -- 172.21.15.78:0/1467359485 learned_addr learned my addr 172.21.15.78:0/1467359485 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809f400640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a0153f60 0x7f80a01407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f8097e00640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a0154340 0x7f80a015e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 -- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a0154340 msgr2=0x7f80a015e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a0154340 0x7f80a015e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 -- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a0153f60 msgr2=0x7f80a01407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a0153f60 0x7f80a01407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 -- 172.21.15.78:0/1467359485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a0140d00 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f809fe00640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f809800cce0 tx=0x7f809800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809800f450 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f809800f700 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809800fb50 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a015f1c0 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a015f780 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80a015a170 con 0x7f80a015a6b0 2024-09-11T12:55:49.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.062+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f809800fcf0 con 0x7f80a015a6b0 2024-09-11T12:55:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.066+0000 7f8097400640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 0x7f807c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.066+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f80980932f0 con 0x7f80a015a6b0 2024-09-11T12:55:49.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.066+0000 7f809f400640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 0x7f807c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.069 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.066+0000 7f809f400640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 0x7f807c0740e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80900ae000 tx=0x7f809001b8c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.070+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8098014030 con 0x7f80a015a6b0 2024-09-11T12:55:49.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f80a00697c0 con 0x7f80a015a6b0 2024-09-11T12:55:49.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f8097400640 1 -- 172.21.15.78:0/1467359485 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5227 (secure 0 0 0) 0x7f809805e150 con 0x7f80a015a6b0 2024-09-11T12:55:49.273 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:55:49.273 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":91,"btime":"2024-09-11T12:55:46:980902+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:55:49.273 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 91 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 msgr2=0x7f807c0740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 0x7f807c0740e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f80900ae000 tx=0x7f809001b8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 msgr2=0x7f80a00b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f809800cce0 tx=0x7f809800a620 comp rx=0 tx=0).stop 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 shutdown_connections 2024-09-11T12:55:49.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f80a015a6b0 0x7f80a00b0ab0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f80a0154340 0x7f80a015e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f807c071c20 0x7f807c0740e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 --2- 172.21.15.78:0/1467359485 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f80a0153f60 0x7f80a01407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 >> 172.21.15.78:0/1467359485 conn(0x7f80a0075610 msgr2=0x7f80a006eb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 shutdown_connections 2024-09-11T12:55:49.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.270+0000 7f80a5000640 1 -- 172.21.15.78:0/1467359485 wait complete. 2024-09-11T12:55:49.697 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:55:43.353328+0000', 'modified': '2024-09-11T12:55:46.366514+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7776}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7776': {'gid': 7776, 'name': 'a', 'rank': 0, 'incarnation': 87, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/4229548436', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6832', 'nonce': 4229548436}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 4229548436}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7776, 'qdb_cluster': [7776]} 2024-09-11T12:55:49.698 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:55:49.698 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-11T12:55:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- 172.21.15.78:0/2197944144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 msgr2=0x7fee340c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- 172.21.15.78:0/2197944144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee340c5a20 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fee24009840 tx=0x7fee2401c820 comp rx=0 tx=0).stop 2024-09-11T12:55:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- 172.21.15.78:0/2197944144 shutdown_connections 2024-09-11T12:55:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- 172.21.15.78:0/2197944144 >> v2:172.21.15.157:3300/0 conn(0x7fee3415b7c0 0x7fee3413dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- 172.21.15.78:0/2197944144 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee340c5a20 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- 172.21.15.78:0/2197944144 >> 172.21.15.78:0/2197944144 conn(0x7fee34073b70 msgr2=0x7fee34073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- 172.21.15.78:0/2197944144 shutdown_connections 2024-09-11T12:55:49.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- 172.21.15.78:0/2197944144 wait complete. 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 Processor -- start 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- start start 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee3413b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee3415b7c0 0x7fee3406faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee341678f0 con 0x7fee340c5f60 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee3413f060 con 0x7fee3415b7c0 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee3bc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee34167a70 con 0x7fee3413fb30 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee39200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee39200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55584/0 (socket says 172.21.15.78:55584) 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee39200640 1 -- 172.21.15.78:0/1631500612 learned_addr learned my addr 172.21.15.78:0/1631500612 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee33e00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee3413b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.842+0000 7fee39c00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee3415b7c0 0x7fee3406faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 -- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 msgr2=0x7fee3413b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee3413b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 -- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee3415b7c0 msgr2=0x7fee3406faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee3415b7c0 0x7fee3406faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 -- 172.21.15.78:0/1631500612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee340cc040 con 0x7fee340c5f60 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee39200640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fee2401a450 tx=0x7fee240a66d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee30c00640 1 -- 172.21.15.78:0/1631500612 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee240040c0 con 0x7fee340c5f60 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee30c00640 1 -- 172.21.15.78:0/1631500612 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee24004350 con 0x7fee340c5f60 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee30c00640 1 -- 172.21.15.78:0/1631500612 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24005280 con 0x7fee340c5f60 2024-09-11T12:55:49.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee341f8260 con 0x7fee340c5f60 2024-09-11T12:55:49.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee34159180 con 0x7fee340c5f60 2024-09-11T12:55:49.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.846+0000 7fee30c00640 1 -- 172.21.15.78:0/1631500612 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fee240c2020 con 0x7fee340c5f60 2024-09-11T12:55:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee30c00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 0x7fee100740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 Processor -- start 2024-09-11T12:55:49.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee30c00640 1 -- 172.21.15.78:0/1631500612 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fee2412d300 con 0x7fee340c5f60 2024-09-11T12:55:49.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- start start 2024-09-11T12:55:49.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee33e00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 0x7fee100740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee33e00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 0x7fee100740e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee3415ca50 tx=0x7fee1c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f4886c4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c53c0 0x55f488677e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4884d5270 con 0x55f488678340 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4884c4770 con 0x55f4886c53c0 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f4884c87b0 con 0x55f4886c4aa0 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55586/0 (socket says 172.21.15.78:55586) 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 -- 172.21.15.78:0/949953507 learned_addr learned my addr 172.21.15.78:0/949953507 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39200640 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f4886c4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee33e00640 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c53c0 0x55f488677e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 -- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 msgr2=0x55f4886c4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f4886c4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 -- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c53c0 msgr2=0x55f488677e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c53c0 0x55f488677e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 -- 172.21.15.78:0/949953507 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f488463f90 con 0x55f488678340 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee39c00640 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009f60 tx=0x7fee2c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27d6f6b08495e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee29600640 1 -- 172.21.15.78:0/949953507 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00d8b0 con 0x55f488678340 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee29600640 1 -- 172.21.15.78:0/949953507 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee2c01c070 con 0x55f488678340 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee29600640 1 -- 172.21.15.78:0/949953507 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00dc90 con 0x55f488678340 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- 172.21.15.78:0/949953507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 msgr2=0x55f488678720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fee2c009f60 tx=0x7fee2c0095c0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- 172.21.15.78:0/949953507 shutdown_connections 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f488678340 0x55f488678720 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c53c0 0x55f488677e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 --2- 172.21.15.78:0/949953507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f4886c4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- 172.21.15.78:0/949953507 >> 172.21.15.78:0/949953507 conn(0x55f4886a5b60 msgr2=0x55f4886a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- 172.21.15.78:0/949953507 shutdown_connections 2024-09-11T12:55:49.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.850+0000 7fee3f053000 1 -- 172.21.15.78:0/949953507 wait complete. 2024-09-11T12:55:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- start start 2024-09-11T12:55:49.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 Processor -- start 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488685d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c53c0 0x55f4886867d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f488594290 con 0x55f4886c4aa0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f488470ab0 con 0x55f488678340 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f48846e170 con 0x55f4886c53c0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 10 client.0 _ll_register_callbacks cb 0x55f488665830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55588/0 (socket says 172.21.15.78:55588) 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 -- 172.21.15.78:0/892017922 learned_addr learned my addr 172.21.15.78:0/892017922 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee39200640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488685d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee39c00640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c53c0 0x55f4886867d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 -- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c53c0 msgr2=0x55f4886867d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c53c0 0x55f4886867d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 -- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 msgr2=0x55f488685d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488685d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 -- 172.21.15.78:0/892017922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4885496f0 con 0x55f4886c4aa0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee33e00640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0146d0 tx=0x7fee1c000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc61d2cf032c7d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee1c0135c0 con 0x55f4886c4aa0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fee1c013760 con 0x55f4886c4aa0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f48845df10 con 0x55f4886c4aa0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fee1c02e050 con 0x55f4886c4aa0 2024-09-11T12:55:49.863 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 10 client.7905 fetch_fsmap learned FSMap version 91 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f488520c40 con 0x55f4886c4aa0 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f48845df10 con 0x55f4886c4aa0 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fee1c031020 con 0x55f4886c4aa0 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 10 client.7905 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 10 client.7905 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 20 client.7905 populate_metadata read hostname 'smithi078' 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f488687480 0x55f4886d6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 4 client.7905 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 --> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4884d71c0 con 0x55f488687480 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.854+0000 7fee39200640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f488687480 0x55f4886d6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee22000640 10 client.7905 ms_handle_connect on v2:172.21.15.78:6832/4229548436 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee39200640 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f488687480 0x55f4886d6b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e722f50fb5d0c4 server_cookie=775f7b8eddd5a707 in_seq=0 out_seq=0 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee22000640 1 -- 172.21.15.78:0/892017922 <== mds.0 v2:172.21.15.78:6832/4229548436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f4884d71c0 con 0x55f488687480 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee22000640 10 client.7905 handle_command_reply: tid=0 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 client.7905 shutdown 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 msgr2=0x55f488686290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7fee1c0146d0 tx=0x7fee1c000ee0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 shutdown_connections 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f488687480 0x55f4886d6b50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c53c0 0x55f4886867d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee22000640 0 client.7905 ms_handle_reset on v2:172.21.15.78:6832/4229548436 2024-09-11T12:55:49.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c4aa0 0x55f488686290 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 --2- 172.21.15.78:0/892017922 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488685d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 >> 172.21.15.78:0/892017922 conn(0x55f4886a5b60 msgr2=0x55f488677500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 shutdown_connections 2024-09-11T12:55:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 1 -- 172.21.15.78:0/892017922 wait complete. 2024-09-11T12:55:49.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.858+0000 7fee3f053000 20 client.7905 trim_cache size 0 max 16384 2024-09-11T12:55:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 Processor -- start 2024-09-11T12:55:49.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- start start 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488735240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f488735780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fee34202cf0 con 0x55f4886c53c0 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fee341fac10 con 0x55f488678340 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fee341fb260 con 0x55f4886c4aa0 2024-09-11T12:55:49.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55594/0 (socket says 172.21.15.78:55594) 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 -- 172.21.15.78:0/1146079447 learned_addr learned my addr 172.21.15.78:0/1146079447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39200640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488735240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee33e00640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f488735780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 -- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 msgr2=0x55f488735780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f488735780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 -- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 msgr2=0x55f488735240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488735240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 -- 172.21.15.78:0/1146079447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f488479e70 con 0x55f4886c53c0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39200640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488735240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee39c00640 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fee2c018000 tx=0x7fee2c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d9b2524d30dc1c server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee28c00640 1 -- 172.21.15.78:0/1146079447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00d8b0 con 0x55f4886c53c0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee28c00640 1 -- 172.21.15.78:0/1146079447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee2c01c040 con 0x55f4886c53c0 2024-09-11T12:55:49.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee28c00640 1 -- 172.21.15.78:0/1146079447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00dd10 con 0x55f4886c53c0 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- 172.21.15.78:0/1146079447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 msgr2=0x55f488735cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7fee2c018000 tx=0x7fee2c004a50 comp rx=0 tx=0).stop 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- 172.21.15.78:0/1146079447 shutdown_connections 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f488735cc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f4886c4aa0 0x55f488735780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 --2- 172.21.15.78:0/1146079447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f488678340 0x55f488735240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- 172.21.15.78:0/1146079447 >> 172.21.15.78:0/1146079447 conn(0x55f4886a5b60 msgr2=0x55f48867cfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- 172.21.15.78:0/1146079447 shutdown_connections 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.878+0000 7fee3f053000 1 -- 172.21.15.78:0/1146079447 wait complete. 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- start start 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 Processor -- start 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f488678340 0x55f4886d9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c4aa0 0x55f4886da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f4884ccde0 con 0x55f4886c53c0 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f4884c7ac0 con 0x55f4886c4aa0 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f4884d7ea0 con 0x55f488678340 2024-09-11T12:55:49.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 10 client.0 _ll_register_callbacks cb 0x7fee1c017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55606/0 (socket says 172.21.15.78:55606) 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 -- 172.21.15.78:0/3317270693 learned_addr learned my addr 172.21.15.78:0/3317270693 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39200640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f488678340 0x55f4886d9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee33e00640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c4aa0 0x55f4886da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 -- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f488678340 msgr2=0x55f4886d9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f488678340 0x55f4886d9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 -- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c4aa0 msgr2=0x55f4886da2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c4aa0 0x55f4886da2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 -- 172.21.15.78:0/3317270693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48844ebf0 con 0x55f4886c53c0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39c00640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000f90 tx=0x7fee2c00d8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb981f48f23c73e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:55:49.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c004b30 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fee2c004cd0 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee2c01c040 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fee2c03c050 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 10 client.7911 fetch_fsmap learned FSMap version 91 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55f48846ac60 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee2c01c040 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fee2c03f020 con 0x55f4886c53c0 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 10 client.7911 fetch_fsmap finished waiting for FSMap version 91 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 10 client.7911 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 20 client.7911 populate_metadata read hostname 'smithi078' 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f4886f5800 0x55f4887390e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 4 client.7911 mds_command: new command op to 7776 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:55:49.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 --> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] -- command(tid 0: {"prefix": "status"}) -- 0x55f4884d2c60 con 0x55f4886f5800 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39200640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f4886f5800 0x55f4887390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee39200640 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f4886f5800 0x55f4887390e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d71c68a837c7e759 server_cookie=ac4459b4c5ac0991 in_seq=0 out_seq=0 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 10 client.7911 ms_handle_connect on v2:172.21.15.78:6832/4229548436 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 1 -- 172.21.15.78:0/3317270693 <== mds.0 v2:172.21.15.78:6832/4229548436 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4884d2c60 con 0x55f4886f5800 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee22000640 10 client.7911 handle_command_reply: tid=0 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.882+0000 7fee3f053000 1 client.7911 shutdown 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 msgr2=0x55f4886f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000f90 tx=0x7fee2c00d8e0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 shutdown_connections 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:6832/4229548436,v1:172.21.15.78:6835/4229548436] conn(0x55f4886f5800 0x55f4887390e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f4886c53c0 0x55f4886f4e40 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f4886c4aa0 0x55f4886da2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 --2- 172.21.15.78:0/3317270693 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f488678340 0x55f4886d9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee22000640 0 client.7911 ms_handle_reset on v2:172.21.15.78:6832/4229548436 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 >> 172.21.15.78:0/3317270693 conn(0x55f4886a5b60 msgr2=0x55f4886a3e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 shutdown_connections 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 1 -- 172.21.15.78:0/3317270693 wait complete. 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3f053000 20 client.7911 trim_cache size 0 max 16384 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:55:49.893 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 7776, 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9167152539999996, 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 91, 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 129, 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 129, 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.423542024 2024-09-11T12:55:49.894 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:55:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 msgr2=0x7fee100740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 0x7fee100740e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fee3415ca50 tx=0x7fee1c009040 comp rx=0 tx=0).stop 2024-09-11T12:55:49.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 msgr2=0x7fee34158af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7fee2401a450 tx=0x7fee240a66d0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 shutdown_connections 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fee10071c20 0x7fee100740e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fee3415b7c0 0x7fee3406faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fee3413fb30 0x7fee3413b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 --2- 172.21.15.78:0/1631500612 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fee340c5f60 0x7fee34158af0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 >> 172.21.15.78:0/1631500612 conn(0x7fee34073b70 msgr2=0x7fee340b3a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 shutdown_connections 2024-09-11T12:55:49.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:49.886+0000 7fee3bc00640 1 -- 172.21.15.78:0/1631500612 wait complete. 2024-09-11T12:55:49.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:55:50.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- 172.21.15.78:0/428836422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 msgr2=0x7f95b40736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 --2- 172.21.15.78:0/428836422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b40736d0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f95ac00ad90 tx=0x7f95ac01dc20 comp rx=0 tx=0).stop 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- 172.21.15.78:0/428836422 shutdown_connections 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 --2- 172.21.15.78:0/428836422 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b40736d0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- 172.21.15.78:0/428836422 >> 172.21.15.78:0/428836422 conn(0x7f95b406ca80 msgr2=0x7f95b40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- 172.21.15.78:0/428836422 shutdown_connections 2024-09-11T12:55:50.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- 172.21.15.78:0/428836422 wait complete. 2024-09-11T12:55:50.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 Processor -- start 2024-09-11T12:55:50.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.070+0000 7f95b9a00640 1 -- start start 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b4070a00 0x7f95b415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 0x7f95b415d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95b4167f20 con 0x7f95b40b54c0 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95b406f760 con 0x7f95b4070a00 2024-09-11T12:55:50.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95b41680a0 con 0x7f95b4157ae0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55630/0 (socket says 172.21.15.78:55630) 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 -- 172.21.15.78:0/3226844784 learned_addr learned my addr 172.21.15.78:0/3226844784 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b2a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b4070a00 0x7f95b415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b3400640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 0x7f95b415d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 -- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 msgr2=0x7f95b415d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 0x7f95b415d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 -- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b4070a00 msgr2=0x7f95b415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b4070a00 0x7f95b415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 -- 172.21.15.78:0/3226844784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b41f7dc0 con 0x7f95b40b54c0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b3400640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 0x7f95b415d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95abe00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c530 tx=0x7f959c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:50.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c017070 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f959c0053c0 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959c012890 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b41f80a0 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b41f8660 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b40736d0 con 0x7f95b40b54c0 2024-09-11T12:55:50.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.074+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f959c01a070 con 0x7f95b40b54c0 2024-09-11T12:55:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.078+0000 7f95aaa00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 0x7f95900740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.078+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f959c091a70 con 0x7f95b40b54c0 2024-09-11T12:55:50.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.078+0000 7f95b2a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 0x7f95900740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:50.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.078+0000 7f95b2a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 0x7f95900740e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007fd0 tx=0x7f95a4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:50.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.082+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f959c05ca70 con 0x7f95b40b54c0 2024-09-11T12:55:50.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f95b415b1a0 con 0x7f95b40b54c0 2024-09-11T12:55:50.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95aaa00640 1 -- 172.21.15.78:0/3226844784 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7f959c065300 con 0x7f95b40b54c0 2024-09-11T12:55:50.285 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:55:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 msgr2=0x7f95900740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:50.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 0x7f95900740e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f95a4007fd0 tx=0x7f95a4009040 comp rx=0 tx=0).stop 2024-09-11T12:55:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 msgr2=0x7f95b4142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f959c00c530 tx=0x7f959c00c9f0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 shutdown_connections 2024-09-11T12:55:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b4157ae0 0x7f95b415d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b40b54c0 0x7f95b4142900 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9590071c20 0x7f95900740e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 --2- 172.21.15.78:0/3226844784 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b4070a00 0x7f95b415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 >> 172.21.15.78:0/3226844784 conn(0x7f95b406ca80 msgr2=0x7f95b40cea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 shutdown_connections 2024-09-11T12:55:50.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50.282+0000 7f95b9a00640 1 -- 172.21.15.78:0/3226844784 wait complete. 2024-09-11T12:55:50.306 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:55:50.306 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:55:50.306 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:55:50.306 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:55:50.306 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:50.307 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:55:50.307 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:55:50.307 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:55:50.307 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:50.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.308 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:55:50.317 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:55:50.318 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42084sec preferred_lft 42084sec 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.319 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:55:50.320 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.321 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:50.321 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:55:50.321 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:55:50.321 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:50.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.322 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:55:50.381 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:55:50.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.384 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:55:50.442 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:55:50.443 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:55:50.443 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:50.443 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:50.443 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:50.443 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:55:50.443 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:55:50.443 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:55:50.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:55:50.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:55:50.444 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:55:50.444 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:55:50.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:55:50.569 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:50.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.576 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:55:50.585 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:55:50.585 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:55:50.585 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:50.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:55:50.647 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:50.647 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:50.695 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:55:50.695 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:50.695 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:50.747 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:55:50.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:50.836 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:50.836 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:50.848 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:50.848 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:50.848 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:55:50.848 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:55:50.848 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:55:50.852 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:55:50.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:50.852 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:55:50.873 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:55:50.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:55:50.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:50.983 INFO:teuthology.orchestra.run.smithi078.stdout:client7480 2024-09-11T12:55:50.983 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:55:50.983 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:50.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:55:51.176 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- 172.21.15.78:0/2375870520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae0157ae0 msgr2=0x7f7ae0070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 --2- 172.21.15.78:0/2375870520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0070a00 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400ad90 tx=0x7f7ad401dc20 comp rx=0 tx=0).stop 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- 172.21.15.78:0/2375870520 shutdown_connections 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 --2- 172.21.15.78:0/2375870520 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0070a00 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- 172.21.15.78:0/2375870520 >> 172.21.15.78:0/2375870520 conn(0x7f7ae006ca80 msgr2=0x7f7ae00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- 172.21.15.78:0/2375870520 shutdown_connections 2024-09-11T12:55:51.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- 172.21.15.78:0/2375870520 wait complete. 2024-09-11T12:55:51.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 Processor -- start 2024-09-11T12:55:51.178 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- start start 2024-09-11T12:55:51.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 0x7f7ae0151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:51.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:51.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:51.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae01680a0 con 0x7f7ae00b4db0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ae0167f20 con 0x7f7ae0157ae0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae8600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ae0168220 con 0x7f7ae00736d0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae5200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55636/0 (socket says 172.21.15.78:55636) 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae5200640 1 -- 172.21.15.78:0/482430496 learned_addr learned my addr 172.21.15.78:0/482430496 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae5c00640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 0x7f7ae0151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.174+0000 7f7ae6600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 -- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 msgr2=0x7f7ae0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 0x7f7ae0151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 -- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae0157ae0 msgr2=0x7f7ae0155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 -- 172.21.15.78:0/482430496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ae0148e30 con 0x7f7ae00b4db0 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5c00640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 0x7f7ae0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:51.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae5200640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c3c0 tx=0x7f7acc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc017070 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7acc0053c0 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7acc003860 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ae00ae840 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ae015c390 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ae015c6d0 con 0x7f7ae00b4db0 2024-09-11T12:55:51.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f7acc01a070 con 0x7f7ae00b4db0 2024-09-11T12:55:51.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 0x7f7ac00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:51.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.178+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f7acc091b10 con 0x7f7ae00b4db0 2024-09-11T12:55:51.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.182+0000 7f7ae5c00640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 0x7f7ac00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:51.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.182+0000 7f7ae5c00640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 0x7f7ac00740e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b0000 tx=0x7f7ad401c460 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:51.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.182+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7ae015c6d0 con 0x7f7ae00b4db0 2024-09-11T12:55:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7ae01598d0 con 0x7f7ae00b4db0 2024-09-11T12:55:51.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7add600640 1 -- 172.21.15.78:0/482430496 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v91) ==== 53+0+73 (secure 0 0 0) 0x7f7acc093e20 con 0x7f7ae00b4db0 2024-09-11T12:55:51.389 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:55:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 msgr2=0x7f7ac00740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:51.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 0x7f7ac00740e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b0000 tx=0x7f7ad401c460 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 msgr2=0x7f7ae01488f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f7acc00c3c0 tx=0x7f7acc00c880 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 shutdown_connections 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7ac0071c20 0x7f7ac00740e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7ae0157ae0 0x7f7ae0155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ae00b4db0 0x7f7ae01488f0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 --2- 172.21.15.78:0/482430496 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ae00736d0 0x7f7ae0151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 >> 172.21.15.78:0/482430496 conn(0x7f7ae006ca80 msgr2=0x7f7ae0072f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 shutdown_connections 2024-09-11T12:55:51.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:51.386+0000 7f7ae8600640 1 -- 172.21.15.78:0/482430496 wait complete. 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:51.411 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:55:51.412 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:55:51.412 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:55:51.412 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:51.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.412 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.423 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42092sec preferred_lft 42092sec 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:55:51.424 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:55:51.425 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:55:51.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.426 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:55:51.490 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:55:51.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.493 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:55:51.546 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:55:51.546 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:55:51.546 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:55:51.546 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:55:51.546 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:55:51.547 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:55:51.547 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:55:51.646 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:51 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:55:51.690 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:51.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.697 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:55:51.705 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:55:51.706 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:55:51.706 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:51.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.707 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:55:51.774 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:51.775 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:51.826 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:55:51.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:55:51.827 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:51.878 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:55:51.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.878 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:51.966 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:51.967 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:51.980 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:55:51.980 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:55:51.980 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:55:51.980 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:55:51.980 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:55:51.985 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:55:51.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:55:51.985 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:55:52.005 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:55:52.093 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:52 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:55:52.097 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:55:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:55:52.103 INFO:teuthology.orchestra.run.smithi157.stdout:client7932 2024-09-11T12:55:52.104 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:55:52.104 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:55:52.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 -- 172.21.15.78:0/3334775518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3300b7950 msgr2=0x7fd3300b7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 --2- 172.21.15.78:0/3334775518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3300b7950 0x7fd3300b7d30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd320009c80 tx=0x7fd32001cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 -- 172.21.15.78:0/3334775518 shutdown_connections 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 --2- 172.21.15.78:0/3334775518 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd330070e50 0x7fd33006e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 --2- 172.21.15.78:0/3334775518 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd3300b7950 0x7fd3300b7d30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 -- 172.21.15.78:0/3334775518 >> 172.21.15.78:0/3334775518 conn(0x7fd3300ce980 msgr2=0x7fd3300ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 -- 172.21.15.78:0/3334775518 shutdown_connections 2024-09-11T12:55:52.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.302+0000 7fd335000640 1 -- 172.21.15.78:0/3334775518 wait complete. 2024-09-11T12:55:52.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 Processor -- start 2024-09-11T12:55:52.308 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- start start 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3300b4520 0x7fd33014f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300b7950 0x7fd33014fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd330168e10 con 0x7fd330070e50 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd330168c90 con 0x7fd3300b7950 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd330168f90 con 0x7fd3300b4520 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55654/0 (socket says 172.21.15.78:55654) 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 -- 172.21.15.78:0/1367546954 learned_addr learned my addr 172.21.15.78:0/1367546954 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:52.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32d400640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3300b4520 0x7fd33014f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32e800640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300b7950 0x7fd33014fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 -- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3300b4520 msgr2=0x7fd33014f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3300b4520 0x7fd33014f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 -- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300b7950 msgr2=0x7fd33014fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300b7950 0x7fd33014fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 -- 172.21.15.78:0/1367546954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33015e930 con 0x7fd330070e50 2024-09-11T12:55:52.311 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd32de00640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd320009bd0 tx=0x7fd320009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3200bc070 con 0x7fd330070e50 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3200ad8e0 con 0x7fd330070e50 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33015ec10 con 0x7fd330070e50 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3200b2040 con 0x7fd330070e50 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd330156be0 con 0x7fd330070e50 2024-09-11T12:55:52.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.306+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3301566f0 con 0x7fd330070e50 2024-09-11T12:55:52.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.310+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fd3200bf070 con 0x7fd330070e50 2024-09-11T12:55:52.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.310+0000 7fd326000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 0x7fd3100740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:52.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.310+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fd32012c4e0 con 0x7fd330070e50 2024-09-11T12:55:52.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.310+0000 7fd32d400640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 0x7fd3100740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:52.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.310+0000 7fd32d400640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 0x7fd3100740e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3301593a0 tx=0x7fd31c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:52.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.314+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd320016990 con 0x7fd330070e50 2024-09-11T12:55:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:52.514+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd33006e960 con 0x7fd330070e50 2024-09-11T12:55:53.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.010+0000 7fd326000640 1 -- 172.21.15.78:0/1367546954 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v92) ==== 136+0+0 (secure 0 0 0) 0x7fd3200f7450 con 0x7fd330070e50 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 msgr2=0x7fd3100740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 0x7fd3100740e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd3301593a0 tx=0x7fd31c009040 comp rx=0 tx=0).stop 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 msgr2=0x7fd33014ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd320009bd0 tx=0x7fd320009340 comp rx=0 tx=0).stop 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 shutdown_connections 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd310071c20 0x7fd3100740e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd3300b7950 0x7fd33014fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd3300b4520 0x7fd33014f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 --2- 172.21.15.78:0/1367546954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd330070e50 0x7fd33014ed00 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 >> 172.21.15.78:0/1367546954 conn(0x7fd3300ce980 msgr2=0x7fd330152650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 shutdown_connections 2024-09-11T12:55:53.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.014+0000 7fd335000640 1 -- 172.21.15.78:0/1367546954 wait complete. 2024-09-11T12:55:53.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:55:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- 172.21.15.78:0/3723133815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15ae40 msgr2=0x7ff73c0cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- 172.21.15.78:0/3723133815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cf7a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7ff73c074f10 tx=0x7ff72c01c800 comp rx=0 tx=0).stop 2024-09-11T12:55:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- 172.21.15.78:0/3723133815 shutdown_connections 2024-09-11T12:55:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- 172.21.15.78:0/3723133815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cf7a0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- 172.21.15.78:0/3723133815 >> v2:172.21.15.157:3300/0 conn(0x7ff73c1518f0 0x7ff73c073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- 172.21.15.78:0/3723133815 >> 172.21.15.78:0/3723133815 conn(0x7ff73c06f3a0 msgr2=0x7ff73c0cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- 172.21.15.78:0/3723133815 shutdown_connections 2024-09-11T12:55:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- 172.21.15.78:0/3723133815 wait complete. 2024-09-11T12:55:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 Processor -- start 2024-09-11T12:55:53.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- start start 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73c1518f0 0x7ff73c147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c167bc0 con 0x7ff73c0cfce0 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c167a40 con 0x7ff73c15ae40 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff744800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c167d40 con 0x7ff73c1518f0 2024-09-11T12:55:53.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55660/0 (socket says 172.21.15.78:55660) 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 -- 172.21.15.78:0/1270198500 learned_addr learned my addr 172.21.15.78:0/1270198500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741400640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73c1518f0 0x7ff73c147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff742800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 -- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73c1518f0 msgr2=0x7ff73c147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73c1518f0 0x7ff73c147850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 -- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c15ae40 msgr2=0x7ff73c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.182+0000 7ff741e00640 1 -- 172.21.15.78:0/1270198500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c14d000 con 0x7ff73c0cfce0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff741e00640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9f0 tx=0x7ff73000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73000cd80 con 0x7ff73c0cfce0 2024-09-11T12:55:53.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff730010430 con 0x7ff73c0cfce0 2024-09-11T12:55:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c14d2a0 con 0x7ff73c0cfce0 2024-09-11T12:55:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730010860 con 0x7ff73c0cfce0 2024-09-11T12:55:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c14d860 con 0x7ff73c0cfce0 2024-09-11T12:55:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73c073c70 con 0x7ff73c0cfce0 2024-09-11T12:55:53.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7ff730010a00 con 0x7ff73c0cfce0 2024-09-11T12:55:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 0x7ff7200740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7ff7300933e0 con 0x7ff73c0cfce0 2024-09-11T12:55:53.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.186+0000 7ff741400640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 0x7ff7200740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.190+0000 7ff741400640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 0x7ff7200740e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0af000 tx=0x7ff72c01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.190+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff73005e300 con 0x7ff73c0cfce0 2024-09-11T12:55:53.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.378+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff73c159eb0 con 0x7ff720071c20 2024-09-11T12:55:53.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.378+0000 7ff736000640 1 -- 172.21.15.78:0/1270198500 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff73c159eb0 con 0x7ff720071c20 2024-09-11T12:55:53.382 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:55:53.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:55:53.383 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:55:53.383 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:55:53.383 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 msgr2=0x7ff7200740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 0x7ff7200740e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0af000 tx=0x7ff72c01b040 comp rx=0 tx=0).stop 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 msgr2=0x7ff73c152fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7ff73000e9f0 tx=0x7ff73000eeb0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 shutdown_connections 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff720071c20 0x7ff7200740e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff73c15ae40 0x7ff73c0cee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff73c1518f0 0x7ff73c147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 --2- 172.21.15.78:0/1270198500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff73c0cfce0 0x7ff73c152fb0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 >> 172.21.15.78:0/1270198500 conn(0x7ff73c06f3a0 msgr2=0x7ff73c072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 shutdown_connections 2024-09-11T12:55:53.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.382+0000 7ff744800640 1 -- 172.21.15.78:0/1270198500 wait complete. 2024-09-11T12:55:53.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:55:53.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- 172.21.15.78:0/2547610942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440cd770 msgr2=0x7fcb440cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- 172.21.15.78:0/2547610942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440cd770 0x7fcb440cdba0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fcb4406d1a0 tx=0x7fcb3c01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- 172.21.15.78:0/2547610942 shutdown_connections 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- 172.21.15.78:0/2547610942 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440cd770 0x7fcb440cdba0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- 172.21.15.78:0/2547610942 >> v2:172.21.15.157:3300/0 conn(0x7fcb440b3040 0x7fcb440cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- 172.21.15.78:0/2547610942 >> 172.21.15.78:0/2547610942 conn(0x7fcb44071dd0 msgr2=0x7fcb440721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- 172.21.15.78:0/2547610942 shutdown_connections 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- 172.21.15.78:0/2547610942 wait complete. 2024-09-11T12:55:53.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 Processor -- start 2024-09-11T12:55:53.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- start start 2024-09-11T12:55:53.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb440b3420 0x7fcb440ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb440cd770 0x7fcb440adc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440b1e10 con 0x7fcb440b3040 2024-09-11T12:55:53.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcb440b1c90 con 0x7fcb440b3420 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb4be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcb440b1f90 con 0x7fcb440cd770 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55672/0 (socket says 172.21.15.78:55672) 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 -- 172.21.15.78:0/3953481767 learned_addr learned my addr 172.21.15.78:0/3953481767 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb48a00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb440b3420 0x7fcb440ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49e00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb440cd770 0x7fcb440adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 -- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb440cd770 msgr2=0x7fcb440adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb440cd770 0x7fcb440adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 -- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb440b3420 msgr2=0x7fcb440ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb440b3420 0x7fcb440ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.546+0000 7fcb49400640 1 -- 172.21.15.78:0/3953481767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb440a9500 con 0x7fcb440b3040 2024-09-11T12:55:53.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb49400640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800eba0 tx=0x7fcb3800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3800eea0 con 0x7fcb440b3040 2024-09-11T12:55:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb38004d50 con 0x7fcb440b3040 2024-09-11T12:55:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb38005280 con 0x7fcb440b3040 2024-09-11T12:55:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb440a97e0 con 0x7fcb440b3040 2024-09-11T12:55:53.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb440a9da0 con 0x7fcb440b3040 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb440cf6e0 con 0x7fcb440b3040 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fcb38005420 con 0x7fcb440b3040 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 0x7fcb240740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fcb380934e0 con 0x7fcb440b3040 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.550+0000 7fcb48a00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 0x7fcb240740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.554+0000 7fcb48a00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 0x7fcb240740e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb441416c0 tx=0x7fcb30009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.554+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcb3805e3d0 con 0x7fcb440b3040 2024-09-11T12:55:53.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.722+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fcb440cc7b0 con 0x7fcb440b3040 2024-09-11T12:55:53.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.722+0000 7fcb41600640 1 -- 172.21.15.78:0/3953481767 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7fcb38062920 con 0x7fcb440b3040 2024-09-11T12:55:53.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 msgr2=0x7fcb240740e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 0x7fcb240740e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb441416c0 tx=0x7fcb30009040 comp rx=0 tx=0).stop 2024-09-11T12:55:53.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 msgr2=0x7fcb441424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800eba0 tx=0x7fcb3800c6a0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 shutdown_connections 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcb440cd770 0x7fcb440adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb24071c20 0x7fcb240740e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcb440b3420 0x7fcb440ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 --2- 172.21.15.78:0/3953481767 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcb440b3040 0x7fcb441424b0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 >> 172.21.15.78:0/3953481767 conn(0x7fcb44071dd0 msgr2=0x7fcb440721b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 shutdown_connections 2024-09-11T12:55:53.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.726+0000 7fcb4be00640 1 -- 172.21.15.78:0/3953481767 wait complete. 2024-09-11T12:55:53.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- 172.21.15.78:0/4070989058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c0ce150 msgr2=0x7f3a6c06fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- 172.21.15.78:0/4070989058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c0ce150 0x7f3a6c06fb80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009bc0 tx=0x7f3a6401cac0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- 172.21.15.78:0/4070989058 shutdown_connections 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- 172.21.15.78:0/4070989058 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c0ce150 0x7f3a6c06fb80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- 172.21.15.78:0/4070989058 >> v2:172.21.15.157:3300/0 conn(0x7f3a6c0b2120 0x7f3a6c0cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- 172.21.15.78:0/4070989058 >> 172.21.15.78:0/4070989058 conn(0x7f3a6c071dd0 msgr2=0x7f3a6c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- 172.21.15.78:0/4070989058 shutdown_connections 2024-09-11T12:55:53.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- 172.21.15.78:0/4070989058 wait complete. 2024-09-11T12:55:53.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 Processor -- start 2024-09-11T12:55:53.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- start start 2024-09-11T12:55:53.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c0b2120 0x7f3a6c130f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a6c0ce150 0x7f3a6c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c1518b0 0x7f3a6c1f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c167960 con 0x7f3a6c0ce150 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3a6c1677e0 con 0x7f3a6c0b2120 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a73e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3a6c167ae0 con 0x7f3a6c1518b0 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.890+0000 7f3a70a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a6c0ce150 0x7f3a6c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a6c0ce150 0x7f3a6c14d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56018/0 (socket says 172.21.15.78:56018) 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 -- 172.21.15.78:0/1193418614 learned_addr learned my addr 172.21.15.78:0/1193418614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a71400640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c0b2120 0x7f3a6c130f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.896 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a71e00640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c1518b0 0x7f3a6c1f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 -- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c1518b0 msgr2=0x7f3a6c1f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3a6c1518b0 0x7f3a6c1f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.897 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 -- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c0b2120 msgr2=0x7f3a6c130f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3a6c0b2120 0x7f3a6c130f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 -- 172.21.15.78:0/1193418614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a6c133ec0 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a70a00640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3a6c0ce150 0x7f3a6c14d520 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800e940 tx=0x7f3a5800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a5800cd80 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a58005020 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a73e00640 1 -- 172.21.15.78:0/1193418614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a6c148450 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a58005450 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a73e00640 1 -- 172.21.15.78:0/1193418614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a6c148a10 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a73e00640 1 -- 172.21.15.78:0/1193418614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a6c148c10 con 0x7f3a6c0ce150 2024-09-11T12:55:53.898 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.894+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f3a580047b0 con 0x7f3a6c0ce150 2024-09-11T12:55:53.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.898+0000 7f3a69600640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a4c071c20 0x7f3a4c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:53.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.898+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f3a58093240 con 0x7f3a6c0ce150 2024-09-11T12:55:53.900 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.898+0000 7f3a71400640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a4c071c20 0x7f3a4c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:53.901 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.898+0000 7f3a71400640 1 --2- 172.21.15.78:0/1193418614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3a4c071c20 0x7f3a4c0740e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c155510 tx=0x7f3a60001da0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:53.904 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:53.902+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3a5805e0b0 con 0x7f3a6c0ce150 2024-09-11T12:55:54.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.094+0000 7f3a73e00640 1 -- 172.21.15.78:0/1193418614 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f3a6c0cdc10 con 0x7f3a4c071c20 2024-09-11T12:55:54.099 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:54.094+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.094+0000 7f3a69600640 1 -- 172.21.15.78:0/1193418614 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3a6c0cdc10 con 0x7f3a4c071c20 2024-09-11T12:55:54.099 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.108 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:54.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-11T12:55:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 -- 172.21.15.78:0/1931843084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ac005920 msgr2=0x7f83ac005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 --2- 172.21.15.78:0/1931843084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ac005920 0x7f83ac005da0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f83b40016c0 tx=0x7f83b401c730 comp rx=0 tx=0).stop 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 -- 172.21.15.78:0/1931843084 shutdown_connections 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 --2- 172.21.15.78:0/1931843084 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83ac005920 0x7f83ac005da0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 -- 172.21.15.78:0/1931843084 >> 172.21.15.78:0/1931843084 conn(0x7f83c8071dd0 msgr2=0x7f83c80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 -- 172.21.15.78:0/1931843084 shutdown_connections 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.250+0000 7f83cd000640 1 -- 172.21.15.78:0/1931843084 wait complete. 2024-09-11T12:55:54.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 Processor -- start 2024-09-11T12:55:54.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- start start 2024-09-11T12:55:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac005920 0x7f83c813c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c8153a10 0x7f83c8158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c81587b0 0x7f83c8158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f83c815aae0 con 0x7f83c8153a10 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f83c815a960 con 0x7f83c81587b0 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f83c815ac60 con 0x7f83ac005920 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c8153a10 0x7f83c8158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c8153a10 0x7f83c8158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56034/0 (socket says 172.21.15.78:56034) 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 -- 172.21.15.78:0/821272287 learned_addr learned my addr 172.21.15.78:0/821272287 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5e00640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac005920 0x7f83c813c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c6800640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c81587b0 0x7f83c8158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 -- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac005920 msgr2=0x7f83c813c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f83ac005920 0x7f83c813c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 -- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c81587b0 msgr2=0x7f83c8158c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f83c81587b0 0x7f83c8158c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 -- 172.21.15.78:0/821272287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c813d940 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83c5400640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f83c8153a10 0x7f83c8158270 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f83b401ae30 tx=0x7f83b4003be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40b9070 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83b40ae040 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- 172.21.15.78:0/821272287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c8140140 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83b40ad070 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.254+0000 7f83cd000640 1 -- 172.21.15.78:0/821272287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c813d660 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.258+0000 7f83cd000640 1 -- 172.21.15.78:0/821272287 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c80b1e60 con 0x7f83c8153a10 2024-09-11T12:55:54.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.258+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f83b4006350 con 0x7f83c8153a10 2024-09-11T12:55:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.258+0000 7f83be000640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83a8071c20 0x7f83a80740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.258+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f83b412baa0 con 0x7f83c8153a10 2024-09-11T12:55:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.258+0000 7f83c5e00640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83a8071c20 0x7f83a80740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.262+0000 7f83c5e00640 1 --2- 172.21.15.78:0/821272287 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f83a8071c20 0x7f83a80740e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f83b8006c50 tx=0x7f83b80067b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.262+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f83b4016490 con 0x7f83c8153a10 2024-09-11T12:55:54.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.454+0000 7f83cd000640 1 -- 172.21.15.78:0/821272287 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f83c8154580 con 0x7f83a8071c20 2024-09-11T12:55:54.457 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:54.454+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.454+0000 7f83be000640 1 -- 172.21.15.78:0/821272287 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f83c8154580 con 0x7f83a8071c20 2024-09-11T12:55:54.458 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.466 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:54.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-11T12:55:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- 172.21.15.78:0/2830505795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 msgr2=0x7f0a940b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 --2- 172.21.15.78:0/2830505795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a940b1a10 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f0a940af400 tx=0x7f0a8401caa0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- 172.21.15.78:0/2830505795 shutdown_connections 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 --2- 172.21.15.78:0/2830505795 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a940ce150 0x7f0a9415e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 --2- 172.21.15.78:0/2830505795 >> v2:172.21.15.157:3300/0 conn(0x7f0a940b1f50 0x7f0a940cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 --2- 172.21.15.78:0/2830505795 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a940b1a10 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- 172.21.15.78:0/2830505795 >> 172.21.15.78:0/2830505795 conn(0x7f0a94071dd0 msgr2=0x7f0a940721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- 172.21.15.78:0/2830505795 shutdown_connections 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- 172.21.15.78:0/2830505795 wait complete. 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 Processor -- start 2024-09-11T12:55:54.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.610+0000 7f0a99400640 1 -- start start 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a940b1f50 0x7f0a9413d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a940ce150 0x7f0a9415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a9415d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0a94070130 con 0x7f0a941518b0 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9406ffb0 con 0x7f0a940ce150 2024-09-11T12:55:54.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0a940702b0 con 0x7f0a940b1f50 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a9415d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a9415d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56050/0 (socket says 172.21.15.78:56050) 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 -- 172.21.15.78:0/4164116186 learned_addr learned my addr 172.21.15.78:0/4164116186 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a92800640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a940b1f50 0x7f0a9413d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a91e00640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a940ce150 0x7f0a9415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 -- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a940b1f50 msgr2=0x7f0a9413d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0a940b1f50 0x7f0a9413d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 -- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a940ce150 msgr2=0x7f0a9415d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0a940ce150 0x7f0a9415d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 -- 172.21.15.78:0/4164116186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9413f680 con 0x7f0a941518b0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a93200640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0a941518b0 0x7f0a9415d5a0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00e9a0 tx=0x7f0a8c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c00ccc0 con 0x7f0a941518b0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8c00ce60 con 0x7f0a941518b0 2024-09-11T12:55:54.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- 172.21.15.78:0/4164116186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9413f960 con 0x7f0a941518b0 2024-09-11T12:55:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c005280 con 0x7f0a941518b0 2024-09-11T12:55:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- 172.21.15.78:0/4164116186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9413ff20 con 0x7f0a941518b0 2024-09-11T12:55:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.614+0000 7f0a99400640 1 -- 172.21.15.78:0/4164116186 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a9415e4d0 con 0x7f0a941518b0 2024-09-11T12:55:54.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.618+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f0a8c005420 con 0x7f0a941518b0 2024-09-11T12:55:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.618+0000 7f0a8aa00640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0a74071c50 0x7f0a74074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.618+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f0a8c093460 con 0x7f0a941518b0 2024-09-11T12:55:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.618+0000 7f0a92800640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0a74071c50 0x7f0a74074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.621 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.618+0000 7f0a92800640 1 --2- 172.21.15.78:0/4164116186 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0a74071c50 0x7f0a74074110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0a840af000 tx=0x7f0a8401b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.623 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.622+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0a8c05e490 con 0x7f0a941518b0 2024-09-11T12:55:54.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.814+0000 7f0a99400640 1 -- 172.21.15.78:0/4164116186 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f0a94069610 con 0x7f0a74071c50 2024-09-11T12:55:54.817 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:54.814+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.814+0000 7f0a8aa00640 1 -- 172.21.15.78:0/4164116186 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0a94069610 con 0x7f0a74071c50 2024-09-11T12:55:54.817 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:54.826 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:54.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 -- 172.21.15.78:0/1241423360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20400cd680 msgr2=0x7f20400cdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 --2- 172.21.15.78:0/1241423360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20400cd680 0x7f20400cdac0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f202c009840 tx=0x7f202c01c820 comp rx=0 tx=0).stop 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 -- 172.21.15.78:0/1241423360 shutdown_connections 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 --2- 172.21.15.78:0/1241423360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f20400cd680 0x7f20400cdac0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 -- 172.21.15.78:0/1241423360 >> 172.21.15.78:0/1241423360 conn(0x7f2040073700 msgr2=0x7f2040073b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 -- 172.21.15.78:0/1241423360 shutdown_connections 2024-09-11T12:55:54.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.966+0000 7f2046a00640 1 -- 172.21.15.78:0/1241423360 wait complete. 2024-09-11T12:55:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 Processor -- start 2024-09-11T12:55:54.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- start start 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f204006c650 0x7f204014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20400cd680 0x7f204015ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2040155ed0 0x7f2040157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f204016a340 con 0x7f2040155ed0 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f204016a1c0 con 0x7f20400cd680 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f204016a4c0 con 0x7f204006c650 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2040155ed0 0x7f2040157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2040155ed0 0x7f2040157bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56066/0 (socket says 172.21.15.78:56066) 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 -- 172.21.15.78:0/2389704407 learned_addr learned my addr 172.21.15.78:0/2389704407 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f203f400640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20400cd680 0x7f204015ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f203fe00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f204006c650 0x7f204014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 -- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f204006c650 msgr2=0x7f204014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f204006c650 0x7f204014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 -- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20400cd680 msgr2=0x7f204015ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f20400cd680 0x7f204015ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 -- 172.21.15.78:0/2389704407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f204013bc00 con 0x7f2040155ed0 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2044a00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2040155ed0 0x7f2040157bf0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f203800c500 tx=0x7f203800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20380052d0 con 0x7f2040155ed0 2024-09-11T12:55:54.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2038005470 con 0x7f2040155ed0 2024-09-11T12:55:54.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2038012870 con 0x7f2040155ed0 2024-09-11T12:55:54.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- 172.21.15.78:0/2389704407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f204013bee0 con 0x7f2040155ed0 2024-09-11T12:55:54.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- 172.21.15.78:0/2389704407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f204013c4a0 con 0x7f2040155ed0 2024-09-11T12:55:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.970+0000 7f2046a00640 1 -- 172.21.15.78:0/2389704407 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20400cdac0 con 0x7f2040155ed0 2024-09-11T12:55:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.974+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f2038003600 con 0x7f2040155ed0 2024-09-11T12:55:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.974+0000 7f2037e00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2020071c20 0x7f20200740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.974+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f20380922f0 con 0x7f2040155ed0 2024-09-11T12:55:54.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.974+0000 7f203fe00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2020071c20 0x7f20200740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:54.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.974+0000 7f203fe00640 1 --2- 172.21.15.78:0/2389704407 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2020071c20 0x7f20200740e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f20400ae710 tx=0x7f2030009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:54.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:54.978+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f203805d8c0 con 0x7f2040155ed0 2024-09-11T12:55:55.175 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.170+0000 7f2046a00640 1 -- 172.21.15.78:0/2389704407 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f2040073c80 con 0x7f2020071c20 2024-09-11T12:55:55.177 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:55.174+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.177 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.174+0000 7f2037e00640 1 -- 172.21.15.78:0/2389704407 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f2040073c80 con 0x7f2020071c20 2024-09-11T12:55:55.177 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.186 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:55.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- 172.21.15.78:0/2739130962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06fd80 msgr2=0x7f400c0703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 --2- 172.21.15.78:0/2739130962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06fd80 0x7f400c0703e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f4004009bc0 tx=0x7f400401cba0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- 172.21.15.78:0/2739130962 shutdown_connections 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 --2- 172.21.15.78:0/2739130962 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06fd80 0x7f400c0703e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- 172.21.15.78:0/2739130962 >> 172.21.15.78:0/2739130962 conn(0x7f400c0b2c70 msgr2=0x7f400c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- 172.21.15.78:0/2739130962 shutdown_connections 2024-09-11T12:55:55.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- 172.21.15.78:0/2739130962 wait complete. 2024-09-11T12:55:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 Processor -- start 2024-09-11T12:55:55.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.330+0000 7f400be00640 1 -- start start 2024-09-11T12:55:55.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06f8f0 0x7f400c1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c06fd80 0x7f400c13bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c0cf3e0 0x7f400c136e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f400c0737b0 con 0x7f400c06f8f0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f400c073630 con 0x7f400c06fd80 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f400c168cb0 con 0x7f400c0cf3e0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06f8f0 0x7f400c1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06f8f0 0x7f400c1598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56084/0 (socket says 172.21.15.78:56084) 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 -- 172.21.15.78:0/3264922675 learned_addr learned my addr 172.21.15.78:0/3264922675 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f4003e00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c06fd80 0x7f400c13bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400b400640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c0cf3e0 0x7f400c136e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 -- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c0cf3e0 msgr2=0x7f400c136e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c0cf3e0 0x7f400c136e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 -- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c06fd80 msgr2=0x7f400c13bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c06fd80 0x7f400c13bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 -- 172.21.15.78:0/3264922675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f400c154f20 con 0x7f400c06f8f0 2024-09-11T12:55:55.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f4003e00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f400c06fd80 0x7f400c13bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400b400640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f400c0cf3e0 0x7f400c136e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400aa00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f400c06f8f0 0x7f400c1598e0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f3ffc00c3c0 tx=0x7f3ffc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc017070 con 0x7f400c06f8f0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ffc003600 con 0x7f400c06f8f0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ffc003a50 con 0x7f400c06f8f0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- 172.21.15.78:0/3264922675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f400c138470 con 0x7f400c06f8f0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- 172.21.15.78:0/3264922675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f400c138a30 con 0x7f400c06f8f0 2024-09-11T12:55:55.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.334+0000 7f400be00640 1 -- 172.21.15.78:0/3264922675 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f400c0703e0 con 0x7f400c06f8f0 2024-09-11T12:55:55.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.338+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f3ffc01a070 con 0x7f400c06f8f0 2024-09-11T12:55:55.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.338+0000 7f4002a00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe8071c20 0x7f3fe80740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.338+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f3ffc091af0 con 0x7f400c06f8f0 2024-09-11T12:55:55.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.338+0000 7f4003e00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe8071c20 0x7f3fe80740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.338+0000 7f4003e00640 1 --2- 172.21.15.78:0/3264922675 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3fe8071c20 0x7f3fe80740e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f400c0cc530 tx=0x7f3ff4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:55.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.342+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3ffc05d0b0 con 0x7f400c06f8f0 2024-09-11T12:55:55.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.542+0000 7f400be00640 1 -- 172.21.15.78:0/3264922675 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7f400c15d240 con 0x7f3fe8071c20 2024-09-11T12:55:55.546 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:55.542+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.542+0000 7f4002a00640 1 -- 172.21.15.78:0/3264922675 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f400c15d240 con 0x7f3fe8071c20 2024-09-11T12:55:55.547 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.555 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:55.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-11T12:55:55.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- 172.21.15.78:0/1157825193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98e4003fe0 msgr2=0x7f98e4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 --2- 172.21.15.78:0/1157825193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f98e4003fe0 0x7f98e4004810 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009b70 tx=0x7f98ec01d250 comp rx=0 tx=0).stop 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f98ffe00640 1 -- 172.21.15.78:0/1157825193 reap_dead start 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- 172.21.15.78:0/1157825193 shutdown_connections 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- 172.21.15.78:0/1157825193 >> 172.21.15.78:0/1157825193 conn(0x7f99000b3050 msgr2=0x7f99000b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- 172.21.15.78:0/1157825193 shutdown_connections 2024-09-11T12:55:55.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- 172.21.15.78:0/1157825193 wait complete. 2024-09-11T12:55:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 Processor -- start 2024-09-11T12:55:55.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- start start 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98e4003fe0 0x7f99001e7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990014a290 0x7f9900154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99001f8b80 0x7f9900154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9900072e80 con 0x7f990014a290 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9900072d00 con 0x7f98e4003fe0 2024-09-11T12:55:55.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f9906400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f990015afc0 con 0x7f99001f8b80 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f98fea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990014a290 0x7f9900154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f98fea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990014a290 0x7f9900154020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56102/0 (socket says 172.21.15.78:56102) 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.702+0000 7f98fea00640 1 -- 172.21.15.78:0/1434973706 learned_addr learned my addr 172.21.15.78:0/1434973706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98ffe00640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99001f8b80 0x7f9900154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98ff400640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98e4003fe0 0x7f99001e7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 -- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99001f8b80 msgr2=0x7f9900154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f99001f8b80 0x7f9900154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 -- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98e4003fe0 msgr2=0x7f99001e7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f98e4003fe0 0x7f99001e7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 -- 172.21.15.78:0/1434973706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9900152770 con 0x7f990014a290 2024-09-11T12:55:55.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98fea00640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f990014a290 0x7f9900154020 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f98ec009b70 tx=0x7f98ec0050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0bb030 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98ec004840 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f9906400640 1 -- 172.21.15.78:0/1434973706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9900152a50 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ec004c70 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f9906400640 1 -- 172.21.15.78:0/1434973706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9900153010 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f9906400640 1 -- 172.21.15.78:0/1434973706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99000723f0 con 0x7f990014a290 2024-09-11T12:55:55.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f98ec0be030 con 0x7f990014a290 2024-09-11T12:55:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.706+0000 7f98f7400640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f98e0071c50 0x7f98e0074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.710+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f98ec12dcf0 con 0x7f990014a290 2024-09-11T12:55:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.710+0000 7f98ff400640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f98e0071c50 0x7f98e0074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:55.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.710+0000 7f98ff400640 1 --2- 172.21.15.78:0/1434973706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f98e0071c50 0x7f98e0074110 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ce0 tx=0x7f98f00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:55.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.710+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f98ec016fa0 con 0x7f990014a290 2024-09-11T12:55:55.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.910+0000 7f9906400640 1 -- 172.21.15.78:0/1434973706 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7f990015def0 con 0x7f98e0071c50 2024-09-11T12:55:55.915 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:55.914+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:55.914+0000 7f98f7400640 1 -- 172.21.15.78:0/1434973706 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f990015def0 con 0x7f98e0071c50 2024-09-11T12:55:55.916 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:55.924 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:55.925 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-11T12:55:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- 172.21.15.78:0/2107074045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 msgr2=0x7f7644070200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- 172.21.15.78:0/2107074045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f7644070200 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f7644159aa0 tx=0x7f763001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- 172.21.15.78:0/2107074045 shutdown_connections 2024-09-11T12:55:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- 172.21.15.78:0/2107074045 >> v2:172.21.15.157:3300/0 conn(0x7f7644070740 0x7f764413f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.072 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- 172.21.15.78:0/2107074045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f7644070200 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- 172.21.15.78:0/2107074045 >> 172.21.15.78:0/2107074045 conn(0x7f7644075610 msgr2=0x7f7644075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- 172.21.15.78:0/2107074045 shutdown_connections 2024-09-11T12:55:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- 172.21.15.78:0/2107074045 wait complete. 2024-09-11T12:55:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 Processor -- start 2024-09-11T12:55:56.073 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- start start 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7644070740 0x7f764413a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f764406d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764413e830 0x7f764415b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7644168a50 con 0x7f76440d04a0 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f76441688d0 con 0x7f7644070740 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764c600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7644168bd0 con 0x7f764413e830 2024-09-11T12:55:56.076 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f764406d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f764406d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56118/0 (socket says 172.21.15.78:56118) 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 -- 172.21.15.78:0/750241450 learned_addr learned my addr 172.21.15.78:0/750241450 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649c00640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7644070740 0x7f764413a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f764a600640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764413e830 0x7f764415b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 -- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764413e830 msgr2=0x7f764415b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f764413e830 0x7f764415b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 -- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7644070740 msgr2=0x7f764413a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7644070740 0x7f764413a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.070+0000 7f7649200640 1 -- 172.21.15.78:0/750241450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f764413a630 con 0x7f76440d04a0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7649200640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f76440d04a0 0x7f764406d830 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f76300b0040 tx=0x7f7630004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300b2040 con 0x7f76440d04a0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7630005180 con 0x7f76440d04a0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76300055d0 con 0x7f76440d04a0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f764c600640 1 -- 172.21.15.78:0/750241450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f764406c650 con 0x7f76440d04a0 2024-09-11T12:55:56.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f764c600640 1 -- 172.21.15.78:0/750241450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f764406cc10 con 0x7f76440d04a0 2024-09-11T12:55:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f764c600640 1 -- 172.21.15.78:0/750241450 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f764415ed90 con 0x7f76440d04a0 2024-09-11T12:55:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f76300046d0 con 0x7f76440d04a0 2024-09-11T12:55:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7624071c50 0x7f7624074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f763012b800 con 0x7f76440d04a0 2024-09-11T12:55:56.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.074+0000 7f7649c00640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7624071c50 0x7f7624074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.078+0000 7f7649c00640 1 --2- 172.21.15.78:0/750241450 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7624071c50 0x7f7624074110 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f76440b14c0 tx=0x7f7638009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.078+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f764413a630 con 0x7f76440d04a0 2024-09-11T12:55:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.270+0000 7f764c600640 1 -- 172.21.15.78:0/750241450 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f764413f0c0 con 0x7f7624071c50 2024-09-11T12:55:56.273 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:56.270+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.270+0000 7f7641600640 1 -- 172.21.15.78:0/750241450 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f764413f0c0 con 0x7f7624071c50 2024-09-11T12:55:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:56.282 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:56.283 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- 172.21.15.78:0/3341444591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade0151340 msgr2=0x7fade01517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 --2- 172.21.15.78:0/3341444591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade0151340 0x7fade01517d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009c80 tx=0x7fadd801cda0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- 172.21.15.78:0/3341444591 shutdown_connections 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 --2- 172.21.15.78:0/3341444591 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade0151340 0x7fade01517d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 --2- 172.21.15.78:0/3341444591 >> v2:172.21.15.157:3300/0 conn(0x7fade00b5790 0x7fade00723f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- 172.21.15.78:0/3341444591 >> 172.21.15.78:0/3341444591 conn(0x7fade00b3050 msgr2=0x7fade00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- 172.21.15.78:0/3341444591 shutdown_connections 2024-09-11T12:55:56.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- 172.21.15.78:0/3341444591 wait complete. 2024-09-11T12:55:56.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 Processor -- start 2024-09-11T12:55:56.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.426+0000 7fade8200640 1 -- start start 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade00b5790 0x7fade0150a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade00b5b70 0x7fade0153910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0151340 0x7fade014e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fade0167ce0 con 0x7fade00b5790 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fade0167b60 con 0x7fade0151340 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fade0167e60 con 0x7fade00b5b70 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade00b5790 0x7fade0150a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade00b5790 0x7fade0150a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56122/0 (socket says 172.21.15.78:56122) 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 -- 172.21.15.78:0/3151963714 learned_addr learned my addr 172.21.15.78:0/3151963714 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade4e00640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade00b5b70 0x7fade0153910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade6200640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0151340 0x7fade014e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 -- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade00b5b70 msgr2=0x7fade0153910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fade00b5b70 0x7fade0153910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 -- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0151340 msgr2=0x7fade014e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0151340 0x7fade014e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 -- 172.21.15.78:0/3151963714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01502c0 con 0x7fade00b5790 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade6200640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fade0151340 0x7fade014e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade5800640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fade00b5790 0x7fade0150a80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fadd400e940 tx=0x7fadd400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd400cd80 con 0x7fade00b5790 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadd4005020 con 0x7fade00b5790 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- 172.21.15.78:0/3151963714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade014b2c0 con 0x7fade00b5790 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd4005450 con 0x7fade00b5790 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- 172.21.15.78:0/3151963714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade014b880 con 0x7fade00b5790 2024-09-11T12:55:56.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.430+0000 7fade8200640 1 -- 172.21.15.78:0/3151963714 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade01517d0 con 0x7fade00b5790 2024-09-11T12:55:56.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.434+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fadd40047b0 con 0x7fade00b5790 2024-09-11T12:55:56.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.434+0000 7faddd600640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fadc0071c20 0x7fadc00740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.434+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fadd4093330 con 0x7fade00b5790 2024-09-11T12:55:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.434+0000 7fade4e00640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fadc0071c20 0x7fadc00740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.434+0000 7fade4e00640 1 --2- 172.21.15.78:0/3151963714 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fadc0071c20 0x7fadc00740e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fadcc007fd0 tx=0x7fadcc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.438+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fadd405e7e0 con 0x7fade00b5790 2024-09-11T12:55:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.630+0000 7fade8200640 1 -- 172.21.15.78:0/3151963714 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fade0072010 con 0x7fadc0071c20 2024-09-11T12:55:56.636 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:56.634+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.634+0000 7faddd600640 1 -- 172.21.15.78:0/3151963714 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fade0072010 con 0x7fadc0071c20 2024-09-11T12:55:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:56.645 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:56.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-11T12:55:56.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- 172.21.15.78:0/3352612360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 msgr2=0x7f28d0071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- 172.21.15.78:0/3352612360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d0071e10 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f28d01581d0 tx=0x7f28bc01c890 comp rx=0 tx=0).stop 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- 172.21.15.78:0/3352612360 shutdown_connections 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- 172.21.15.78:0/3352612360 >> v2:172.21.15.157:3300/0 conn(0x7f28d0072350 0x7f28d006f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- 172.21.15.78:0/3352612360 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d0071e10 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- 172.21.15.78:0/3352612360 >> 172.21.15.78:0/3352612360 conn(0x7f28d00b3050 msgr2=0x7f28d00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- 172.21.15.78:0/3352612360 shutdown_connections 2024-09-11T12:55:56.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- 172.21.15.78:0/3352612360 wait complete. 2024-09-11T12:55:56.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 Processor -- start 2024-09-11T12:55:56.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- start start 2024-09-11T12:55:56.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d0072350 0x7f28d0149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d00b00d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d0151050 0x7f28d00b0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0167230 con 0x7f28d0074060 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f28d00708b0 con 0x7f28d0151050 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28d6000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f28d01673b0 con 0x7f28d0072350 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28cea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d00b00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28cea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d00b00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56132/0 (socket says 172.21.15.78:56132) 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28cea00640 1 -- 172.21.15.78:0/2490188513 learned_addr learned my addr 172.21.15.78:0/2490188513 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.790+0000 7f28cf400640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d0072350 0x7f28d0149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 -- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d0072350 msgr2=0x7f28d0149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cfe00640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d0151050 0x7f28d00b0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d0072350 0x7f28d0149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 -- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d0151050 msgr2=0x7f28d00b0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d0151050 0x7f28d00b0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 -- 172.21.15.78:0/2490188513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d01fb420 con 0x7f28d0074060 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cf400640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f28d0072350 0x7f28d0149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cfe00640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f28d0151050 0x7f28d00b0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28cea00640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f28d0074060 0x7f28d00b00d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f28bc0b0040 tx=0x7f28bc0a5770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0bc070 con 0x7f28d0074060 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28bc0053e0 con 0x7f28d0074060 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc0b2040 con 0x7f28d0074060 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28d6000640 1 -- 172.21.15.78:0/2490188513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d01fb700 con 0x7f28d0074060 2024-09-11T12:55:56.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28d6000640 1 -- 172.21.15.78:0/2490188513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01fbcc0 con 0x7f28d0074060 2024-09-11T12:55:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28d6000640 1 -- 172.21.15.78:0/2490188513 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d006f5d0 con 0x7f28d0074060 2024-09-11T12:55:56.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f28bc0046d0 con 0x7f28d0074060 2024-09-11T12:55:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f28b0071c10 0x7f28b00740d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.794+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f28bc12c520 con 0x7f28d0074060 2024-09-11T12:55:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.798+0000 7f28cf400640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f28b0071c10 0x7f28b00740d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:56.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.798+0000 7f28cf400640 1 --2- 172.21.15.78:0/2490188513 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f28b0071c10 0x7f28b00740d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f28c0001630 tx=0x7f28c0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:56.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.798+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f28bc0164b0 con 0x7f28d0074060 2024-09-11T12:55:57.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.998+0000 7f28d6000640 1 -- 172.21.15.78:0/2490188513 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f28d0070ce0 con 0x7f28b0071c10 2024-09-11T12:55:57.002 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:56.998+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:56.998+0000 7f28c7400640 1 -- 172.21.15.78:0/2490188513 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f28d0070ce0 con 0x7f28b0071c10 2024-09-11T12:55:57.002 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.011 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:57.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-11T12:55:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 -- 172.21.15.78:0/2502059627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 msgr2=0x7f3b54071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 --2- 172.21.15.78:0/2502059627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b54071e10 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009900 tx=0x7f3b4001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 -- 172.21.15.78:0/2502059627 shutdown_connections 2024-09-11T12:55:57.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 --2- 172.21.15.78:0/2502059627 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b54071e10 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 --2- 172.21.15.78:0/2502059627 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b54151050 0x7f3b54073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 -- 172.21.15.78:0/2502059627 >> 172.21.15.78:0/2502059627 conn(0x7f3b540b3050 msgr2=0x7f3b540b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 -- 172.21.15.78:0/2502059627 shutdown_connections 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.154+0000 7f3b5b200640 1 -- 172.21.15.78:0/2502059627 wait complete. 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 Processor -- start 2024-09-11T12:55:57.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- start start 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b54072350 0x7f3b5414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b5414d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b54151050 0x7f3b541477b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5415e970 con 0x7f3b54073e90 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5415e7f0 con 0x7f3b54072350 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5415eaf0 con 0x7f3b54151050 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b5414d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b5414d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56144/0 (socket says 172.21.15.78:56144) 2024-09-11T12:55:57.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 -- 172.21.15.78:0/2264449438 learned_addr learned my addr 172.21.15.78:0/2264449438 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:57.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53e00640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b54072350 0x7f3b5414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b59200640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b54151050 0x7f3b541477b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 -- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b54151050 msgr2=0x7f3b541477b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b54151050 0x7f3b541477b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 -- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b54072350 msgr2=0x7f3b5414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b54072350 0x7f3b5414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 -- 172.21.15.78:0/2264449438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b5414d2e0 con 0x7f3b54073e90 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b53400640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b54073e90 0x7f3b5414d6d0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009850 tx=0x7f3b40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400b2040 con 0x7f3b54073e90 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b40005070 con 0x7f3b54073e90 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- 172.21.15.78:0/2264449438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b5414d070 con 0x7f3b54073e90 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400054c0 con 0x7f3b54073e90 2024-09-11T12:55:57.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.158+0000 7f3b5b200640 1 -- 172.21.15.78:0/2264449438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b541f7990 con 0x7f3b54073e90 2024-09-11T12:55:57.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.162+0000 7f3b5b200640 1 -- 172.21.15.78:0/2264449438 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b54071e10 con 0x7f3b54073e90 2024-09-11T12:55:57.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.162+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f3b400046d0 con 0x7f3b54073e90 2024-09-11T12:55:57.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.162+0000 7f3b4be00640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b34071c50 0x7f3b34074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.162+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f3b4012bf40 con 0x7f3b54073e90 2024-09-11T12:55:57.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.162+0000 7f3b53e00640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b34071c50 0x7f3b34074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.166+0000 7f3b53e00640 1 --2- 172.21.15.78:0/2264449438 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b34071c50 0x7f3b34074110 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f3b540ced90 tx=0x7f3b44009630 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.166+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3b400bc060 con 0x7f3b54073e90 2024-09-11T12:55:57.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.362+0000 7f3b5b200640 1 -- 172.21.15.78:0/2264449438 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f3b54070ce0 con 0x7f3b34071c50 2024-09-11T12:55:57.366 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:57.362+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.362+0000 7f3b4be00640 1 -- 172.21.15.78:0/2264449438 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3b54070ce0 con 0x7f3b34071c50 2024-09-11T12:55:57.366 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.375 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:57.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 -- 172.21.15.78:0/3208288303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5440706b0 msgr2=0x7fe5440b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 --2- 172.21.15.78:0/3208288303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5440706b0 0x7fe5440b5530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe53c009bc0 tx=0x7fe53c01cba0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 -- 172.21.15.78:0/3208288303 shutdown_connections 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 --2- 172.21.15.78:0/3208288303 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5440706b0 0x7fe5440b5530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 --2- 172.21.15.78:0/3208288303 >> v2:172.21.15.157:3300/0 conn(0x7fe544074bc0 0x7fe544159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 -- 172.21.15.78:0/3208288303 >> 172.21.15.78:0/3208288303 conn(0x7fe5440b3050 msgr2=0x7fe5440b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 -- 172.21.15.78:0/3208288303 shutdown_connections 2024-09-11T12:55:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.518+0000 7fe548e00640 1 -- 172.21.15.78:0/3208288303 wait complete. 2024-09-11T12:55:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 Processor -- start 2024-09-11T12:55:57.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- start start 2024-09-11T12:55:57.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5440706b0 0x7fe544147260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe544074bc0 0x7fe5440b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe544159970 0x7fe5440ce880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe544167be0 con 0x7fe5440706b0 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe544167a60 con 0x7fe544159970 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe544167d60 con 0x7fe544074bc0 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5440706b0 0x7fe544147260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5440706b0 0x7fe544147260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56152/0 (socket says 172.21.15.78:56152) 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 -- 172.21.15.78:0/1178520903 learned_addr learned my addr 172.21.15.78:0/1178520903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe542800640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe544159970 0x7fe5440ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541400640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe544074bc0 0x7fe5440b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 -- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe544074bc0 msgr2=0x7fe5440b0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe544074bc0 0x7fe5440b0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 -- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe544159970 msgr2=0x7fe5440ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe544159970 0x7fe5440ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 -- 172.21.15.78:0/1178520903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe544148600 con 0x7fe5440706b0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe542800640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe544159970 0x7fe5440ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe541e00640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5440706b0 0x7fe544147260 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7fe53400c430 tx=0x7fe53400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534011790 con 0x7fe5440706b0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe534017070 con 0x7fe5440706b0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- 172.21.15.78:0/1178520903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5440b1930 con 0x7fe5440706b0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534011ce0 con 0x7fe5440706b0 2024-09-11T12:55:57.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.522+0000 7fe548e00640 1 -- 172.21.15.78:0/1178520903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe544148b50 con 0x7fe5440706b0 2024-09-11T12:55:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe548e00640 1 -- 172.21.15.78:0/1178520903 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5440b5530 con 0x7fe5440706b0 2024-09-11T12:55:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fe53401c460 con 0x7fe5440706b0 2024-09-11T12:55:57.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe53a000640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe524071c50 0x7fe524074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fe534092690 con 0x7fe5440706b0 2024-09-11T12:55:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe541400640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe524071c50 0x7fe524074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.526+0000 7fe541400640 1 --2- 172.21.15.78:0/1178520903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe524071c50 0x7fe524074110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe5441f8a20 tx=0x7fe5300091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.530+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe5440b1930 con 0x7fe5440706b0 2024-09-11T12:55:57.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.726+0000 7fe548e00640 1 -- 172.21.15.78:0/1178520903 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fe54414ebd0 con 0x7fe524071c50 2024-09-11T12:55:57.731 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:57.726+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.726+0000 7fe53a000640 1 -- 172.21.15.78:0/1178520903 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe54414ebd0 con 0x7fe524071c50 2024-09-11T12:55:57.731 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:57.739 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:57.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 -- 172.21.15.78:0/16171943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 msgr2=0x7f4dc815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 --2- 172.21.15.78:0/16171943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc815f780 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80d16c0 tx=0x7f4dc001cd80 comp rx=0 tx=0).stop 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 -- 172.21.15.78:0/16171943 shutdown_connections 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 --2- 172.21.15.78:0/16171943 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc815f780 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 --2- 172.21.15.78:0/16171943 >> v2:172.21.15.157:3300/0 conn(0x7f4dc815c860 0x7f4dc813edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 -- 172.21.15.78:0/16171943 >> 172.21.15.78:0/16171943 conn(0x7f4dc8074610 msgr2=0x7f4dc80d1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:57.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 -- 172.21.15.78:0/16171943 shutdown_connections 2024-09-11T12:55:57.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.882+0000 7f4dc7200640 1 -- 172.21.15.78:0/16171943 wait complete. 2024-09-11T12:55:57.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 Processor -- start 2024-09-11T12:55:57.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- start start 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815c480 0x7f4dc81fb390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc815c860 0x7f4dc813c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc813cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc813e0a0 con 0x7f4dc815f350 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc813df20 con 0x7f4dc815c860 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4dc813e220 con 0x7f4dc815c480 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc813cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc813cf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56174/0 (socket says 172.21.15.78:56174) 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 -- 172.21.15.78:0/956552341 learned_addr learned my addr 172.21.15.78:0/956552341 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc5e00640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815c480 0x7f4dc81fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc5400640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc815c860 0x7f4dc813c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 -- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815c480 msgr2=0x7f4dc81fb390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4dc815c480 0x7f4dc81fb390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 -- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc815c860 msgr2=0x7f4dc813c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4dc815c860 0x7f4dc813c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 -- 172.21.15.78:0/956552341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc8153610 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc6800640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4dc815f350 0x7f4dc813cf00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001b430 tx=0x7f4dc0005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00bb070 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4dc0004d40 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- 172.21.15.78:0/956552341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc81559c0 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc00b2040 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.886+0000 7f4dc7200640 1 -- 172.21.15.78:0/956552341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc81fab90 con 0x7f4dc815f350 2024-09-11T12:55:57.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dc7200640 1 -- 172.21.15.78:0/956552341 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dc813e7e0 con 0x7f4dc815f350 2024-09-11T12:55:57.893 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f4dc00be070 con 0x7f4dc815f350 2024-09-11T12:55:57.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dbe000640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4da4071c20 0x7f4da40740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:57.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f4dc012c470 con 0x7f4dc815f350 2024-09-11T12:55:57.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dc5e00640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4da4071c20 0x7f4da40740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:57.895 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.890+0000 7f4dc5e00640 1 --2- 172.21.15.78:0/956552341 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4da4071c20 0x7f4da40740e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4db0007fd0 tx=0x7f4db0007e40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:57.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:57.894+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4dc0016970 con 0x7f4dc815f350 2024-09-11T12:55:58.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.094+0000 7f4dc7200640 1 -- 172.21.15.78:0/956552341 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f4dc813cd30 con 0x7f4da4071c20 2024-09-11T12:55:58.101 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.098+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.098+0000 7f4dbe000640 1 -- 172.21.15.78:0/956552341 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f4dc813cd30 con 0x7f4da4071c20 2024-09-11T12:55:58.101 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.111 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:58.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-11T12:55:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- 172.21.15.78:0/351525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8152990 msgr2=0x7f08b815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- 172.21.15.78:0/351525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8152990 0x7f08b815d060 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f08b80749c0 tx=0x7f08a801cb80 comp rx=0 tx=0).stop 2024-09-11T12:55:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- 172.21.15.78:0/351525082 shutdown_connections 2024-09-11T12:55:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- 172.21.15.78:0/351525082 >> v2:172.21.15.157:3300/0 conn(0x7f08b80cf520 0x7f08b80cf990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- 172.21.15.78:0/351525082 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b8152990 0x7f08b815d060 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- 172.21.15.78:0/351525082 >> 172.21.15.78:0/351525082 conn(0x7f08b80b3050 msgr2=0x7f08b80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- 172.21.15.78:0/351525082 shutdown_connections 2024-09-11T12:55:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- 172.21.15.78:0/351525082 wait complete. 2024-09-11T12:55:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 Processor -- start 2024-09-11T12:55:58.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- start start 2024-09-11T12:55:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b80b5530 0x7f08b81f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80cf520 0x7f08b8145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8152990 0x7f08b80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f08b8169110 con 0x7f08b80b5530 2024-09-11T12:55:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f08b815d7b0 con 0x7f08b8152990 2024-09-11T12:55:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08bd000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f08b8169290 con 0x7f08b80cf520 2024-09-11T12:55:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.254+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b80b5530 0x7f08b81f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b80b5530 0x7f08b81f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56190/0 (socket says 172.21.15.78:56190) 2024-09-11T12:55:58.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 -- 172.21.15.78:0/3387504085 learned_addr learned my addr 172.21.15.78:0/3387504085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b6800640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8152990 0x7f08b80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5400640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80cf520 0x7f08b8145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 -- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80cf520 msgr2=0x7f08b8145930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80cf520 0x7f08b8145930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 -- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8152990 msgr2=0x7f08b80ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8152990 0x7f08b80ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 -- 172.21.15.78:0/3387504085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08b81f8d40 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5400640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f08b80cf520 0x7f08b8145930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b6800640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f08b8152990 0x7f08b80ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08b5e00640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f08b80b5530 0x7f08b81f8800 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f08a8000f90 tx=0x7f08a80092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80bb070 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08a8004e00 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08a80b2040 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08bd000640 1 -- 172.21.15.78:0/3387504085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08b8073a20 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08bd000640 1 -- 172.21.15.78:0/3387504085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08b8153c40 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08bd000640 1 -- 172.21.15.78:0/3387504085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08b80cf990 con 0x7f08b80b5530 2024-09-11T12:55:58.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f08a80be070 con 0x7f08b80b5530 2024-09-11T12:55:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.258+0000 7f08ae000640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0898071c50 0x7f0898074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.262+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f08a812c690 con 0x7f08b80b5530 2024-09-11T12:55:58.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.262+0000 7f08b5400640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0898071c50 0x7f0898074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.262+0000 7f08b5400640 1 --2- 172.21.15.78:0/3387504085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0898071c50 0x7f0898074110 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f08b8068c50 tx=0x7f08a40091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:58.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.262+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f08a8016770 con 0x7f08b80b5530 2024-09-11T12:55:58.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.462+0000 7f08bd000640 1 -- 172.21.15.78:0/3387504085 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f08b8070620 con 0x7f0898071c50 2024-09-11T12:55:58.465 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.462+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.462+0000 7f08ae000640 1 -- 172.21.15.78:0/3387504085 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f08b8070620 con 0x7f0898071c50 2024-09-11T12:55:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.474 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:58.474 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- 172.21.15.78:0/1675840033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 msgr2=0x7fe9640cd8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 --2- 172.21.15.78:0/1675840033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 0x7fe9640cd8b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe95c009bf0 tx=0x7fe95c01cbb0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- 172.21.15.78:0/1675840033 shutdown_connections 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 --2- 172.21.15.78:0/1675840033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 0x7fe9640cd8b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- 172.21.15.78:0/1675840033 >> 172.21.15.78:0/1675840033 conn(0x7fe9640cf6d0 msgr2=0x7fe9640cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:58.624 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- 172.21.15.78:0/1675840033 shutdown_connections 2024-09-11T12:55:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- 172.21.15.78:0/1675840033 wait complete. 2024-09-11T12:55:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 Processor -- start 2024-09-11T12:55:58.625 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- start start 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 0x7fe96414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe964144950 0x7fe9641417e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe964144d30 0x7fe9640afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe96416a520 con 0x7fe964144950 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe96416a3a0 con 0x7fe964144d30 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe969a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe96416a6a0 con 0x7fe964070c30 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe964144950 0x7fe9641417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe964144950 0x7fe9641417e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56202/0 (socket says 172.21.15.78:56202) 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 -- 172.21.15.78:0/2721693244 learned_addr learned my addr 172.21.15.78:0/2721693244 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962a00640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 0x7fe96414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe963400640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe964144d30 0x7fe9640afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 -- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 msgr2=0x7fe96414c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe964070c30 0x7fe96414c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 -- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe964144d30 msgr2=0x7fe9640afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe964144d30 0x7fe9640afe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.622+0000 7fe962000640 1 -- 172.21.15.78:0/2721693244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe96413c730 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe962000640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe964144950 0x7fe9641417e0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fe95000c440 tx=0x7fe95000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950016020 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe950012920 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe969a00640 1 -- 172.21.15.78:0/2721693244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe964138e70 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe950012d70 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe969a00640 1 -- 172.21.15.78:0/2721693244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe96413ccb0 con 0x7fe964144950 2024-09-11T12:55:58.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe969a00640 1 -- 172.21.15.78:0/2721693244 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe964070520 con 0x7fe964144950 2024-09-11T12:55:58.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fe950019020 con 0x7fe964144950 2024-09-11T12:55:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe944071c50 0x7fe944074110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fe950092140 con 0x7fe964144950 2024-09-11T12:55:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.626+0000 7fe962a00640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe944071c50 0x7fe944074110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:58.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.630+0000 7fe962a00640 1 --2- 172.21.15.78:0/2721693244 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe944071c50 0x7fe944074110 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe954000c10 tx=0x7fe954009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:58.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.630+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe950020070 con 0x7fe964144950 2024-09-11T12:55:58.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.830+0000 7fe969a00640 1 -- 172.21.15.78:0/2721693244 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7fe9640b3730 con 0x7fe944071c50 2024-09-11T12:55:58.839 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.834+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:58.839 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.834+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:58.839 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.834+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:58.839 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.834+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:58.839 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.834+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:55:58.845 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:58.842+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.842+0000 7fe95aa00640 1 -- 172.21.15.78:0/2721693244 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe9640b3730 con 0x7fe944071c50 2024-09-11T12:55:58.845 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:58.854 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:58.855 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-11T12:55:59.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 -- 172.21.15.78:0/187616011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff268005920 msgr2=0x7ff268005da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 --2- 172.21.15.78:0/187616011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff268005920 0x7ff268005da0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7ff2700016f0 tx=0x7ff27001c750 comp rx=0 tx=0).stop 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 -- 172.21.15.78:0/187616011 shutdown_connections 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 --2- 172.21.15.78:0/187616011 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff268005920 0x7ff268005da0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 -- 172.21.15.78:0/187616011 >> 172.21.15.78:0/187616011 conn(0x7ff284071dd0 msgr2=0x7ff2840721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 -- 172.21.15.78:0/187616011 shutdown_connections 2024-09-11T12:55:59.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:58.998+0000 7ff28ba00640 1 -- 172.21.15.78:0/187616011 wait complete. 2024-09-11T12:55:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 Processor -- start 2024-09-11T12:55:59.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- start start 2024-09-11T12:55:59.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff268005920 0x7ff284150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff284142290 0x7ff284142710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff284134470 0x7ff2841348d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff2840cdf80 con 0x7ff284134470 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff2840cde00 con 0x7ff268005920 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff2840ce100 con 0x7ff284142290 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff284134470 0x7ff2841348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff268005920 0x7ff284150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff284134470 0x7ff2841348d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56238/0 (socket says 172.21.15.78:56238) 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 -- 172.21.15.78:0/1471018952 learned_addr learned my addr 172.21.15.78:0/1471018952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff283e00640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff284142290 0x7ff284142710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 -- 172.21.15.78:0/1471018952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff284142290 msgr2=0x7ff284142710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff284142290 0x7ff284142710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 -- 172.21.15.78:0/1471018952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff268005920 msgr2=0x7ff284150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff268005920 0x7ff284150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 -- 172.21.15.78:0/1471018952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff284154ed0 con 0x7ff284134470 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff289a00640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff284134470 0x7ff2841348d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7ff2840af9c0 tx=0x7ff27c0129e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c013860 con 0x7ff284134470 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff27c013a40 con 0x7ff284134470 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- 172.21.15.78:0/1471018952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2841551b0 con 0x7ff284134470 2024-09-11T12:55:59.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27c010480 con 0x7ff284134470 2024-09-11T12:55:59.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.002+0000 7ff28ba00640 1 -- 172.21.15.78:0/1471018952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841514c0 con 0x7ff284134470 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff28ba00640 1 -- 172.21.15.78:0/1471018952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff28415aae0 con 0x7ff284134470 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7ff27c010620 con 0x7ff284134470 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff280c00640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff264071c20 0x7ff2640740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7ff27c096440 con 0x7ff284134470 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff289000640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff264071c20 0x7ff2640740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.006+0000 7ff289000640 1 --2- 172.21.15.78:0/1471018952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff264071c20 0x7ff2640740e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff284052540 tx=0x7ff2740091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.010+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff27c061440 con 0x7ff284134470 2024-09-11T12:55:59.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.214+0000 7ff28ba00640 1 -- 172.21.15.78:0/1471018952 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7ff284143ae0 con 0x7ff264071c20 2024-09-11T12:55:59.217 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:59.214+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.214+0000 7ff280c00640 1 -- 172.21.15.78:0/1471018952 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff284143ae0 con 0x7ff264071c20 2024-09-11T12:55:59.217 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.226 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:59.226 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-11T12:55:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 -- 172.21.15.78:0/807647907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0003f60 msgr2=0x7f89a0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 --2- 172.21.15.78:0/807647907 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89a0003f60 0x7f89a0004810 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c390 tx=0x7f89a801ce10 comp rx=0 tx=0).stop 2024-09-11T12:55:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89bbe00640 1 -- 172.21.15.78:0/807647907 reap_dead start 2024-09-11T12:55:59.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 -- 172.21.15.78:0/807647907 shutdown_connections 2024-09-11T12:55:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 -- 172.21.15.78:0/807647907 >> 172.21.15.78:0/807647907 conn(0x7f89bc071dd0 msgr2=0x7f89bc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 -- 172.21.15.78:0/807647907 shutdown_connections 2024-09-11T12:55:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.366+0000 7f89c2400640 1 -- 172.21.15.78:0/807647907 wait complete. 2024-09-11T12:55:59.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 Processor -- start 2024-09-11T12:55:59.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- start start 2024-09-11T12:55:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a0003f60 0x7f89bc152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89bc1f95b0 0x7f89bc1445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89bc1f89e0 0x7f89bc1f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc0b2100 con 0x7f89bc1f95b0 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f89bc0b1f80 con 0x7f89a0003f60 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f89bc0b2280 con 0x7f89bc1f89e0 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89bc1f95b0 0x7f89bc1445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89bc1f95b0 0x7f89bc1445e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56244/0 (socket says 172.21.15.78:56244) 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 -- 172.21.15.78:0/3698102704 learned_addr learned my addr 172.21.15.78:0/3698102704 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89bbe00640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89bc1f89e0 0x7f89bc1f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89bb400640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a0003f60 0x7f89bc152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 -- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89bc1f89e0 msgr2=0x7f89bc1f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f89bc1f89e0 0x7f89bc1f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 -- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a0003f60 msgr2=0x7f89bc152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f89a0003f60 0x7f89bc152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 -- 172.21.15.78:0/3698102704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89bc136030 con 0x7f89bc1f95b0 2024-09-11T12:55:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89baa00640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89bc1f95b0 0x7f89bc1445e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f89a800c390 tx=0x7f89a8000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a80060c0 con 0x7f89bc1f95b0 2024-09-11T12:55:59.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a8006260 con 0x7f89bc1f95b0 2024-09-11T12:55:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- 172.21.15.78:0/3698102704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89bc136310 con 0x7f89bc1f95b0 2024-09-11T12:55:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a8006670 con 0x7f89bc1f95b0 2024-09-11T12:55:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.370+0000 7f89c2400640 1 -- 172.21.15.78:0/3698102704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89bc152d90 con 0x7f89bc1f95b0 2024-09-11T12:55:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.374+0000 7f89c2400640 1 -- 172.21.15.78:0/3698102704 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89bc0b2de0 con 0x7f89bc1f95b0 2024-09-11T12:55:59.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.374+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7f89a8003600 con 0x7f89bc1f95b0 2024-09-11T12:55:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.374+0000 7f89b3400640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f899c071c20 0x7f899c0740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.374+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f89a812c670 con 0x7f89bc1f95b0 2024-09-11T12:55:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.374+0000 7f89bb400640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f899c071c20 0x7f899c0740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.378+0000 7f89bb400640 1 --2- 172.21.15.78:0/3698102704 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f899c071c20 0x7f899c0740e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89ac011550 tx=0x7f89ac009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.378+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f89a80be060 con 0x7f89bc1f95b0 2024-09-11T12:55:59.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.566+0000 7f89c2400640 1 -- 172.21.15.78:0/3698102704 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f89bc0ccf60 con 0x7f899c071c20 2024-09-11T12:55:59.572 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:59.570+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.570+0000 7f89b3400640 1 -- 172.21.15.78:0/3698102704 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f89bc0ccf60 con 0x7f899c071c20 2024-09-11T12:55:59.572 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.580 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:59.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-11T12:55:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 -- 172.21.15.78:0/2339971739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 msgr2=0x7fa27015a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 --2- 172.21.15.78:0/2339971739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa27015a700 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa270158240 tx=0x7fa25c01c690 comp rx=0 tx=0).stop 2024-09-11T12:55:59.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 -- 172.21.15.78:0/2339971739 shutdown_connections 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 --2- 172.21.15.78:0/2339971739 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa27015a700 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 --2- 172.21.15.78:0/2339971739 >> v2:172.21.15.157:3300/0 conn(0x7fa2700b3040 0x7fa2700cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 -- 172.21.15.78:0/2339971739 >> 172.21.15.78:0/2339971739 conn(0x7fa270071dd0 msgr2=0x7fa2700721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 -- 172.21.15.78:0/2339971739 shutdown_connections 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.722+0000 7fa274e00640 1 -- 172.21.15.78:0/2339971739 wait complete. 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 Processor -- start 2024-09-11T12:55:59.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- start start 2024-09-11T12:55:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2700b3040 0x7fa270157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa270150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2700cd770 0x7fa270158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700af050 con 0x7fa2700b3420 2024-09-11T12:55:59.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2700aeed0 con 0x7fa2700b3040 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2700af1d0 con 0x7fa2700cd770 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa270150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa270150930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56252/0 (socket says 172.21.15.78:56252) 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 -- 172.21.15.78:0/993659274 learned_addr learned my addr 172.21.15.78:0/993659274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26e800640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2700cd770 0x7fa270158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 -- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2700cd770 msgr2=0x7fa270158790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26de00640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2700b3040 0x7fa270157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2700cd770 0x7fa270158790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 -- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2700b3040 msgr2=0x7fa270157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2700b3040 0x7fa270157bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 -- 172.21.15.78:0/993659274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa27013f5d0 con 0x7fa2700b3420 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26e800640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2700cd770 0x7fa270158790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:59.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26de00640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2700b3040 0x7fa270157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa26d400640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2700b3420 0x7fa270150930 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fa25c0b0040 tx=0x7fa25c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c004430 con 0x7fa2700b3420 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa25c005120 con 0x7fa2700b3420 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa25c005570 con 0x7fa2700b3420 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- 172.21.15.78:0/993659274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa270156e10 con 0x7fa2700b3420 2024-09-11T12:55:59.732 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- 172.21.15.78:0/993659274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2701573d0 con 0x7fa2700b3420 2024-09-11T12:55:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.726+0000 7fa274e00640 1 -- 172.21.15.78:0/993659274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2700cf6e0 con 0x7fa2700b3420 2024-09-11T12:55:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.730+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 20) ==== 95442+0+0 (secure 0 0 0) 0x7fa25c0046d0 con 0x7fa2700b3420 2024-09-11T12:55:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.730+0000 7fa266000640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa250071c20 0x7fa2500740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:55:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.730+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fa25c12d7f0 con 0x7fa2700b3420 2024-09-11T12:55:59.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.730+0000 7fa26de00640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa250071c20 0x7fa2500740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:55:59.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.730+0000 7fa26de00640 1 --2- 172.21.15.78:0/993659274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa250071c20 0x7fa2500740e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa26000a8b0 tx=0x7fa260008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:55:59.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.734+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa27015ac40 con 0x7fa2700b3420 2024-09-11T12:55:59.927 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.922+0000 7fa274e00640 1 -- 172.21.15.78:0/993659274 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fa2700725a0 con 0x7fa250071c20 2024-09-11T12:55:59.928 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:55:59.926+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:55:59.926+0000 7fa266000640 1 -- 172.21.15.78:0/993659274 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fa2700725a0 con 0x7fa250071c20 2024-09-11T12:55:59.929 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:55:59.937 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:55:59.938 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-11T12:56:00.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 -- 172.21.15.78:0/3825998389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415ec50 msgr2=0x7f939415c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 --2- 172.21.15.78:0/3825998389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415ec50 0x7f939415c8c0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f93940a9460 tx=0x7f938401ccf0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 -- 172.21.15.78:0/3825998389 shutdown_connections 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 --2- 172.21.15.78:0/3825998389 >> v2:172.21.15.157:3300/0 conn(0x7f93940aad70 0x7f93941520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 --2- 172.21.15.78:0/3825998389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415ec50 0x7f939415c8c0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 -- 172.21.15.78:0/3825998389 >> 172.21.15.78:0/3825998389 conn(0x7f9394075610 msgr2=0x7f9394075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 -- 172.21.15.78:0/3825998389 shutdown_connections 2024-09-11T12:56:00.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.086+0000 7f939b400640 1 -- 172.21.15.78:0/3825998389 wait complete. 2024-09-11T12:56:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 Processor -- start 2024-09-11T12:56:00.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- start start 2024-09-11T12:56:00.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940aad70 0x7f9394143630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f939415ec50 0x7f9394142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415f030 0x7f93940b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f939415db90 con 0x7f939415f030 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f939415da10 con 0x7f939415ec50 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f939415dd10 con 0x7f93940aad70 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415f030 0x7f93940b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415f030 0x7f93940b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56282/0 (socket says 172.21.15.78:56282) 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 -- 172.21.15.78:0/2895533633 learned_addr learned my addr 172.21.15.78:0/2895533633 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9398a00640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940aad70 0x7f9394143630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9393e00640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f939415ec50 0x7f9394142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 -- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940aad70 msgr2=0x7f9394143630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f93940aad70 0x7f9394143630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 -- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f939415ec50 msgr2=0x7f9394142fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f939415ec50 0x7f9394142fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 -- 172.21.15.78:0/2895533633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93941faa70 con 0x7f939415f030 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9399400640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f939415f030 0x7f93940b0ca0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f938c00e8c0 tx=0x7f938c00ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c00ccc0 con 0x7f939415f030 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f938c00ce60 con 0x7f939415f030 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- 172.21.15.78:0/2895533633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93941f8300 con 0x7f939415f030 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c005280 con 0x7f939415f030 2024-09-11T12:56:00.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.090+0000 7f939b400640 1 -- 172.21.15.78:0/2895533633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93941faff0 con 0x7f939415f030 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f939b400640 1 -- 172.21.15.78:0/2895533633 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f939415e240 con 0x7f939415f030 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f938c005420 con 0x7f939415f030 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f9390c00640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9370071cb0 0x7f9370074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f938c093340 con 0x7f939415f030 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f9398a00640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9370071cb0 0x7f9370074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.094+0000 7f9398a00640 1 --2- 172.21.15.78:0/2895533633 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9370071cb0 0x7f9370074170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f93840af000 tx=0x7f938401b530 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.098+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f938c05e0d0 con 0x7f939415f030 2024-09-11T12:56:00.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.290+0000 7f939b400640 1 -- 172.21.15.78:0/2895533633 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f93940707f0 con 0x7f9370071cb0 2024-09-11T12:56:00.293 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:00.290+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.290+0000 7f9390c00640 1 -- 172.21.15.78:0/2895533633 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f93940707f0 con 0x7f9370071cb0 2024-09-11T12:56:00.293 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:00.301 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:00.302 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- 172.21.15.78:0/1391552152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048074060 msgr2=0x7f5048071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 --2- 172.21.15.78:0/1391552152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048074060 0x7f5048071e10 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f503400a950 tx=0x7f503401d910 comp rx=0 tx=0).stop 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- 172.21.15.78:0/1391552152 shutdown_connections 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 --2- 172.21.15.78:0/1391552152 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048074060 0x7f5048071e10 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- 172.21.15.78:0/1391552152 >> 172.21.15.78:0/1391552152 conn(0x7f50480b3050 msgr2=0x7f50480b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:00.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- 172.21.15.78:0/1391552152 shutdown_connections 2024-09-11T12:56:00.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- 172.21.15.78:0/1391552152 wait complete. 2024-09-11T12:56:00.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 Processor -- start 2024-09-11T12:56:00.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- start start 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5048072350 0x7f504815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5048074060 0x7f5048146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048151050 0x7f504814a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50480705a0 con 0x7f5048074060 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5048070420 con 0x7f5048151050 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.446+0000 7f504f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5048070720 con 0x7f5048072350 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5048074060 0x7f5048146b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5048074060 0x7f5048146b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56290/0 (socket says 172.21.15.78:56290) 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 -- 172.21.15.78:0/1400033089 learned_addr learned my addr 172.21.15.78:0/1400033089 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:00.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047e00640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5048072350 0x7f504815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f504d200640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048151050 0x7f504814a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 -- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5048072350 msgr2=0x7f504815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5048072350 0x7f504815ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 -- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048151050 msgr2=0x7f504814a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5048151050 0x7f504814a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 -- 172.21.15.78:0/1400033089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50481476e0 con 0x7f5048074060 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f5047400640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5048074060 0x7f5048146b20 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f503401bfc0 tx=0x7f5034004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50340b3040 con 0x7f5048074060 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5034003600 con 0x7f5048074060 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f504f200640 1 -- 172.21.15.78:0/1400033089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5048147400 con 0x7f5048074060 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5034003a50 con 0x7f5048074060 2024-09-11T12:56:00.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f504f200640 1 -- 172.21.15.78:0/1400033089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f504815a830 con 0x7f5048074060 2024-09-11T12:56:00.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f504f200640 1 -- 172.21.15.78:0/1400033089 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5048071e10 con 0x7f5048074060 2024-09-11T12:56:00.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f50340c0060 con 0x7f5048074060 2024-09-11T12:56:00.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.450+0000 7f503fe00640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5028071cb0 0x7f5028074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.454+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f503412c5c0 con 0x7f5048074060 2024-09-11T12:56:00.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.454+0000 7f5047e00640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5028071cb0 0x7f5028074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.454+0000 7f5047e00640 1 --2- 172.21.15.78:0/1400033089 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5028071cb0 0x7f5028074170 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f50480b0290 tx=0x7f5038009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.454+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5034017680 con 0x7f5048074060 2024-09-11T12:56:00.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.646+0000 7f504f200640 1 -- 172.21.15.78:0/1400033089 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f5048070ce0 con 0x7f5028071cb0 2024-09-11T12:56:00.649 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:00.646+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.646+0000 7f503fe00640 1 -- 172.21.15.78:0/1400033089 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5048070ce0 con 0x7f5028071cb0 2024-09-11T12:56:00.650 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:00.658 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:00.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-11T12:56:00.802 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 -- 172.21.15.78:0/4119197182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cde00 msgr2=0x7fa9f006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 --2- 172.21.15.78:0/4119197182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cde00 0x7fa9f006fd50 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009840 tx=0x7fa9dc01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 -- 172.21.15.78:0/4119197182 shutdown_connections 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 --2- 172.21.15.78:0/4119197182 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cde00 0x7fa9f006fd50 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 -- 172.21.15.78:0/4119197182 >> 172.21.15.78:0/4119197182 conn(0x7fa9f00b2260 msgr2=0x7fa9f00b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 -- 172.21.15.78:0/4119197182 shutdown_connections 2024-09-11T12:56:00.803 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.798+0000 7fa9f6e00640 1 -- 172.21.15.78:0/4119197182 wait complete. 2024-09-11T12:56:00.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 Processor -- start 2024-09-11T12:56:00.804 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- start start 2024-09-11T12:56:00.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0073200 0x7fa9f0144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cde00 0x7fa9f014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cf480 0x7fa9f0132740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.806 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f016c680 con 0x7fa9f00cf480 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f016c500 con 0x7fa9f0073200 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa9f016c800 con 0x7fa9f00cde00 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cf480 0x7fa9f0132740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cf480 0x7fa9f0132740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56308/0 (socket says 172.21.15.78:56308) 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9efe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0073200 0x7fa9f0144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 -- 172.21.15.78:0/2597747500 learned_addr learned my addr 172.21.15.78:0/2597747500 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9ef400640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cde00 0x7fa9f014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 -- 172.21.15.78:0/2597747500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cde00 msgr2=0x7fa9f014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9f00cde00 0x7fa9f014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 -- 172.21.15.78:0/2597747500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0073200 msgr2=0x7fa9f0144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9f0073200 0x7fa9f0144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 -- 172.21.15.78:0/2597747500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f01316a0 con 0x7fa9f00cf480 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f4e00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9f00cf480 0x7fa9f0132740 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c410 tx=0x7fa9e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011790 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9e8017070 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- 172.21.15.78:0/2597747500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f0131980 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8011cc0 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- 172.21.15.78:0/2597747500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f0131f40 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.802+0000 7fa9f6e00640 1 -- 172.21.15.78:0/2597747500 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f00ced70 con 0x7fa9f00cf480 2024-09-11T12:56:00.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.806+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fa9e801c460 con 0x7fa9f00cf480 2024-09-11T12:56:00.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.806+0000 7fa9e7e00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa9d0071ce0 0x7fa9d00741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:00.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.806+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fa9e80925a0 con 0x7fa9f00cf480 2024-09-11T12:56:00.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.806+0000 7fa9efe00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa9d0071ce0 0x7fa9d00741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:00.810 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.806+0000 7fa9efe00640 1 --2- 172.21.15.78:0/2597747500 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa9d0071ce0 0x7fa9d00741a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000b450 tx=0x7fa9e000d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:00.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.810+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa9e805d4d0 con 0x7fa9f00cf480 2024-09-11T12:56:01.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:00.998+0000 7fa9f6e00640 1 -- 172.21.15.78:0/2597747500 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7fa9f00b0330 con 0x7fa9d0071ce0 2024-09-11T12:56:01.003 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:01.002+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.002+0000 7fa9e7e00640 1 -- 172.21.15.78:0/2597747500 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fa9f00b0330 con 0x7fa9d0071ce0 2024-09-11T12:56:01.004 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.012 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:01.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- 172.21.15.78:0/1067533787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 msgr2=0x7fd0f415af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- 172.21.15.78:0/1067533787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 0x7fd0f415af50 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4009bc0 tx=0x7fd0e401cb70 comp rx=0 tx=0).stop 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- 172.21.15.78:0/1067533787 shutdown_connections 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- 172.21.15.78:0/1067533787 >> v2:172.21.15.157:3300/0 conn(0x7fd0f4073cb0 0x7fd0f415b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- 172.21.15.78:0/1067533787 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 0x7fd0f415af50 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- 172.21.15.78:0/1067533787 >> 172.21.15.78:0/1067533787 conn(0x7fd0f40cd190 msgr2=0x7fd0f40cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- 172.21.15.78:0/1067533787 shutdown_connections 2024-09-11T12:56:01.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- 172.21.15.78:0/1067533787 wait complete. 2024-09-11T12:56:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 Processor -- start 2024-09-11T12:56:01.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- start start 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 0x7fd0f415a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0f4073cb0 0x7fd0f412a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0f40c5dd0 0x7fd0f4141530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41676b0 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f415def0 con 0x7fd0f4073cb0 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0fac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0f4167830 con 0x7fd0f40738d0 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0f8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0f40c5dd0 0x7fd0f4141530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0f8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0f40c5dd0 0x7fd0f4141530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56320/0 (socket says 172.21.15.78:56320) 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0f8c00640 1 -- 172.21.15.78:0/1125637391 learned_addr learned my addr 172.21.15.78:0/1125637391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.158+0000 7fd0f3e00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 0x7fd0f415a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f3400640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0f4073cb0 0x7fd0f412a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 -- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 msgr2=0x7fd0f415a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0f40738d0 0x7fd0f415a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 -- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0f4073cb0 msgr2=0x7fd0f412a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0f4073cb0 0x7fd0f412a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 -- 172.21.15.78:0/1125637391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f4140490 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0f8c00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0f40c5dd0 0x7fd0f4141530 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec00c3c0 tx=0x7fd0ec00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec011790 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0ec017070 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0fac00640 1 -- 172.21.15.78:0/1125637391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f4140770 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0ec011ce0 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0fac00640 1 -- 172.21.15.78:0/1125637391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f4140d30 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0fac00640 1 -- 172.21.15.78:0/1125637391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f40c6260 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.162+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fd0ec01c460 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.166+0000 7fd0ebe00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0d4071cb0 0x7fd0d4074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.166+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fd0ec092510 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.166+0000 7fd0f3e00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0d4071cb0 0x7fd0d4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.166+0000 7fd0f3e00640 1 --2- 172.21.15.78:0/1125637391 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0d4071cb0 0x7fd0d4074170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40af000 tx=0x7fd0e401b3b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.166+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd0ec05d440 con 0x7fd0f40c5dd0 2024-09-11T12:56:01.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.354+0000 7fd0fac00640 1 -- 172.21.15.78:0/1125637391 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fd0f40c6680 con 0x7fd0d4071cb0 2024-09-11T12:56:01.360 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:01.358+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.358+0000 7fd0ebe00640 1 -- 172.21.15.78:0/1125637391 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fd0f40c6680 con 0x7fd0d4071cb0 2024-09-11T12:56:01.361 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.370 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:01.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 -- 172.21.15.78:0/2317968936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f00b5b70 msgr2=0x7f84f0072b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 --2- 172.21.15.78:0/2317968936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f00b5b70 0x7f84f0072b00 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f84f00cd910 tx=0x7f84dc01c800 comp rx=0 tx=0).stop 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 -- 172.21.15.78:0/2317968936 shutdown_connections 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 --2- 172.21.15.78:0/2317968936 >> v2:172.21.15.157:3300/0 conn(0x7f84f0151340 0x7f84f01517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 --2- 172.21.15.78:0/2317968936 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f00b5b70 0x7f84f0072b00 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 -- 172.21.15.78:0/2317968936 >> 172.21.15.78:0/2317968936 conn(0x7f84f00b3050 msgr2=0x7f84f00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 -- 172.21.15.78:0/2317968936 shutdown_connections 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 -- 172.21.15.78:0/2317968936 wait complete. 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.514+0000 7f84f6200640 1 Processor -- start 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- start start 2024-09-11T12:56:01.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f00b5790 0x7f84f014a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f00b5b70 0x7f84f015d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f0151340 0x7f84f0146800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0167ce0 con 0x7f84f0151340 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84f0167b60 con 0x7f84f00b5b70 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84f0167e60 con 0x7f84f00b5790 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f0151340 0x7f84f0146800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f0151340 0x7f84f0146800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56328/0 (socket says 172.21.15.78:56328) 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 -- 172.21.15.78:0/3081899101 learned_addr learned my addr 172.21.15.78:0/3081899101 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84ef400640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f00b5790 0x7f84f014a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84eea00640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f00b5b70 0x7f84f015d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 -- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f00b5790 msgr2=0x7f84f014a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f84f00b5790 0x7f84f014a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 -- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f00b5b70 msgr2=0x7f84f015d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f84f00b5b70 0x7f84f015d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 -- 172.21.15.78:0/3081899101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f014cd90 con 0x7f84f0151340 2024-09-11T12:56:01.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84efe00640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f84f0151340 0x7f84f0146800 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f84e800b3c0 tx=0x7f84e800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011830 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84e8016070 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84e8011d60 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- 172.21.15.78:0/3081899101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f014bfd0 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- 172.21.15.78:0/3081899101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f014c590 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.518+0000 7f84f6200640 1 -- 172.21.15.78:0/3081899101 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f01517d0 con 0x7f84f0151340 2024-09-11T12:56:01.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.522+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f84e801b460 con 0x7f84f0151340 2024-09-11T12:56:01.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.522+0000 7f84e7400640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84d0071cb0 0x7f84d0074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.522+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f84e8092460 con 0x7f84f0151340 2024-09-11T12:56:01.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.522+0000 7f84ef400640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84d0071cb0 0x7f84d0074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.522+0000 7f84ef400640 1 --2- 172.21.15.78:0/3081899101 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84d0071cb0 0x7f84d0074170 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f84e000f290 tx=0x7f84e0001aa0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.529 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.526+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f84e805d390 con 0x7f84f0151340 2024-09-11T12:56:01.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.714+0000 7f84f6200640 1 -- 172.21.15.78:0/3081899101 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f84f0072010 con 0x7f84d0071cb0 2024-09-11T12:56:01.720 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:01.718+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.718+0000 7f84e7400640 1 -- 172.21.15.78:0/3081899101 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f84f0072010 con 0x7f84d0071cb0 2024-09-11T12:56:01.720 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:01.728 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:01.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 -- 172.21.15.78:0/2630180769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00707c0 msgr2=0x7fc8d013fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 --2- 172.21.15.78:0/2630180769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00707c0 0x7fc8d013fe70 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009c80 tx=0x7fc8c801cda0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 -- 172.21.15.78:0/2630180769 shutdown_connections 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 --2- 172.21.15.78:0/2630180769 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00707c0 0x7fc8d013fe70 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 --2- 172.21.15.78:0/2630180769 >> v2:172.21.15.157:3300/0 conn(0x7fc8d00703e0 0x7fc8d015da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 -- 172.21.15.78:0/2630180769 >> 172.21.15.78:0/2630180769 conn(0x7fc8d00b2c70 msgr2=0x7fc8d00b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 -- 172.21.15.78:0/2630180769 shutdown_connections 2024-09-11T12:56:01.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.870+0000 7fc8d5800640 1 -- 172.21.15.78:0/2630180769 wait complete. 2024-09-11T12:56:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 Processor -- start 2024-09-11T12:56:01.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 -- start start 2024-09-11T12:56:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d00703e0 0x7fc8d0133490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8d00707c0 0x7fc8d0152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00cf3e0 0x7fc8d01523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d00737d0 con 0x7fc8d00707c0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d0073650 con 0x7fc8d00703e0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d5800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc8d0168eb0 con 0x7fc8d00cf3e0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8d00707c0 0x7fc8d0152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d00703e0 0x7fc8d0133490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d00703e0 0x7fc8d0133490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:37710/0 (socket says 172.21.15.78:37710) 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cfe00640 1 -- 172.21.15.78:0/219447081 learned_addr learned my addr 172.21.15.78:0/219447081 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d4e00640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00cf3e0 0x7fc8d01523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 -- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00cf3e0 msgr2=0x7fc8d01523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00cf3e0 0x7fc8d01523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 -- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d00703e0 msgr2=0x7fc8d0133490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc8d00703e0 0x7fc8d0133490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 -- 172.21.15.78:0/219447081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d0153270 con 0x7fc8d00707c0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8d4e00640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc8d00cf3e0 0x7fc8d01523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8cf400640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc8d00707c0 0x7fc8d0152be0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc00c3c0 tx=0x7fc8bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.874+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc017070 con 0x7fc8d00707c0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8d5800640 1 -- 172.21.15.78:0/219447081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d0153550 con 0x7fc8d00707c0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8bc0053c0 con 0x7fc8d00707c0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc003860 con 0x7fc8d00707c0 2024-09-11T12:56:01.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8d5800640 1 -- 172.21.15.78:0/219447081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d0153b10 con 0x7fc8d00707c0 2024-09-11T12:56:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8d5800640 1 -- 172.21.15.78:0/219447081 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d015b820 con 0x7fc8d00707c0 2024-09-11T12:56:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc8bc01a070 con 0x7fc8d00707c0 2024-09-11T12:56:01.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8c7e00640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8b0071cb0 0x7fc8b0074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fc8bc091cd0 con 0x7fc8d00707c0 2024-09-11T12:56:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8cfe00640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8b0071cb0 0x7fc8b0074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:01.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.878+0000 7fc8cfe00640 1 --2- 172.21.15.78:0/219447081 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc8b0071cb0 0x7fc8b0074170 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009ce0 tx=0x7fc8c00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:01.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:01.882+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc8bc05cbb0 con 0x7fc8d00707c0 2024-09-11T12:56:02.078 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.074+0000 7fc8d5800640 1 -- 172.21.15.78:0/219447081 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7fc8d013b5f0 con 0x7fc8b0071cb0 2024-09-11T12:56:02.079 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:02.078+0000 7f1eccc00640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-11T12:56:02.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.078+0000 7fc8c7e00640 1 -- 172.21.15.78:0/219447081 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc8d013b5f0 con 0x7fc8b0071cb0 2024-09-11T12:56:02.080 INFO:teuthology.orchestra.run.smithi078.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-11T12:56:02.089 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-11T12:56:02.089 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:56:02.090 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:02.198 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:02.293 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:56:02.293 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:02.294 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:02.294 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:02.302 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:02.303 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:02.352 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- 172.21.15.78:0/3208243958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157a50 msgr2=0x7f38c4070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 --2- 172.21.15.78:0/3208243958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157a50 0x7f38c4070780 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f38c40afa30 tx=0x7f38b801c800 comp rx=0 tx=0).stop 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- 172.21.15.78:0/3208243958 shutdown_connections 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 --2- 172.21.15.78:0/3208243958 >> v2:172.21.15.157:3300/0 conn(0x7f38c4157e30 0x7f38c40b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 --2- 172.21.15.78:0/3208243958 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157a50 0x7f38c4070780 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- 172.21.15.78:0/3208243958 >> 172.21.15.78:0/3208243958 conn(0x7f38c406ca80 msgr2=0x7f38c40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- 172.21.15.78:0/3208243958 shutdown_connections 2024-09-11T12:56:02.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- 172.21.15.78:0/3208243958 wait complete. 2024-09-11T12:56:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 Processor -- start 2024-09-11T12:56:02.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.534+0000 7f38cc000640 1 -- start start 2024-09-11T12:56:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 0x7f38c414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40b1be0 0x7f38c41fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:02.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40b4b80 con 0x7f38c4157e30 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f38c40b4a00 con 0x7f38c4157a50 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f38c4167ed0 con 0x7f38c40b1be0 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56350/0 (socket says 172.21.15.78:56350) 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 -- 172.21.15.78:0/2643077390 learned_addr learned my addr 172.21.15.78:0/2643077390 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c9600640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 0x7f38c414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38ca000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40b1be0 0x7f38c41fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 -- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40b1be0 msgr2=0x7f38c41fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40b1be0 0x7f38c41fb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 -- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 msgr2=0x7f38c414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 0x7f38c414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 -- 172.21.15.78:0/2643077390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c41f8980 con 0x7f38c4157e30 2024-09-11T12:56:02.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c9600640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 0x7f38c414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c8c00640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f38b000c480 tx=0x7f38b000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0012830 con 0x7f38c4157e30 2024-09-11T12:56:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b0017070 con 0x7f38c4157e30 2024-09-11T12:56:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c41f7bc0 con 0x7f38c4157e30 2024-09-11T12:56:02.543 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b0012d80 con 0x7f38c4157e30 2024-09-11T12:56:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c41f8180 con 0x7f38c4157e30 2024-09-11T12:56:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.538+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c40b1ff0 con 0x7f38c4157e30 2024-09-11T12:56:02.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.542+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f38b001c460 con 0x7f38c4157e30 2024-09-11T12:56:02.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.542+0000 7f38c1600640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 0x7f38a40741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.542+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f38b0092610 con 0x7f38c4157e30 2024-09-11T12:56:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.542+0000 7f38c9600640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 0x7f38a40741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:02.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.542+0000 7f38c9600640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 0x7f38a40741d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38b80af000 tx=0x7f38b801b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:02.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.546+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f38c40b1ff0 con 0x7f38c4157e30 2024-09-11T12:56:02.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.722+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f38c4070780 con 0x7f38c4157e30 2024-09-11T12:56:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.722+0000 7f38c1600640 1 -- 172.21.15.78:0/2643077390 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v129) ==== 80+0+0 (secure 0 0 0) 0x7f38b005dcd0 con 0x7f38c4157e30 2024-09-11T12:56:02.726 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:56:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 msgr2=0x7f38a40741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 0x7f38a40741d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38b80af000 tx=0x7f38b801b040 comp rx=0 tx=0).stop 2024-09-11T12:56:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 msgr2=0x7f38c40b16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f38b000c480 tx=0x7f38b000c940 comp rx=0 tx=0).stop 2024-09-11T12:56:02.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 shutdown_connections 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f38c40b1be0 0x7f38c41fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f38c4157e30 0x7f38c40b16a0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f38a4071d10 0x7f38a40741d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 --2- 172.21.15.78:0/2643077390 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f38c4157a50 0x7f38c414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 >> 172.21.15.78:0/2643077390 conn(0x7f38c406ca80 msgr2=0x7f38c40cf430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 shutdown_connections 2024-09-11T12:56:02.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:02.726+0000 7f38cc000640 1 -- 172.21.15.78:0/2643077390 wait complete. 2024-09-11T12:56:02.746 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:56:02.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:02.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:02.772 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:56:02.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:02.772 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:56:02.824 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:02.824 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:02.824 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:02.834 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:02.834 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:02.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 -- 172.21.15.78:0/1414862169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c415ba80 msgr2=0x7fa1c406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 --2- 172.21.15.78:0/1414862169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c415ba80 0x7fa1c406fc50 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4152090 tx=0x7fa1b001c890 comp rx=0 tx=0).stop 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 -- 172.21.15.78:0/1414862169 shutdown_connections 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 --2- 172.21.15.78:0/1414862169 >> v2:172.21.15.157:3300/0 conn(0x7fa1c4070190 0x7fa1c4071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 --2- 172.21.15.78:0/1414862169 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c415ba80 0x7fa1c406fc50 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 -- 172.21.15.78:0/1414862169 >> 172.21.15.78:0/1414862169 conn(0x7fa1c406ca80 msgr2=0x7fa1c40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 -- 172.21.15.78:0/1414862169 shutdown_connections 2024-09-11T12:56:03.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.022+0000 7fa1cac00640 1 -- 172.21.15.78:0/1414862169 wait complete. 2024-09-11T12:56:03.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 Processor -- start 2024-09-11T12:56:03.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- start start 2024-09-11T12:56:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1c4074240 0x7fa1c414f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c415ba80 0x7fa1c41fa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c41658b0 con 0x7fa1c4070190 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa1c4165730 con 0x7fa1c415ba80 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa1c4165a30 con 0x7fa1c4074240 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56366/0 (socket says 172.21.15.78:56366) 2024-09-11T12:56:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 -- 172.21.15.78:0/3248617347 learned_addr learned my addr 172.21.15.78:0/3248617347 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3400640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1c4074240 0x7fa1c414f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c8c00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c415ba80 0x7fa1c41fa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 -- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1c4074240 msgr2=0x7fa1c414f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1c4074240 0x7fa1c414f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 -- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c415ba80 msgr2=0x7fa1c41fa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c415ba80 0x7fa1c41fa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 -- 172.21.15.78:0/3248617347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1c4152330 con 0x7fa1c4070190 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1c3e00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c870 tx=0x7fa1b400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b4003b80 con 0x7fa1c4070190 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1b4003d20 con 0x7fa1c4070190 2024-09-11T12:56:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1c4152610 con 0x7fa1c4070190 2024-09-11T12:56:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1b400aab0 con 0x7fa1c4070190 2024-09-11T12:56:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.026+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1c414ead0 con 0x7fa1c4070190 2024-09-11T12:56:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1c4071d60 con 0x7fa1c4070190 2024-09-11T12:56:03.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fa1b400ac50 con 0x7fa1c4070190 2024-09-11T12:56:03.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1bbe00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 0x7fa1a40741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fa1b4092610 con 0x7fa1c4070190 2024-09-11T12:56:03.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1c3400640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 0x7fa1a40741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.030+0000 7fa1c3400640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 0x7fa1a40741d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00af000 tx=0x7fa1b001b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.034+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa1c40705a0 con 0x7fa1c4070190 2024-09-11T12:56:03.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa1c415a750 con 0x7fa1c4070190 2024-09-11T12:56:03.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1bbe00640 1 -- 172.21.15.78:0/3248617347 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v129) ==== 80+0+0 (secure 0 0 0) 0x7fa1b405d580 con 0x7fa1c4070190 2024-09-11T12:56:03.216 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 msgr2=0x7fa1a40741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 0x7fa1a40741d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa1b00af000 tx=0x7fa1b001b0d0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 msgr2=0x7fa1c414e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa1b400c870 tx=0x7fa1b400cd30 comp rx=0 tx=0).stop 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 shutdown_connections 2024-09-11T12:56:03.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa1a4071d10 0x7fa1a40741d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa1c415ba80 0x7fa1c41fa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa1c4074240 0x7fa1c414f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 --2- 172.21.15.78:0/3248617347 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa1c4070190 0x7fa1c414e590 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 >> 172.21.15.78:0/3248617347 conn(0x7fa1c406ca80 msgr2=0x7fa1c4072b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:03.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 shutdown_connections 2024-09-11T12:56:03.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.214+0000 7fa1cac00640 1 -- 172.21.15.78:0/3248617347 wait complete. 2024-09-11T12:56:03.236 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:56:03.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:03.237 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:03.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:56:03.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:03.267 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:56:03.320 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:03.320 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:03.320 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:03.329 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:03.329 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:03.329 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:03.329 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:03.370 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:03.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.506+0000 7f7068a00640 1 -- 172.21.15.78:0/1702654661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064073710 msgr2=0x7f706415ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.506+0000 7f7068a00640 1 --2- 172.21.15.78:0/1702654661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064073710 0x7f706415ae20 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f70640afa30 tx=0x7f705001c800 comp rx=0 tx=0).stop 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- 172.21.15.78:0/1702654661 shutdown_connections 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 --2- 172.21.15.78:0/1702654661 >> v2:172.21.15.157:3300/0 conn(0x7f706415b360 0x7f70640b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 --2- 172.21.15.78:0/1702654661 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064073710 0x7f706415ae20 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- 172.21.15.78:0/1702654661 >> 172.21.15.78:0/1702654661 conn(0x7f706406ca80 msgr2=0x7f70640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- 172.21.15.78:0/1702654661 shutdown_connections 2024-09-11T12:56:03.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- 172.21.15.78:0/1702654661 wait complete. 2024-09-11T12:56:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 Processor -- start 2024-09-11T12:56:03.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- start start 2024-09-11T12:56:03.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 0x7f7064152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70640cb940 0x7f70641517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f706406f4b0 con 0x7f7064156960 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f706406f330 con 0x7f70640cb940 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7064164db0 con 0x7f7064157a50 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56376/0 (socket says 172.21.15.78:56376) 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 -- 172.21.15.78:0/2578203234 learned_addr learned my addr 172.21.15.78:0/2578203234 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7061e00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 0x7f7064152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7061400640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70640cb940 0x7f70641517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 -- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 msgr2=0x7f7064152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 0x7f7064152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 -- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70640cb940 msgr2=0x7f70641517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70640cb940 0x7f70641517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 -- 172.21.15.78:0/2578203234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641fb1a0 con 0x7f7064156960 2024-09-11T12:56:03.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7061e00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 0x7f7064152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7062800640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c970 tx=0x7f705c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c007bb0 con 0x7f7064156960 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f705c00f450 con 0x7f7064156960 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c007e60 con 0x7f7064156960 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.510+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641fb480 con 0x7f7064156960 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641fba40 con 0x7f7064156960 2024-09-11T12:56:03.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f706415ae20 con 0x7f7064156960 2024-09-11T12:56:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f705c00f8a0 con 0x7f7064156960 2024-09-11T12:56:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f705a000640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 0x7f70440741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f705c093010 con 0x7f7064156960 2024-09-11T12:56:03.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f7061e00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 0x7f70440741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.514+0000 7f7061e00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 0x7f70440741a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70640af460 tx=0x7f7054009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.522 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.518+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f705c014030 con 0x7f7064156960 2024-09-11T12:56:03.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.718+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70640b2880 con 0x7f7064156960 2024-09-11T12:56:03.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.718+0000 7f705a000640 1 -- 172.21.15.78:0/2578203234 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f705c05e480 con 0x7f7064156960 2024-09-11T12:56:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:03.723 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"btime":"2024-09-11T12:55:53:010575+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:56:03.723 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 92 2024-09-11T12:56:03.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 msgr2=0x7f70440741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 0x7f70440741a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f70640af460 tx=0x7f7054009040 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 msgr2=0x7f7064156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c970 tx=0x7f705c00ce30 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 shutdown_connections 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7064156960 0x7f7064156df0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7044071ce0 0x7f70440741a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f70640cb940 0x7f70641517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 --2- 172.21.15.78:0/2578203234 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7064157a50 0x7f7064152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 >> 172.21.15.78:0/2578203234 conn(0x7f706406ca80 msgr2=0x7f70640cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 shutdown_connections 2024-09-11T12:56:03.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.722+0000 7f7068a00640 1 -- 172.21.15.78:0/2578203234 wait complete. 2024-09-11T12:56:03.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- 172.21.15.78:0/1443373495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60415bc30 msgr2=0x7fc604072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 --2- 172.21.15.78:0/1443373495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60415bc30 0x7fc604072ba0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc009900 tx=0x7fc5fc01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- 172.21.15.78:0/1443373495 shutdown_connections 2024-09-11T12:56:03.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 --2- 172.21.15.78:0/1443373495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc60415bc30 0x7fc604072ba0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 --2- 172.21.15.78:0/1443373495 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415b850 0x7fc604072660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- 172.21.15.78:0/1443373495 >> 172.21.15.78:0/1443373495 conn(0x7fc604073b70 msgr2=0x7fc604073f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- 172.21.15.78:0/1443373495 shutdown_connections 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- 172.21.15.78:0/1443373495 wait complete. 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 Processor -- start 2024-09-11T12:56:03.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.882+0000 7fc60ac00640 1 -- start start 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60415b850 0x7fc60414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415bc30 0x7fc60414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6041690d0 con 0x7fc604070110 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc604168f50 con 0x7fc60415b850 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc604169250 con 0x7fc60415bc30 2024-09-11T12:56:03.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36010/0 (socket says 172.21.15.78:36010) 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 -- 172.21.15.78:0/437258274 learned_addr learned my addr 172.21.15.78:0/437258274 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc608c00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415bc30 0x7fc60414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603400640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60415b850 0x7fc60414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 -- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415bc30 msgr2=0x7fc60414d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415bc30 0x7fc60414d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 -- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60415b850 msgr2=0x7fc60414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60415b850 0x7fc60414d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:03.890 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 -- 172.21.15.78:0/437258274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc604154080 con 0x7fc604070110 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc603e00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400ead0 tx=0x7fc5f400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400ce40 con 0x7fc604070110 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f4004c20 con 0x7fc604070110 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc604154360 con 0x7fc604070110 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4005280 con 0x7fc604070110 2024-09-11T12:56:03.891 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041f8120 con 0x7fc604070110 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc60415f650 con 0x7fc604070110 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.886+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc5f4005420 con 0x7fc604070110 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.890+0000 7fc5fbe00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 0x7fc5e4074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.890+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7fc5f40933f0 con 0x7fc604070110 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.890+0000 7fc603400640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 0x7fc5e4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:03.892 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.890+0000 7fc603400640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 0x7fc5e4074170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc60415a0c0 tx=0x7fc5f0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:03.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:03.894+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc5f4016030 con 0x7fc604070110 2024-09-11T12:56:04.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc6040c4350 con 0x7fc604070110 2024-09-11T12:56:04.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc5fbe00640 1 -- 172.21.15.78:0/437258274 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7fc5f405e7d0 con 0x7fc604070110 2024-09-11T12:56:04.097 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:04.097 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"btime":"2024-09-11T12:55:53:010575+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:56:04.098 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 92 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 msgr2=0x7fc5e4074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 0x7fc5e4074170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc60415a0c0 tx=0x7fc5f0009040 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 msgr2=0x7fc6041f8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400ead0 tx=0x7fc5f400ef90 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 shutdown_connections 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc60415bc30 0x7fc60414d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5e4071cb0 0x7fc5e4074170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc60415b850 0x7fc60414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 --2- 172.21.15.78:0/437258274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc604070110 0x7fc6041f8920 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 >> 172.21.15.78:0/437258274 conn(0x7fc604073b70 msgr2=0x7fc604073f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.094+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 shutdown_connections 2024-09-11T12:56:04.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.098+0000 7fc60ac00640 1 -- 172.21.15.78:0/437258274 wait complete. 2024-09-11T12:56:04.122 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 -- 172.21.15.78:0/1230920268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 msgr2=0x7f495c074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 --2- 172.21.15.78:0/1230920268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c074260 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f494c009c80 tx=0x7f494c01cd70 comp rx=0 tx=0).stop 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 -- 172.21.15.78:0/1230920268 shutdown_connections 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 --2- 172.21.15.78:0/1230920268 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c074260 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 -- 172.21.15.78:0/1230920268 >> 172.21.15.78:0/1230920268 conn(0x7f495c06ca80 msgr2=0x7f495c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 -- 172.21.15.78:0/1230920268 shutdown_connections 2024-09-11T12:56:04.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.258+0000 7f4963200640 1 -- 172.21.15.78:0/1230920268 wait complete. 2024-09-11T12:56:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 Processor -- start 2024-09-11T12:56:04.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- start start 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495c06fc50 0x7f495c14a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495c15aec0 0x7f495c1fad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1519a0 con 0x7f495c0cf760 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f495c151820 con 0x7f495c15aec0 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f495c151b20 con 0x7f495c06fc50 2024-09-11T12:56:04.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36036/0 (socket says 172.21.15.78:36036) 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 -- 172.21.15.78:0/354130184 learned_addr learned my addr 172.21.15.78:0/354130184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495be00640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495c06fc50 0x7f495c14a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4961200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495c15aec0 0x7f495c1fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 -- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495c06fc50 msgr2=0x7f495c14a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495c06fc50 0x7f495c14a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 -- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495c15aec0 msgr2=0x7f495c1fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495c15aec0 0x7f495c1fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 -- 172.21.15.78:0/354130184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c144780 con 0x7f495c0cf760 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f495b400640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f494800c480 tx=0x7f494800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49480052d0 con 0x7f495c0cf760 2024-09-11T12:56:04.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4948005470 con 0x7f495c0cf760 2024-09-11T12:56:04.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c1fb240 con 0x7f495c0cf760 2024-09-11T12:56:04.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4948012870 con 0x7f495c0cf760 2024-09-11T12:56:04.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.262+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c1fb7f0 con 0x7f495c0cf760 2024-09-11T12:56:04.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f495c0cc370 con 0x7f495c0cf760 2024-09-11T12:56:04.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f4948003600 con 0x7f495c0cf760 2024-09-11T12:56:04.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f4953e00640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 0x7f493c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f4948092490 con 0x7f495c0cf760 2024-09-11T12:56:04.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f495be00640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 0x7f493c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.266+0000 7f495be00640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 0x7f493c074170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f494c0af000 tx=0x7f494c01b5b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.270+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f494805da50 con 0x7f495c0cf760 2024-09-11T12:56:04.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f495c074260 con 0x7f495c0cf760 2024-09-11T12:56:04.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4953e00640 1 -- 172.21.15.78:0/354130184 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16739 (secure 0 0 0) 0x7f494805d350 con 0x7f495c0cf760 2024-09-11T12:56:04.441 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:04.441 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:44.968878+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:41.069903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:42.319816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:04.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 msgr2=0x7f493c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 0x7f493c074170 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f494c0af000 tx=0x7f494c01b5b0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 msgr2=0x7f495c144240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f494800c480 tx=0x7f494800c940 comp rx=0 tx=0).stop 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 shutdown_connections 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f493c071cb0 0x7f493c074170 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f495c15aec0 0x7f495c1fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f495c0cf760 0x7f495c144240 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 --2- 172.21.15.78:0/354130184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f495c06fc50 0x7f495c14a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 >> 172.21.15.78:0/354130184 conn(0x7f495c06ca80 msgr2=0x7f495c149700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 shutdown_connections 2024-09-11T12:56:04.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.438+0000 7f4963200640 1 -- 172.21.15.78:0/354130184 wait complete. 2024-09-11T12:56:04.463 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:56:04.464 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 -- 172.21.15.78:0/722192338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53601518f0 msgr2=0x7f5360073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 --2- 172.21.15.78:0/722192338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53601518f0 0x7f5360073c70 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f536006ab40 tx=0x7f535001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 -- 172.21.15.78:0/722192338 shutdown_connections 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 --2- 172.21.15.78:0/722192338 >> v2:172.21.15.157:3300/0 conn(0x7f53600cfeb0 0x7f536015c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 --2- 172.21.15.78:0/722192338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53601518f0 0x7f5360073c70 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 -- 172.21.15.78:0/722192338 >> 172.21.15.78:0/722192338 conn(0x7f536006f3a0 msgr2=0x7f53600cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 -- 172.21.15.78:0/722192338 shutdown_connections 2024-09-11T12:56:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 -- 172.21.15.78:0/722192338 wait complete. 2024-09-11T12:56:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.602+0000 7f5367800640 1 Processor -- start 2024-09-11T12:56:04.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- start start 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53601518f0 0x7f5360159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 0x7f5360159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5360167740 con 0x7f53600cfeb0 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f53601675c0 con 0x7f536015ae40 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f53601678c0 con 0x7f53601518f0 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36056/0 (socket says 172.21.15.78:36056) 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 -- 172.21.15.78:0/3667134155 learned_addr learned my addr 172.21.15.78:0/3667134155 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5365800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 0x7f5360159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f535fe00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53601518f0 0x7f5360159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 -- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53601518f0 msgr2=0x7f5360159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53601518f0 0x7f5360159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 -- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 msgr2=0x7f5360159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 0x7f5360159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 -- 172.21.15.78:0/3667134155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5360050ba0 con 0x7f53600cfeb0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5365800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 0x7f5360159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5364e00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f535001af70 tx=0x7f5350004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500ba020 con 0x7f53600cfeb0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5350005030 con 0x7f53600cfeb0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536014dd70 con 0x7f53600cfeb0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53500b2040 con 0x7f53600cfeb0 2024-09-11T12:56:04.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536014e330 con 0x7f53600cfeb0 2024-09-11T12:56:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.606+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536015c630 con 0x7f53600cfeb0 2024-09-11T12:56:04.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.610+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f53500046d0 con 0x7f53600cfeb0 2024-09-11T12:56:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.610+0000 7f535cc00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 0x7f53400741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.610+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f535012cbb0 con 0x7f53600cfeb0 2024-09-11T12:56:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.610+0000 7f535fe00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 0x7f53400741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.610+0000 7f535fe00640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 0x7f53400741a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f536015a8e0 tx=0x7f534c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.614+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f53500c4060 con 0x7f53600cfeb0 2024-09-11T12:56:04.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.810+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5360152810 con 0x7f53600cfeb0 2024-09-11T12:56:04.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.810+0000 7f535cc00640 1 -- 172.21.15.78:0/3667134155 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f53500164b0 con 0x7f53600cfeb0 2024-09-11T12:56:04.814 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:04.814 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"btime":"2024-09-11T12:55:53:010575+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:56:04.815 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 92 2024-09-11T12:56:04.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 msgr2=0x7f53400741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 0x7f53400741a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f536015a8e0 tx=0x7f534c008040 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 msgr2=0x7f5360142000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f535001af70 tx=0x7f5350004040 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 shutdown_connections 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5340071ce0 0x7f53400741a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f536015ae40 0x7f5360159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f53601518f0 0x7f5360159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 --2- 172.21.15.78:0/3667134155 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f53600cfeb0 0x7f5360142000 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 >> 172.21.15.78:0/3667134155 conn(0x7f536006f3a0 msgr2=0x7f5360142960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 shutdown_connections 2024-09-11T12:56:04.817 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.814+0000 7f5367800640 1 -- 172.21.15.78:0/3667134155 wait complete. 2024-09-11T12:56:04.836 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:04.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 -- 172.21.15.78:0/3769236297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c003c70 msgr2=0x7f4e4c004070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 --2- 172.21.15.78:0/3769236297 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e4c003c70 0x7f4e4c004070 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f4e58001860 tx=0x7f4e5801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6a000640 1 -- 172.21.15.78:0/3769236297 reap_dead start 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 -- 172.21.15.78:0/3769236297 shutdown_connections 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 -- 172.21.15.78:0/3769236297 >> 172.21.15.78:0/3769236297 conn(0x7f4e6406ca80 msgr2=0x7f4e640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 -- 172.21.15.78:0/3769236297 shutdown_connections 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.974+0000 7f4e6c000640 1 -- 172.21.15.78:0/3769236297 wait complete. 2024-09-11T12:56:04.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 Processor -- start 2024-09-11T12:56:04.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- start start 2024-09-11T12:56:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e4c003c70 0x7f4e641467a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e641f97f0 0x7f4e640b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64167b90 con 0x7f4e640b1030 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4e64167a10 con 0x7f4e4c003c70 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4e64167d10 con 0x7f4e641f97f0 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36074/0 (socket says 172.21.15.78:36074) 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 -- 172.21.15.78:0/145697599 learned_addr learned my addr 172.21.15.78:0/145697599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e68c00640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e641f97f0 0x7f4e640b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e69600640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e4c003c70 0x7f4e641467a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 -- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e641f97f0 msgr2=0x7f4e640b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e641f97f0 0x7f4e640b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 -- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e4c003c70 msgr2=0x7f4e641467a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e4c003c70 0x7f4e641467a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 -- 172.21.15.78:0/145697599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e6414f160 con 0x7f4e640b1030 2024-09-11T12:56:04.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6a000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4e6415c040 tx=0x7f4e5c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c004b30 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e5c004cd0 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e6414ee80 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c003be0 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e64152650 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.978+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e6415b550 con 0x7f4e640b1030 2024-09-11T12:56:04.984 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.982+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f4e5c004030 con 0x7f4e640b1030 2024-09-11T12:56:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.982+0000 7f4e61600640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 0x7f4e44074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.982+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f4e5c092f60 con 0x7f4e640b1030 2024-09-11T12:56:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.982+0000 7f4e69600640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 0x7f4e44074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:04.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.982+0000 7f4e69600640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 0x7f4e44074170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e58001830 tx=0x7f4e580196f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:04.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:04.986+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4e5c05e2f0 con 0x7f4e640b1030 2024-09-11T12:56:05.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.186+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e64155f20 con 0x7f4e640b1030 2024-09-11T12:56:05.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.186+0000 7f4e61600640 1 -- 172.21.15.78:0/145697599 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+5227 (secure 0 0 0) 0x7f4e5c05dbf0 con 0x7f4e640b1030 2024-09-11T12:56:05.190 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:05.190 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":92,"btime":"2024-09-11T12:55:53:010575+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:55:43.353328+0000","modified":"2024-09-11T12:55:46.366514+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7776},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7776":{"gid":7776,"name":"a","rank":0,"incarnation":87,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/4229548436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":4229548436},{"type":"v1","addr":"172.21.15.78:6835","nonce":4229548436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7776,"qdb_cluster":[7776]},"id":10}]} 2024-09-11T12:56:05.190 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 92 2024-09-11T12:56:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 msgr2=0x7f4e44074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 0x7f4e44074170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e58001830 tx=0x7f4e580196f0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 msgr2=0x7f4e641595e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f4e6415c040 tx=0x7f4e5c00f3c0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 shutdown_connections 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4e640b1030 0x7f4e641595e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4e641f97f0 0x7f4e640b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4e44071cb0 0x7f4e44074170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 --2- 172.21.15.78:0/145697599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4e4c003c70 0x7f4e641467a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 >> 172.21.15.78:0/145697599 conn(0x7f4e6406ca80 msgr2=0x7f4e640b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 shutdown_connections 2024-09-11T12:56:05.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.190+0000 7f4e6c000640 1 -- 172.21.15.78:0/145697599 wait complete. 2024-09-11T12:56:05.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- 172.21.15.78:0/674610280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd8159fc0 msgr2=0x7f5bd80706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/674610280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd8159fc0 0x7f5bd80706f0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc009840 tx=0x7f5bcc01c820 comp rx=0 tx=0).stop 2024-09-11T12:56:05.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- 172.21.15.78:0/674610280 shutdown_connections 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/674610280 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd8159fc0 0x7f5bd80706f0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- 172.21.15.78:0/674610280 >> 172.21.15.78:0/674610280 conn(0x7f5bd806ca80 msgr2=0x7f5bd80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- 172.21.15.78:0/674610280 shutdown_connections 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- 172.21.15.78:0/674610280 wait complete. 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 Processor -- start 2024-09-11T12:56:05.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.350+0000 7f5bdfe00640 1 -- start start 2024-09-11T12:56:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd806f200 0x7f5bd8142800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd8159fc0 0x7f5bd8153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd81689d0 con 0x7f5bd814f7c0 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd8168850 con 0x7f5bd806f200 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd8168b50 con 0x7f5bd8159fc0 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36086/0 (socket says 172.21.15.78:36086) 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 -- 172.21.15.78:0/2475216272 learned_addr learned my addr 172.21.15.78:0/2475216272 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdd400640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd806f200 0x7f5bd8142800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdca00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd8159fc0 0x7f5bd8153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 -- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd8159fc0 msgr2=0x7f5bd8153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd8159fc0 0x7f5bd8153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 -- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd806f200 msgr2=0x7f5bd8142800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd806f200 0x7f5bd8142800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 -- 172.21.15.78:0/2475216272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd8142d40 con 0x7f5bd814f7c0 2024-09-11T12:56:05.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdde00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c7f0 tx=0x7f5bd000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0003b80 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bd0003d20 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd000aab0 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd8152bd0 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd8153890 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd80706f0 con 0x7f5bd814f7c0 2024-09-11T12:56:05.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.354+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f5bd000ac50 con 0x7f5bd814f7c0 2024-09-11T12:56:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.358+0000 7f5bd5600640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 0x7f5bb8074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.358+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f5bd0092860 con 0x7f5bd814f7c0 2024-09-11T12:56:05.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.358+0000 7f5bdd400640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 0x7f5bb8074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.358+0000 7f5bdd400640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 0x7f5bb8074170 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0af000 tx=0x7f5bcc01b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:05.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.362+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5bd005dd50 con 0x7f5bd814f7c0 2024-09-11T12:56:05.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.530+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5bd80cc620 con 0x7f5bd814f7c0 2024-09-11T12:56:05.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.530+0000 7f5bd5600640 1 -- 172.21.15.78:0/2475216272 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) ==== 74+0+16739 (secure 0 0 0) 0x7f5bd005d650 con 0x7f5bd814f7c0 2024-09-11T12:56:05.535 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:05.535 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":129,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:55:44.968878+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:55:41.069903+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-11T12:55:42.319816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 msgr2=0x7f5bb8074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 0x7f5bb8074170 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc0af000 tx=0x7f5bcc01b060 comp rx=0 tx=0).stop 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 msgr2=0x7f5bd80ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000c7f0 tx=0x7f5bd000ccb0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 shutdown_connections 2024-09-11T12:56:05.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5bd814f7c0 0x7f5bd80ad870 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5bd8159fc0 0x7f5bd8153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5bb8071cb0 0x7f5bb8074170 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 --2- 172.21.15.78:0/2475216272 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5bd806f200 0x7f5bd8142800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 >> 172.21.15.78:0/2475216272 conn(0x7f5bd806ca80 msgr2=0x7f5bd80af730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 shutdown_connections 2024-09-11T12:56:05.538 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.534+0000 7f5bdfe00640 1 -- 172.21.15.78:0/2475216272 wait complete. 2024-09-11T12:56:05.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:56:05.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 -- 172.21.15.78:0/765445514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 msgr2=0x7f466c15c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 --2- 172.21.15.78:0/765445514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c15c100 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f465c009bc0 tx=0x7f465c01cac0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 -- 172.21.15.78:0/765445514 shutdown_connections 2024-09-11T12:56:05.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 --2- 172.21.15.78:0/765445514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 --2- 172.21.15.78:0/765445514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c15c100 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 -- 172.21.15.78:0/765445514 >> 172.21.15.78:0/765445514 conn(0x7f466c074b90 msgr2=0x7f466c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:05.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 -- 172.21.15.78:0/765445514 shutdown_connections 2024-09-11T12:56:05.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.694+0000 7f4671400640 1 -- 172.21.15.78:0/765445514 wait complete. 2024-09-11T12:56:05.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 Processor -- start 2024-09-11T12:56:05.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- start start 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c06e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c15ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f466c168e60 con 0x7f466c06fdf0 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f466c168ce0 con 0x7f466c0b14b0 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f466c168fe0 con 0x7f466c0d0bf0 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36090/0 (socket says 172.21.15.78:36090) 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 -- 172.21.15.78:0/1396376138 learned_addr learned my addr 172.21.15.78:0/1396376138 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466b200640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c15ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4669e00640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c06e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 -- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 msgr2=0x7f466c15ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c15ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 -- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 msgr2=0x7f466c06e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c06e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 -- 172.21.15.78:0/1396376138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c143480 con 0x7f466c06fdf0 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466b200640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4669e00640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:05.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f466a800640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f465c01c530 tx=0x7f465c005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:05.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b2040 con 0x7f466c06fdf0 2024-09-11T12:56:05.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f465c004b40 con 0x7f466c06fdf0 2024-09-11T12:56:05.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c14a850 con 0x7f466c06fdf0 2024-09-11T12:56:05.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a5a30 con 0x7f466c06fdf0 2024-09-11T12:56:05.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.698+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c14ae10 con 0x7f466c06fdf0 2024-09-11T12:56:05.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c15c100 con 0x7f466c06fdf0 2024-09-11T12:56:05.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f465c0bd020 con 0x7f466c06fdf0 2024-09-11T12:56:05.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4662a00640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 0x7f464c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:05.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5709+0+0 (secure 0 0 0) 0x7f465c12c690 con 0x7f466c06fdf0 2024-09-11T12:56:05.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4669e00640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 0x7f464c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:05.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.702+0000 7f4669e00640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 0x7f464c074170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f466c14cf30 tx=0x7f4658009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:05.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.706+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f465c0166b0 con 0x7f466c06fdf0 2024-09-11T12:56:05.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.906+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f466c14c6c0 con 0x7f466c06fdf0 2024-09-11T12:56:05.909 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:56:05.906+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:56:05.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4662a00640 1 -- 172.21.15.78:0/1396376138 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v93) ==== 161+0+0 (secure 0 0 0) 0x7f465c0f7c50 con 0x7f466c06fdf0 2024-09-11T12:56:05.912 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:56:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 msgr2=0x7f464c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 0x7f464c074170 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f466c14cf30 tx=0x7f4658009040 comp rx=0 tx=0).stop 2024-09-11T12:56:05.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 msgr2=0x7f466c14a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f465c01c530 tx=0x7f465c005aa0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 shutdown_connections 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f466c0d0bf0 0x7f466c15ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f464c071cb0 0x7f464c074170 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f466c0b14b0 0x7f466c06e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 --2- 172.21.15.78:0/1396376138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f466c06fdf0 0x7f466c14a270 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 >> 172.21.15.78:0/1396376138 conn(0x7f466c074b90 msgr2=0x7f466c0b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 shutdown_connections 2024-09-11T12:56:05.915 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:05.910+0000 7f4671400640 1 -- 172.21.15.78:0/1396376138 wait complete. 2024-09-11T12:56:05.921 INFO:tasks.ceph.mds.a.smithi078.stderr: -2912> 2024-09-11T12:55:45.938+0000 7ff2b5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:56:05.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:56:05.968 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:56:06.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 -- 172.21.15.78:0/1623391358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 msgr2=0x7fd2c8075610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:06.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1623391358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8075610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009840 tx=0x7fd2b401c6d0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 -- 172.21.15.78:0/1623391358 shutdown_connections 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1623391358 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c80ced90 0x7fd2c80cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1623391358 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8075610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 -- 172.21.15.78:0/1623391358 >> 172.21.15.78:0/1623391358 conn(0x7fd2c80b2260 msgr2=0x7fd2c80b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 -- 172.21.15.78:0/1623391358 shutdown_connections 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.078+0000 7fd2cca00640 1 -- 172.21.15.78:0/1623391358 wait complete. 2024-09-11T12:56:06.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 Processor -- start 2024-09-11T12:56:06.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- start start 2024-09-11T12:56:06.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:06.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c815bb90 0x7fd2c80b1350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8168f10 con 0x7fd2c80ced90 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c8168d90 con 0x7fd2c80cc950 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c8169090 con 0x7fd2c815bb90 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36104/0 (socket says 172.21.15.78:36104) 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 -- 172.21.15.78:0/1190069544 learned_addr learned my addr 172.21.15.78:0/1190069544 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5e00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c6800640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c815bb90 0x7fd2c80b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 -- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c815bb90 msgr2=0x7fd2c80b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c815bb90 0x7fd2c80b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 -- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 msgr2=0x7fd2c8153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:06.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8153480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 -- 172.21.15.78:0/1190069544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c81505f0 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2c5400640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fd2b401a300 tx=0x7fd2b40a5590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b40b2040 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2b4004380 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c8151810 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b4003860 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c8151dd0 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.082+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c8075610 con 0x7fd2c80ced90 2024-09-11T12:56:06.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.086+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fd2b4003a00 con 0x7fd2c80ced90 2024-09-11T12:56:06.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.086+0000 7fd2be000640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 0x7fd2a8074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:06.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.086+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5795+0+0 (secure 0 0 0) 0x7fd2b412caf0 con 0x7fd2c80ced90 2024-09-11T12:56:06.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.086+0000 7fd2c5e00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 0x7fd2a8074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:06.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.086+0000 7fd2c5e00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 0x7fd2a8074170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8007fd0 tx=0x7fd2b8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:06.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.090+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd2b4016410 con 0x7fd2c80ced90 2024-09-11T12:56:06.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.290+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd2c80b2f80 con 0x7fd2c80ced90 2024-09-11T12:56:06.914 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.910+0000 7fd2be000640 1 -- 172.21.15.78:0/1190069544 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v94) ==== 104+0+0 (secure 0 0 0) 0x7fd2b40f79c0 con 0x7fd2c80ced90 2024-09-11T12:56:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 msgr2=0x7fd2a8074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 0x7fd2a8074170 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2b8007fd0 tx=0x7fd2b8009040 comp rx=0 tx=0).stop 2024-09-11T12:56:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 msgr2=0x7fd2c8142ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:06.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fd2b401a300 tx=0x7fd2b40a5590 comp rx=0 tx=0).stop 2024-09-11T12:56:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 shutdown_connections 2024-09-11T12:56:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd2c815bb90 0x7fd2c80b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd2c80ced90 0x7fd2c8142ed0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd2a8071cb0 0x7fd2a8074170 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 --2- 172.21.15.78:0/1190069544 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2c80cc950 0x7fd2c8153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 >> 172.21.15.78:0/1190069544 conn(0x7fd2c80b2260 msgr2=0x7fd2c815ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 shutdown_connections 2024-09-11T12:56:06.918 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:06.914+0000 7fd2cca00640 1 -- 172.21.15.78:0/1190069544 wait complete. 2024-09-11T12:56:06.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:56:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 -- 172.21.15.78:0/3278049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c0b5c40 msgr2=0x7f835c0cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:07.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 --2- 172.21.15.78:0/3278049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c0b5c40 0x7f835c0cf990 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f8354009840 tx=0x7f835401c820 comp rx=0 tx=0).stop 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 -- 172.21.15.78:0/3278049750 shutdown_connections 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 --2- 172.21.15.78:0/3278049750 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c0b5c40 0x7f835c0cf990 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 -- 172.21.15.78:0/3278049750 >> 172.21.15.78:0/3278049750 conn(0x7f835c0b3050 msgr2=0x7f835c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 -- 172.21.15.78:0/3278049750 shutdown_connections 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.134+0000 7f8363c00640 1 -- 172.21.15.78:0/3278049750 wait complete. 2024-09-11T12:56:07.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 Processor -- start 2024-09-11T12:56:07.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- start start 2024-09-11T12:56:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 0x7f835c1f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f835c0cf520 0x7f835c150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:07.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f835c169b40 con 0x7f835c152990 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f835c1699c0 con 0x7f835c0b5c40 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f835c169cc0 con 0x7f835c0cf520 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36120/0 (socket says 172.21.15.78:36120) 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 -- 172.21.15.78:0/2022121309 learned_addr learned my addr 172.21.15.78:0/2022121309 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361200640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 0x7f835c1f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 -- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f835c0cf520 msgr2=0x7f835c150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f835c0cf520 0x7f835c150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 -- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 msgr2=0x7f835c1f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 0x7f835c1f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:07.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 -- 172.21.15.78:0/2022121309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835c1565a0 con 0x7f835c152990 2024-09-11T12:56:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361200640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 0x7f835c1f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8361c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f835401a450 tx=0x7f83540a5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:07.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83540b2040 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83540043b0 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8354005260 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f835c156880 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835c156e40 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.138+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f835c159890 con 0x7f835c152990 2024-09-11T12:56:07.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.142+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f83540be060 con 0x7f835c152990 2024-09-11T12:56:07.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.142+0000 7f8358c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 0x7f833c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:07.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.142+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5795+0+0 (secure 0 0 0) 0x7f835412c220 con 0x7f835c152990 2024-09-11T12:56:07.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.142+0000 7f8361200640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 0x7f833c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:07.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.142+0000 7f8361200640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 0x7f833c074170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f834c0015b0 tx=0x7f834c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.146+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8354016410 con 0x7f835c152990 2024-09-11T12:56:07.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:07.314+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f835c0cf970 con 0x7f835c152990 2024-09-11T12:56:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.074+0000 7f8358c00640 1 -- 172.21.15.78:0/2022121309 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) ==== 181+0+0 (secure 0 0 0) 0x7f83540f7780 con 0x7f835c152990 2024-09-11T12:56:08.079 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 msgr2=0x7f833c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 0x7f833c074170 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f834c0015b0 tx=0x7f834c009040 comp rx=0 tx=0).stop 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 msgr2=0x7f835c153000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f835401a450 tx=0x7f83540a5570 comp rx=0 tx=0).stop 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 shutdown_connections 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f835c152990 0x7f835c153000 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f835c0cf520 0x7f835c150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f833c071cb0 0x7f833c074170 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 --2- 172.21.15.78:0/2022121309 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f835c0b5c40 0x7f835c1f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 >> 172.21.15.78:0/2022121309 conn(0x7f835c0b3050 msgr2=0x7f835c15eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 shutdown_connections 2024-09-11T12:56:08.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.078+0000 7f8363c00640 1 -- 172.21.15.78:0/2022121309 wait complete. 2024-09-11T12:56:08.106 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:56:08.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- 172.21.15.78:0/52674300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980cf520 msgr2=0x7fd5980cf990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:08.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 --2- 172.21.15.78:0/52674300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980cf520 0x7fd5980cf990 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fd5980af670 tx=0x7fd58801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- 172.21.15.78:0/52674300 shutdown_connections 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 --2- 172.21.15.78:0/52674300 >> v2:172.21.15.157:3300/0 conn(0x7fd5980b5530 0x7fd5980b59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 --2- 172.21.15.78:0/52674300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980cf520 0x7fd5980cf990 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- 172.21.15.78:0/52674300 >> 172.21.15.78:0/52674300 conn(0x7fd5980b3050 msgr2=0x7fd5980b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- 172.21.15.78:0/52674300 shutdown_connections 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- 172.21.15.78:0/52674300 wait complete. 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 Processor -- start 2024-09-11T12:56:08.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 -- start start 2024-09-11T12:56:08.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.250+0000 7fd5a0800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 0x7fd598073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 0x7fd59815db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd598168870 con 0x7fd5980b5530 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981686f0 con 0x7fd598152990 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd5981689f0 con 0x7fd5980cf520 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36136/0 (socket says 172.21.15.78:36136) 2024-09-11T12:56:08.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 -- 172.21.15.78:0/587288327 learned_addr learned my addr 172.21.15.78:0/587288327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59d400640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 0x7fd598073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:08.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59e800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 0x7fd59815db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 -- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 msgr2=0x7fd598073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 0x7fd598073490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 -- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 msgr2=0x7fd59815db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 0x7fd59815db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 -- 172.21.15.78:0/587288327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5980ad9a0 con 0x7fd5980b5530 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59e800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 0x7fd59815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59d400640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 0x7fd598073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd59de00640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b440 tx=0x7fd58c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c015020 con 0x7fd5980b5530 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd58c011980 con 0x7fd5980b5530 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58c011dd0 con 0x7fd5980b5530 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5980adc80 con 0x7fd5980b5530 2024-09-11T12:56:08.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5980739d0 con 0x7fd5980b5530 2024-09-11T12:56:08.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5980b59a0 con 0x7fd5980b5530 2024-09-11T12:56:08.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.254+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fd58c018020 con 0x7fd5980b5530 2024-09-11T12:56:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.258+0000 7fd592000640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 0x7fd57c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.258+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5357+0+0 (secure 0 0 0) 0x7fd58c0923d0 con 0x7fd5980b5530 2024-09-11T12:56:08.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.258+0000 7fd59d400640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 0x7fd57c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:08.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.258+0000 7fd59d400640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 0x7fd57c074170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd5880af000 tx=0x7fd58801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:08.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.262+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd58c01f070 con 0x7fd5980b5530 2024-09-11T12:56:08.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:08.426+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd59814cb30 con 0x7fd5980b5530 2024-09-11T12:56:09.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.082+0000 7fd592000640 1 -- 172.21.15.78:0/587288327 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) ==== 169+0+0 (secure 0 0 0) 0x7fd58c00d040 con 0x7fd5980b5530 2024-09-11T12:56:09.086 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 msgr2=0x7fd57c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 0x7fd57c074170 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd5880af000 tx=0x7fd58801b100 comp rx=0 tx=0).stop 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 msgr2=0x7fd5980ae760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fd58c00b440 tx=0x7fd58c00b900 comp rx=0 tx=0).stop 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 shutdown_connections 2024-09-11T12:56:09.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd57c071cb0 0x7fd57c074170 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd598152990 0x7fd59815db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd5980cf520 0x7fd598073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 --2- 172.21.15.78:0/587288327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd5980b5530 0x7fd5980ae760 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 >> 172.21.15.78:0/587288327 conn(0x7fd5980b3050 msgr2=0x7fd59806ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 shutdown_connections 2024-09-11T12:56:09.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.086+0000 7fd5a0800640 1 -- 172.21.15.78:0/587288327 wait complete. 2024-09-11T12:56:09.113 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:56:09.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 -- 172.21.15.78:0/521730858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9508074240 msgr2=0x7f950815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 --2- 172.21.15.78:0/521730858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9508074240 0x7f950815ba80 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f9508152090 tx=0x7f94f801cb50 comp rx=0 tx=0).stop 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 -- 172.21.15.78:0/521730858 shutdown_connections 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 --2- 172.21.15.78:0/521730858 >> v2:172.21.15.157:3300/0 conn(0x7f9508070360 0x7f950815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 --2- 172.21.15.78:0/521730858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9508074240 0x7f950815ba80 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 -- 172.21.15.78:0/521730858 >> 172.21.15.78:0/521730858 conn(0x7f950806ca80 msgr2=0x7f95080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 -- 172.21.15.78:0/521730858 shutdown_connections 2024-09-11T12:56:09.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.250+0000 7f950ea00640 1 -- 172.21.15.78:0/521730858 wait complete. 2024-09-11T12:56:09.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 Processor -- start 2024-09-11T12:56:09.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 -- start start 2024-09-11T12:56:09.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9508070360 0x7f9508158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9508074240 0x7f9508150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95080729b0 con 0x7f950815e5b0 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9508072830 con 0x7f9508074240 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9508167740 con 0x7f9508070360 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36148/0 (socket says 172.21.15.78:36148) 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 -- 172.21.15.78:0/505328546 learned_addr learned my addr 172.21.15.78:0/505328546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f9507e00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9508070360 0x7f9508158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f9507400640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9508074240 0x7f9508150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 -- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9508070360 msgr2=0x7f9508158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9508070360 0x7f9508158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 -- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9508074240 msgr2=0x7f9508150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9508074240 0x7f9508150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 -- 172.21.15.78:0/505328546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95081583a0 con 0x7f950815e5b0 2024-09-11T12:56:09.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ca00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f950000c9a0 tx=0x7f950000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500007bb0 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f950000f450 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9500007e60 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.254+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95081545b0 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9508154b70 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f950815ea20 con 0x7f950815e5b0 2024-09-11T12:56:09.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f950000f8a0 con 0x7f950815e5b0 2024-09-11T12:56:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f94ffe00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 0x7f94e80741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7f9500092da0 con 0x7f950815e5b0 2024-09-11T12:56:09.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f9507e00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 0x7f94e80741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.258+0000 7f9507e00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 0x7f94e80741a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94f80af000 tx=0x7f94f801b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:09.266 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.262+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9500014030 con 0x7f950815e5b0 2024-09-11T12:56:09.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.430+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f95080695e0 con 0x7f950815e5b0 2024-09-11T12:56:09.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f94ffe00640 1 -- 172.21.15.78:0/505328546 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7f950005de30 con 0x7f950815e5b0 2024-09-11T12:56:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 msgr2=0x7f94e80741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 0x7f94e80741a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f94f80af000 tx=0x7f94f801b390 comp rx=0 tx=0).stop 2024-09-11T12:56:09.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 msgr2=0x7f950815d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f950000c9a0 tx=0x7f950000ce60 comp rx=0 tx=0).stop 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 shutdown_connections 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f950815e5b0 0x7f950815d200 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f94e8071ce0 0x7f94e80741a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9508074240 0x7f9508150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 --2- 172.21.15.78:0/505328546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9508070360 0x7f9508158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 >> 172.21.15.78:0/505328546 conn(0x7f950806ca80 msgr2=0x7f9508164ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 shutdown_connections 2024-09-11T12:56:09.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.434+0000 7f950ea00640 1 -- 172.21.15.78:0/505328546 wait complete. 2024-09-11T12:56:09.458 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- 172.21.15.78:0/1945273195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ec0051d0 msgr2=0x7fc2ec00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 --2- 172.21.15.78:0/1945273195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ec0051d0 0x7fc2ec00b9d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc00c2e0 tx=0x7fc2fc01cf60 comp rx=0 tx=0).stop 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- 172.21.15.78:0/1945273195 shutdown_connections 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 --2- 172.21.15.78:0/1945273195 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc2ec0051d0 0x7fc2ec00b9d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- 172.21.15.78:0/1945273195 >> 172.21.15.78:0/1945273195 conn(0x7fc30406ca80 msgr2=0x7fc3040b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- 172.21.15.78:0/1945273195 shutdown_connections 2024-09-11T12:56:09.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- 172.21.15.78:0/1945273195 wait complete. 2024-09-11T12:56:09.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 Processor -- start 2024-09-11T12:56:09.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- start start 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ec0051d0 0x7fc3041422e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30415b050 0x7fc304152560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc30415d3a0 con 0x7fc304152aa0 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc30415d220 con 0x7fc30415b050 2024-09-11T12:56:09.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.598+0000 7fc30c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3040cf280 con 0x7fc2ec0051d0 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36172/0 (socket says 172.21.15.78:36172) 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 -- 172.21.15.78:0/1403064085 learned_addr learned my addr 172.21.15.78:0/1403064085 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc309800640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ec0051d0 0x7fc3041422e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc308e00640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30415b050 0x7fc304152560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 -- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ec0051d0 msgr2=0x7fc3041422e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ec0051d0 0x7fc3041422e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 -- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30415b050 msgr2=0x7fc304152560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30415b050 0x7fc304152560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 -- 172.21.15.78:0/1403064085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3041f4d10 con 0x7fc304152aa0 2024-09-11T12:56:09.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30a200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc01d800 tx=0x7fc2fc006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:09.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc001500 con 0x7fc304152aa0 2024-09-11T12:56:09.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2fc003600 con 0x7fc304152aa0 2024-09-11T12:56:09.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3041532d0 con 0x7fc304152aa0 2024-09-11T12:56:09.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2fc003a50 con 0x7fc304152aa0 2024-09-11T12:56:09.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc30415b4d0 con 0x7fc304152aa0 2024-09-11T12:56:09.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3040b4930 con 0x7fc304152aa0 2024-09-11T12:56:09.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.602+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc2fc003bf0 con 0x7fc304152aa0 2024-09-11T12:56:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.606+0000 7fc301600640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 0x7fc2e40741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.606+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7fc2fc12c210 con 0x7fc304152aa0 2024-09-11T12:56:09.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.606+0000 7fc309800640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 0x7fc2e40741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:09.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.606+0000 7fc309800640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 0x7fc2e40741a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000f90 tx=0x7fc2f8004a10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:09.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.610+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc2fc016d10 con 0x7fc304152aa0 2024-09-11T12:56:09.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:09.842+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fc30415d730 con 0x7fc304152aa0 2024-09-11T12:56:10.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.082+0000 7fc301600640 1 -- 172.21.15.78:0/1403064085 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v364) ==== 151+0+0 (secure 0 0 0) 0x7fc2fc0f7560 con 0x7fc304152aa0 2024-09-11T12:56:10.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 msgr2=0x7fc2e40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 0x7fc2e40741a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8000f90 tx=0x7fc2f8004a10 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 msgr2=0x7fc3041f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fc2fc01d800 tx=0x7fc2fc006ca0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 shutdown_connections 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc304152aa0 0x7fc3041f4000 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc2e4071ce0 0x7fc2e40741a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc30415b050 0x7fc304152560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 --2- 172.21.15.78:0/1403064085 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc2ec0051d0 0x7fc3041422e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 >> 172.21.15.78:0/1403064085 conn(0x7fc30406ca80 msgr2=0x7fc30415df30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 shutdown_connections 2024-09-11T12:56:10.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.086+0000 7fc30c200640 1 -- 172.21.15.78:0/1403064085 wait complete. 2024-09-11T12:56:10.110 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-11T12:56:10.111 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-11T12:56:10.111 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:56:10.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:10.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 -- 172.21.15.78:0/2571657032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fbc003a40 msgr2=0x7f1fbc003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 --2- 172.21.15.78:0/2571657032 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fbc003a40 0x7f1fbc003e60 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f1fc8009b40 tx=0x7f1fc801dc50 comp rx=0 tx=0).stop 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fd7e00640 1 -- 172.21.15.78:0/2571657032 reap_dead start 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 -- 172.21.15.78:0/2571657032 shutdown_connections 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 -- 172.21.15.78:0/2571657032 >> 172.21.15.78:0/2571657032 conn(0x7f1fd80cfeb0 msgr2=0x7f1fd80d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 -- 172.21.15.78:0/2571657032 shutdown_connections 2024-09-11T12:56:10.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.250+0000 7f1fdca00640 1 -- 172.21.15.78:0/2571657032 wait complete. 2024-09-11T12:56:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 Processor -- start 2024-09-11T12:56:10.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- start start 2024-09-11T12:56:10.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc003a40 0x7f1fd8151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fd80b2410 0x7f1fd80ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd80720f0 con 0x7f1fd80b19d0 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1fd8071f70 con 0x7f1fbc003a40 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1fd8072270 con 0x7f1fd80b2410 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36190/0 (socket says 172.21.15.78:36190) 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 -- 172.21.15.78:0/3566988313 learned_addr learned my addr 172.21.15.78:0/3566988313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7400640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc003a40 0x7f1fd8151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd6a00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fd80b2410 0x7f1fd80ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 -- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fd80b2410 msgr2=0x7f1fd80ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fd80b2410 0x7f1fd80ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 -- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc003a40 msgr2=0x7f1fd8151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc003a40 0x7f1fd8151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 -- 172.21.15.78:0/3566988313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fd81f7220 con 0x7f1fd80b19d0 2024-09-11T12:56:10.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fd7e00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fd815e380 tx=0x7f1fd000eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000c970 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fd0018070 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fd81fb220 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd000cea0 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fd81f74a0 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.254+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fd80738d0 con 0x7f1fd80b19d0 2024-09-11T12:56:10.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.258+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f1fd001d460 con 0x7f1fd80b19d0 2024-09-11T12:56:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.258+0000 7f1fcf400640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 0x7f1fb80741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.258+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7f1fd0092970 con 0x7f1fd80b19d0 2024-09-11T12:56:10.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.258+0000 7f1fd7400640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 0x7f1fb80741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.258+0000 7f1fd7400640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 0x7f1fb80741a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80ad000 tx=0x7f1fc8019ff0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.262+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1fd81f7220 con 0x7f1fd80b19d0 2024-09-11T12:56:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.426+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fd8070490 con 0x7f1fd80b19d0 2024-09-11T12:56:10.431 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.426+0000 7f1fcf400640 1 -- 172.21.15.78:0/3566988313 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+13042 (secure 0 0 0) 0x7f1fd0066290 con 0x7f1fd80b19d0 2024-09-11T12:56:10.432 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:10.432 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":132,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:09.081542+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/4229548436":"2024-09-12T12:56:05.911261+0000","172.21.15.78:6832/4229548436":"2024-09-12T12:56:05.911261+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 msgr2=0x7f1fb80741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 0x7f1fb80741a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f1fc80ad000 tx=0x7f1fc8019ff0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 msgr2=0x7f1fd80addb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f1fd815e380 tx=0x7f1fd000eae0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 shutdown_connections 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1fd80b19d0 0x7f1fd80addb0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1fd80b2410 0x7f1fd80ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1fb8071ce0 0x7f1fb80741a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 --2- 172.21.15.78:0/3566988313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1fbc003a40 0x7f1fd8151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 >> 172.21.15.78:0/3566988313 conn(0x7f1fd80cfeb0 msgr2=0x7f1fd806eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 shutdown_connections 2024-09-11T12:56:10.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.430+0000 7f1fdca00640 1 -- 172.21.15.78:0/3566988313 wait complete. 2024-09-11T12:56:10.453 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:56:10.454 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:56:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- 172.21.15.78:0/710487274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80b2120 msgr2=0x7fcba80cdc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.600 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 --2- 172.21.15.78:0/710487274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80b2120 0x7fcba80cdc10 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009870 tx=0x7fcb9001c740 comp rx=0 tx=0).stop 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- 172.21.15.78:0/710487274 shutdown_connections 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 --2- 172.21.15.78:0/710487274 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80b2120 0x7fcba80cdc10 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- 172.21.15.78:0/710487274 >> 172.21.15.78:0/710487274 conn(0x7fcba8071dd0 msgr2=0x7fcba80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- 172.21.15.78:0/710487274 shutdown_connections 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- 172.21.15.78:0/710487274 wait complete. 2024-09-11T12:56:10.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 Processor -- start 2024-09-11T12:56:10.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- start start 2024-09-11T12:56:10.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcba80b2120 0x7fcba815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcba81518b0 0x7fcba8138540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8168e90 con 0x7fcba80ce150 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcba8168d10 con 0x7fcba81518b0 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbaec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcba8169010 con 0x7fcba80b2120 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcb9f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcb9f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36206/0 (socket says 172.21.15.78:36206) 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcb9f400640 1 -- 172.21.15.78:0/1639328174 learned_addr learned my addr 172.21.15.78:0/1639328174 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcbacc00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcba81518b0 0x7fcba8138540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.598+0000 7fcba7e00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcba80b2120 0x7fcba815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 -- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcba80b2120 msgr2=0x7fcba815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcba80b2120 0x7fcba815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 -- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcba81518b0 msgr2=0x7fcba8138540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcba81518b0 0x7fcba8138540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 -- 172.21.15.78:0/1639328174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba8138a80 con 0x7fcba80ce150 2024-09-11T12:56:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcb9f400640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcb90019350 tx=0x7fcb900052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb900b0070 con 0x7fcba80ce150 2024-09-11T12:56:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb900b1040 con 0x7fcba80ce150 2024-09-11T12:56:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba815b4f0 con 0x7fcba80ce150 2024-09-11T12:56:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb900046d0 con 0x7fcba80ce150 2024-09-11T12:56:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba815bab0 con 0x7fcba80ce150 2024-09-11T12:56:10.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba80cdc10 con 0x7fcba80ce150 2024-09-11T12:56:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fcb900bd020 con 0x7fcba80ce150 2024-09-11T12:56:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 0x7fcb840741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.602+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7fcb9012bd40 con 0x7fcba80ce150 2024-09-11T12:56:10.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.606+0000 7fcba7e00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 0x7fcb840741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.606+0000 7fcba7e00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 0x7fcb840741a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb98001870 tx=0x7fcb980016a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.606+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcb900c4020 con 0x7fcba80ce150 2024-09-11T12:56:10.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.774+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcba80b3260 con 0x7fcba80ce150 2024-09-11T12:56:10.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.774+0000 7fcba4c00640 1 -- 172.21.15.78:0/1639328174 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) ==== 93+0+10 (secure 0 0 0) 0x7fcb90016370 con 0x7fcba80ce150 2024-09-11T12:56:10.779 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:56:10.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 msgr2=0x7fcb840741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 0x7fcb840741a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcb98001870 tx=0x7fcb980016a0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 msgr2=0x7fcba815c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fcb90019350 tx=0x7fcb900052d0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 shutdown_connections 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcb84071ce0 0x7fcb840741a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcba81518b0 0x7fcba8138540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcba80ce150 0x7fcba815c590 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 --2- 172.21.15.78:0/1639328174 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcba80b2120 0x7fcba815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 >> 172.21.15.78:0/1639328174 conn(0x7fcba8071dd0 msgr2=0x7fcba814d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 shutdown_connections 2024-09-11T12:56:10.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.778+0000 7fcbaec00640 1 -- 172.21.15.78:0/1639328174 wait complete. 2024-09-11T12:56:10.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-11T12:56:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 -- 172.21.15.78:0/2939348644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3015a1c0 msgr2=0x7fde300cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 --2- 172.21.15.78:0/2939348644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3015a1c0 0x7fde300cbed0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fde18009900 tx=0x7fde1801c820 comp rx=0 tx=0).stop 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 -- 172.21.15.78:0/2939348644 shutdown_connections 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 --2- 172.21.15.78:0/2939348644 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3013d250 0x7fde300cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 --2- 172.21.15.78:0/2939348644 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3015a1c0 0x7fde300cbed0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 -- 172.21.15.78:0/2939348644 >> 172.21.15.78:0/2939348644 conn(0x7fde3006ca80 msgr2=0x7fde300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 -- 172.21.15.78:0/2939348644 shutdown_connections 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.938+0000 7fde36200640 1 -- 172.21.15.78:0/2939348644 wait complete. 2024-09-11T12:56:10.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 Processor -- start 2024-09-11T12:56:10.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- start start 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 0x7fde301f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3015a1c0 0x7fde30142c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fde3013ebe0 con 0x7fde3013d250 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fde3013ea60 con 0x7fde300b4780 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fde3013ed60 con 0x7fde3015a1c0 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36226/0 (socket says 172.21.15.78:36226) 2024-09-11T12:56:10.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 -- 172.21.15.78:0/4136779476 learned_addr learned my addr 172.21.15.78:0/4136779476 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 -- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3015a1c0 msgr2=0x7fde30142c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3015a1c0 0x7fde30142c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 -- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 msgr2=0x7fde301f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2f400640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 0x7fde301f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 0x7fde301f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 -- 172.21.15.78:0/4136779476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde30151820 con 0x7fde3013d250 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2f400640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 0x7fde301f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde2ea00640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde18009850 tx=0x7fde180a5820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180b2040 con 0x7fde3013d250 2024-09-11T12:56:10.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde180043c0 con 0x7fde3013d250 2024-09-11T12:56:10.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18005270 con 0x7fde3013d250 2024-09-11T12:56:10.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde30143180 con 0x7fde3013d250 2024-09-11T12:56:10.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301fb010 con 0x7fde3013d250 2024-09-11T12:56:10.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.942+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde3013f230 con 0x7fde3013d250 2024-09-11T12:56:10.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.946+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fde18005410 con 0x7fde3013d250 2024-09-11T12:56:10.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.946+0000 7fde27400640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 0x7fde0c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:10.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.946+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7fde1812c220 con 0x7fde3013d250 2024-09-11T12:56:10.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.946+0000 7fde2f400640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 0x7fde0c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:10.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.946+0000 7fde2f400640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 0x7fde0c074170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fde20007fd0 tx=0x7fde20009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:10.953 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:10.950+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fde180164d0 con 0x7fde3013d250 2024-09-11T12:56:11.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:11.178+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7fde300cc0e0 con 0x7fde3013d250 2024-09-11T12:56:12.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.094+0000 7fde27400640 1 -- 172.21.15.78:0/4136779476 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v366) ==== 152+0+0 (secure 0 0 0) 0x7fde180f7a10 con 0x7fde3013d250 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 msgr2=0x7fde0c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 0x7fde0c074170 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fde20007fd0 tx=0x7fde20009040 comp rx=0 tx=0).stop 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 msgr2=0x7fde301512e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fde18009850 tx=0x7fde180a5820 comp rx=0 tx=0).stop 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 shutdown_connections 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fde3015a1c0 0x7fde30142c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fde3013d250 0x7fde301512e0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fde0c071cb0 0x7fde0c074170 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 --2- 172.21.15.78:0/4136779476 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fde300b4780 0x7fde301f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 >> 172.21.15.78:0/4136779476 conn(0x7fde3006ca80 msgr2=0x7fde300b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:12.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 shutdown_connections 2024-09-11T12:56:12.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.098+0000 7fde36200640 1 -- 172.21.15.78:0/4136779476 wait complete. 2024-09-11T12:56:12.124 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 -- 172.21.15.78:0/1338362957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 msgr2=0x7fe7080b0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 --2- 172.21.15.78:0/1338362957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe7080b0fe0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fe6f4009900 tx=0x7fe6f401c820 comp rx=0 tx=0).stop 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 -- 172.21.15.78:0/1338362957 shutdown_connections 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 --2- 172.21.15.78:0/1338362957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe7080b0fe0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 --2- 172.21.15.78:0/1338362957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe708161210 0x7fe70806e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 -- 172.21.15.78:0/1338362957 >> 172.21.15.78:0/1338362957 conn(0x7fe7080cf830 msgr2=0x7fe7080cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:12.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 -- 172.21.15.78:0/1338362957 shutdown_connections 2024-09-11T12:56:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.262+0000 7fe70ec00640 1 -- 172.21.15.78:0/1338362957 wait complete. 2024-09-11T12:56:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 Processor -- start 2024-09-11T12:56:12.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- start start 2024-09-11T12:56:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7080a6ce0 0x7fe70814a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe708161210 0x7fe7081517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe70816b370 con 0x7fe7081615f0 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe70816b1f0 con 0x7fe708161210 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe70816b4f0 con 0x7fe7080a6ce0 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe707e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7080a6ce0 0x7fe70814a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe707400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe708161210 0x7fe7081517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36234/0 (socket says 172.21.15.78:36234) 2024-09-11T12:56:12.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 -- 172.21.15.78:0/3301615461 learned_addr learned my addr 172.21.15.78:0/3301615461 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 -- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7080a6ce0 msgr2=0x7fe70814a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7080a6ce0 0x7fe70814a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 -- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe708161210 msgr2=0x7fe7081517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe708161210 0x7fe7081517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 -- 172.21.15.78:0/3301615461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70814b730 con 0x7fe7081615f0 2024-09-11T12:56:12.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70cc00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe70000c7a0 tx=0x7fe70000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe700003b80 con 0x7fe7081615f0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe700003d40 con 0x7fe7081615f0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70814ae90 con 0x7fe7081615f0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70000aab0 con 0x7fe7081615f0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.266+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70814b450 con 0x7fe7081615f0 2024-09-11T12:56:12.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70806e4c0 con 0x7fe7081615f0 2024-09-11T12:56:12.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fe70000ac50 con 0x7fe7081615f0 2024-09-11T12:56:12.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe6ffe00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 0x7fe6e8074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7fe700092330 con 0x7fe7081615f0 2024-09-11T12:56:12.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe707e00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 0x7fe6e8074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.270+0000 7fe707e00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 0x7fe6e8074170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004630 tx=0x7fe6f800b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:12.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.274+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe70005d590 con 0x7fe7081615f0 2024-09-11T12:56:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe7080a6ac0 con 0x7fe7081615f0 2024-09-11T12:56:12.444 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe6ffe00640 1 -- 172.21.15.78:0/3301615461 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+1998 (secure 0 0 0) 0x7fe700061ae0 con 0x7fe7081615f0 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:56:12.445 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:56:12.446 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:56:12.447 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:56:12.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 msgr2=0x7fe6e8074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 0x7fe6e8074170 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8004630 tx=0x7fe6f800b390 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 msgr2=0x7fe708151d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fe70000c7a0 tx=0x7fe70000cc60 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 shutdown_connections 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe7081615f0 0x7fe708151d00 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6e8071cb0 0x7fe6e8074170 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe708161210 0x7fe7081517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 --2- 172.21.15.78:0/3301615461 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe7080a6ce0 0x7fe70814a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 >> 172.21.15.78:0/3301615461 conn(0x7fe7080cf830 msgr2=0x7fe7080b1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 shutdown_connections 2024-09-11T12:56:12.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.442+0000 7fe70ec00640 1 -- 172.21.15.78:0/3301615461 wait complete. 2024-09-11T12:56:12.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 -- 172.21.15.78:0/1780171042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8914004d50 msgr2=0x7f89140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 --2- 172.21.15.78:0/1780171042 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8914004d50 0x7f89140051d0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8928006fd0 tx=0x7f892801cb80 comp rx=0 tx=0).stop 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8935800640 1 -- 172.21.15.78:0/1780171042 reap_dead start 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 -- 172.21.15.78:0/1780171042 shutdown_connections 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 -- 172.21.15.78:0/1780171042 >> 172.21.15.78:0/1780171042 conn(0x7f89300d0410 msgr2=0x7f89300d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 -- 172.21.15.78:0/1780171042 shutdown_connections 2024-09-11T12:56:12.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.606+0000 7f8937800640 1 -- 172.21.15.78:0/1780171042 wait complete. 2024-09-11T12:56:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 Processor -- start 2024-09-11T12:56:12.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- start start 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8914004d50 0x7f8930142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f893013eda0 0x7f893013f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8930144e00 con 0x7f89301501e0 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8930144c80 con 0x7f8914004d50 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8930144f80 con 0x7f893013eda0 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36254/0 (socket says 172.21.15.78:36254) 2024-09-11T12:56:12.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 -- 172.21.15.78:0/374099217 learned_addr learned my addr 172.21.15.78:0/374099217 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8934e00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8914004d50 0x7f8930142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8935800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f893013eda0 0x7f893013f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 -- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f893013eda0 msgr2=0x7f893013f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f893013eda0 0x7f893013f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 -- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8914004d50 msgr2=0x7f8930142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8914004d50 0x7f8930142f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 -- 172.21.15.78:0/374099217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89301f33c0 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892fe00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f891c001870 tx=0x7f891c012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c010b40 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f891c017030 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89301f3660 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891c012e60 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.610+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89301f3c20 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8930073370 con 0x7f89301501e0 2024-09-11T12:56:12.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f891c013450 con 0x7f89301501e0 2024-09-11T12:56:12.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f892cc00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 0x7f89100741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:12.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7f891c096110 con 0x7f89301501e0 2024-09-11T12:56:12.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f8934e00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 0x7f89100741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:12.619 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.614+0000 7f8934e00640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 0x7f89100741a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8920009aa0 tx=0x7f89200091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:12.622 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.618+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f891c061990 con 0x7f89301501e0 2024-09-11T12:56:12.676 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:56:12.679 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:56:12.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.790+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f89300afef0 con 0x7f89301501e0 2024-09-11T12:56:12.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f892cc00640 1 -- 172.21.15.78:0/374099217 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f891c061290 con 0x7f89301501e0 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 msgr2=0x7f89100741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 0x7f89100741a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8920009aa0 tx=0x7f89200091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 msgr2=0x7f8930150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f891c001870 tx=0x7f891c012b20 comp rx=0 tx=0).stop 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 shutdown_connections 2024-09-11T12:56:12.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f893013eda0 0x7f893013f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f89301501e0 0x7f8930150660 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8910071ce0 0x7f89100741a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 --2- 172.21.15.78:0/374099217 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8914004d50 0x7f8930142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 >> 172.21.15.78:0/374099217 conn(0x7f89300d0410 msgr2=0x7f893015e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 shutdown_connections 2024-09-11T12:56:12.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:12.794+0000 7f8937800640 1 -- 172.21.15.78:0/374099217 wait complete. 2024-09-11T12:56:12.818 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:56:12.818 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:12.930 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:13.029 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:56:13.030 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:13.030 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:13.038 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:13.038 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:13.038 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:13.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:13.047 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:13.047 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:13.086 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:13.087 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:13.087 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:13.098 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:13.098 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:56:13.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:13.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 -- 172.21.15.78:0/2502982468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b4157ae0 msgr2=0x7f33b4070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 --2- 172.21.15.78:0/2502982468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b4157ae0 0x7f33b4070a00 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009bc0 tx=0x7f33a801cb10 comp rx=0 tx=0).stop 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 -- 172.21.15.78:0/2502982468 shutdown_connections 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 --2- 172.21.15.78:0/2502982468 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b4157ae0 0x7f33b4070a00 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 -- 172.21.15.78:0/2502982468 >> 172.21.15.78:0/2502982468 conn(0x7f33b406ca80 msgr2=0x7f33b40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 -- 172.21.15.78:0/2502982468 shutdown_connections 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.262+0000 7f33bc600640 1 -- 172.21.15.78:0/2502982468 wait complete. 2024-09-11T12:56:13.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 Processor -- start 2024-09-11T12:56:13.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- start start 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40b4db0 0x7f33b41649d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b4157ae0 0x7f33b41582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f33b4168960 con 0x7f33b40736d0 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f33b41687e0 con 0x7f33b4157ae0 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f33b4168ae0 con 0x7f33b40b4db0 2024-09-11T12:56:13.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36268/0 (socket says 172.21.15.78:36268) 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 -- 172.21.15.78:0/1820163369 learned_addr learned my addr 172.21.15.78:0/1820163369 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9200640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40b4db0 0x7f33b41649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33ba600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b4157ae0 0x7f33b41582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 -- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40b4db0 msgr2=0x7f33b41649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40b4db0 0x7f33b41649d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 -- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b4157ae0 msgr2=0x7f33b41582a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b4157ae0 0x7f33b41582a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 -- 172.21.15.78:0/1820163369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b41f7880 con 0x7f33b40736d0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b9c00640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009b10 tx=0x7f33a80a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80bb070 con 0x7f33b40736d0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33a8004d40 con 0x7f33b40736d0 2024-09-11T12:56:13.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80b2040 con 0x7f33b40736d0 2024-09-11T12:56:13.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b41f7b20 con 0x7f33b40736d0 2024-09-11T12:56:13.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.266+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b41f80e0 con 0x7f33b40736d0 2024-09-11T12:56:13.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b4070a00 con 0x7f33b40736d0 2024-09-11T12:56:13.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f33a80a5970 con 0x7f33b40736d0 2024-09-11T12:56:13.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33b1600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 0x7f3394074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7f33a812c610 con 0x7f33b40736d0 2024-09-11T12:56:13.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33b9200640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 0x7f3394074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.270+0000 7f33b9200640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 0x7f3394074170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f33b41642b0 tx=0x7f33a0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:13.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.274+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f33a8016790 con 0x7f33b40736d0 2024-09-11T12:56:13.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.474+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33b40ce720 con 0x7f33b40736d0 2024-09-11T12:56:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.474+0000 7f33b1600640 1 -- 172.21.15.78:0/1820163369 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) ==== 94+0+3911 (secure 0 0 0) 0x7f33a8100080 con 0x7f33b40736d0 2024-09-11T12:56:13.478 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:13.478 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":94,"btime":"2024-09-11T12:56:06:913871+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6575,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1810887097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1810887097},{"type":"v1","addr":"172.21.15.157:6801","nonce":1810887097}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7357,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/11896162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":11896162},{"type":"v1","addr":"172.21.15.157:6835","nonce":11896162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":86},{"gid":7773,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1316948904","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":1316948904},{"type":"v1","addr":"172.21.15.78:6834","nonce":1316948904}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8178,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3214051970","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":3214051970},{"type":"v1","addr":"172.21.15.78:6835","nonce":3214051970}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94}],"filesystems":[]} 2024-09-11T12:56:13.478 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 94 2024-09-11T12:56:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 msgr2=0x7f3394074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 0x7f3394074170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f33b41642b0 tx=0x7f33a0009040 comp rx=0 tx=0).stop 2024-09-11T12:56:13.480 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 msgr2=0x7f33b41517c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009b10 tx=0x7f33a80a5880 comp rx=0 tx=0).stop 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 shutdown_connections 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3394071cb0 0x7f3394074170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f33b4157ae0 0x7f33b41582a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f33b40b4db0 0x7f33b41649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 --2- 172.21.15.78:0/1820163369 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f33b40736d0 0x7f33b41517c0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 >> 172.21.15.78:0/1820163369 conn(0x7f33b406ca80 msgr2=0x7f33b4156d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 shutdown_connections 2024-09-11T12:56:13.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.478+0000 7f33bc600640 1 -- 172.21.15.78:0/1820163369 wait complete. 2024-09-11T12:56:13.499 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:56:13.499 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:56:13.500 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:56:13.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:13.500 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:56:13.500 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:56:13.500 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:56:13.500 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:13.500 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:56:13.500 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:56:13.501 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:56:13.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:13.501 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:56:13.501 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:56:13.501 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:56:13.501 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:13.513 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:56:13.513 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:56:13.514 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:56:13.514 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:56:13.517 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:56:13.517 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:56:13.518 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:56:13.518 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:56:13.558 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:56:13.558 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:13.559 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:56:13.560 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:13.561 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:56:13.561 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:13.563 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:56:13.563 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:13.563 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:56:13.644 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:56:13.695 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:56:13.695 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:56:13.696 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:56:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 -- 172.21.15.78:0/2542909593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840b3990 msgr2=0x7ff98415eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 --2- 172.21.15.78:0/2542909593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840b3990 0x7ff98415eb10 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff974009930 tx=0x7ff97401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 -- 172.21.15.78:0/2542909593 shutdown_connections 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 --2- 172.21.15.78:0/2542909593 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840cfdf0 0x7ff98413ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 --2- 172.21.15.78:0/2542909593 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840b3990 0x7ff98415eb10 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 -- 172.21.15.78:0/2542909593 >> 172.21.15.78:0/2542909593 conn(0x7ff984073770 msgr2=0x7ff984073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 -- 172.21.15.78:0/2542909593 shutdown_connections 2024-09-11T12:56:13.715 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.710+0000 7ff983200640 1 -- 172.21.15.78:0/2542909593 wait complete. 2024-09-11T12:56:13.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 Processor -- start 2024-09-11T12:56:13.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- start start 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9840c2f90 0x7ff98412d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840cfdf0 0x7ff9840cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff9840ccc80 con 0x7ff9840c2f90 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff9840ccb00 con 0x7ff9840cfdf0 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff9840cce00 con 0x7ff9840b3990 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3301/0 says I am v2:172.21.15.78:35764/0 (socket says 172.21.15.78:35764) 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 -- 172.21.15.78:0/2401114903 learned_addr learned my addr 172.21.15.78:0/2401114903 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff982800640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840cfdf0 0x7ff9840cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.718 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff979400640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9840c2f90 0x7ff98412d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 -- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840cfdf0 msgr2=0x7ff9840cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840cfdf0 0x7ff9840cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 -- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9840c2f90 msgr2=0x7ff98412d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9840c2f90 0x7ff98412d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:13.719 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 -- 172.21.15.78:0/2401114903 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9840b40d0 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff981e00640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff974009850 tx=0x7ff9740a55a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9740b2040 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff974004380 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff974005290 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff984152010 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.714+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9840cdfb0 con 0x7ff9840b3990 2024-09-11T12:56:13.720 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9840cd380 con 0x7ff9840b3990 2024-09-11T12:56:13.721 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7ff974005430 con 0x7ff9840b3990 2024-09-11T12:56:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff97aa00640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 0x7ff960074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(132..132 src has 1..132) ==== 4943+0+0 (secure 0 0 0) 0x7ff97412bf30 con 0x7ff9840b3990 2024-09-11T12:56:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff979400640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 0x7ff960074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:13.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.718+0000 7ff979400640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 0x7ff960074170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff984166260 tx=0x7ff96c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:13.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.722+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff9740164d0 con 0x7ff9840b3990 2024-09-11T12:56:13.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:13.898+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7ff98415c1c0 con 0x7ff9840b3990 2024-09-11T12:56:14.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.114+0000 7ff97aa00640 1 -- 172.21.15.78:0/2401114903 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) ==== 97+0+0 (secure 0 0 0) 0x7ff9740f71e0 con 0x7ff9840b3990 2024-09-11T12:56:14.119 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:56:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 msgr2=0x7ff960074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 0x7ff960074170 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff984166260 tx=0x7ff96c009040 comp rx=0 tx=0).stop 2024-09-11T12:56:14.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 msgr2=0x7ff9841f74a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff974009850 tx=0x7ff9740a55a0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 shutdown_connections 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff960071cb0 0x7ff960074170 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff9840cfdf0 0x7ff9840cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff9840c2f90 0x7ff98412d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 --2- 172.21.15.78:0/2401114903 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff9840b3990 0x7ff9841f74a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 >> 172.21.15.78:0/2401114903 conn(0x7ff984073770 msgr2=0x7ff984139f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 shutdown_connections 2024-09-11T12:56:14.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.118+0000 7ff983200640 1 -- 172.21.15.78:0/2401114903 wait complete. 2024-09-11T12:56:14.142 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:56:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 -- 172.21.15.78:0/1664757328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 msgr2=0x7fc24c0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 --2- 172.21.15.78:0/1664757328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c0a9d00 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fc24c0ce790 tx=0x7fc23801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 -- 172.21.15.78:0/1664757328 shutdown_connections 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 --2- 172.21.15.78:0/1664757328 >> v2:172.21.15.157:3300/0 conn(0x7fc24c0d0bf0 0x7fc24c0d1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 --2- 172.21.15.78:0/1664757328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c0a9d00 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 -- 172.21.15.78:0/1664757328 >> 172.21.15.78:0/1664757328 conn(0x7fc24c074b90 msgr2=0x7fc24c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 -- 172.21.15.78:0/1664757328 shutdown_connections 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.282+0000 7fc253a00640 1 -- 172.21.15.78:0/1664757328 wait complete. 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 Processor -- start 2024-09-11T12:56:14.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- start start 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24c0b14b0 0x7fc24c13b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc24c0d0bf0 0x7fc24c1fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c168e60 con 0x7fc24c06fdf0 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc24c168ce0 con 0x7fc24c0b14b0 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc24c168fe0 con 0x7fc24c0d0bf0 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43726/0 (socket says 172.21.15.78:43726) 2024-09-11T12:56:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 -- 172.21.15.78:0/1430977184 learned_addr learned my addr 172.21.15.78:0/1430977184 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc24be00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24c0b14b0 0x7fc24c13b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc24c0d0bf0 0x7fc24c1fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 -- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc24c0d0bf0 msgr2=0x7fc24c1fafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc24c0d0bf0 0x7fc24c1fafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 -- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24c0b14b0 msgr2=0x7fc24c13b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24c0b14b0 0x7fc24c13b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 -- 172.21.15.78:0/1430977184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc24c1556b0 con 0x7fc24c06fdf0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc251000640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c500 tx=0x7fc23c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c017070 con 0x7fc24c06fdf0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc23c0053c0 con 0x7fc24c06fdf0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc23c012870 con 0x7fc24c06fdf0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc24c155970 con 0x7fc24c06fdf0 2024-09-11T12:56:14.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc24c155f30 con 0x7fc24c06fdf0 2024-09-11T12:56:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.286+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc24c0a9d00 con 0x7fc24c06fdf0 2024-09-11T12:56:14.292 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.290+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc23c01a070 con 0x7fc24c06fdf0 2024-09-11T12:56:14.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.290+0000 7fc248c00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 0x7fc22c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.290+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7fc23c091910 con 0x7fc24c06fdf0 2024-09-11T12:56:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.290+0000 7fc24be00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 0x7fc22c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.294 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.290+0000 7fc24be00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 0x7fc22c074170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2380af000 tx=0x7fc23801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:14.297 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.294+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc23c05cb10 con 0x7fc24c06fdf0 2024-09-11T12:56:14.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.518+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc24c06c650 con 0x7fc24c06fdf0 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc248c00640 1 -- 172.21.15.78:0/1430977184 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7fc23c0653a0 con 0x7fc24c06fdf0 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:56:14.524 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.525 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:14.526 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.527 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.528 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:56:14.530 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.531 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.532 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.533 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.535 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.536 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.537 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:56:14.538 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 msgr2=0x7fc22c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 0x7fc22c074170 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc2380af000 tx=0x7fc23801b100 comp rx=0 tx=0).stop 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 msgr2=0x7fc24c153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fc23c00c500 tx=0x7fc23c00c9c0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 shutdown_connections 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc24c0d0bf0 0x7fc24c1fafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.539 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc22c071cb0 0x7fc22c074170 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc24c0b14b0 0x7fc24c13b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 --2- 172.21.15.78:0/1430977184 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc24c06fdf0 0x7fc24c153010 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 >> 172.21.15.78:0/1430977184 conn(0x7fc24c074b90 msgr2=0x7fc24c06ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 shutdown_connections 2024-09-11T12:56:14.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.522+0000 7fc253a00640 1 -- 172.21.15.78:0/1430977184 wait complete. 2024-09-11T12:56:14.545 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 -- 172.21.15.78:0/2752684919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204074260 msgr2=0x7fd20415aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 --2- 172.21.15.78:0/2752684919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204074260 0x7fd20415aec0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fd1f0009930 tx=0x7fd1f001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 -- 172.21.15.78:0/2752684919 shutdown_connections 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 --2- 172.21.15.78:0/2752684919 >> v2:172.21.15.157:3300/0 conn(0x7fd20406fc50 0x7fd204070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 --2- 172.21.15.78:0/2752684919 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd204074260 0x7fd20415aec0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 -- 172.21.15.78:0/2752684919 >> 172.21.15.78:0/2752684919 conn(0x7fd20406ca80 msgr2=0x7fd2040b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 -- 172.21.15.78:0/2752684919 shutdown_connections 2024-09-11T12:56:14.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.682+0000 7fd20c200640 1 -- 172.21.15.78:0/2752684919 wait complete. 2024-09-11T12:56:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 Processor -- start 2024-09-11T12:56:14.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- start start 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20406fc50 0x7fd204136ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040cf760 0x7fd204159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd204151010 con 0x7fd204074260 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd204151190 con 0x7fd2040cf760 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd204151310 con 0x7fd20406fc50 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43740/0 (socket says 172.21.15.78:43740) 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 -- 172.21.15.78:0/2265896070 learned_addr learned my addr 172.21.15.78:0/2265896070 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd209800640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20406fc50 0x7fd204136ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20a200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040cf760 0x7fd204159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 -- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20406fc50 msgr2=0x7fd204136ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20406fc50 0x7fd204136ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 -- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040cf760 msgr2=0x7fd204159900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040cf760 0x7fd204159900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.691 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 -- 172.21.15.78:0/2265896070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2040b36f0 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd208e00640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00b0040 tx=0x7fd1f00a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00b2040 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1f0005070 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f00054c0 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd204151fb0 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.686+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd204151cd0 con 0x7fd204074260 2024-09-11T12:56:14.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd204070100 con 0x7fd204074260 2024-09-11T12:56:14.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fd1f00bd060 con 0x7fd204074260 2024-09-11T12:56:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd201600640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 0x7fd1e4074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7fd1f012bd20 con 0x7fd204074260 2024-09-11T12:56:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd209800640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 0x7fd1e4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:14.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.690+0000 7fd209800640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 0x7fd1e4074170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001e40 tx=0x7fd1f8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:14.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.694+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd1f00164d0 con 0x7fd204074260 2024-09-11T12:56:14.866 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.862+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd20415caa0 con 0x7fd204074260 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.862+0000 7fd201600640 1 -- 172.21.15.78:0/2265896070 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) ==== 84+0+65 (secure 0 0 0) 0x7fd1f00f6f20 con 0x7fd204074260 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:14.867 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 msgr2=0x7fd1e4074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 0x7fd1e4074170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8001e40 tx=0x7fd1f8009040 comp rx=0 tx=0).stop 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 msgr2=0x7fd20414a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd1f00b0040 tx=0x7fd1f00a57a0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 shutdown_connections 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1e4071cb0 0x7fd1e4074170 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.869 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd2040cf760 0x7fd204159900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd204074260 0x7fd20414a780 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 --2- 172.21.15.78:0/2265896070 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd20406fc50 0x7fd204136ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 >> 172.21.15.78:0/2265896070 conn(0x7fd20406ca80 msgr2=0x7fd20415c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 shutdown_connections 2024-09-11T12:56:14.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:14.866+0000 7fd20c200640 1 -- 172.21.15.78:0/2265896070 wait complete. 2024-09-11T12:56:14.888 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:56:14.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- 172.21.15.78:0/2325729478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 msgr2=0x7f74a815af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- 172.21.15.78:0/2325729478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a815af50 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7498009840 tx=0x7f749801c7f0 comp rx=0 tx=0).stop 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- 172.21.15.78:0/2325729478 shutdown_connections 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- 172.21.15.78:0/2325729478 >> v2:172.21.15.157:3300/0 conn(0x7f74a80c64e0 0x7f74a815b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- 172.21.15.78:0/2325729478 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a815af50 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- 172.21.15.78:0/2325729478 >> 172.21.15.78:0/2325729478 conn(0x7f74a80cd190 msgr2=0x7f74a80cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- 172.21.15.78:0/2325729478 shutdown_connections 2024-09-11T12:56:15.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- 172.21.15.78:0/2325729478 wait complete. 2024-09-11T12:56:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 Processor -- start 2024-09-11T12:56:15.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- start start 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a81fb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a8073cb0 0x7f74a813d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74a81676b0 con 0x7f74a80c64e0 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74a815de80 con 0x7f74a8073cb0 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74ace00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74a8167830 con 0x7f74a80738d0 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43742/0 (socket says 172.21.15.78:43742) 2024-09-11T12:56:15.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 -- 172.21.15.78:0/2113557558 learned_addr learned my addr 172.21.15.78:0/2113557558 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a5e00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a81fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a5400640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a8073cb0 0x7f74a813d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 -- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 msgr2=0x7f74a81fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a81fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 -- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a8073cb0 msgr2=0x7f74a813d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a8073cb0 0x7f74a813d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 -- 172.21.15.78:0/2113557558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a80cba40 con 0x7f74a80c64e0 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a5e00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a81fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:15.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f74a6800640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009fd0 tx=0x7f74a000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.034+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a000ce20 con 0x7f74a80c64e0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74a0004b30 con 0x7f74a80c64e0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a813e2c0 con 0x7f74a80c64e0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0005020 con 0x7f74a80c64e0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a813e880 con 0x7f74a80c64e0 2024-09-11T12:56:15.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a80c5dd0 con 0x7f74a80c64e0 2024-09-11T12:56:15.042 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f74a00051c0 con 0x7f74a80c64e0 2024-09-11T12:56:15.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f749e000640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 0x7f7484074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:15.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 4857+0+0 (secure 0 0 0) 0x7f74a00932d0 con 0x7f74a80c64e0 2024-09-11T12:56:15.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.038+0000 7f74a5e00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 0x7f7484074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:15.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.042+0000 7f74a5e00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 0x7f7484074170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74980af000 tx=0x7f749801b030 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:15.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.042+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f74a005ea00 con 0x7f74a80c64e0 2024-09-11T12:56:15.218 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:15.214+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f74a80cff60 con 0x7f74a80c64e0 2024-09-11T12:56:16.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.122+0000 7f749e000640 1 -- 172.21.15.78:0/2113557558 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) ==== 138+0+0 (secure 0 0 0) 0x7f74a005e300 con 0x7f74a80c64e0 2024-09-11T12:56:16.127 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:56:16.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 msgr2=0x7f7484074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:16.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 0x7f7484074170 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74980af000 tx=0x7f749801b030 comp rx=0 tx=0).stop 2024-09-11T12:56:16.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 msgr2=0x7f74a80c3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:16.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009fd0 tx=0x7f74a000ef10 comp rx=0 tx=0).stop 2024-09-11T12:56:16.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 shutdown_connections 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a80c64e0 0x7f74a80c3980 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7484071cb0 0x7f7484074170 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a8073cb0 0x7f74a813d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 --2- 172.21.15.78:0/2113557558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a80738d0 0x7f74a81fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 >> 172.21.15.78:0/2113557558 conn(0x7f74a80cd190 msgr2=0x7f74a80cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 shutdown_connections 2024-09-11T12:56:16.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.126+0000 7f74ace00640 1 -- 172.21.15.78:0/2113557558 wait complete. 2024-09-11T12:56:16.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:56:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- 172.21.15.78:0/2750814110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc0a9460 msgr2=0x7f60dc0a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 --2- 172.21.15.78:0/2750814110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc0a9460 0x7f60dc0a98f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f60c8009870 tx=0x7f60c801c740 comp rx=0 tx=0).stop 2024-09-11T12:56:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- 172.21.15.78:0/2750814110 shutdown_connections 2024-09-11T12:56:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 --2- 172.21.15.78:0/2750814110 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc0a9460 0x7f60dc0a98f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- 172.21.15.78:0/2750814110 >> 172.21.15.78:0/2750814110 conn(0x7f60dc071cf0 msgr2=0x7f60dc072100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- 172.21.15.78:0/2750814110 shutdown_connections 2024-09-11T12:56:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- 172.21.15.78:0/2750814110 wait complete. 2024-09-11T12:56:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 Processor -- start 2024-09-11T12:56:16.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 -- start start 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.306+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc141200 0x7f60dc06fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 0x7f60dc1598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc169aa0 con 0x7f60dc0a9460 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60dc169920 con 0x7f60dc15c990 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60dc169c20 con 0x7f60dc141200 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43752/0 (socket says 172.21.15.78:43752) 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 -- 172.21.15.78:0/1327955804 learned_addr learned my addr 172.21.15.78:0/1327955804 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:16.313 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60dbe00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc141200 0x7f60dc06fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1400640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 0x7f60dc1598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 -- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc141200 msgr2=0x7f60dc06fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc141200 0x7f60dc06fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 -- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 msgr2=0x7f60dc1598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 0x7f60dc1598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 -- 172.21.15.78:0/1327955804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60dc15a310 con 0x7f60dc0a9460 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1400640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 0x7f60dc1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e0a00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c870 tx=0x7f60cc00cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc003b80 con 0x7f60dc0a9460 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60dc15a5f0 con 0x7f60dc0a9460 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60cc003d40 con 0x7f60dc0a9460 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00aa50 con 0x7f60dc0a9460 2024-09-11T12:56:16.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60dc13d5f0 con 0x7f60dc0a9460 2024-09-11T12:56:16.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60dc0a98f0 con 0x7f60dc0a9460 2024-09-11T12:56:16.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.310+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f60cc00abf0 con 0x7f60dc0a9460 2024-09-11T12:56:16.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.314+0000 7f60d8c00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 0x7f60bc074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:16.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.314+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 5227+0+0 (secure 0 0 0) 0x7f60cc093180 con 0x7f60dc0a9460 2024-09-11T12:56:16.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.314+0000 7f60dbe00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 0x7f60bc074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:16.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.314+0000 7f60dbe00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 0x7f60bc074170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f60c80af000 tx=0x7f60c80b2040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:16.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.318+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f60cc05e920 con 0x7f60dc0a9460 2024-09-11T12:56:16.487 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:16.482+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f60dc15a7c0 con 0x7f60dc0a9460 2024-09-11T12:56:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.130+0000 7f60d8c00640 1 -- 172.21.15.78:0/1327955804 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) ==== 170+0+0 (secure 0 0 0) 0x7f60cc05e220 con 0x7f60dc0a9460 2024-09-11T12:56:17.135 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 msgr2=0x7f60bc074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 0x7f60bc074170 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f60c80af000 tx=0x7f60c80b2040 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 msgr2=0x7f60dc153970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f60cc00c870 tx=0x7f60cc00cd30 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 shutdown_connections 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60bc071cb0 0x7f60bc074170 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60dc15c990 0x7f60dc1598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60dc141200 0x7f60dc06fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 --2- 172.21.15.78:0/1327955804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60dc0a9460 0x7f60dc153970 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 >> 172.21.15.78:0/1327955804 conn(0x7f60dc071cf0 msgr2=0x7f60dc15d470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:17.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 shutdown_connections 2024-09-11T12:56:17.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.134+0000 7f60e1e00640 1 -- 172.21.15.78:0/1327955804 wait complete. 2024-09-11T12:56:17.172 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- 172.21.15.78:0/2606542602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 msgr2=0x7f6b3c0b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 --2- 172.21.15.78:0/2606542602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c0b2de0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c15bfb0 tx=0x7f6b2c01c830 comp rx=0 tx=0).stop 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- 172.21.15.78:0/2606542602 shutdown_connections 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 --2- 172.21.15.78:0/2606542602 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 0x7f6b3c13f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 --2- 172.21.15.78:0/2606542602 >> v2:172.21.15.157:3300/0 conn(0x7f6b3c0cc5d0 0x7f6b3c15ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 --2- 172.21.15.78:0/2606542602 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c0b2de0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- 172.21.15.78:0/2606542602 >> 172.21.15.78:0/2606542602 conn(0x7f6b3c071dd0 msgr2=0x7f6b3c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- 172.21.15.78:0/2606542602 shutdown_connections 2024-09-11T12:56:17.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- 172.21.15.78:0/2606542602 wait complete. 2024-09-11T12:56:17.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 Processor -- start 2024-09-11T12:56:17.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.314+0000 7f6b42a00640 1 -- start start 2024-09-11T12:56:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b3c0cc5d0 0x7f6b3c15a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 0x7f6b3c15af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:17.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c15fc30 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c15fab0 con 0x7f6b3c0cc5d0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6b3c15fdb0 con 0x7f6b3c13f000 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43772/0 (socket says 172.21.15.78:43772) 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 -- 172.21.15.78:0/465282846 learned_addr learned my addr 172.21.15.78:0/465282846 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3b400640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b3c0cc5d0 0x7f6b3c15a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b40a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 0x7f6b3c15af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 -- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 msgr2=0x7f6b3c15af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 0x7f6b3c15af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 -- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b3c0cc5d0 msgr2=0x7f6b3c15a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b3c0cc5d0 0x7f6b3c15a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 -- 172.21.15.78:0/465282846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c13eac0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b3be00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b0040 tx=0x7f6b2c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:17.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0b2040 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b2c004340 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c13a230 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0052d0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c06ebe0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.318+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b3c1602e0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.322+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f6b2c005470 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.322+0000 7f6b33e00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 0x7f6b1c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.322+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 5609+0+0 (secure 0 0 0) 0x7f6b2c12c380 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.322+0000 7f6b3b400640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 0x7f6b1c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:17.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.322+0000 7f6b3b400640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 0x7f6b1c074170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c137290 tx=0x7f6b2800b310 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:17.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.326+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6b2c0164b0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:17.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:17.526+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f6b3c140640 con 0x7f6b3c0cc1f0 2024-09-11T12:56:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.134+0000 7f6b33e00640 1 -- 172.21.15.78:0/465282846 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v97) ==== 174+0+0 (secure 0 0 0) 0x7f6b2c0f79e0 con 0x7f6b3c0cc1f0 2024-09-11T12:56:18.137 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 22 and data pool 23 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 msgr2=0x7f6b1c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 0x7f6b1c074170 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c137290 tx=0x7f6b2800b310 comp rx=0 tx=0).stop 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 msgr2=0x7f6b3c1334d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0b0040 tx=0x7f6b2c0a56b0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 shutdown_connections 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6b3c13f000 0x7f6b3c15af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6b1c071cb0 0x7f6b1c074170 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6b3c0cc5d0 0x7f6b3c15a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 --2- 172.21.15.78:0/465282846 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6b3c0cc1f0 0x7f6b3c1334d0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 >> 172.21.15.78:0/465282846 conn(0x7f6b3c071dd0 msgr2=0x7f6b3c15c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 shutdown_connections 2024-09-11T12:56:18.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.138+0000 7f6b42a00640 1 -- 172.21.15.78:0/465282846 wait complete. 2024-09-11T12:56:18.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- 172.21.15.78:0/2419824549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 msgr2=0x7f37640cc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- 172.21.15.78:0/2419824549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f37640cc5e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f376406cf90 tx=0x7f375c01cae0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- 172.21.15.78:0/2419824549 shutdown_connections 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- 172.21.15.78:0/2419824549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f37640cc5e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- 172.21.15.78:0/2419824549 >> v2:172.21.15.157:3300/0 conn(0x7f37640b4780 0x7f376415e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- 172.21.15.78:0/2419824549 >> 172.21.15.78:0/2419824549 conn(0x7f376406ca80 msgr2=0x7f37640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- 172.21.15.78:0/2419824549 shutdown_connections 2024-09-11T12:56:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- 172.21.15.78:0/2419824549 wait complete. 2024-09-11T12:56:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 Processor -- start 2024-09-11T12:56:18.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- start start 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3764074bd0 0x7f376413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37640b4780 0x7f3764140260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f376413ebe0 con 0x7f376415a1c0 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f376413ea60 con 0x7f3764074bd0 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f376bc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f376413ed60 con 0x7f37640b4780 2024-09-11T12:56:18.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43796/0 (socket says 172.21.15.78:43796) 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 -- 172.21.15.78:0/3679409834 learned_addr learned my addr 172.21.15.78:0/3679409834 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769200640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3764074bd0 0x7f376413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3763e00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37640b4780 0x7f3764140260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 -- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37640b4780 msgr2=0x7f3764140260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37640b4780 0x7f3764140260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 -- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3764074bd0 msgr2=0x7f376413ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3764074bd0 0x7f376413ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 -- 172.21.15.78:0/3679409834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3764143230 con 0x7f376415a1c0 2024-09-11T12:56:18.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.330+0000 7f3769c00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f375c01b190 tx=0x7f375c005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0bb070 con 0x7f376415a1c0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f375c004d40 con 0x7f376415a1c0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f375c0b2040 con 0x7f376415a1c0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3764142990 con 0x7f376415a1c0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3764142f50 con 0x7f376415a1c0 2024-09-11T12:56:18.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f376413f230 con 0x7f376415a1c0 2024-09-11T12:56:18.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f375c0a5a70 con 0x7f376415a1c0 2024-09-11T12:56:18.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 0x7f3744074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7f375c12c690 con 0x7f376415a1c0 2024-09-11T12:56:18.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3769200640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 0x7f3744074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.334+0000 7f3769200640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 0x7f3744074170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3754004630 tx=0x7f375400b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:18.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.338+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f376415e700 con 0x7f376415a1c0 2024-09-11T12:56:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.506+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3764159840 con 0x7f376415a1c0 2024-09-11T12:56:18.509 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.506+0000 7f3760c00640 1 -- 172.21.15.78:0/3679409834 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16744 (secure 0 0 0) 0x7f375c0f75c0 con 0x7f376415a1c0 2024-09-11T12:56:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:18.510 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":139,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:17.530221+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:15.230647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:16.498978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 msgr2=0x7f3744074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 0x7f3744074170 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3754004630 tx=0x7f375400b390 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 msgr2=0x7f376406eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f375c01b190 tx=0x7f375c005b00 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 shutdown_connections 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f376415a1c0 0x7f376406eb40 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37640b4780 0x7f3764140260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3744071cb0 0x7f3744074170 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 --2- 172.21.15.78:0/3679409834 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3764074bd0 0x7f376413ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 >> 172.21.15.78:0/3679409834 conn(0x7f376406ca80 msgr2=0x7f3764073200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:18.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 shutdown_connections 2024-09-11T12:56:18.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.510+0000 7f376bc00640 1 -- 172.21.15.78:0/3679409834 wait complete. 2024-09-11T12:56:18.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:18.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 -- 172.21.15.78:0/3087302211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd40b0ff0 msgr2=0x7fabd415ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 --2- 172.21.15.78:0/3087302211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd40b0ff0 0x7fabd415ff40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009840 tx=0x7fabc801c820 comp rx=0 tx=0).stop 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 -- 172.21.15.78:0/3087302211 shutdown_connections 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 --2- 172.21.15.78:0/3087302211 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40d0ec0 0x7fabd40d1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 --2- 172.21.15.78:0/3087302211 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd40b0ff0 0x7fabd415ff40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 -- 172.21.15.78:0/3087302211 >> 172.21.15.78:0/3087302211 conn(0x7fabd40ceab0 msgr2=0x7fabd40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 -- 172.21.15.78:0/3087302211 shutdown_connections 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.670+0000 7fabdbe00640 1 -- 172.21.15.78:0/3087302211 wait complete. 2024-09-11T12:56:18.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 Processor -- start 2024-09-11T12:56:18.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- start start 2024-09-11T12:56:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40b0ff0 0x7fabd413da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd41520b0 0x7fabd413f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fabd40aa4e0 con 0x7fabd40d0ec0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fabd40aa360 con 0x7fabd41520b0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fabd40aa660 con 0x7fabd40b0ff0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43800/0 (socket says 172.21.15.78:43800) 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 -- 172.21.15.78:0/1645849306 learned_addr learned my addr 172.21.15.78:0/1645849306 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd9400640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40b0ff0 0x7fabd413da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd9e00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd41520b0 0x7fabd413f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 -- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40b0ff0 msgr2=0x7fabd413da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40b0ff0 0x7fabd413da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 -- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd41520b0 msgr2=0x7fabd413f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd41520b0 0x7fabd413f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 -- 172.21.15.78:0/1645849306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabd413e710 con 0x7fabd40d0ec0 2024-09-11T12:56:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd8a00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9d0 tx=0x7fabc000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc000cde0 con 0x7fabd40d0ec0 2024-09-11T12:56:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabc0004020 con 0x7fabd40d0ec0 2024-09-11T12:56:18.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0004450 con 0x7fabd40d0ec0 2024-09-11T12:56:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabd413e9f0 con 0x7fabd40d0ec0 2024-09-11T12:56:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.674+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabd413efb0 con 0x7fabd40d0ec0 2024-09-11T12:56:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabd413f3f0 con 0x7fabd40d0ec0 2024-09-11T12:56:18.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fabc0003600 con 0x7fabd40d0ec0 2024-09-11T12:56:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabd1600640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 0x7fabb4074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7fabc0093280 con 0x7fabd40d0ec0 2024-09-11T12:56:18.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabd9400640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 0x7fabb4074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:18.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.678+0000 7fabd9400640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 0x7fabb4074170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc80af000 tx=0x7fabc801b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:18.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.682+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fabc005e020 con 0x7fabd40d0ec0 2024-09-11T12:56:18.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.850+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabd40aab10 con 0x7fabd40d0ec0 2024-09-11T12:56:18.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.850+0000 7fabd1600640 1 -- 172.21.15.78:0/1645849306 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) ==== 74+0+16744 (secure 0 0 0) 0x7fabc0062570 con 0x7fabd40d0ec0 2024-09-11T12:56:18.853 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:18.854 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":139,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:17.530221+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:15.230647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:16.498978+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.850+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 msgr2=0x7fabb4074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.850+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 0x7fabb4074170 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fabc80af000 tx=0x7fabc801b060 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 msgr2=0x7fabd4141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fabc000e9d0 tx=0x7fabc000ee90 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 shutdown_connections 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fabb4071cb0 0x7fabb4074170 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fabd41520b0 0x7fabd413f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fabd40d0ec0 0x7fabd4141820 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 --2- 172.21.15.78:0/1645849306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fabd40b0ff0 0x7fabd413da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 >> 172.21.15.78:0/1645849306 conn(0x7fabd40ceab0 msgr2=0x7fabd415d070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:18.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 shutdown_connections 2024-09-11T12:56:18.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:18.854+0000 7fabdbe00640 1 -- 172.21.15.78:0/1645849306 wait complete. 2024-09-11T12:56:18.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:56:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 -- 172.21.15.78:0/417401899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 msgr2=0x7eff3c072f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 --2- 172.21.15.78:0/417401899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c072f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7eff2c009840 tx=0x7eff2c01c820 comp rx=0 tx=0).stop 2024-09-11T12:56:19.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 -- 172.21.15.78:0/417401899 shutdown_connections 2024-09-11T12:56:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 --2- 172.21.15.78:0/417401899 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 0x7eff3c0734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 --2- 172.21.15.78:0/417401899 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c072f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 -- 172.21.15.78:0/417401899 >> 172.21.15.78:0/417401899 conn(0x7eff3c0d0410 msgr2=0x7eff3c0d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 -- 172.21.15.78:0/417401899 shutdown_connections 2024-09-11T12:56:19.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.018+0000 7eff42a00640 1 -- 172.21.15.78:0/417401899 wait complete. 2024-09-11T12:56:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 Processor -- start 2024-09-11T12:56:19.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- start start 2024-09-11T12:56:19.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 0x7eff3c1fb810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:19.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c13fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c168f30 con 0x7eff3c144c80 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7eff3c168db0 con 0x7eff3c144640 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7eff3c1690b0 con 0x7eff3c071d90 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43808/0 (socket says 172.21.15.78:43808) 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 -- 172.21.15.78:0/1718500616 learned_addr learned my addr 172.21.15.78:0/1718500616 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff3be00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 0x7eff3c1fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff33e00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c13fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 -- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 msgr2=0x7eff3c1fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 0x7eff3c1fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:19.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 -- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 msgr2=0x7eff3c13fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c13fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 -- 172.21.15.78:0/1718500616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3c140080 con 0x7eff3c144c80 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff40a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ca60 tx=0x7eff3400cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff34016020 con 0x7eff3c144c80 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff3400a930 con 0x7eff3c144c80 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff3400ad60 con 0x7eff3c144c80 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3c164dd0 con 0x7eff3c144c80 2024-09-11T12:56:19.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff3c06f950 con 0x7eff3c144c80 2024-09-11T12:56:19.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.022+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff3c15f590 con 0x7eff3c144c80 2024-09-11T12:56:19.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.026+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7eff34019020 con 0x7eff3c144c80 2024-09-11T12:56:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.026+0000 7eff38c00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 0x7eff18074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.026+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(139..139 src has 1..139) ==== 5709+0+0 (secure 0 0 0) 0x7eff340926a0 con 0x7eff3c144c80 2024-09-11T12:56:19.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.026+0000 7eff3be00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 0x7eff18074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:19.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.026+0000 7eff3be00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 0x7eff18074170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0af000 tx=0x7eff2c01b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:19.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.030+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7eff34020030 con 0x7eff3c144c80 2024-09-11T12:56:19.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:19.234+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7eff3c073410 con 0x7eff3c144c80 2024-09-11T12:56:20.122 INFO:tasks.ceph.mds.c.smithi078.stderr:2024-09-11T12:56:20.122+0000 7f9dcd000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:56:20.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.158+0000 7eff38c00640 1 -- 172.21.15.78:0/1718500616 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) ==== 118+0+0 (secure 0 0 0) 0x7eff3c140080 con 0x7eff3c144c80 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 msgr2=0x7eff18074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 0x7eff18074170 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7eff2c0af000 tx=0x7eff2c01b060 comp rx=0 tx=0).stop 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 msgr2=0x7eff3c13eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7eff3400ca60 tx=0x7eff3400cb50 comp rx=0 tx=0).stop 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 shutdown_connections 2024-09-11T12:56:20.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7eff3c144c80 0x7eff3c13eae0 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7eff18071cb0 0x7eff18074170 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7eff3c144640 0x7eff3c13fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 --2- 172.21.15.78:0/1718500616 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7eff3c071d90 0x7eff3c1fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 >> 172.21.15.78:0/1718500616 conn(0x7eff3c0d0410 msgr2=0x7eff3c15e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 shutdown_connections 2024-09-11T12:56:20.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.162+0000 7eff42a00640 1 -- 172.21.15.78:0/1718500616 wait complete. 2024-09-11T12:56:20.189 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:56:20.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:56:20.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- 172.21.15.78:0/4131620507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cde50 msgr2=0x7f81cc0b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:20.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- 172.21.15.78:0/4131620507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cde50 0x7f81cc0b2f40 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f81cc072e20 tx=0x7f81c401cd80 comp rx=0 tx=0).stop 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- 172.21.15.78:0/4131620507 shutdown_connections 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- 172.21.15.78:0/4131620507 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cde50 0x7f81cc0b2f40 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- 172.21.15.78:0/4131620507 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc0cda10 0x7f81cc0b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- 172.21.15.78:0/4131620507 >> v2:172.21.15.157:3300/0 conn(0x7f81cc13d6d0 0x7f81cc06e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- 172.21.15.78:0/4131620507 >> 172.21.15.78:0/4131620507 conn(0x7f81cc075c10 msgr2=0x7f81cc0b16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- 172.21.15.78:0/4131620507 shutdown_connections 2024-09-11T12:56:20.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- 172.21.15.78:0/4131620507 wait complete. 2024-09-11T12:56:20.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 Processor -- start 2024-09-11T12:56:20.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- start start 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 0x7f81cc158400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc13d6d0 0x7f81cc166f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc06f9c0 con 0x7f81cc0cda10 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f81cc06f840 con 0x7f81cc0cde50 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.334+0000 7f81d0e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f81cc06fb40 con 0x7f81cc13d6d0 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43832/0 (socket says 172.21.15.78:43832) 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 -- 172.21.15.78:0/193497477 learned_addr learned my addr 172.21.15.78:0/193497477 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81caa00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 0x7f81cc158400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cbe00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc13d6d0 0x7f81cc166f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:20.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 -- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc13d6d0 msgr2=0x7f81cc166f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc13d6d0 0x7f81cc166f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 -- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 msgr2=0x7f81cc158400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 0x7f81cc158400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 -- 172.21.15.78:0/193497477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81cc152870 con 0x7f81cc0cda10 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81caa00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 0x7f81cc158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81cb400640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d950 tx=0x7f81bc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc017070 con 0x7f81cc0cda10 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81bc009d20 con 0x7f81cc0cda10 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81cc152b10 con 0x7f81cc0cda10 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0106c0 con 0x7f81cc0cda10 2024-09-11T12:56:20.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81cc13ce10 con 0x7f81cc0cda10 2024-09-11T12:56:20.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81cc06e720 con 0x7f81cc0cda10 2024-09-11T12:56:20.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.338+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f81bc01a070 con 0x7f81cc0cda10 2024-09-11T12:56:20.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.342+0000 7f81c3400640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 0x7f81ac074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:20.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.342+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f81bc092aa0 con 0x7f81cc0cda10 2024-09-11T12:56:20.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.342+0000 7f81caa00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 0x7f81ac074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:20.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.342+0000 7f81caa00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 0x7f81ac074170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005fd0 tx=0x7f81b8005f20 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:20.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.346+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f81bc05d830 con 0x7f81cc0cda10 2024-09-11T12:56:20.549 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:20.546+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f81cc073dc0 con 0x7f81cc0cda10 2024-09-11T12:56:21.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.162+0000 7f81c3400640 1 -- 172.21.15.78:0/193497477 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) ==== 122+0+0 (secure 0 0 0) 0x7f81bc061d80 con 0x7f81cc0cda10 2024-09-11T12:56:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 msgr2=0x7f81ac074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 0x7f81ac074170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f81b8005fd0 tx=0x7f81b8005f20 comp rx=0 tx=0).stop 2024-09-11T12:56:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 msgr2=0x7f81cc157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d950 tx=0x7f81bc00de10 comp rx=0 tx=0).stop 2024-09-11T12:56:21.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 shutdown_connections 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f81cc13d6d0 0x7f81cc166f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f81ac071cb0 0x7f81ac074170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f81cc0cde50 0x7f81cc158400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 --2- 172.21.15.78:0/193497477 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f81cc0cda10 0x7f81cc157e20 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 >> 172.21.15.78:0/193497477 conn(0x7f81cc075c10 msgr2=0x7f81cc15e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 shutdown_connections 2024-09-11T12:56:21.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.166+0000 7f81d0e00640 1 -- 172.21.15.78:0/193497477 wait complete. 2024-09-11T12:56:21.190 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:56:21.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- 172.21.15.78:0/3189320472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80aed20 msgr2=0x7f5db80cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3189320472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80aed20 0x7f5db80cf280 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009c80 tx=0x7f5da801cd70 comp rx=0 tx=0).stop 2024-09-11T12:56:21.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- 172.21.15.78:0/3189320472 shutdown_connections 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3189320472 >> v2:172.21.15.157:3300/0 conn(0x7f5db815d1e0 0x7f5db80b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3189320472 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80aed20 0x7f5db80cf280 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- 172.21.15.78:0/3189320472 >> 172.21.15.78:0/3189320472 conn(0x7f5db806ca80 msgr2=0x7f5db80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- 172.21.15.78:0/3189320472 shutdown_connections 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- 172.21.15.78:0/3189320472 wait complete. 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 Processor -- start 2024-09-11T12:56:21.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.330+0000 7f5dbce00640 1 -- start start 2024-09-11T12:56:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 0x7f5db8145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80cf7c0 0x7f5db8141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5db80cd5a0 con 0x7f5db815d1e0 2024-09-11T12:56:21.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5db80cd420 con 0x7f5db80aed20 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5db80cd720 con 0x7f5db80cf7c0 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43838/0 (socket says 172.21.15.78:43838) 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 -- 172.21.15.78:0/3830966683 learned_addr learned my addr 172.21.15.78:0/3830966683 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db5e00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 0x7f5db8145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db5400640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80cf7c0 0x7f5db8141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 -- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80cf7c0 msgr2=0x7f5db8141980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80cf7c0 0x7f5db8141980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 -- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 msgr2=0x7f5db8145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 0x7f5db8145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 -- 172.21.15.78:0/3830966683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db8155f00 con 0x7f5db815d1e0 2024-09-11T12:56:21.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db5e00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 0x7f5db8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5db6800640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e970 tx=0x7f5db000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000ccc0 con 0x7f5db815d1e0 2024-09-11T12:56:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5db0004020 con 0x7f5db815d1e0 2024-09-11T12:56:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0004470 con 0x7f5db815d1e0 2024-09-11T12:56:21.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db8155940 con 0x7f5db815d1e0 2024-09-11T12:56:21.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db814fca0 con 0x7f5db815d1e0 2024-09-11T12:56:21.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db80cdc40 con 0x7f5db815d1e0 2024-09-11T12:56:21.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.334+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f5db0003600 con 0x7f5db815d1e0 2024-09-11T12:56:21.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.338+0000 7f5dae000640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 0x7f5d98074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.338+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f5db0093190 con 0x7f5db815d1e0 2024-09-11T12:56:21.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.338+0000 7f5db5e00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 0x7f5d98074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.338+0000 7f5db5e00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 0x7f5d98074170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5da80af000 tx=0x7f5da801b5b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:21.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.342+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5db005e600 con 0x7f5db815d1e0 2024-09-11T12:56:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.538+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db814b9d0 con 0x7f5db815d1e0 2024-09-11T12:56:21.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.538+0000 7f5dae000640 1 -- 172.21.15.78:0/3830966683 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5229 (secure 0 0 0) 0x7f5db005df00 con 0x7f5db815d1e0 2024-09-11T12:56:21.542 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:21.542 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":101,"btime":"2024-09-11T12:56:21:163184+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:21.542 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 101 2024-09-11T12:56:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 msgr2=0x7f5d98074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 0x7f5d98074170 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5da80af000 tx=0x7f5da801b5b0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 msgr2=0x7f5db8154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f5db000e970 tx=0x7f5db000ee30 comp rx=0 tx=0).stop 2024-09-11T12:56:21.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 shutdown_connections 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5db815d1e0 0x7f5db8154ed0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5db80cf7c0 0x7f5db8141980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5d98071cb0 0x7f5d98074170 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 --2- 172.21.15.78:0/3830966683 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5db80aed20 0x7f5db8145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 >> 172.21.15.78:0/3830966683 conn(0x7f5db806ca80 msgr2=0x7f5db80af870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 shutdown_connections 2024-09-11T12:56:21.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.542+0000 7f5dbce00640 1 -- 172.21.15.78:0/3830966683 wait complete. 2024-09-11T12:56:21.563 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:21.705 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- 172.21.15.78:0/1228942577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 msgr2=0x7fc364161e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 --2- 172.21.15.78:0/1228942577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc364161e80 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fc350009930 tx=0x7fc35001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- 172.21.15.78:0/1228942577 shutdown_connections 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 --2- 172.21.15.78:0/1228942577 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 0x7fc3640c2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 --2- 172.21.15.78:0/1228942577 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc364161e80 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- 172.21.15.78:0/1228942577 >> 172.21.15.78:0/1228942577 conn(0x7fc364073770 msgr2=0x7fc364073b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- 172.21.15.78:0/1228942577 shutdown_connections 2024-09-11T12:56:21.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- 172.21.15.78:0/1228942577 wait complete. 2024-09-11T12:56:21.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 Processor -- start 2024-09-11T12:56:21.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.702+0000 7fc36a200640 1 -- start start 2024-09-11T12:56:21.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 0x7fc3640cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc364154890 0x7fc3641602d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc36416a810 con 0x7fc364072d70 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3640c3fb0 con 0x7fc364154890 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc36416a990 con 0x7fc364070600 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43864/0 (socket says 172.21.15.78:43864) 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 -- 172.21.15.78:0/65305503 learned_addr learned my addr 172.21.15.78:0/65305503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc368e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 0x7fc3640cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc369800640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc364154890 0x7fc3641602d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 -- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 msgr2=0x7fc3640cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 0x7fc3640cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 -- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc364154890 msgr2=0x7fc3641602d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc364154890 0x7fc3641602d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 -- 172.21.15.78:0/65305503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc36414c400 con 0x7fc364072d70 2024-09-11T12:56:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc363e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc35001a4e0 tx=0x7fc3500040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:21.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3500ba020 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc350005030 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3641483e0 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3500b2040 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc36414a2a0 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.706+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3640c2f90 con 0x7fc364072d70 2024-09-11T12:56:21.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.710+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc3500054a0 con 0x7fc364072d70 2024-09-11T12:56:21.712 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.710+0000 7fc360c00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 0x7fc3440741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:21.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.710+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc35012c810 con 0x7fc364072d70 2024-09-11T12:56:21.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.710+0000 7fc368e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 0x7fc3440741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:21.713 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.710+0000 7fc368e00640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 0x7fc3440741a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc354001e40 tx=0x7fc354009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:21.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.714+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc3500c4060 con 0x7fc364072d70 2024-09-11T12:56:21.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.878+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc36406fea0 con 0x7fc364072d70 2024-09-11T12:56:21.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc360c00640 1 -- 172.21.15.78:0/65305503 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16731 (secure 0 0 0) 0x7fc3500164d0 con 0x7fc364072d70 2024-09-11T12:56:21.884 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:21.884 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":140,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:19.149907+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:15.230647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:16.498978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:21.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 msgr2=0x7fc3440741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 0x7fc3440741a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fc354001e40 tx=0x7fc354009040 comp rx=0 tx=0).stop 2024-09-11T12:56:21.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 msgr2=0x7fc3641fb1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fc35001a4e0 tx=0x7fc3500040c0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 shutdown_connections 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc344071ce0 0x7fc3440741a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc364154890 0x7fc3641602d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc364072d70 0x7fc3641fb1f0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 --2- 172.21.15.78:0/65305503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc364070600 0x7fc3640cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 >> 172.21.15.78:0/65305503 conn(0x7fc364073770 msgr2=0x7fc36415f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 shutdown_connections 2024-09-11T12:56:21.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:21.882+0000 7fc36a200640 1 -- 172.21.15.78:0/65305503 wait complete. 2024-09-11T12:56:21.905 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:56:21.905 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:22.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 -- 172.21.15.78:0/1286287152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 msgr2=0x7fba1015b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 --2- 172.21.15.78:0/1286287152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba1015b550 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009900 tx=0x7fb9f801c8a0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 -- 172.21.15.78:0/1286287152 shutdown_connections 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 --2- 172.21.15.78:0/1286287152 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba1015b550 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 --2- 172.21.15.78:0/1286287152 >> v2:172.21.15.157:3300/0 conn(0x7fba100b4b60 0x7fba1015ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 --2- 172.21.15.78:0/1286287152 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 0x7fba10073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 -- 172.21.15.78:0/1286287152 >> 172.21.15.78:0/1286287152 conn(0x7fba1006ca80 msgr2=0x7fba100b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 -- 172.21.15.78:0/1286287152 shutdown_connections 2024-09-11T12:56:22.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.042+0000 7fba16000640 1 -- 172.21.15.78:0/1286287152 wait complete. 2024-09-11T12:56:22.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 Processor -- start 2024-09-11T12:56:22.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- start start 2024-09-11T12:56:22.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 0x7fba10139400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba100b4b60 0x7fba100b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fba1006f100 con 0x7fba100ccb20 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fba1006ef80 con 0x7fba100b4b60 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fba1006f280 con 0x7fba100b4780 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43872/0 (socket says 172.21.15.78:43872) 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 -- 172.21.15.78:0/2885825399 learned_addr learned my addr 172.21.15.78:0/2885825399 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba07e00640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba100b4b60 0x7fba100b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0f400640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 0x7fba10139400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 -- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 msgr2=0x7fba10139400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 0x7fba10139400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 -- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba100b4b60 msgr2=0x7fba100b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba100b4b60 0x7fba100b0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 -- 172.21.15.78:0/2885825399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba101f5820 con 0x7fba100ccb20 2024-09-11T12:56:22.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba0fe00640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cce0 tx=0x7fba0800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800f450 con 0x7fba100ccb20 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba0800f700 con 0x7fba100ccb20 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0800fb50 con 0x7fba100ccb20 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba101f5b00 con 0x7fba100ccb20 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba10070920 con 0x7fba100ccb20 2024-09-11T12:56:22.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.046+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba10073f70 con 0x7fba100ccb20 2024-09-11T12:56:22.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.050+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fba0800fcf0 con 0x7fba100ccb20 2024-09-11T12:56:22.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.050+0000 7fba07400640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 0x7fb9ec074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.050+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fba080934c0 con 0x7fba100ccb20 2024-09-11T12:56:22.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.050+0000 7fba0f400640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 0x7fb9ec074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.050+0000 7fba0f400640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 0x7fb9ec074170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fba00005fd0 tx=0x7fba00005e60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.054+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fba08014030 con 0x7fba100ccb20 2024-09-11T12:56:22.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.218+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fba101389b0 con 0x7fb9ec071cb0 2024-09-11T12:56:22.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.222+0000 7fba07400640 1 -- 172.21.15.78:0/2885825399 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7fba101389b0 con 0x7fb9ec071cb0 2024-09-11T12:56:22.227 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:22.229 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:22.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 msgr2=0x7fb9ec074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 0x7fb9ec074170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fba00005fd0 tx=0x7fba00005e60 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 msgr2=0x7fba101f4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fba0800cce0 tx=0x7fba0800a620 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 shutdown_connections 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fba100ccb20 0x7fba101f4d60 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb9ec071cb0 0x7fb9ec074170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fba100b4b60 0x7fba100b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 --2- 172.21.15.78:0/2885825399 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fba100b4780 0x7fba10139400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 >> 172.21.15.78:0/2885825399 conn(0x7fba1006ca80 msgr2=0x7fba10156ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 shutdown_connections 2024-09-11T12:56:22.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.226+0000 7fba16000640 1 -- 172.21.15.78:0/2885825399 wait complete. 2024-09-11T12:56:22.248 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-11T12:56:20.502301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276924,"kb_used_data":14896,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697796,"statfs":{"total":773094113280,"available":772810543104,"internally_reserved":0,"allocated":15253504,"data_stored":7335908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.372016"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595977+0000","last_change":"2024-09-11T12:56:18.173943+0000","last_active":"2024-09-11T12:56:19.595977+0000","last_peered":"2024-09-11T12:56:19.595977+0000","last_clean":"2024-09-11T12:56:19.595977+0000","last_became_active":"2024-09-11T12:56:18.173652+0000","last_became_peered":"2024-09-11T12:56:18.173652+0000","last_unstale":"2024-09-11T12:56:19.595977+0000","last_undegraded":"2024-09-11T12:56:19.595977+0000","last_fullsized":"2024-09-11T12:56:19.595977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:18.772831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596430+0000","last_change":"2024-09-11T12:56:18.164866+0000","last_active":"2024-09-11T12:56:19.596430+0000","last_peered":"2024-09-11T12:56:19.596430+0000","last_clean":"2024-09-11T12:56:19.596430+0000","last_became_active":"2024-09-11T12:56:18.164605+0000","last_became_peered":"2024-09-11T12:56:18.164605+0000","last_unstale":"2024-09-11T12:56:19.596430+0000","last_undegraded":"2024-09-11T12:56:19.596430+0000","last_fullsized":"2024-09-11T12:56:19.596430+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:23.148160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160240+0000","last_change":"2024-09-11T12:56:18.159219+0000","last_active":"2024-09-11T12:56:19.160240+0000","last_peered":"2024-09-11T12:56:19.160240+0000","last_clean":"2024-09-11T12:56:19.160240+0000","last_became_active":"2024-09-11T12:56:18.158933+0000","last_became_peered":"2024-09-11T12:56:18.158933+0000","last_unstale":"2024-09-11T12:56:19.160240+0000","last_undegraded":"2024-09-11T12:56:19.160240+0000","last_fullsized":"2024-09-11T12:56:19.160240+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:52.766319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594935+0000","last_change":"2024-09-11T12:56:18.163856+0000","last_active":"2024-09-11T12:56:19.594935+0000","last_peered":"2024-09-11T12:56:19.594935+0000","last_clean":"2024-09-11T12:56:19.594935+0000","last_became_active":"2024-09-11T12:56:18.163556+0000","last_became_peered":"2024-09-11T12:56:18.163556+0000","last_unstale":"2024-09-11T12:56:19.594935+0000","last_undegraded":"2024-09-11T12:56:19.594935+0000","last_fullsized":"2024-09-11T12:56:19.594935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:23.489680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596493+0000","last_change":"2024-09-11T12:56:18.172839+0000","last_active":"2024-09-11T12:56:19.596493+0000","last_peered":"2024-09-11T12:56:19.596493+0000","last_clean":"2024-09-11T12:56:19.596493+0000","last_became_active":"2024-09-11T12:56:18.172579+0000","last_became_peered":"2024-09-11T12:56:18.172579+0000","last_unstale":"2024-09-11T12:56:19.596493+0000","last_undegraded":"2024-09-11T12:56:19.596493+0000","last_fullsized":"2024-09-11T12:56:19.596493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595269+0000","last_change":"2024-09-11T12:56:18.173161+0000","last_active":"2024-09-11T12:56:19.595269+0000","last_peered":"2024-09-11T12:56:19.595269+0000","last_clean":"2024-09-11T12:56:19.595269+0000","last_became_active":"2024-09-11T12:56:18.172846+0000","last_became_peered":"2024-09-11T12:56:18.172846+0000","last_unstale":"2024-09-11T12:56:19.595269+0000","last_undegraded":"2024-09-11T12:56:19.595269+0000","last_fullsized":"2024-09-11T12:56:19.595269+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:23.412771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594607+0000","last_change":"2024-09-11T12:56:18.179661+0000","last_active":"2024-09-11T12:56:19.594607+0000","last_peered":"2024-09-11T12:56:19.594607+0000","last_clean":"2024-09-11T12:56:19.594607+0000","last_became_active":"2024-09-11T12:56:18.179403+0000","last_became_peered":"2024-09-11T12:56:18.179403+0000","last_unstale":"2024-09-11T12:56:19.594607+0000","last_undegraded":"2024-09-11T12:56:19.594607+0000","last_fullsized":"2024-09-11T12:56:19.594607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:42:46.556219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595447+0000","last_change":"2024-09-11T12:56:18.166706+0000","last_active":"2024-09-11T12:56:19.595447+0000","last_peered":"2024-09-11T12:56:19.595447+0000","last_clean":"2024-09-11T12:56:19.595447+0000","last_became_active":"2024-09-11T12:56:18.166394+0000","last_became_peered":"2024-09-11T12:56:18.166394+0000","last_unstale":"2024-09-11T12:56:19.595447+0000","last_undegraded":"2024-09-11T12:56:19.595447+0000","last_fullsized":"2024-09-11T12:56:19.595447+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:36.186648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596698+0000","last_change":"2024-09-11T12:56:18.162390+0000","last_active":"2024-09-11T12:56:19.596698+0000","last_peered":"2024-09-11T12:56:19.596698+0000","last_clean":"2024-09-11T12:56:19.596698+0000","last_became_active":"2024-09-11T12:56:18.162072+0000","last_became_peered":"2024-09-11T12:56:18.162072+0000","last_unstale":"2024-09-11T12:56:19.596698+0000","last_undegraded":"2024-09-11T12:56:19.596698+0000","last_fullsized":"2024-09-11T12:56:19.596698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596601+0000","last_change":"2024-09-11T12:56:18.175354+0000","last_active":"2024-09-11T12:56:19.596601+0000","last_peered":"2024-09-11T12:56:19.596601+0000","last_clean":"2024-09-11T12:56:19.596601+0000","last_became_active":"2024-09-11T12:56:18.175161+0000","last_became_peered":"2024-09-11T12:56:18.175161+0000","last_unstale":"2024-09-11T12:56:19.596601+0000","last_undegraded":"2024-09-11T12:56:19.596601+0000","last_fullsized":"2024-09-11T12:56:19.596601+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596468+0000","last_change":"2024-09-11T12:56:18.173609+0000","last_active":"2024-09-11T12:56:19.596468+0000","last_peered":"2024-09-11T12:56:19.596468+0000","last_clean":"2024-09-11T12:56:19.596468+0000","last_became_active":"2024-09-11T12:56:18.173274+0000","last_became_peered":"2024-09-11T12:56:18.173274+0000","last_unstale":"2024-09-11T12:56:19.596468+0000","last_undegraded":"2024-09-11T12:56:19.596468+0000","last_fullsized":"2024-09-11T12:56:19.596468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:36.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596373+0000","last_change":"2024-09-11T12:56:18.168941+0000","last_active":"2024-09-11T12:56:19.596373+0000","last_peered":"2024-09-11T12:56:19.596373+0000","last_clean":"2024-09-11T12:56:19.596373+0000","last_became_active":"2024-09-11T12:56:18.168672+0000","last_became_peered":"2024-09-11T12:56:18.168672+0000","last_unstale":"2024-09-11T12:56:19.596373+0000","last_undegraded":"2024-09-11T12:56:19.596373+0000","last_fullsized":"2024-09-11T12:56:19.596373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:47.937582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160530+0000","last_change":"2024-09-11T12:56:18.179560+0000","last_active":"2024-09-11T12:56:19.160530+0000","last_peered":"2024-09-11T12:56:19.160530+0000","last_clean":"2024-09-11T12:56:19.160530+0000","last_became_active":"2024-09-11T12:56:18.179206+0000","last_became_peered":"2024-09-11T12:56:18.179206+0000","last_unstale":"2024-09-11T12:56:19.160530+0000","last_undegraded":"2024-09-11T12:56:19.160530+0000","last_fullsized":"2024-09-11T12:56:19.160530+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:09.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159646+0000","last_change":"2024-09-11T12:56:18.160292+0000","last_active":"2024-09-11T12:56:19.159646+0000","last_peered":"2024-09-11T12:56:19.159646+0000","last_clean":"2024-09-11T12:56:19.159646+0000","last_became_active":"2024-09-11T12:56:18.160043+0000","last_became_peered":"2024-09-11T12:56:18.160043+0000","last_unstale":"2024-09-11T12:56:19.159646+0000","last_undegraded":"2024-09-11T12:56:19.159646+0000","last_fullsized":"2024-09-11T12:56:19.159646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:17.045517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595721+0000","last_change":"2024-09-11T12:56:18.177991+0000","last_active":"2024-09-11T12:56:19.595721+0000","last_peered":"2024-09-11T12:56:19.595721+0000","last_clean":"2024-09-11T12:56:19.595721+0000","last_became_active":"2024-09-11T12:56:18.177360+0000","last_became_peered":"2024-09-11T12:56:18.177360+0000","last_unstale":"2024-09-11T12:56:19.595721+0000","last_undegraded":"2024-09-11T12:56:19.595721+0000","last_fullsized":"2024-09-11T12:56:19.595721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:04.083800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593870+0000","last_change":"2024-09-11T12:56:18.170484+0000","last_active":"2024-09-11T12:56:19.593870+0000","last_peered":"2024-09-11T12:56:19.593870+0000","last_clean":"2024-09-11T12:56:19.593870+0000","last_became_active":"2024-09-11T12:56:18.170026+0000","last_became_peered":"2024-09-11T12:56:18.170026+0000","last_unstale":"2024-09-11T12:56:19.593870+0000","last_undegraded":"2024-09-11T12:56:19.593870+0000","last_fullsized":"2024-09-11T12:56:19.593870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:43.440169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595768+0000","last_change":"2024-09-11T12:56:18.173193+0000","last_active":"2024-09-11T12:56:19.595768+0000","last_peered":"2024-09-11T12:56:19.595768+0000","last_clean":"2024-09-11T12:56:19.595768+0000","last_became_active":"2024-09-11T12:56:18.172909+0000","last_became_peered":"2024-09-11T12:56:18.172909+0000","last_unstale":"2024-09-11T12:56:19.595768+0000","last_undegraded":"2024-09-11T12:56:19.595768+0000","last_fullsized":"2024-09-11T12:56:19.595768+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:12:05.294584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157965+0000","last_change":"2024-09-11T12:56:18.168531+0000","last_active":"2024-09-11T12:56:19.157965+0000","last_peered":"2024-09-11T12:56:19.157965+0000","last_clean":"2024-09-11T12:56:19.157965+0000","last_became_active":"2024-09-11T12:56:18.164779+0000","last_became_peered":"2024-09-11T12:56:18.164779+0000","last_unstale":"2024-09-11T12:56:19.157965+0000","last_undegraded":"2024-09-11T12:56:19.157965+0000","last_fullsized":"2024-09-11T12:56:19.157965+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:16.193698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595206+0000","last_change":"2024-09-11T12:56:18.164474+0000","last_active":"2024-09-11T12:56:19.595206+0000","last_peered":"2024-09-11T12:56:19.595206+0000","last_clean":"2024-09-11T12:56:19.595206+0000","last_became_active":"2024-09-11T12:56:18.164227+0000","last_became_peered":"2024-09-11T12:56:18.164227+0000","last_unstale":"2024-09-11T12:56:19.595206+0000","last_undegraded":"2024-09-11T12:56:19.595206+0000","last_fullsized":"2024-09-11T12:56:19.595206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:06.935670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160150+0000","last_change":"2024-09-11T12:56:18.168807+0000","last_active":"2024-09-11T12:56:19.160150+0000","last_peered":"2024-09-11T12:56:19.160150+0000","last_clean":"2024-09-11T12:56:19.160150+0000","last_became_active":"2024-09-11T12:56:18.168550+0000","last_became_peered":"2024-09-11T12:56:18.168550+0000","last_unstale":"2024-09-11T12:56:19.160150+0000","last_undegraded":"2024-09-11T12:56:19.160150+0000","last_fullsized":"2024-09-11T12:56:19.160150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:05.868695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595528+0000","last_change":"2024-09-11T12:56:18.161363+0000","last_active":"2024-09-11T12:56:19.595528+0000","last_peered":"2024-09-11T12:56:19.595528+0000","last_clean":"2024-09-11T12:56:19.595528+0000","last_became_active":"2024-09-11T12:56:18.160507+0000","last_became_peered":"2024-09-11T12:56:18.160507+0000","last_unstale":"2024-09-11T12:56:19.595528+0000","last_undegraded":"2024-09-11T12:56:19.595528+0000","last_fullsized":"2024-09-11T12:56:19.595528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:48:30.459921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596369+0000","last_change":"2024-09-11T12:56:18.166544+0000","last_active":"2024-09-11T12:56:19.596369+0000","last_peered":"2024-09-11T12:56:19.596369+0000","last_clean":"2024-09-11T12:56:19.596369+0000","last_became_active":"2024-09-11T12:56:18.166205+0000","last_became_peered":"2024-09-11T12:56:18.166205+0000","last_unstale":"2024-09-11T12:56:19.596369+0000","last_undegraded":"2024-09-11T12:56:19.596369+0000","last_fullsized":"2024-09-11T12:56:19.596369+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:44.887180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594481+0000","last_change":"2024-09-11T12:56:18.163075+0000","last_active":"2024-09-11T12:56:19.594481+0000","last_peered":"2024-09-11T12:56:19.594481+0000","last_clean":"2024-09-11T12:56:19.594481+0000","last_became_active":"2024-09-11T12:56:18.162571+0000","last_became_peered":"2024-09-11T12:56:18.162571+0000","last_unstale":"2024-09-11T12:56:19.594481+0000","last_undegraded":"2024-09-11T12:56:19.594481+0000","last_fullsized":"2024-09-11T12:56:19.594481+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:59.976927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595484+0000","last_change":"2024-09-11T12:56:18.160752+0000","last_active":"2024-09-11T12:56:19.595484+0000","last_peered":"2024-09-11T12:56:19.595484+0000","last_clean":"2024-09-11T12:56:19.595484+0000","last_became_active":"2024-09-11T12:56:18.160442+0000","last_became_peered":"2024-09-11T12:56:18.160442+0000","last_unstale":"2024-09-11T12:56:19.595484+0000","last_undegraded":"2024-09-11T12:56:19.595484+0000","last_fullsized":"2024-09-11T12:56:19.595484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:54.583895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160890+0000","last_change":"2024-09-11T12:56:18.157288+0000","last_active":"2024-09-11T12:56:19.160890+0000","last_peered":"2024-09-11T12:56:19.160890+0000","last_clean":"2024-09-11T12:56:19.160890+0000","last_became_active":"2024-09-11T12:56:18.156999+0000","last_became_peered":"2024-09-11T12:56:18.156999+0000","last_unstale":"2024-09-11T12:56:19.160890+0000","last_undegraded":"2024-09-11T12:56:19.160890+0000","last_fullsized":"2024-09-11T12:56:19.160890+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:54.103076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595686+0000","last_change":"2024-09-11T12:56:18.175598+0000","last_active":"2024-09-11T12:56:19.595686+0000","last_peered":"2024-09-11T12:56:19.595686+0000","last_clean":"2024-09-11T12:56:19.595686+0000","last_became_active":"2024-09-11T12:56:18.175172+0000","last_became_peered":"2024-09-11T12:56:18.175172+0000","last_unstale":"2024-09-11T12:56:19.595686+0000","last_undegraded":"2024-09-11T12:56:19.595686+0000","last_fullsized":"2024-09-11T12:56:19.595686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:52.123801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160460+0000","last_change":"2024-09-11T12:56:18.151813+0000","last_active":"2024-09-11T12:56:19.160460+0000","last_peered":"2024-09-11T12:56:19.160460+0000","last_clean":"2024-09-11T12:56:19.160460+0000","last_became_active":"2024-09-11T12:56:18.150867+0000","last_became_peered":"2024-09-11T12:56:18.150867+0000","last_unstale":"2024-09-11T12:56:19.160460+0000","last_undegraded":"2024-09-11T12:56:19.160460+0000","last_fullsized":"2024-09-11T12:56:19.160460+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:36.449221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595009+0000","last_change":"2024-09-11T12:56:18.151526+0000","last_active":"2024-09-11T12:56:19.595009+0000","last_peered":"2024-09-11T12:56:19.595009+0000","last_clean":"2024-09-11T12:56:19.595009+0000","last_became_active":"2024-09-11T12:56:18.150668+0000","last_became_peered":"2024-09-11T12:56:18.150668+0000","last_unstale":"2024-09-11T12:56:19.595009+0000","last_undegraded":"2024-09-11T12:56:19.595009+0000","last_fullsized":"2024-09-11T12:56:19.595009+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:26.951757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160488+0000","last_change":"2024-09-11T12:56:18.150169+0000","last_active":"2024-09-11T12:56:19.160488+0000","last_peered":"2024-09-11T12:56:19.160488+0000","last_clean":"2024-09-11T12:56:19.160488+0000","last_became_active":"2024-09-11T12:56:18.149782+0000","last_became_peered":"2024-09-11T12:56:18.149782+0000","last_unstale":"2024-09-11T12:56:19.160488+0000","last_undegraded":"2024-09-11T12:56:19.160488+0000","last_fullsized":"2024-09-11T12:56:19.160488+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:44:48.408023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160931+0000","last_change":"2024-09-11T12:56:18.161857+0000","last_active":"2024-09-11T12:56:19.160931+0000","last_peered":"2024-09-11T12:56:19.160931+0000","last_clean":"2024-09-11T12:56:19.160931+0000","last_became_active":"2024-09-11T12:56:18.161576+0000","last_became_peered":"2024-09-11T12:56:18.161576+0000","last_unstale":"2024-09-11T12:56:19.160931+0000","last_undegraded":"2024-09-11T12:56:19.160931+0000","last_fullsized":"2024-09-11T12:56:19.160931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160736+0000","last_change":"2024-09-11T12:56:18.165489+0000","last_active":"2024-09-11T12:56:19.160736+0000","last_peered":"2024-09-11T12:56:19.160736+0000","last_clean":"2024-09-11T12:56:19.160736+0000","last_became_active":"2024-09-11T12:56:18.165067+0000","last_became_peered":"2024-09-11T12:56:18.165067+0000","last_unstale":"2024-09-11T12:56:19.160736+0000","last_undegraded":"2024-09-11T12:56:19.160736+0000","last_fullsized":"2024-09-11T12:56:19.160736+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:36.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157533+0000","last_change":"2024-09-11T12:56:18.163192+0000","last_active":"2024-09-11T12:56:19.157533+0000","last_peered":"2024-09-11T12:56:19.157533+0000","last_clean":"2024-09-11T12:56:19.157533+0000","last_became_active":"2024-09-11T12:56:18.162309+0000","last_became_peered":"2024-09-11T12:56:18.162309+0000","last_unstale":"2024-09-11T12:56:19.157533+0000","last_undegraded":"2024-09-11T12:56:19.157533+0000","last_fullsized":"2024-09-11T12:56:19.157533+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595726+0000","last_change":"2024-09-11T12:56:18.175389+0000","last_active":"2024-09-11T12:56:19.595726+0000","last_peered":"2024-09-11T12:56:19.595726+0000","last_clean":"2024-09-11T12:56:19.595726+0000","last_became_active":"2024-09-11T12:56:18.174789+0000","last_became_peered":"2024-09-11T12:56:18.174789+0000","last_unstale":"2024-09-11T12:56:19.595726+0000","last_undegraded":"2024-09-11T12:56:19.595726+0000","last_fullsized":"2024-09-11T12:56:19.595726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:10:13.350548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"139'4","reported_seq":19,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596051+0000","last_change":"2024-09-11T12:56:17.156091+0000","last_active":"2024-09-11T12:56:19.596051+0000","last_peered":"2024-09-11T12:56:19.596051+0000","last_clean":"2024-09-11T12:56:19.596051+0000","last_became_active":"2024-09-11T12:56:17.155801+0000","last_became_peered":"2024-09-11T12:56:17.155801+0000","last_unstale":"2024-09-11T12:56:19.596051+0000","last_undegraded":"2024-09-11T12:56:19.596051+0000","last_fullsized":"2024-09-11T12:56:19.596051+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:59:48.944107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160417+0000","last_change":"2024-09-11T12:56:18.163805+0000","last_active":"2024-09-11T12:56:19.160417+0000","last_peered":"2024-09-11T12:56:19.160417+0000","last_clean":"2024-09-11T12:56:19.160417+0000","last_became_active":"2024-09-11T12:56:18.163525+0000","last_became_peered":"2024-09-11T12:56:18.163525+0000","last_unstale":"2024-09-11T12:56:19.160417+0000","last_undegraded":"2024-09-11T12:56:19.160417+0000","last_fullsized":"2024-09-11T12:56:19.160417+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:23.148160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159914+0000","last_change":"2024-09-11T12:56:17.160978+0000","last_active":"2024-09-11T12:56:19.159914+0000","last_peered":"2024-09-11T12:56:19.159914+0000","last_clean":"2024-09-11T12:56:19.159914+0000","last_became_active":"2024-09-11T12:56:17.160669+0000","last_became_peered":"2024-09-11T12:56:17.160669+0000","last_unstale":"2024-09-11T12:56:19.159914+0000","last_undegraded":"2024-09-11T12:56:19.159914+0000","last_fullsized":"2024-09-11T12:56:19.159914+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595552+0000","last_change":"2024-09-11T12:56:18.170070+0000","last_active":"2024-09-11T12:56:19.595552+0000","last_peered":"2024-09-11T12:56:19.595552+0000","last_clean":"2024-09-11T12:56:19.595552+0000","last_became_active":"2024-09-11T12:56:18.169794+0000","last_became_peered":"2024-09-11T12:56:18.169794+0000","last_unstale":"2024-09-11T12:56:19.595552+0000","last_undegraded":"2024-09-11T12:56:19.595552+0000","last_fullsized":"2024-09-11T12:56:19.595552+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:24:51.485085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594917+0000","last_change":"2024-09-11T12:56:17.163157+0000","last_active":"2024-09-11T12:56:19.594917+0000","last_peered":"2024-09-11T12:56:19.594917+0000","last_clean":"2024-09-11T12:56:19.594917+0000","last_became_active":"2024-09-11T12:56:17.162872+0000","last_became_peered":"2024-09-11T12:56:17.162872+0000","last_unstale":"2024-09-11T12:56:19.594917+0000","last_undegraded":"2024-09-11T12:56:19.594917+0000","last_fullsized":"2024-09-11T12:56:19.594917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:33.412968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160658+0000","last_change":"2024-09-11T12:56:18.173594+0000","last_active":"2024-09-11T12:56:19.160658+0000","last_peered":"2024-09-11T12:56:19.160658+0000","last_clean":"2024-09-11T12:56:19.160658+0000","last_became_active":"2024-09-11T12:56:18.173288+0000","last_became_peered":"2024-09-11T12:56:18.173288+0000","last_unstale":"2024-09-11T12:56:19.160658+0000","last_undegraded":"2024-09-11T12:56:19.160658+0000","last_fullsized":"2024-09-11T12:56:19.160658+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.996637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595106+0000","last_change":"2024-09-11T12:56:17.158690+0000","last_active":"2024-09-11T12:56:19.595106+0000","last_peered":"2024-09-11T12:56:19.595106+0000","last_clean":"2024-09-11T12:56:19.595106+0000","last_became_active":"2024-09-11T12:56:17.158420+0000","last_became_peered":"2024-09-11T12:56:17.158420+0000","last_unstale":"2024-09-11T12:56:19.595106+0000","last_undegraded":"2024-09-11T12:56:19.595106+0000","last_fullsized":"2024-09-11T12:56:19.595106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:51:36.763247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594523+0000","last_change":"2024-09-11T12:56:18.168507+0000","last_active":"2024-09-11T12:56:19.594523+0000","last_peered":"2024-09-11T12:56:19.594523+0000","last_clean":"2024-09-11T12:56:19.594523+0000","last_became_active":"2024-09-11T12:56:18.168251+0000","last_became_peered":"2024-09-11T12:56:18.168251+0000","last_unstale":"2024-09-11T12:56:19.594523+0000","last_undegraded":"2024-09-11T12:56:19.594523+0000","last_fullsized":"2024-09-11T12:56:19.594523+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:39.866623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160638+0000","last_change":"2024-09-11T12:56:17.157989+0000","last_active":"2024-09-11T12:56:19.160638+0000","last_peered":"2024-09-11T12:56:19.160638+0000","last_clean":"2024-09-11T12:56:19.160638+0000","last_became_active":"2024-09-11T12:56:17.157690+0000","last_became_peered":"2024-09-11T12:56:17.157690+0000","last_unstale":"2024-09-11T12:56:19.160638+0000","last_undegraded":"2024-09-11T12:56:19.160638+0000","last_fullsized":"2024-09-11T12:56:19.160638+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:33.414909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594889+0000","last_change":"2024-09-11T12:56:18.170686+0000","last_active":"2024-09-11T12:56:19.594889+0000","last_peered":"2024-09-11T12:56:19.594889+0000","last_clean":"2024-09-11T12:56:19.594889+0000","last_became_active":"2024-09-11T12:56:18.170422+0000","last_became_peered":"2024-09-11T12:56:18.170422+0000","last_unstale":"2024-09-11T12:56:19.594889+0000","last_undegraded":"2024-09-11T12:56:19.594889+0000","last_fullsized":"2024-09-11T12:56:19.594889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593926+0000","last_change":"2024-09-11T12:56:17.167482+0000","last_active":"2024-09-11T12:56:19.593926+0000","last_peered":"2024-09-11T12:56:19.593926+0000","last_clean":"2024-09-11T12:56:19.593926+0000","last_became_active":"2024-09-11T12:56:17.167173+0000","last_became_peered":"2024-09-11T12:56:17.167173+0000","last_unstale":"2024-09-11T12:56:19.593926+0000","last_undegraded":"2024-09-11T12:56:19.593926+0000","last_fullsized":"2024-09-11T12:56:19.593926+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:15.618071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593896+0000","last_change":"2024-09-11T12:56:18.159865+0000","last_active":"2024-09-11T12:56:19.593896+0000","last_peered":"2024-09-11T12:56:19.593896+0000","last_clean":"2024-09-11T12:56:19.593896+0000","last_became_active":"2024-09-11T12:56:18.159659+0000","last_became_peered":"2024-09-11T12:56:18.159659+0000","last_unstale":"2024-09-11T12:56:19.593896+0000","last_undegraded":"2024-09-11T12:56:19.593896+0000","last_fullsized":"2024-09-11T12:56:19.593896+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595940+0000","last_change":"2024-09-11T12:56:17.163158+0000","last_active":"2024-09-11T12:56:19.595940+0000","last_peered":"2024-09-11T12:56:19.595940+0000","last_clean":"2024-09-11T12:56:19.595940+0000","last_became_active":"2024-09-11T12:56:17.162847+0000","last_became_peered":"2024-09-11T12:56:17.162847+0000","last_unstale":"2024-09-11T12:56:19.595940+0000","last_undegraded":"2024-09-11T12:56:19.595940+0000","last_fullsized":"2024-09-11T12:56:19.595940+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:40.658294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594647+0000","last_change":"2024-09-11T12:56:18.174176+0000","last_active":"2024-09-11T12:56:19.594647+0000","last_peered":"2024-09-11T12:56:19.594647+0000","last_clean":"2024-09-11T12:56:19.594647+0000","last_became_active":"2024-09-11T12:56:18.173540+0000","last_became_peered":"2024-09-11T12:56:18.173540+0000","last_unstale":"2024-09-11T12:56:19.594647+0000","last_undegraded":"2024-09-11T12:56:19.594647+0000","last_fullsized":"2024-09-11T12:56:19.594647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:05.388020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594030+0000","last_change":"2024-09-11T12:56:17.163727+0000","last_active":"2024-09-11T12:56:19.594030+0000","last_peered":"2024-09-11T12:56:19.594030+0000","last_clean":"2024-09-11T12:56:19.594030+0000","last_became_active":"2024-09-11T12:56:17.163494+0000","last_became_peered":"2024-09-11T12:56:17.163494+0000","last_unstale":"2024-09-11T12:56:19.594030+0000","last_undegraded":"2024-09-11T12:56:19.594030+0000","last_fullsized":"2024-09-11T12:56:19.594030+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:27:27.281118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158343+0000","last_change":"2024-09-11T12:56:18.170654+0000","last_active":"2024-09-11T12:56:19.158343+0000","last_peered":"2024-09-11T12:56:19.158343+0000","last_clean":"2024-09-11T12:56:19.158343+0000","last_became_active":"2024-09-11T12:56:18.169026+0000","last_became_peered":"2024-09-11T12:56:18.169026+0000","last_unstale":"2024-09-11T12:56:19.158343+0000","last_undegraded":"2024-09-11T12:56:19.158343+0000","last_fullsized":"2024-09-11T12:56:19.158343+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:05.309532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595392+0000","last_change":"2024-09-11T12:56:17.162318+0000","last_active":"2024-09-11T12:56:19.595392+0000","last_peered":"2024-09-11T12:56:19.595392+0000","last_clean":"2024-09-11T12:56:19.595392+0000","last_became_active":"2024-09-11T12:56:17.161960+0000","last_became_peered":"2024-09-11T12:56:17.161960+0000","last_unstale":"2024-09-11T12:56:19.595392+0000","last_undegraded":"2024-09-11T12:56:19.595392+0000","last_fullsized":"2024-09-11T12:56:19.595392+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:20:41.080476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157872+0000","last_change":"2024-09-11T12:56:18.161522+0000","last_active":"2024-09-11T12:56:19.157872+0000","last_peered":"2024-09-11T12:56:19.157872+0000","last_clean":"2024-09-11T12:56:19.157872+0000","last_became_active":"2024-09-11T12:56:18.161212+0000","last_became_peered":"2024-09-11T12:56:18.161212+0000","last_unstale":"2024-09-11T12:56:19.157872+0000","last_undegraded":"2024-09-11T12:56:19.157872+0000","last_fullsized":"2024-09-11T12:56:19.157872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:56.363389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"139'3","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595321+0000","last_change":"2024-09-11T12:56:17.156655+0000","last_active":"2024-09-11T12:56:19.595321+0000","last_peered":"2024-09-11T12:56:19.595321+0000","last_clean":"2024-09-11T12:56:19.595321+0000","last_became_active":"2024-09-11T12:56:17.156333+0000","last_became_peered":"2024-09-11T12:56:17.156333+0000","last_unstale":"2024-09-11T12:56:19.595321+0000","last_undegraded":"2024-09-11T12:56:19.595321+0000","last_fullsized":"2024-09-11T12:56:19.595321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:33:55.130625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160065+0000","last_change":"2024-09-11T12:56:18.155842+0000","last_active":"2024-09-11T12:56:19.160065+0000","last_peered":"2024-09-11T12:56:19.160065+0000","last_clean":"2024-09-11T12:56:19.160065+0000","last_became_active":"2024-09-11T12:56:18.154935+0000","last_became_peered":"2024-09-11T12:56:18.154935+0000","last_unstale":"2024-09-11T12:56:19.160065+0000","last_undegraded":"2024-09-11T12:56:19.160065+0000","last_fullsized":"2024-09-11T12:56:19.160065+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:47.937582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"139'6","reported_seq":21,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160009+0000","last_change":"2024-09-11T12:56:17.152914+0000","last_active":"2024-09-11T12:56:19.160009+0000","last_peered":"2024-09-11T12:56:19.160009+0000","last_clean":"2024-09-11T12:56:19.160009+0000","last_became_active":"2024-09-11T12:56:17.152624+0000","last_became_peered":"2024-09-11T12:56:17.152624+0000","last_unstale":"2024-09-11T12:56:19.160009+0000","last_undegraded":"2024-09-11T12:56:19.160009+0000","last_fullsized":"2024-09-11T12:56:19.160009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:15.618071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157623+0000","last_change":"2024-09-11T12:56:18.162721+0000","last_active":"2024-09-11T12:56:19.157623+0000","last_peered":"2024-09-11T12:56:19.157623+0000","last_clean":"2024-09-11T12:56:19.157623+0000","last_became_active":"2024-09-11T12:56:18.160923+0000","last_became_peered":"2024-09-11T12:56:18.160923+0000","last_unstale":"2024-09-11T12:56:19.157623+0000","last_undegraded":"2024-09-11T12:56:19.157623+0000","last_fullsized":"2024-09-11T12:56:19.157623+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595002+0000","last_change":"2024-09-11T12:56:17.159279+0000","last_active":"2024-09-11T12:56:19.595002+0000","last_peered":"2024-09-11T12:56:19.595002+0000","last_clean":"2024-09-11T12:56:19.595002+0000","last_became_active":"2024-09-11T12:56:17.159020+0000","last_became_peered":"2024-09-11T12:56:17.159020+0000","last_unstale":"2024-09-11T12:56:19.595002+0000","last_undegraded":"2024-09-11T12:56:19.595002+0000","last_fullsized":"2024-09-11T12:56:19.595002+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:36.233519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594609+0000","last_change":"2024-09-11T12:56:18.180134+0000","last_active":"2024-09-11T12:56:19.594609+0000","last_peered":"2024-09-11T12:56:19.594609+0000","last_clean":"2024-09-11T12:56:19.594609+0000","last_became_active":"2024-09-11T12:56:18.175229+0000","last_became_peered":"2024-09-11T12:56:18.175229+0000","last_unstale":"2024-09-11T12:56:19.594609+0000","last_undegraded":"2024-09-11T12:56:19.594609+0000","last_fullsized":"2024-09-11T12:56:19.594609+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:05.947746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157723+0000","last_change":"2024-09-11T12:56:17.159986+0000","last_active":"2024-09-11T12:56:19.157723+0000","last_peered":"2024-09-11T12:56:19.157723+0000","last_clean":"2024-09-11T12:56:19.157723+0000","last_became_active":"2024-09-11T12:56:17.159714+0000","last_became_peered":"2024-09-11T12:56:17.159714+0000","last_unstale":"2024-09-11T12:56:19.157723+0000","last_undegraded":"2024-09-11T12:56:19.157723+0000","last_fullsized":"2024-09-11T12:56:19.157723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594020+0000","last_change":"2024-09-11T12:56:18.152532+0000","last_active":"2024-09-11T12:56:19.594020+0000","last_peered":"2024-09-11T12:56:19.594020+0000","last_clean":"2024-09-11T12:56:19.594020+0000","last_became_active":"2024-09-11T12:56:18.152263+0000","last_became_peered":"2024-09-11T12:56:18.152263+0000","last_unstale":"2024-09-11T12:56:19.594020+0000","last_undegraded":"2024-09-11T12:56:19.594020+0000","last_fullsized":"2024-09-11T12:56:19.594020+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593923+0000","last_change":"2024-09-11T12:56:17.162210+0000","last_active":"2024-09-11T12:56:19.593923+0000","last_peered":"2024-09-11T12:56:19.593923+0000","last_clean":"2024-09-11T12:56:19.593923+0000","last_became_active":"2024-09-11T12:56:17.161330+0000","last_became_peered":"2024-09-11T12:56:17.161330+0000","last_unstale":"2024-09-11T12:56:19.593923+0000","last_undegraded":"2024-09-11T12:56:19.593923+0000","last_fullsized":"2024-09-11T12:56:19.593923+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159448+0000","last_change":"2024-09-11T12:56:18.171308+0000","last_active":"2024-09-11T12:56:19.159448+0000","last_peered":"2024-09-11T12:56:19.159448+0000","last_clean":"2024-09-11T12:56:19.159448+0000","last_became_active":"2024-09-11T12:56:18.164785+0000","last_became_peered":"2024-09-11T12:56:18.164785+0000","last_unstale":"2024-09-11T12:56:19.159448+0000","last_undegraded":"2024-09-11T12:56:19.159448+0000","last_fullsized":"2024-09-11T12:56:19.159448+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:45.694134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"139'8","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159313+0000","last_change":"2024-09-11T12:56:17.153678+0000","last_active":"2024-09-11T12:56:19.159313+0000","last_peered":"2024-09-11T12:56:19.159313+0000","last_clean":"2024-09-11T12:56:19.159313+0000","last_became_active":"2024-09-11T12:56:17.153151+0000","last_became_peered":"2024-09-11T12:56:17.153151+0000","last_unstale":"2024-09-11T12:56:19.159313+0000","last_undegraded":"2024-09-11T12:56:19.159313+0000","last_fullsized":"2024-09-11T12:56:19.159313+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:36.233519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595360+0000","last_change":"2024-09-11T12:56:18.163250+0000","last_active":"2024-09-11T12:56:19.595360+0000","last_peered":"2024-09-11T12:56:19.595360+0000","last_clean":"2024-09-11T12:56:19.595360+0000","last_became_active":"2024-09-11T12:56:18.162964+0000","last_became_peered":"2024-09-11T12:56:18.162964+0000","last_unstale":"2024-09-11T12:56:19.595360+0000","last_undegraded":"2024-09-11T12:56:19.595360+0000","last_fullsized":"2024-09-11T12:56:19.595360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:16.193698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.161017+0000","last_change":"2024-09-11T12:56:17.162723+0000","last_active":"2024-09-11T12:56:19.161017+0000","last_peered":"2024-09-11T12:56:19.161017+0000","last_clean":"2024-09-11T12:56:19.161017+0000","last_became_active":"2024-09-11T12:56:17.162458+0000","last_became_peered":"2024-09-11T12:56:17.162458+0000","last_unstale":"2024-09-11T12:56:19.161017+0000","last_undegraded":"2024-09-11T12:56:19.161017+0000","last_fullsized":"2024-09-11T12:56:19.161017+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:59.894483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158432+0000","last_change":"2024-09-11T12:56:18.171174+0000","last_active":"2024-09-11T12:56:19.158432+0000","last_peered":"2024-09-11T12:56:19.158432+0000","last_clean":"2024-09-11T12:56:19.158432+0000","last_became_active":"2024-09-11T12:56:18.170881+0000","last_became_peered":"2024-09-11T12:56:18.170881+0000","last_unstale":"2024-09-11T12:56:19.158432+0000","last_undegraded":"2024-09-11T12:56:19.158432+0000","last_fullsized":"2024-09-11T12:56:19.158432+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:47:32.385005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160771+0000","last_change":"2024-09-11T12:56:17.159839+0000","last_active":"2024-09-11T12:56:19.160771+0000","last_peered":"2024-09-11T12:56:19.160771+0000","last_clean":"2024-09-11T12:56:19.160771+0000","last_became_active":"2024-09-11T12:56:17.159568+0000","last_became_peered":"2024-09-11T12:56:17.159568+0000","last_unstale":"2024-09-11T12:56:19.160771+0000","last_undegraded":"2024-09-11T12:56:19.160771+0000","last_fullsized":"2024-09-11T12:56:19.160771+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:11.702594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595106+0000","last_change":"2024-09-11T12:56:18.166966+0000","last_active":"2024-09-11T12:56:19.595106+0000","last_peered":"2024-09-11T12:56:19.595106+0000","last_clean":"2024-09-11T12:56:19.595106+0000","last_became_active":"2024-09-11T12:56:18.166677+0000","last_became_peered":"2024-09-11T12:56:18.166677+0000","last_unstale":"2024-09-11T12:56:19.595106+0000","last_undegraded":"2024-09-11T12:56:19.595106+0000","last_fullsized":"2024-09-11T12:56:19.595106+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:56.363389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596046+0000","last_change":"2024-09-11T12:56:17.158043+0000","last_active":"2024-09-11T12:56:19.596046+0000","last_peered":"2024-09-11T12:56:19.596046+0000","last_clean":"2024-09-11T12:56:19.596046+0000","last_became_active":"2024-09-11T12:56:17.157312+0000","last_became_peered":"2024-09-11T12:56:17.157312+0000","last_unstale":"2024-09-11T12:56:19.596046+0000","last_undegraded":"2024-09-11T12:56:19.596046+0000","last_fullsized":"2024-09-11T12:56:19.596046+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:46.601805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595384+0000","last_change":"2024-09-11T12:56:18.179626+0000","last_active":"2024-09-11T12:56:19.595384+0000","last_peered":"2024-09-11T12:56:19.595384+0000","last_clean":"2024-09-11T12:56:19.595384+0000","last_became_active":"2024-09-11T12:56:18.179343+0000","last_became_peered":"2024-09-11T12:56:18.179343+0000","last_unstale":"2024-09-11T12:56:19.595384+0000","last_undegraded":"2024-09-11T12:56:19.595384+0000","last_fullsized":"2024-09-11T12:56:19.595384+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:29:29.519414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595846+0000","last_change":"2024-09-11T12:56:17.165337+0000","last_active":"2024-09-11T12:56:19.595846+0000","last_peered":"2024-09-11T12:56:19.595846+0000","last_clean":"2024-09-11T12:56:19.595846+0000","last_became_active":"2024-09-11T12:56:17.165011+0000","last_became_peered":"2024-09-11T12:56:17.165011+0000","last_unstale":"2024-09-11T12:56:19.595846+0000","last_undegraded":"2024-09-11T12:56:19.595846+0000","last_fullsized":"2024-09-11T12:56:19.595846+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:47:50.519843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159831+0000","last_change":"2024-09-11T12:56:18.170321+0000","last_active":"2024-09-11T12:56:19.159831+0000","last_peered":"2024-09-11T12:56:19.159831+0000","last_clean":"2024-09-11T12:56:19.159831+0000","last_became_active":"2024-09-11T12:56:18.163860+0000","last_became_peered":"2024-09-11T12:56:18.163860+0000","last_unstale":"2024-09-11T12:56:19.159831+0000","last_undegraded":"2024-09-11T12:56:19.159831+0000","last_fullsized":"2024-09-11T12:56:19.159831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:01.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595105+0000","last_change":"2024-09-11T12:56:17.161102+0000","last_active":"2024-09-11T12:56:19.595105+0000","last_peered":"2024-09-11T12:56:19.595105+0000","last_clean":"2024-09-11T12:56:19.595105+0000","last_became_active":"2024-09-11T12:56:17.160479+0000","last_became_peered":"2024-09-11T12:56:17.160479+0000","last_unstale":"2024-09-11T12:56:19.595105+0000","last_undegraded":"2024-09-11T12:56:19.595105+0000","last_fullsized":"2024-09-11T12:56:19.595105+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:34:20.045407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594766+0000","last_change":"2024-09-11T12:56:18.169504+0000","last_active":"2024-09-11T12:56:19.594766+0000","last_peered":"2024-09-11T12:56:19.594766+0000","last_clean":"2024-09-11T12:56:19.594766+0000","last_became_active":"2024-09-11T12:56:18.169262+0000","last_became_peered":"2024-09-11T12:56:18.169262+0000","last_unstale":"2024-09-11T12:56:19.594766+0000","last_undegraded":"2024-09-11T12:56:19.594766+0000","last_fullsized":"2024-09-11T12:56:19.594766+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:46:27.819654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595692+0000","last_change":"2024-09-11T12:56:17.163874+0000","last_active":"2024-09-11T12:56:19.595692+0000","last_peered":"2024-09-11T12:56:19.595692+0000","last_clean":"2024-09-11T12:56:19.595692+0000","last_became_active":"2024-09-11T12:56:17.161901+0000","last_became_peered":"2024-09-11T12:56:17.161901+0000","last_unstale":"2024-09-11T12:56:19.595692+0000","last_undegraded":"2024-09-11T12:56:19.595692+0000","last_fullsized":"2024-09-11T12:56:19.595692+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:41:28.837956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595580+0000","last_change":"2024-09-11T12:56:18.176205+0000","last_active":"2024-09-11T12:56:19.595580+0000","last_peered":"2024-09-11T12:56:19.595580+0000","last_clean":"2024-09-11T12:56:19.595580+0000","last_became_active":"2024-09-11T12:56:18.174067+0000","last_became_peered":"2024-09-11T12:56:18.174067+0000","last_unstale":"2024-09-11T12:56:19.595580+0000","last_undegraded":"2024-09-11T12:56:19.595580+0000","last_fullsized":"2024-09-11T12:56:19.595580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:19.213356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594017+0000","last_change":"2024-09-11T12:56:17.159432+0000","last_active":"2024-09-11T12:56:19.594017+0000","last_peered":"2024-09-11T12:56:19.594017+0000","last_clean":"2024-09-11T12:56:19.594017+0000","last_became_active":"2024-09-11T12:56:17.159150+0000","last_became_peered":"2024-09-11T12:56:17.159150+0000","last_unstale":"2024-09-11T12:56:19.594017+0000","last_undegraded":"2024-09-11T12:56:19.594017+0000","last_fullsized":"2024-09-11T12:56:19.594017+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:30.773880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594812+0000","last_change":"2024-09-11T12:56:18.162832+0000","last_active":"2024-09-11T12:56:19.594812+0000","last_peered":"2024-09-11T12:56:19.594812+0000","last_clean":"2024-09-11T12:56:19.594812+0000","last_became_active":"2024-09-11T12:56:18.162223+0000","last_became_peered":"2024-09-11T12:56:18.162223+0000","last_unstale":"2024-09-11T12:56:19.594812+0000","last_undegraded":"2024-09-11T12:56:19.594812+0000","last_fullsized":"2024-09-11T12:56:19.594812+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:01.529543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595787+0000","last_change":"2024-09-11T12:56:17.164420+0000","last_active":"2024-09-11T12:56:19.595787+0000","last_peered":"2024-09-11T12:56:19.595787+0000","last_clean":"2024-09-11T12:56:19.595787+0000","last_became_active":"2024-09-11T12:56:17.162913+0000","last_became_peered":"2024-09-11T12:56:17.162913+0000","last_unstale":"2024-09-11T12:56:19.595787+0000","last_undegraded":"2024-09-11T12:56:19.595787+0000","last_fullsized":"2024-09-11T12:56:19.595787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:22.083507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158213+0000","last_change":"2024-09-11T12:56:18.169449+0000","last_active":"2024-09-11T12:56:19.158213+0000","last_peered":"2024-09-11T12:56:19.158213+0000","last_clean":"2024-09-11T12:56:19.158213+0000","last_became_active":"2024-09-11T12:56:18.169199+0000","last_became_peered":"2024-09-11T12:56:18.169199+0000","last_unstale":"2024-09-11T12:56:19.158213+0000","last_undegraded":"2024-09-11T12:56:19.158213+0000","last_fullsized":"2024-09-11T12:56:19.158213+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:33.336616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158441+0000","last_change":"2024-09-11T12:56:17.160318+0000","last_active":"2024-09-11T12:56:19.158441+0000","last_peered":"2024-09-11T12:56:19.158441+0000","last_clean":"2024-09-11T12:56:19.158441+0000","last_became_active":"2024-09-11T12:56:17.160088+0000","last_became_peered":"2024-09-11T12:56:17.160088+0000","last_unstale":"2024-09-11T12:56:19.158441+0000","last_undegraded":"2024-09-11T12:56:19.158441+0000","last_fullsized":"2024-09-11T12:56:19.158441+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:30.773880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595389+0000","last_change":"2024-09-11T12:56:18.165839+0000","last_active":"2024-09-11T12:56:19.595389+0000","last_peered":"2024-09-11T12:56:19.595389+0000","last_clean":"2024-09-11T12:56:19.595389+0000","last_became_active":"2024-09-11T12:56:18.165525+0000","last_became_peered":"2024-09-11T12:56:18.165525+0000","last_unstale":"2024-09-11T12:56:19.595389+0000","last_undegraded":"2024-09-11T12:56:19.595389+0000","last_fullsized":"2024-09-11T12:56:19.595389+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:45:54.412049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"140'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:20.148922+0000","last_change":"2024-09-11T12:56:17.151805+0000","last_active":"2024-09-11T12:56:20.148922+0000","last_peered":"2024-09-11T12:56:20.148922+0000","last_clean":"2024-09-11T12:56:20.148922+0000","last_became_active":"2024-09-11T12:56:17.151520+0000","last_became_peered":"2024-09-11T12:56:17.151520+0000","last_unstale":"2024-09-11T12:56:20.148922+0000","last_undegraded":"2024-09-11T12:56:20.148922+0000","last_fullsized":"2024-09-11T12:56:20.148922+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:04.826106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595633+0000","last_change":"2024-09-11T12:56:18.175754+0000","last_active":"2024-09-11T12:56:19.595633+0000","last_peered":"2024-09-11T12:56:19.595633+0000","last_clean":"2024-09-11T12:56:19.595633+0000","last_became_active":"2024-09-11T12:56:18.175288+0000","last_became_peered":"2024-09-11T12:56:18.175288+0000","last_unstale":"2024-09-11T12:56:19.595633+0000","last_undegraded":"2024-09-11T12:56:19.595633+0000","last_fullsized":"2024-09-11T12:56:19.595633+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:58.892257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596645+0000","last_change":"2024-09-11T12:56:17.167395+0000","last_active":"2024-09-11T12:56:19.596645+0000","last_peered":"2024-09-11T12:56:19.596645+0000","last_clean":"2024-09-11T12:56:19.596645+0000","last_became_active":"2024-09-11T12:56:17.167027+0000","last_became_peered":"2024-09-11T12:56:17.167027+0000","last_unstale":"2024-09-11T12:56:19.596645+0000","last_undegraded":"2024-09-11T12:56:19.596645+0000","last_fullsized":"2024-09-11T12:56:19.596645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:49.645834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157808+0000","last_change":"2024-09-11T12:56:18.176147+0000","last_active":"2024-09-11T12:56:19.157808+0000","last_peered":"2024-09-11T12:56:19.157808+0000","last_clean":"2024-09-11T12:56:19.157808+0000","last_became_active":"2024-09-11T12:56:18.175854+0000","last_became_peered":"2024-09-11T12:56:18.175854+0000","last_unstale":"2024-09-11T12:56:19.157808+0000","last_undegraded":"2024-09-11T12:56:19.157808+0000","last_fullsized":"2024-09-11T12:56:19.157808+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:26.951757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157972+0000","last_change":"2024-09-11T12:56:17.153175+0000","last_active":"2024-09-11T12:56:19.157972+0000","last_peered":"2024-09-11T12:56:19.157972+0000","last_clean":"2024-09-11T12:56:19.157972+0000","last_became_active":"2024-09-11T12:56:17.152884+0000","last_became_peered":"2024-09-11T12:56:17.152884+0000","last_unstale":"2024-09-11T12:56:19.157972+0000","last_undegraded":"2024-09-11T12:56:19.157972+0000","last_fullsized":"2024-09-11T12:56:19.157972+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157649+0000","last_change":"2024-09-11T12:56:18.160943+0000","last_active":"2024-09-11T12:56:19.157649+0000","last_peered":"2024-09-11T12:56:19.157649+0000","last_clean":"2024-09-11T12:56:19.157649+0000","last_became_active":"2024-09-11T12:56:18.160667+0000","last_became_peered":"2024-09-11T12:56:18.160667+0000","last_unstale":"2024-09-11T12:56:19.157649+0000","last_undegraded":"2024-09-11T12:56:19.157649+0000","last_fullsized":"2024-09-11T12:56:19.157649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:23.489680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160000+0000","last_change":"2024-09-11T12:56:17.155507+0000","last_active":"2024-09-11T12:56:19.160000+0000","last_peered":"2024-09-11T12:56:19.160000+0000","last_clean":"2024-09-11T12:56:19.160000+0000","last_became_active":"2024-09-11T12:56:17.155178+0000","last_became_peered":"2024-09-11T12:56:17.155178+0000","last_unstale":"2024-09-11T12:56:19.160000+0000","last_undegraded":"2024-09-11T12:56:19.160000+0000","last_fullsized":"2024-09-11T12:56:19.160000+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:33.412968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159874+0000","last_change":"2024-09-11T12:56:18.169243+0000","last_active":"2024-09-11T12:56:19.159874+0000","last_peered":"2024-09-11T12:56:19.159874+0000","last_clean":"2024-09-11T12:56:19.159874+0000","last_became_active":"2024-09-11T12:56:18.168597+0000","last_became_peered":"2024-09-11T12:56:18.168597+0000","last_unstale":"2024-09-11T12:56:19.159874+0000","last_undegraded":"2024-09-11T12:56:19.159874+0000","last_fullsized":"2024-09-11T12:56:19.159874+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:44.887180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594232+0000","last_change":"2024-09-11T12:56:17.154900+0000","last_active":"2024-09-11T12:56:19.594232+0000","last_peered":"2024-09-11T12:56:19.594232+0000","last_clean":"2024-09-11T12:56:19.594232+0000","last_became_active":"2024-09-11T12:56:17.154587+0000","last_became_peered":"2024-09-11T12:56:17.154587+0000","last_unstale":"2024-09-11T12:56:19.594232+0000","last_undegraded":"2024-09-11T12:56:19.594232+0000","last_fullsized":"2024-09-11T12:56:19.594232+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593869+0000","last_change":"2024-09-11T12:56:18.170263+0000","last_active":"2024-09-11T12:56:19.593869+0000","last_peered":"2024-09-11T12:56:19.593869+0000","last_clean":"2024-09-11T12:56:19.593869+0000","last_became_active":"2024-09-11T12:56:18.169656+0000","last_became_peered":"2024-09-11T12:56:18.169656+0000","last_unstale":"2024-09-11T12:56:19.593869+0000","last_undegraded":"2024-09-11T12:56:19.593869+0000","last_fullsized":"2024-09-11T12:56:19.593869+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:57.503942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596082+0000","last_change":"2024-09-11T12:56:17.164719+0000","last_active":"2024-09-11T12:56:19.596082+0000","last_peered":"2024-09-11T12:56:19.596082+0000","last_clean":"2024-09-11T12:56:19.596082+0000","last_became_active":"2024-09-11T12:56:17.164074+0000","last_became_peered":"2024-09-11T12:56:17.164074+0000","last_unstale":"2024-09-11T12:56:19.596082+0000","last_undegraded":"2024-09-11T12:56:19.596082+0000","last_fullsized":"2024-09-11T12:56:19.596082+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:31.346670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159521+0000","last_change":"2024-09-11T12:56:18.164375+0000","last_active":"2024-09-11T12:56:19.159521+0000","last_peered":"2024-09-11T12:56:19.159521+0000","last_clean":"2024-09-11T12:56:19.159521+0000","last_became_active":"2024-09-11T12:56:18.164043+0000","last_became_peered":"2024-09-11T12:56:18.164043+0000","last_unstale":"2024-09-11T12:56:19.159521+0000","last_undegraded":"2024-09-11T12:56:19.159521+0000","last_fullsized":"2024-09-11T12:56:19.159521+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:08.589118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159433+0000","last_change":"2024-09-11T12:56:17.166196+0000","last_active":"2024-09-11T12:56:19.159433+0000","last_peered":"2024-09-11T12:56:19.159433+0000","last_clean":"2024-09-11T12:56:19.159433+0000","last_became_active":"2024-09-11T12:56:17.165828+0000","last_became_peered":"2024-09-11T12:56:17.165828+0000","last_unstale":"2024-09-11T12:56:19.159433+0000","last_undegraded":"2024-09-11T12:56:19.159433+0000","last_fullsized":"2024-09-11T12:56:19.159433+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:34:20.045407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158095+0000","last_change":"2024-09-11T12:56:18.177890+0000","last_active":"2024-09-11T12:56:19.158095+0000","last_peered":"2024-09-11T12:56:19.158095+0000","last_clean":"2024-09-11T12:56:19.158095+0000","last_became_active":"2024-09-11T12:56:18.177529+0000","last_became_peered":"2024-09-11T12:56:18.177529+0000","last_unstale":"2024-09-11T12:56:19.158095+0000","last_undegraded":"2024-09-11T12:56:19.158095+0000","last_fullsized":"2024-09-11T12:56:19.158095+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:33.818525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160519+0000","last_change":"2024-09-11T12:56:17.156745+0000","last_active":"2024-09-11T12:56:19.160519+0000","last_peered":"2024-09-11T12:56:19.160519+0000","last_clean":"2024-09-11T12:56:19.160519+0000","last_became_active":"2024-09-11T12:56:17.156429+0000","last_became_peered":"2024-09-11T12:56:17.156429+0000","last_unstale":"2024-09-11T12:56:19.160519+0000","last_undegraded":"2024-09-11T12:56:19.160519+0000","last_fullsized":"2024-09-11T12:56:19.160519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":352,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595820+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:19.595820+0000","last_peered":"2024-09-11T12:56:19.595820+0000","last_clean":"2024-09-11T12:56:19.595820+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:19.595820+0000","last_undegraded":"2024-09-11T12:56:19.595820+0000","last_fullsized":"2024-09-11T12:56:19.595820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37652,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334188,"statfs":{"total":96636764160,"available":96598208512,"internally_reserved":0,"allocated":2359296,"data_stored":1356791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":770391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1761280,"data_stored":770973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1769472,"data_stored":772051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1757184,"data_stored":770403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":770459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":770403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2334720,"data_stored":1354437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:22.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:22.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- 172.21.15.78:0/1892732511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e28003fe0 msgr2=0x7f2e28004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 --2- 172.21.15.78:0/1892732511 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e28003fe0 0x7f2e28004810 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b90 tx=0x7f2e3001d2b0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e43400640 1 -- 172.21.15.78:0/1892732511 reap_dead start 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- 172.21.15.78:0/1892732511 shutdown_connections 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- 172.21.15.78:0/1892732511 >> 172.21.15.78:0/1892732511 conn(0x7f2e4406ca80 msgr2=0x7f2e440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- 172.21.15.78:0/1892732511 shutdown_connections 2024-09-11T12:56:22.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- 172.21.15.78:0/1892732511 wait complete. 2024-09-11T12:56:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 Processor -- start 2024-09-11T12:56:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- start start 2024-09-11T12:56:22.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 0x7f2e440c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4413b7b0 0x7f2e4413bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441403d0 con 0x7f2e440ad870 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4413dd60 con 0x7f2e28003fe0 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.430+0000 7f2e49600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2e440ce8a0 con 0x7f2e4413b7b0 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43894/0 (socket says 172.21.15.78:43894) 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 -- 172.21.15.78:0/1007361908 learned_addr learned my addr 172.21.15.78:0/1007361908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:22.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42a00640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 0x7f2e440c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e43400640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4413b7b0 0x7f2e4413bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 -- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4413b7b0 msgr2=0x7f2e4413bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4413b7b0 0x7f2e4413bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 -- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 msgr2=0x7f2e440c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 0x7f2e440c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 -- 172.21.15.78:0/1007361908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4413c1a0 con 0x7f2e440ad870 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42a00640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 0x7f2e440c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e42000640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b90 tx=0x7f2e30005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e300b2070 con 0x7f2e440ad870 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e300b3040 con 0x7f2e440ad870 2024-09-11T12:56:22.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e440ae310 con 0x7f2e440ad870 2024-09-11T12:56:22.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e30009730 con 0x7f2e440ad870 2024-09-11T12:56:22.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e440ae8d0 con 0x7f2e440ad870 2024-09-11T12:56:22.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e440700c0 con 0x7f2e440ad870 2024-09-11T12:56:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.434+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f2e30004290 con 0x7f2e440ad870 2024-09-11T12:56:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.438+0000 7f2e3aa00640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 0x7f2e24074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.438+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f2e3012d4d0 con 0x7f2e440ad870 2024-09-11T12:56:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.438+0000 7f2e42a00640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 0x7f2e24074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.438+0000 7f2e42a00640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 0x7f2e24074170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000fe0 tx=0x7f2e340067b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.442+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2e30017000 con 0x7f2e440ad870 2024-09-11T12:56:22.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.602+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e4413ce60 con 0x7f2e24071cb0 2024-09-11T12:56:22.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.606+0000 7f2e3aa00640 1 -- 172.21.15.78:0/1007361908 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252479 (secure 0 0 0) 0x7f2e4413ce60 con 0x7f2e24071cb0 2024-09-11T12:56:22.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:22.615 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 msgr2=0x7f2e24074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 0x7f2e24074170 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000fe0 tx=0x7f2e340067b0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 msgr2=0x7f2e440adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009b90 tx=0x7f2e30005590 comp rx=0 tx=0).stop 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 shutdown_connections 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2e4413b7b0 0x7f2e4413bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2e440ad870 0x7f2e440adcf0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2e24071cb0 0x7f2e24074170 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 --2- 172.21.15.78:0/1007361908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2e28003fe0 0x7f2e440c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 >> 172.21.15.78:0/1007361908 conn(0x7f2e4406ca80 msgr2=0x7f2e440cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 shutdown_connections 2024-09-11T12:56:22.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.610+0000 7f2e49600640 1 -- 172.21.15.78:0/1007361908 wait complete. 2024-09-11T12:56:22.634 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-11T12:56:22.502699+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":276924,"kb_used_data":14896,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754697796,"statfs":{"total":773094113280,"available":772810543104,"internally_reserved":0,"allocated":15253504,"data_stored":7335908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001065"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595977+0000","last_change":"2024-09-11T12:56:18.173943+0000","last_active":"2024-09-11T12:56:19.595977+0000","last_peered":"2024-09-11T12:56:19.595977+0000","last_clean":"2024-09-11T12:56:19.595977+0000","last_became_active":"2024-09-11T12:56:18.173652+0000","last_became_peered":"2024-09-11T12:56:18.173652+0000","last_unstale":"2024-09-11T12:56:19.595977+0000","last_undegraded":"2024-09-11T12:56:19.595977+0000","last_fullsized":"2024-09-11T12:56:19.595977+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:18.772831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596430+0000","last_change":"2024-09-11T12:56:18.164866+0000","last_active":"2024-09-11T12:56:19.596430+0000","last_peered":"2024-09-11T12:56:19.596430+0000","last_clean":"2024-09-11T12:56:19.596430+0000","last_became_active":"2024-09-11T12:56:18.164605+0000","last_became_peered":"2024-09-11T12:56:18.164605+0000","last_unstale":"2024-09-11T12:56:19.596430+0000","last_undegraded":"2024-09-11T12:56:19.596430+0000","last_fullsized":"2024-09-11T12:56:19.596430+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:23.148160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160240+0000","last_change":"2024-09-11T12:56:18.159219+0000","last_active":"2024-09-11T12:56:19.160240+0000","last_peered":"2024-09-11T12:56:19.160240+0000","last_clean":"2024-09-11T12:56:19.160240+0000","last_became_active":"2024-09-11T12:56:18.158933+0000","last_became_peered":"2024-09-11T12:56:18.158933+0000","last_unstale":"2024-09-11T12:56:19.160240+0000","last_undegraded":"2024-09-11T12:56:19.160240+0000","last_fullsized":"2024-09-11T12:56:19.160240+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:08:52.766319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594935+0000","last_change":"2024-09-11T12:56:18.163856+0000","last_active":"2024-09-11T12:56:19.594935+0000","last_peered":"2024-09-11T12:56:19.594935+0000","last_clean":"2024-09-11T12:56:19.594935+0000","last_became_active":"2024-09-11T12:56:18.163556+0000","last_became_peered":"2024-09-11T12:56:18.163556+0000","last_unstale":"2024-09-11T12:56:19.594935+0000","last_undegraded":"2024-09-11T12:56:19.594935+0000","last_fullsized":"2024-09-11T12:56:19.594935+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:23.489680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596493+0000","last_change":"2024-09-11T12:56:18.172839+0000","last_active":"2024-09-11T12:56:19.596493+0000","last_peered":"2024-09-11T12:56:19.596493+0000","last_clean":"2024-09-11T12:56:19.596493+0000","last_became_active":"2024-09-11T12:56:18.172579+0000","last_became_peered":"2024-09-11T12:56:18.172579+0000","last_unstale":"2024-09-11T12:56:19.596493+0000","last_undegraded":"2024-09-11T12:56:19.596493+0000","last_fullsized":"2024-09-11T12:56:19.596493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595269+0000","last_change":"2024-09-11T12:56:18.173161+0000","last_active":"2024-09-11T12:56:19.595269+0000","last_peered":"2024-09-11T12:56:19.595269+0000","last_clean":"2024-09-11T12:56:19.595269+0000","last_became_active":"2024-09-11T12:56:18.172846+0000","last_became_peered":"2024-09-11T12:56:18.172846+0000","last_unstale":"2024-09-11T12:56:19.595269+0000","last_undegraded":"2024-09-11T12:56:19.595269+0000","last_fullsized":"2024-09-11T12:56:19.595269+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:23.412771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594607+0000","last_change":"2024-09-11T12:56:18.179661+0000","last_active":"2024-09-11T12:56:19.594607+0000","last_peered":"2024-09-11T12:56:19.594607+0000","last_clean":"2024-09-11T12:56:19.594607+0000","last_became_active":"2024-09-11T12:56:18.179403+0000","last_became_peered":"2024-09-11T12:56:18.179403+0000","last_unstale":"2024-09-11T12:56:19.594607+0000","last_undegraded":"2024-09-11T12:56:19.594607+0000","last_fullsized":"2024-09-11T12:56:19.594607+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:42:46.556219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595447+0000","last_change":"2024-09-11T12:56:18.166706+0000","last_active":"2024-09-11T12:56:19.595447+0000","last_peered":"2024-09-11T12:56:19.595447+0000","last_clean":"2024-09-11T12:56:19.595447+0000","last_became_active":"2024-09-11T12:56:18.166394+0000","last_became_peered":"2024-09-11T12:56:18.166394+0000","last_unstale":"2024-09-11T12:56:19.595447+0000","last_undegraded":"2024-09-11T12:56:19.595447+0000","last_fullsized":"2024-09-11T12:56:19.595447+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:36.186648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596698+0000","last_change":"2024-09-11T12:56:18.162390+0000","last_active":"2024-09-11T12:56:19.596698+0000","last_peered":"2024-09-11T12:56:19.596698+0000","last_clean":"2024-09-11T12:56:19.596698+0000","last_became_active":"2024-09-11T12:56:18.162072+0000","last_became_peered":"2024-09-11T12:56:18.162072+0000","last_unstale":"2024-09-11T12:56:19.596698+0000","last_undegraded":"2024-09-11T12:56:19.596698+0000","last_fullsized":"2024-09-11T12:56:19.596698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596601+0000","last_change":"2024-09-11T12:56:18.175354+0000","last_active":"2024-09-11T12:56:19.596601+0000","last_peered":"2024-09-11T12:56:19.596601+0000","last_clean":"2024-09-11T12:56:19.596601+0000","last_became_active":"2024-09-11T12:56:18.175161+0000","last_became_peered":"2024-09-11T12:56:18.175161+0000","last_unstale":"2024-09-11T12:56:19.596601+0000","last_undegraded":"2024-09-11T12:56:19.596601+0000","last_fullsized":"2024-09-11T12:56:19.596601+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596468+0000","last_change":"2024-09-11T12:56:18.173609+0000","last_active":"2024-09-11T12:56:19.596468+0000","last_peered":"2024-09-11T12:56:19.596468+0000","last_clean":"2024-09-11T12:56:19.596468+0000","last_became_active":"2024-09-11T12:56:18.173274+0000","last_became_peered":"2024-09-11T12:56:18.173274+0000","last_unstale":"2024-09-11T12:56:19.596468+0000","last_undegraded":"2024-09-11T12:56:19.596468+0000","last_fullsized":"2024-09-11T12:56:19.596468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:36.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596373+0000","last_change":"2024-09-11T12:56:18.168941+0000","last_active":"2024-09-11T12:56:19.596373+0000","last_peered":"2024-09-11T12:56:19.596373+0000","last_clean":"2024-09-11T12:56:19.596373+0000","last_became_active":"2024-09-11T12:56:18.168672+0000","last_became_peered":"2024-09-11T12:56:18.168672+0000","last_unstale":"2024-09-11T12:56:19.596373+0000","last_undegraded":"2024-09-11T12:56:19.596373+0000","last_fullsized":"2024-09-11T12:56:19.596373+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:47.937582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160530+0000","last_change":"2024-09-11T12:56:18.179560+0000","last_active":"2024-09-11T12:56:19.160530+0000","last_peered":"2024-09-11T12:56:19.160530+0000","last_clean":"2024-09-11T12:56:19.160530+0000","last_became_active":"2024-09-11T12:56:18.179206+0000","last_became_peered":"2024-09-11T12:56:18.179206+0000","last_unstale":"2024-09-11T12:56:19.160530+0000","last_undegraded":"2024-09-11T12:56:19.160530+0000","last_fullsized":"2024-09-11T12:56:19.160530+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:09.833887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159646+0000","last_change":"2024-09-11T12:56:18.160292+0000","last_active":"2024-09-11T12:56:19.159646+0000","last_peered":"2024-09-11T12:56:19.159646+0000","last_clean":"2024-09-11T12:56:19.159646+0000","last_became_active":"2024-09-11T12:56:18.160043+0000","last_became_peered":"2024-09-11T12:56:18.160043+0000","last_unstale":"2024-09-11T12:56:19.159646+0000","last_undegraded":"2024-09-11T12:56:19.159646+0000","last_fullsized":"2024-09-11T12:56:19.159646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:55:17.045517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595721+0000","last_change":"2024-09-11T12:56:18.177991+0000","last_active":"2024-09-11T12:56:19.595721+0000","last_peered":"2024-09-11T12:56:19.595721+0000","last_clean":"2024-09-11T12:56:19.595721+0000","last_became_active":"2024-09-11T12:56:18.177360+0000","last_became_peered":"2024-09-11T12:56:18.177360+0000","last_unstale":"2024-09-11T12:56:19.595721+0000","last_undegraded":"2024-09-11T12:56:19.595721+0000","last_fullsized":"2024-09-11T12:56:19.595721+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:04.083800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593870+0000","last_change":"2024-09-11T12:56:18.170484+0000","last_active":"2024-09-11T12:56:19.593870+0000","last_peered":"2024-09-11T12:56:19.593870+0000","last_clean":"2024-09-11T12:56:19.593870+0000","last_became_active":"2024-09-11T12:56:18.170026+0000","last_became_peered":"2024-09-11T12:56:18.170026+0000","last_unstale":"2024-09-11T12:56:19.593870+0000","last_undegraded":"2024-09-11T12:56:19.593870+0000","last_fullsized":"2024-09-11T12:56:19.593870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:43.440169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595768+0000","last_change":"2024-09-11T12:56:18.173193+0000","last_active":"2024-09-11T12:56:19.595768+0000","last_peered":"2024-09-11T12:56:19.595768+0000","last_clean":"2024-09-11T12:56:19.595768+0000","last_became_active":"2024-09-11T12:56:18.172909+0000","last_became_peered":"2024-09-11T12:56:18.172909+0000","last_unstale":"2024-09-11T12:56:19.595768+0000","last_undegraded":"2024-09-11T12:56:19.595768+0000","last_fullsized":"2024-09-11T12:56:19.595768+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:12:05.294584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157965+0000","last_change":"2024-09-11T12:56:18.168531+0000","last_active":"2024-09-11T12:56:19.157965+0000","last_peered":"2024-09-11T12:56:19.157965+0000","last_clean":"2024-09-11T12:56:19.157965+0000","last_became_active":"2024-09-11T12:56:18.164779+0000","last_became_peered":"2024-09-11T12:56:18.164779+0000","last_unstale":"2024-09-11T12:56:19.157965+0000","last_undegraded":"2024-09-11T12:56:19.157965+0000","last_fullsized":"2024-09-11T12:56:19.157965+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:16.193698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595206+0000","last_change":"2024-09-11T12:56:18.164474+0000","last_active":"2024-09-11T12:56:19.595206+0000","last_peered":"2024-09-11T12:56:19.595206+0000","last_clean":"2024-09-11T12:56:19.595206+0000","last_became_active":"2024-09-11T12:56:18.164227+0000","last_became_peered":"2024-09-11T12:56:18.164227+0000","last_unstale":"2024-09-11T12:56:19.595206+0000","last_undegraded":"2024-09-11T12:56:19.595206+0000","last_fullsized":"2024-09-11T12:56:19.595206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:46:06.935670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160150+0000","last_change":"2024-09-11T12:56:18.168807+0000","last_active":"2024-09-11T12:56:19.160150+0000","last_peered":"2024-09-11T12:56:19.160150+0000","last_clean":"2024-09-11T12:56:19.160150+0000","last_became_active":"2024-09-11T12:56:18.168550+0000","last_became_peered":"2024-09-11T12:56:18.168550+0000","last_unstale":"2024-09-11T12:56:19.160150+0000","last_undegraded":"2024-09-11T12:56:19.160150+0000","last_fullsized":"2024-09-11T12:56:19.160150+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:05.868695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595528+0000","last_change":"2024-09-11T12:56:18.161363+0000","last_active":"2024-09-11T12:56:19.595528+0000","last_peered":"2024-09-11T12:56:19.595528+0000","last_clean":"2024-09-11T12:56:19.595528+0000","last_became_active":"2024-09-11T12:56:18.160507+0000","last_became_peered":"2024-09-11T12:56:18.160507+0000","last_unstale":"2024-09-11T12:56:19.595528+0000","last_undegraded":"2024-09-11T12:56:19.595528+0000","last_fullsized":"2024-09-11T12:56:19.595528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:48:30.459921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596369+0000","last_change":"2024-09-11T12:56:18.166544+0000","last_active":"2024-09-11T12:56:19.596369+0000","last_peered":"2024-09-11T12:56:19.596369+0000","last_clean":"2024-09-11T12:56:19.596369+0000","last_became_active":"2024-09-11T12:56:18.166205+0000","last_became_peered":"2024-09-11T12:56:18.166205+0000","last_unstale":"2024-09-11T12:56:19.596369+0000","last_undegraded":"2024-09-11T12:56:19.596369+0000","last_fullsized":"2024-09-11T12:56:19.596369+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:44.887180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594481+0000","last_change":"2024-09-11T12:56:18.163075+0000","last_active":"2024-09-11T12:56:19.594481+0000","last_peered":"2024-09-11T12:56:19.594481+0000","last_clean":"2024-09-11T12:56:19.594481+0000","last_became_active":"2024-09-11T12:56:18.162571+0000","last_became_peered":"2024-09-11T12:56:18.162571+0000","last_unstale":"2024-09-11T12:56:19.594481+0000","last_undegraded":"2024-09-11T12:56:19.594481+0000","last_fullsized":"2024-09-11T12:56:19.594481+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:59.976927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595484+0000","last_change":"2024-09-11T12:56:18.160752+0000","last_active":"2024-09-11T12:56:19.595484+0000","last_peered":"2024-09-11T12:56:19.595484+0000","last_clean":"2024-09-11T12:56:19.595484+0000","last_became_active":"2024-09-11T12:56:18.160442+0000","last_became_peered":"2024-09-11T12:56:18.160442+0000","last_unstale":"2024-09-11T12:56:19.595484+0000","last_undegraded":"2024-09-11T12:56:19.595484+0000","last_fullsized":"2024-09-11T12:56:19.595484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:15:54.583895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160890+0000","last_change":"2024-09-11T12:56:18.157288+0000","last_active":"2024-09-11T12:56:19.160890+0000","last_peered":"2024-09-11T12:56:19.160890+0000","last_clean":"2024-09-11T12:56:19.160890+0000","last_became_active":"2024-09-11T12:56:18.156999+0000","last_became_peered":"2024-09-11T12:56:18.156999+0000","last_unstale":"2024-09-11T12:56:19.160890+0000","last_undegraded":"2024-09-11T12:56:19.160890+0000","last_fullsized":"2024-09-11T12:56:19.160890+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:03:54.103076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595686+0000","last_change":"2024-09-11T12:56:18.175598+0000","last_active":"2024-09-11T12:56:19.595686+0000","last_peered":"2024-09-11T12:56:19.595686+0000","last_clean":"2024-09-11T12:56:19.595686+0000","last_became_active":"2024-09-11T12:56:18.175172+0000","last_became_peered":"2024-09-11T12:56:18.175172+0000","last_unstale":"2024-09-11T12:56:19.595686+0000","last_undegraded":"2024-09-11T12:56:19.595686+0000","last_fullsized":"2024-09-11T12:56:19.595686+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:00:52.123801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160460+0000","last_change":"2024-09-11T12:56:18.151813+0000","last_active":"2024-09-11T12:56:19.160460+0000","last_peered":"2024-09-11T12:56:19.160460+0000","last_clean":"2024-09-11T12:56:19.160460+0000","last_became_active":"2024-09-11T12:56:18.150867+0000","last_became_peered":"2024-09-11T12:56:18.150867+0000","last_unstale":"2024-09-11T12:56:19.160460+0000","last_undegraded":"2024-09-11T12:56:19.160460+0000","last_fullsized":"2024-09-11T12:56:19.160460+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:55:36.449221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595009+0000","last_change":"2024-09-11T12:56:18.151526+0000","last_active":"2024-09-11T12:56:19.595009+0000","last_peered":"2024-09-11T12:56:19.595009+0000","last_clean":"2024-09-11T12:56:19.595009+0000","last_became_active":"2024-09-11T12:56:18.150668+0000","last_became_peered":"2024-09-11T12:56:18.150668+0000","last_unstale":"2024-09-11T12:56:19.595009+0000","last_undegraded":"2024-09-11T12:56:19.595009+0000","last_fullsized":"2024-09-11T12:56:19.595009+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:26.951757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160488+0000","last_change":"2024-09-11T12:56:18.150169+0000","last_active":"2024-09-11T12:56:19.160488+0000","last_peered":"2024-09-11T12:56:19.160488+0000","last_clean":"2024-09-11T12:56:19.160488+0000","last_became_active":"2024-09-11T12:56:18.149782+0000","last_became_peered":"2024-09-11T12:56:18.149782+0000","last_unstale":"2024-09-11T12:56:19.160488+0000","last_undegraded":"2024-09-11T12:56:19.160488+0000","last_fullsized":"2024-09-11T12:56:19.160488+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:44:48.408023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160931+0000","last_change":"2024-09-11T12:56:18.161857+0000","last_active":"2024-09-11T12:56:19.160931+0000","last_peered":"2024-09-11T12:56:19.160931+0000","last_clean":"2024-09-11T12:56:19.160931+0000","last_became_active":"2024-09-11T12:56:18.161576+0000","last_became_peered":"2024-09-11T12:56:18.161576+0000","last_unstale":"2024-09-11T12:56:19.160931+0000","last_undegraded":"2024-09-11T12:56:19.160931+0000","last_fullsized":"2024-09-11T12:56:19.160931+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160736+0000","last_change":"2024-09-11T12:56:18.165489+0000","last_active":"2024-09-11T12:56:19.160736+0000","last_peered":"2024-09-11T12:56:19.160736+0000","last_clean":"2024-09-11T12:56:19.160736+0000","last_became_active":"2024-09-11T12:56:18.165067+0000","last_became_peered":"2024-09-11T12:56:18.165067+0000","last_unstale":"2024-09-11T12:56:19.160736+0000","last_undegraded":"2024-09-11T12:56:19.160736+0000","last_fullsized":"2024-09-11T12:56:19.160736+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:36.346450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157533+0000","last_change":"2024-09-11T12:56:18.163192+0000","last_active":"2024-09-11T12:56:19.157533+0000","last_peered":"2024-09-11T12:56:19.157533+0000","last_clean":"2024-09-11T12:56:19.157533+0000","last_became_active":"2024-09-11T12:56:18.162309+0000","last_became_peered":"2024-09-11T12:56:18.162309+0000","last_unstale":"2024-09-11T12:56:19.157533+0000","last_undegraded":"2024-09-11T12:56:19.157533+0000","last_fullsized":"2024-09-11T12:56:19.157533+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595726+0000","last_change":"2024-09-11T12:56:18.175389+0000","last_active":"2024-09-11T12:56:19.595726+0000","last_peered":"2024-09-11T12:56:19.595726+0000","last_clean":"2024-09-11T12:56:19.595726+0000","last_became_active":"2024-09-11T12:56:18.174789+0000","last_became_peered":"2024-09-11T12:56:18.174789+0000","last_unstale":"2024-09-11T12:56:19.595726+0000","last_undegraded":"2024-09-11T12:56:19.595726+0000","last_fullsized":"2024-09-11T12:56:19.595726+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:10:13.350548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"139'4","reported_seq":19,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596051+0000","last_change":"2024-09-11T12:56:17.156091+0000","last_active":"2024-09-11T12:56:19.596051+0000","last_peered":"2024-09-11T12:56:19.596051+0000","last_clean":"2024-09-11T12:56:19.596051+0000","last_became_active":"2024-09-11T12:56:17.155801+0000","last_became_peered":"2024-09-11T12:56:17.155801+0000","last_unstale":"2024-09-11T12:56:19.596051+0000","last_undegraded":"2024-09-11T12:56:19.596051+0000","last_fullsized":"2024-09-11T12:56:19.596051+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:59:48.944107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160417+0000","last_change":"2024-09-11T12:56:18.163805+0000","last_active":"2024-09-11T12:56:19.160417+0000","last_peered":"2024-09-11T12:56:19.160417+0000","last_clean":"2024-09-11T12:56:19.160417+0000","last_became_active":"2024-09-11T12:56:18.163525+0000","last_became_peered":"2024-09-11T12:56:18.163525+0000","last_unstale":"2024-09-11T12:56:19.160417+0000","last_undegraded":"2024-09-11T12:56:19.160417+0000","last_fullsized":"2024-09-11T12:56:19.160417+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:18:23.148160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159914+0000","last_change":"2024-09-11T12:56:17.160978+0000","last_active":"2024-09-11T12:56:19.159914+0000","last_peered":"2024-09-11T12:56:19.159914+0000","last_clean":"2024-09-11T12:56:19.159914+0000","last_became_active":"2024-09-11T12:56:17.160669+0000","last_became_peered":"2024-09-11T12:56:17.160669+0000","last_unstale":"2024-09-11T12:56:19.159914+0000","last_undegraded":"2024-09-11T12:56:19.159914+0000","last_fullsized":"2024-09-11T12:56:19.159914+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595552+0000","last_change":"2024-09-11T12:56:18.170070+0000","last_active":"2024-09-11T12:56:19.595552+0000","last_peered":"2024-09-11T12:56:19.595552+0000","last_clean":"2024-09-11T12:56:19.595552+0000","last_became_active":"2024-09-11T12:56:18.169794+0000","last_became_peered":"2024-09-11T12:56:18.169794+0000","last_unstale":"2024-09-11T12:56:19.595552+0000","last_undegraded":"2024-09-11T12:56:19.595552+0000","last_fullsized":"2024-09-11T12:56:19.595552+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:24:51.485085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594917+0000","last_change":"2024-09-11T12:56:17.163157+0000","last_active":"2024-09-11T12:56:19.594917+0000","last_peered":"2024-09-11T12:56:19.594917+0000","last_clean":"2024-09-11T12:56:19.594917+0000","last_became_active":"2024-09-11T12:56:17.162872+0000","last_became_peered":"2024-09-11T12:56:17.162872+0000","last_unstale":"2024-09-11T12:56:19.594917+0000","last_undegraded":"2024-09-11T12:56:19.594917+0000","last_fullsized":"2024-09-11T12:56:19.594917+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:33.412968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160658+0000","last_change":"2024-09-11T12:56:18.173594+0000","last_active":"2024-09-11T12:56:19.160658+0000","last_peered":"2024-09-11T12:56:19.160658+0000","last_clean":"2024-09-11T12:56:19.160658+0000","last_became_active":"2024-09-11T12:56:18.173288+0000","last_became_peered":"2024-09-11T12:56:18.173288+0000","last_unstale":"2024-09-11T12:56:19.160658+0000","last_undegraded":"2024-09-11T12:56:19.160658+0000","last_fullsized":"2024-09-11T12:56:19.160658+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:23.996637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595106+0000","last_change":"2024-09-11T12:56:17.158690+0000","last_active":"2024-09-11T12:56:19.595106+0000","last_peered":"2024-09-11T12:56:19.595106+0000","last_clean":"2024-09-11T12:56:19.595106+0000","last_became_active":"2024-09-11T12:56:17.158420+0000","last_became_peered":"2024-09-11T12:56:17.158420+0000","last_unstale":"2024-09-11T12:56:19.595106+0000","last_undegraded":"2024-09-11T12:56:19.595106+0000","last_fullsized":"2024-09-11T12:56:19.595106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:51:36.763247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594523+0000","last_change":"2024-09-11T12:56:18.168507+0000","last_active":"2024-09-11T12:56:19.594523+0000","last_peered":"2024-09-11T12:56:19.594523+0000","last_clean":"2024-09-11T12:56:19.594523+0000","last_became_active":"2024-09-11T12:56:18.168251+0000","last_became_peered":"2024-09-11T12:56:18.168251+0000","last_unstale":"2024-09-11T12:56:19.594523+0000","last_undegraded":"2024-09-11T12:56:19.594523+0000","last_fullsized":"2024-09-11T12:56:19.594523+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:39.866623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160638+0000","last_change":"2024-09-11T12:56:17.157989+0000","last_active":"2024-09-11T12:56:19.160638+0000","last_peered":"2024-09-11T12:56:19.160638+0000","last_clean":"2024-09-11T12:56:19.160638+0000","last_became_active":"2024-09-11T12:56:17.157690+0000","last_became_peered":"2024-09-11T12:56:17.157690+0000","last_unstale":"2024-09-11T12:56:19.160638+0000","last_undegraded":"2024-09-11T12:56:19.160638+0000","last_fullsized":"2024-09-11T12:56:19.160638+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:04:33.414909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594889+0000","last_change":"2024-09-11T12:56:18.170686+0000","last_active":"2024-09-11T12:56:19.594889+0000","last_peered":"2024-09-11T12:56:19.594889+0000","last_clean":"2024-09-11T12:56:19.594889+0000","last_became_active":"2024-09-11T12:56:18.170422+0000","last_became_peered":"2024-09-11T12:56:18.170422+0000","last_unstale":"2024-09-11T12:56:19.594889+0000","last_undegraded":"2024-09-11T12:56:19.594889+0000","last_fullsized":"2024-09-11T12:56:19.594889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593926+0000","last_change":"2024-09-11T12:56:17.167482+0000","last_active":"2024-09-11T12:56:19.593926+0000","last_peered":"2024-09-11T12:56:19.593926+0000","last_clean":"2024-09-11T12:56:19.593926+0000","last_became_active":"2024-09-11T12:56:17.167173+0000","last_became_peered":"2024-09-11T12:56:17.167173+0000","last_unstale":"2024-09-11T12:56:19.593926+0000","last_undegraded":"2024-09-11T12:56:19.593926+0000","last_fullsized":"2024-09-11T12:56:19.593926+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:15.618071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593896+0000","last_change":"2024-09-11T12:56:18.159865+0000","last_active":"2024-09-11T12:56:19.593896+0000","last_peered":"2024-09-11T12:56:19.593896+0000","last_clean":"2024-09-11T12:56:19.593896+0000","last_became_active":"2024-09-11T12:56:18.159659+0000","last_became_peered":"2024-09-11T12:56:18.159659+0000","last_unstale":"2024-09-11T12:56:19.593896+0000","last_undegraded":"2024-09-11T12:56:19.593896+0000","last_fullsized":"2024-09-11T12:56:19.593896+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:24:47.853230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595940+0000","last_change":"2024-09-11T12:56:17.163158+0000","last_active":"2024-09-11T12:56:19.595940+0000","last_peered":"2024-09-11T12:56:19.595940+0000","last_clean":"2024-09-11T12:56:19.595940+0000","last_became_active":"2024-09-11T12:56:17.162847+0000","last_became_peered":"2024-09-11T12:56:17.162847+0000","last_unstale":"2024-09-11T12:56:19.595940+0000","last_undegraded":"2024-09-11T12:56:19.595940+0000","last_fullsized":"2024-09-11T12:56:19.595940+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:58:40.658294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594647+0000","last_change":"2024-09-11T12:56:18.174176+0000","last_active":"2024-09-11T12:56:19.594647+0000","last_peered":"2024-09-11T12:56:19.594647+0000","last_clean":"2024-09-11T12:56:19.594647+0000","last_became_active":"2024-09-11T12:56:18.173540+0000","last_became_peered":"2024-09-11T12:56:18.173540+0000","last_unstale":"2024-09-11T12:56:19.594647+0000","last_undegraded":"2024-09-11T12:56:19.594647+0000","last_fullsized":"2024-09-11T12:56:19.594647+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:05.388020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594030+0000","last_change":"2024-09-11T12:56:17.163727+0000","last_active":"2024-09-11T12:56:19.594030+0000","last_peered":"2024-09-11T12:56:19.594030+0000","last_clean":"2024-09-11T12:56:19.594030+0000","last_became_active":"2024-09-11T12:56:17.163494+0000","last_became_peered":"2024-09-11T12:56:17.163494+0000","last_unstale":"2024-09-11T12:56:19.594030+0000","last_undegraded":"2024-09-11T12:56:19.594030+0000","last_fullsized":"2024-09-11T12:56:19.594030+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:27:27.281118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158343+0000","last_change":"2024-09-11T12:56:18.170654+0000","last_active":"2024-09-11T12:56:19.158343+0000","last_peered":"2024-09-11T12:56:19.158343+0000","last_clean":"2024-09-11T12:56:19.158343+0000","last_became_active":"2024-09-11T12:56:18.169026+0000","last_became_peered":"2024-09-11T12:56:18.169026+0000","last_unstale":"2024-09-11T12:56:19.158343+0000","last_undegraded":"2024-09-11T12:56:19.158343+0000","last_fullsized":"2024-09-11T12:56:19.158343+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:05.309532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595392+0000","last_change":"2024-09-11T12:56:17.162318+0000","last_active":"2024-09-11T12:56:19.595392+0000","last_peered":"2024-09-11T12:56:19.595392+0000","last_clean":"2024-09-11T12:56:19.595392+0000","last_became_active":"2024-09-11T12:56:17.161960+0000","last_became_peered":"2024-09-11T12:56:17.161960+0000","last_unstale":"2024-09-11T12:56:19.595392+0000","last_undegraded":"2024-09-11T12:56:19.595392+0000","last_fullsized":"2024-09-11T12:56:19.595392+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:20:41.080476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157872+0000","last_change":"2024-09-11T12:56:18.161522+0000","last_active":"2024-09-11T12:56:19.157872+0000","last_peered":"2024-09-11T12:56:19.157872+0000","last_clean":"2024-09-11T12:56:19.157872+0000","last_became_active":"2024-09-11T12:56:18.161212+0000","last_became_peered":"2024-09-11T12:56:18.161212+0000","last_unstale":"2024-09-11T12:56:19.157872+0000","last_undegraded":"2024-09-11T12:56:19.157872+0000","last_fullsized":"2024-09-11T12:56:19.157872+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:56.363389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"139'3","reported_seq":18,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595321+0000","last_change":"2024-09-11T12:56:17.156655+0000","last_active":"2024-09-11T12:56:19.595321+0000","last_peered":"2024-09-11T12:56:19.595321+0000","last_clean":"2024-09-11T12:56:19.595321+0000","last_became_active":"2024-09-11T12:56:17.156333+0000","last_became_peered":"2024-09-11T12:56:17.156333+0000","last_unstale":"2024-09-11T12:56:19.595321+0000","last_undegraded":"2024-09-11T12:56:19.595321+0000","last_fullsized":"2024-09-11T12:56:19.595321+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:33:55.130625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160065+0000","last_change":"2024-09-11T12:56:18.155842+0000","last_active":"2024-09-11T12:56:19.160065+0000","last_peered":"2024-09-11T12:56:19.160065+0000","last_clean":"2024-09-11T12:56:19.160065+0000","last_became_active":"2024-09-11T12:56:18.154935+0000","last_became_peered":"2024-09-11T12:56:18.154935+0000","last_unstale":"2024-09-11T12:56:19.160065+0000","last_undegraded":"2024-09-11T12:56:19.160065+0000","last_fullsized":"2024-09-11T12:56:19.160065+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:25:47.937582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"139'6","reported_seq":21,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160009+0000","last_change":"2024-09-11T12:56:17.152914+0000","last_active":"2024-09-11T12:56:19.160009+0000","last_peered":"2024-09-11T12:56:19.160009+0000","last_clean":"2024-09-11T12:56:19.160009+0000","last_became_active":"2024-09-11T12:56:17.152624+0000","last_became_peered":"2024-09-11T12:56:17.152624+0000","last_unstale":"2024-09-11T12:56:19.160009+0000","last_undegraded":"2024-09-11T12:56:19.160009+0000","last_fullsized":"2024-09-11T12:56:19.160009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:46:15.618071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157623+0000","last_change":"2024-09-11T12:56:18.162721+0000","last_active":"2024-09-11T12:56:19.157623+0000","last_peered":"2024-09-11T12:56:19.157623+0000","last_clean":"2024-09-11T12:56:19.157623+0000","last_became_active":"2024-09-11T12:56:18.160923+0000","last_became_peered":"2024-09-11T12:56:18.160923+0000","last_unstale":"2024-09-11T12:56:19.157623+0000","last_undegraded":"2024-09-11T12:56:19.157623+0000","last_fullsized":"2024-09-11T12:56:19.157623+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:59:25.085255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595002+0000","last_change":"2024-09-11T12:56:17.159279+0000","last_active":"2024-09-11T12:56:19.595002+0000","last_peered":"2024-09-11T12:56:19.595002+0000","last_clean":"2024-09-11T12:56:19.595002+0000","last_became_active":"2024-09-11T12:56:17.159020+0000","last_became_peered":"2024-09-11T12:56:17.159020+0000","last_unstale":"2024-09-11T12:56:19.595002+0000","last_undegraded":"2024-09-11T12:56:19.595002+0000","last_fullsized":"2024-09-11T12:56:19.595002+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:36.233519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594609+0000","last_change":"2024-09-11T12:56:18.180134+0000","last_active":"2024-09-11T12:56:19.594609+0000","last_peered":"2024-09-11T12:56:19.594609+0000","last_clean":"2024-09-11T12:56:19.594609+0000","last_became_active":"2024-09-11T12:56:18.175229+0000","last_became_peered":"2024-09-11T12:56:18.175229+0000","last_unstale":"2024-09-11T12:56:19.594609+0000","last_undegraded":"2024-09-11T12:56:19.594609+0000","last_fullsized":"2024-09-11T12:56:19.594609+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:51:05.947746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157723+0000","last_change":"2024-09-11T12:56:17.159986+0000","last_active":"2024-09-11T12:56:19.157723+0000","last_peered":"2024-09-11T12:56:19.157723+0000","last_clean":"2024-09-11T12:56:19.157723+0000","last_became_active":"2024-09-11T12:56:17.159714+0000","last_became_peered":"2024-09-11T12:56:17.159714+0000","last_unstale":"2024-09-11T12:56:19.157723+0000","last_undegraded":"2024-09-11T12:56:19.157723+0000","last_fullsized":"2024-09-11T12:56:19.157723+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594020+0000","last_change":"2024-09-11T12:56:18.152532+0000","last_active":"2024-09-11T12:56:19.594020+0000","last_peered":"2024-09-11T12:56:19.594020+0000","last_clean":"2024-09-11T12:56:19.594020+0000","last_became_active":"2024-09-11T12:56:18.152263+0000","last_became_peered":"2024-09-11T12:56:18.152263+0000","last_unstale":"2024-09-11T12:56:19.594020+0000","last_undegraded":"2024-09-11T12:56:19.594020+0000","last_fullsized":"2024-09-11T12:56:19.594020+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:07.632426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593923+0000","last_change":"2024-09-11T12:56:17.162210+0000","last_active":"2024-09-11T12:56:19.593923+0000","last_peered":"2024-09-11T12:56:19.593923+0000","last_clean":"2024-09-11T12:56:19.593923+0000","last_became_active":"2024-09-11T12:56:17.161330+0000","last_became_peered":"2024-09-11T12:56:17.161330+0000","last_unstale":"2024-09-11T12:56:19.593923+0000","last_undegraded":"2024-09-11T12:56:19.593923+0000","last_fullsized":"2024-09-11T12:56:19.593923+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159448+0000","last_change":"2024-09-11T12:56:18.171308+0000","last_active":"2024-09-11T12:56:19.159448+0000","last_peered":"2024-09-11T12:56:19.159448+0000","last_clean":"2024-09-11T12:56:19.159448+0000","last_became_active":"2024-09-11T12:56:18.164785+0000","last_became_peered":"2024-09-11T12:56:18.164785+0000","last_unstale":"2024-09-11T12:56:19.159448+0000","last_undegraded":"2024-09-11T12:56:19.159448+0000","last_fullsized":"2024-09-11T12:56:19.159448+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:45.694134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"139'8","reported_seq":23,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159313+0000","last_change":"2024-09-11T12:56:17.153678+0000","last_active":"2024-09-11T12:56:19.159313+0000","last_peered":"2024-09-11T12:56:19.159313+0000","last_clean":"2024-09-11T12:56:19.159313+0000","last_became_active":"2024-09-11T12:56:17.153151+0000","last_became_peered":"2024-09-11T12:56:17.153151+0000","last_unstale":"2024-09-11T12:56:19.159313+0000","last_undegraded":"2024-09-11T12:56:19.159313+0000","last_fullsized":"2024-09-11T12:56:19.159313+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:14:36.233519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595360+0000","last_change":"2024-09-11T12:56:18.163250+0000","last_active":"2024-09-11T12:56:19.595360+0000","last_peered":"2024-09-11T12:56:19.595360+0000","last_clean":"2024-09-11T12:56:19.595360+0000","last_became_active":"2024-09-11T12:56:18.162964+0000","last_became_peered":"2024-09-11T12:56:18.162964+0000","last_unstale":"2024-09-11T12:56:19.595360+0000","last_undegraded":"2024-09-11T12:56:19.595360+0000","last_fullsized":"2024-09-11T12:56:19.595360+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:16.193698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.161017+0000","last_change":"2024-09-11T12:56:17.162723+0000","last_active":"2024-09-11T12:56:19.161017+0000","last_peered":"2024-09-11T12:56:19.161017+0000","last_clean":"2024-09-11T12:56:19.161017+0000","last_became_active":"2024-09-11T12:56:17.162458+0000","last_became_peered":"2024-09-11T12:56:17.162458+0000","last_unstale":"2024-09-11T12:56:19.161017+0000","last_undegraded":"2024-09-11T12:56:19.161017+0000","last_fullsized":"2024-09-11T12:56:19.161017+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:01:59.894483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158432+0000","last_change":"2024-09-11T12:56:18.171174+0000","last_active":"2024-09-11T12:56:19.158432+0000","last_peered":"2024-09-11T12:56:19.158432+0000","last_clean":"2024-09-11T12:56:19.158432+0000","last_became_active":"2024-09-11T12:56:18.170881+0000","last_became_peered":"2024-09-11T12:56:18.170881+0000","last_unstale":"2024-09-11T12:56:19.158432+0000","last_undegraded":"2024-09-11T12:56:19.158432+0000","last_fullsized":"2024-09-11T12:56:19.158432+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:47:32.385005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160771+0000","last_change":"2024-09-11T12:56:17.159839+0000","last_active":"2024-09-11T12:56:19.160771+0000","last_peered":"2024-09-11T12:56:19.160771+0000","last_clean":"2024-09-11T12:56:19.160771+0000","last_became_active":"2024-09-11T12:56:17.159568+0000","last_became_peered":"2024-09-11T12:56:17.159568+0000","last_unstale":"2024-09-11T12:56:19.160771+0000","last_undegraded":"2024-09-11T12:56:19.160771+0000","last_fullsized":"2024-09-11T12:56:19.160771+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:11.702594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595106+0000","last_change":"2024-09-11T12:56:18.166966+0000","last_active":"2024-09-11T12:56:19.595106+0000","last_peered":"2024-09-11T12:56:19.595106+0000","last_clean":"2024-09-11T12:56:19.595106+0000","last_became_active":"2024-09-11T12:56:18.166677+0000","last_became_peered":"2024-09-11T12:56:18.166677+0000","last_unstale":"2024-09-11T12:56:19.595106+0000","last_undegraded":"2024-09-11T12:56:19.595106+0000","last_fullsized":"2024-09-11T12:56:19.595106+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:56.363389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"139'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596046+0000","last_change":"2024-09-11T12:56:17.158043+0000","last_active":"2024-09-11T12:56:19.596046+0000","last_peered":"2024-09-11T12:56:19.596046+0000","last_clean":"2024-09-11T12:56:19.596046+0000","last_became_active":"2024-09-11T12:56:17.157312+0000","last_became_peered":"2024-09-11T12:56:17.157312+0000","last_unstale":"2024-09-11T12:56:19.596046+0000","last_undegraded":"2024-09-11T12:56:19.596046+0000","last_fullsized":"2024-09-11T12:56:19.596046+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:09:46.601805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595384+0000","last_change":"2024-09-11T12:56:18.179626+0000","last_active":"2024-09-11T12:56:19.595384+0000","last_peered":"2024-09-11T12:56:19.595384+0000","last_clean":"2024-09-11T12:56:19.595384+0000","last_became_active":"2024-09-11T12:56:18.179343+0000","last_became_peered":"2024-09-11T12:56:18.179343+0000","last_unstale":"2024-09-11T12:56:19.595384+0000","last_undegraded":"2024-09-11T12:56:19.595384+0000","last_fullsized":"2024-09-11T12:56:19.595384+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:29:29.519414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595846+0000","last_change":"2024-09-11T12:56:17.165337+0000","last_active":"2024-09-11T12:56:19.595846+0000","last_peered":"2024-09-11T12:56:19.595846+0000","last_clean":"2024-09-11T12:56:19.595846+0000","last_became_active":"2024-09-11T12:56:17.165011+0000","last_became_peered":"2024-09-11T12:56:17.165011+0000","last_unstale":"2024-09-11T12:56:19.595846+0000","last_undegraded":"2024-09-11T12:56:19.595846+0000","last_fullsized":"2024-09-11T12:56:19.595846+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:47:50.519843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159831+0000","last_change":"2024-09-11T12:56:18.170321+0000","last_active":"2024-09-11T12:56:19.159831+0000","last_peered":"2024-09-11T12:56:19.159831+0000","last_clean":"2024-09-11T12:56:19.159831+0000","last_became_active":"2024-09-11T12:56:18.163860+0000","last_became_peered":"2024-09-11T12:56:18.163860+0000","last_unstale":"2024-09-11T12:56:19.159831+0000","last_undegraded":"2024-09-11T12:56:19.159831+0000","last_fullsized":"2024-09-11T12:56:19.159831+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:01.614155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595105+0000","last_change":"2024-09-11T12:56:17.161102+0000","last_active":"2024-09-11T12:56:19.595105+0000","last_peered":"2024-09-11T12:56:19.595105+0000","last_clean":"2024-09-11T12:56:19.595105+0000","last_became_active":"2024-09-11T12:56:17.160479+0000","last_became_peered":"2024-09-11T12:56:17.160479+0000","last_unstale":"2024-09-11T12:56:19.595105+0000","last_undegraded":"2024-09-11T12:56:19.595105+0000","last_fullsized":"2024-09-11T12:56:19.595105+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:34:20.045407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594766+0000","last_change":"2024-09-11T12:56:18.169504+0000","last_active":"2024-09-11T12:56:19.594766+0000","last_peered":"2024-09-11T12:56:19.594766+0000","last_clean":"2024-09-11T12:56:19.594766+0000","last_became_active":"2024-09-11T12:56:18.169262+0000","last_became_peered":"2024-09-11T12:56:18.169262+0000","last_unstale":"2024-09-11T12:56:19.594766+0000","last_undegraded":"2024-09-11T12:56:19.594766+0000","last_fullsized":"2024-09-11T12:56:19.594766+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:46:27.819654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595692+0000","last_change":"2024-09-11T12:56:17.163874+0000","last_active":"2024-09-11T12:56:19.595692+0000","last_peered":"2024-09-11T12:56:19.595692+0000","last_clean":"2024-09-11T12:56:19.595692+0000","last_became_active":"2024-09-11T12:56:17.161901+0000","last_became_peered":"2024-09-11T12:56:17.161901+0000","last_unstale":"2024-09-11T12:56:19.595692+0000","last_undegraded":"2024-09-11T12:56:19.595692+0000","last_fullsized":"2024-09-11T12:56:19.595692+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:41:28.837956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595580+0000","last_change":"2024-09-11T12:56:18.176205+0000","last_active":"2024-09-11T12:56:19.595580+0000","last_peered":"2024-09-11T12:56:19.595580+0000","last_clean":"2024-09-11T12:56:19.595580+0000","last_became_active":"2024-09-11T12:56:18.174067+0000","last_became_peered":"2024-09-11T12:56:18.174067+0000","last_unstale":"2024-09-11T12:56:19.595580+0000","last_undegraded":"2024-09-11T12:56:19.595580+0000","last_fullsized":"2024-09-11T12:56:19.595580+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:19.213356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594017+0000","last_change":"2024-09-11T12:56:17.159432+0000","last_active":"2024-09-11T12:56:19.594017+0000","last_peered":"2024-09-11T12:56:19.594017+0000","last_clean":"2024-09-11T12:56:19.594017+0000","last_became_active":"2024-09-11T12:56:17.159150+0000","last_became_peered":"2024-09-11T12:56:17.159150+0000","last_unstale":"2024-09-11T12:56:19.594017+0000","last_undegraded":"2024-09-11T12:56:19.594017+0000","last_fullsized":"2024-09-11T12:56:19.594017+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:30.773880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594812+0000","last_change":"2024-09-11T12:56:18.162832+0000","last_active":"2024-09-11T12:56:19.594812+0000","last_peered":"2024-09-11T12:56:19.594812+0000","last_clean":"2024-09-11T12:56:19.594812+0000","last_became_active":"2024-09-11T12:56:18.162223+0000","last_became_peered":"2024-09-11T12:56:18.162223+0000","last_unstale":"2024-09-11T12:56:19.594812+0000","last_undegraded":"2024-09-11T12:56:19.594812+0000","last_fullsized":"2024-09-11T12:56:19.594812+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:01.529543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595787+0000","last_change":"2024-09-11T12:56:17.164420+0000","last_active":"2024-09-11T12:56:19.595787+0000","last_peered":"2024-09-11T12:56:19.595787+0000","last_clean":"2024-09-11T12:56:19.595787+0000","last_became_active":"2024-09-11T12:56:17.162913+0000","last_became_peered":"2024-09-11T12:56:17.162913+0000","last_unstale":"2024-09-11T12:56:19.595787+0000","last_undegraded":"2024-09-11T12:56:19.595787+0000","last_fullsized":"2024-09-11T12:56:19.595787+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:22.083507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158213+0000","last_change":"2024-09-11T12:56:18.169449+0000","last_active":"2024-09-11T12:56:19.158213+0000","last_peered":"2024-09-11T12:56:19.158213+0000","last_clean":"2024-09-11T12:56:19.158213+0000","last_became_active":"2024-09-11T12:56:18.169199+0000","last_became_peered":"2024-09-11T12:56:18.169199+0000","last_unstale":"2024-09-11T12:56:19.158213+0000","last_undegraded":"2024-09-11T12:56:19.158213+0000","last_fullsized":"2024-09-11T12:56:19.158213+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:55:33.336616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"139'2","reported_seq":17,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158441+0000","last_change":"2024-09-11T12:56:17.160318+0000","last_active":"2024-09-11T12:56:19.158441+0000","last_peered":"2024-09-11T12:56:19.158441+0000","last_clean":"2024-09-11T12:56:19.158441+0000","last_became_active":"2024-09-11T12:56:17.160088+0000","last_became_peered":"2024-09-11T12:56:17.160088+0000","last_unstale":"2024-09-11T12:56:19.158441+0000","last_undegraded":"2024-09-11T12:56:19.158441+0000","last_fullsized":"2024-09-11T12:56:19.158441+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:30.773880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595389+0000","last_change":"2024-09-11T12:56:18.165839+0000","last_active":"2024-09-11T12:56:19.595389+0000","last_peered":"2024-09-11T12:56:19.595389+0000","last_clean":"2024-09-11T12:56:19.595389+0000","last_became_active":"2024-09-11T12:56:18.165525+0000","last_became_peered":"2024-09-11T12:56:18.165525+0000","last_unstale":"2024-09-11T12:56:19.595389+0000","last_undegraded":"2024-09-11T12:56:19.595389+0000","last_fullsized":"2024-09-11T12:56:19.595389+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:45:54.412049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"140'1","reported_seq":16,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:20.148922+0000","last_change":"2024-09-11T12:56:17.151805+0000","last_active":"2024-09-11T12:56:20.148922+0000","last_peered":"2024-09-11T12:56:20.148922+0000","last_clean":"2024-09-11T12:56:20.148922+0000","last_became_active":"2024-09-11T12:56:17.151520+0000","last_became_peered":"2024-09-11T12:56:17.151520+0000","last_unstale":"2024-09-11T12:56:20.148922+0000","last_undegraded":"2024-09-11T12:56:20.148922+0000","last_fullsized":"2024-09-11T12:56:20.148922+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:04.826106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595633+0000","last_change":"2024-09-11T12:56:18.175754+0000","last_active":"2024-09-11T12:56:19.595633+0000","last_peered":"2024-09-11T12:56:19.595633+0000","last_clean":"2024-09-11T12:56:19.595633+0000","last_became_active":"2024-09-11T12:56:18.175288+0000","last_became_peered":"2024-09-11T12:56:18.175288+0000","last_unstale":"2024-09-11T12:56:19.595633+0000","last_undegraded":"2024-09-11T12:56:19.595633+0000","last_fullsized":"2024-09-11T12:56:19.595633+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:15:58.892257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596645+0000","last_change":"2024-09-11T12:56:17.167395+0000","last_active":"2024-09-11T12:56:19.596645+0000","last_peered":"2024-09-11T12:56:19.596645+0000","last_clean":"2024-09-11T12:56:19.596645+0000","last_became_active":"2024-09-11T12:56:17.167027+0000","last_became_peered":"2024-09-11T12:56:17.167027+0000","last_unstale":"2024-09-11T12:56:19.596645+0000","last_undegraded":"2024-09-11T12:56:19.596645+0000","last_fullsized":"2024-09-11T12:56:19.596645+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:49.645834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157808+0000","last_change":"2024-09-11T12:56:18.176147+0000","last_active":"2024-09-11T12:56:19.157808+0000","last_peered":"2024-09-11T12:56:19.157808+0000","last_clean":"2024-09-11T12:56:19.157808+0000","last_became_active":"2024-09-11T12:56:18.175854+0000","last_became_peered":"2024-09-11T12:56:18.175854+0000","last_unstale":"2024-09-11T12:56:19.157808+0000","last_undegraded":"2024-09-11T12:56:19.157808+0000","last_fullsized":"2024-09-11T12:56:19.157808+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:26.951757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157972+0000","last_change":"2024-09-11T12:56:17.153175+0000","last_active":"2024-09-11T12:56:19.157972+0000","last_peered":"2024-09-11T12:56:19.157972+0000","last_clean":"2024-09-11T12:56:19.157972+0000","last_became_active":"2024-09-11T12:56:17.152884+0000","last_became_peered":"2024-09-11T12:56:17.152884+0000","last_unstale":"2024-09-11T12:56:19.157972+0000","last_undegraded":"2024-09-11T12:56:19.157972+0000","last_fullsized":"2024-09-11T12:56:19.157972+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.157649+0000","last_change":"2024-09-11T12:56:18.160943+0000","last_active":"2024-09-11T12:56:19.157649+0000","last_peered":"2024-09-11T12:56:19.157649+0000","last_clean":"2024-09-11T12:56:19.157649+0000","last_became_active":"2024-09-11T12:56:18.160667+0000","last_became_peered":"2024-09-11T12:56:18.160667+0000","last_unstale":"2024-09-11T12:56:19.157649+0000","last_undegraded":"2024-09-11T12:56:19.157649+0000","last_fullsized":"2024-09-11T12:56:19.157649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:37:23.489680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160000+0000","last_change":"2024-09-11T12:56:17.155507+0000","last_active":"2024-09-11T12:56:19.160000+0000","last_peered":"2024-09-11T12:56:19.160000+0000","last_clean":"2024-09-11T12:56:19.160000+0000","last_became_active":"2024-09-11T12:56:17.155178+0000","last_became_peered":"2024-09-11T12:56:17.155178+0000","last_unstale":"2024-09-11T12:56:19.160000+0000","last_undegraded":"2024-09-11T12:56:19.160000+0000","last_fullsized":"2024-09-11T12:56:19.160000+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:07:33.412968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159874+0000","last_change":"2024-09-11T12:56:18.169243+0000","last_active":"2024-09-11T12:56:19.159874+0000","last_peered":"2024-09-11T12:56:19.159874+0000","last_clean":"2024-09-11T12:56:19.159874+0000","last_became_active":"2024-09-11T12:56:18.168597+0000","last_became_peered":"2024-09-11T12:56:18.168597+0000","last_unstale":"2024-09-11T12:56:19.159874+0000","last_undegraded":"2024-09-11T12:56:19.159874+0000","last_fullsized":"2024-09-11T12:56:19.159874+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:04:44.887180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.594232+0000","last_change":"2024-09-11T12:56:17.154900+0000","last_active":"2024-09-11T12:56:19.594232+0000","last_peered":"2024-09-11T12:56:19.594232+0000","last_clean":"2024-09-11T12:56:19.594232+0000","last_became_active":"2024-09-11T12:56:17.154587+0000","last_became_peered":"2024-09-11T12:56:17.154587+0000","last_unstale":"2024-09-11T12:56:19.594232+0000","last_undegraded":"2024-09-11T12:56:19.594232+0000","last_fullsized":"2024-09-11T12:56:19.594232+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:22.742019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.593869+0000","last_change":"2024-09-11T12:56:18.170263+0000","last_active":"2024-09-11T12:56:19.593869+0000","last_peered":"2024-09-11T12:56:19.593869+0000","last_clean":"2024-09-11T12:56:19.593869+0000","last_became_active":"2024-09-11T12:56:18.169656+0000","last_became_peered":"2024-09-11T12:56:18.169656+0000","last_unstale":"2024-09-11T12:56:19.593869+0000","last_undegraded":"2024-09-11T12:56:19.593869+0000","last_fullsized":"2024-09-11T12:56:19.593869+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:44:57.503942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.596082+0000","last_change":"2024-09-11T12:56:17.164719+0000","last_active":"2024-09-11T12:56:19.596082+0000","last_peered":"2024-09-11T12:56:19.596082+0000","last_clean":"2024-09-11T12:56:19.596082+0000","last_became_active":"2024-09-11T12:56:17.164074+0000","last_became_peered":"2024-09-11T12:56:17.164074+0000","last_unstale":"2024-09-11T12:56:19.596082+0000","last_undegraded":"2024-09-11T12:56:19.596082+0000","last_fullsized":"2024-09-11T12:56:19.596082+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:18:31.346670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159521+0000","last_change":"2024-09-11T12:56:18.164375+0000","last_active":"2024-09-11T12:56:19.159521+0000","last_peered":"2024-09-11T12:56:19.159521+0000","last_clean":"2024-09-11T12:56:19.159521+0000","last_became_active":"2024-09-11T12:56:18.164043+0000","last_became_peered":"2024-09-11T12:56:18.164043+0000","last_unstale":"2024-09-11T12:56:19.159521+0000","last_undegraded":"2024-09-11T12:56:19.159521+0000","last_fullsized":"2024-09-11T12:56:19.159521+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:42:08.589118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.159433+0000","last_change":"2024-09-11T12:56:17.166196+0000","last_active":"2024-09-11T12:56:19.159433+0000","last_peered":"2024-09-11T12:56:19.159433+0000","last_clean":"2024-09-11T12:56:19.159433+0000","last_became_active":"2024-09-11T12:56:17.165828+0000","last_became_peered":"2024-09-11T12:56:17.165828+0000","last_unstale":"2024-09-11T12:56:19.159433+0000","last_undegraded":"2024-09-11T12:56:19.159433+0000","last_fullsized":"2024-09-11T12:56:19.159433+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:34:20.045407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.158095+0000","last_change":"2024-09-11T12:56:18.177890+0000","last_active":"2024-09-11T12:56:19.158095+0000","last_peered":"2024-09-11T12:56:19.158095+0000","last_clean":"2024-09-11T12:56:19.158095+0000","last_became_active":"2024-09-11T12:56:18.177529+0000","last_became_peered":"2024-09-11T12:56:18.177529+0000","last_unstale":"2024-09-11T12:56:19.158095+0000","last_undegraded":"2024-09-11T12:56:19.158095+0000","last_fullsized":"2024-09-11T12:56:19.158095+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:17.129869+0000","last_clean_scrub_stamp":"2024-09-11T12:56:17.129869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:32:33.818525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.160519+0000","last_change":"2024-09-11T12:56:17.156745+0000","last_active":"2024-09-11T12:56:19.160519+0000","last_peered":"2024-09-11T12:56:19.160519+0000","last_clean":"2024-09-11T12:56:19.160519+0000","last_became_active":"2024-09-11T12:56:17.156429+0000","last_became_peered":"2024-09-11T12:56:17.156429+0000","last_unstale":"2024-09-11T12:56:19.160519+0000","last_undegraded":"2024-09-11T12:56:19.160519+0000","last_fullsized":"2024-09-11T12:56:19.160519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:16.127369+0000","last_clean_scrub_stamp":"2024-09-11T12:56:16.127369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:57:54.969118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":352,"reported_epoch":140,"state":"active+clean","last_fresh":"2024-09-11T12:56:19.595820+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:19.595820+0000","last_peered":"2024-09-11T12:56:19.595820+0000","last_clean":"2024-09-11T12:56:19.595820+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:19.595820+0000","last_undegraded":"2024-09-11T12:56:19.595820+0000","last_fullsized":"2024-09-11T12:56:19.595820+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509525,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37652,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334188,"statfs":{"total":96636764160,"available":96598208512,"internally_reserved":0,"allocated":2359296,"data_stored":1356791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":770391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37076,"kb_used_data":1720,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334764,"statfs":{"total":96636764160,"available":96598798336,"internally_reserved":0,"allocated":1761280,"data_stored":770973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37084,"kb_used_data":1728,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334756,"statfs":{"total":96636764160,"available":96598790144,"internally_reserved":0,"allocated":1769472,"data_stored":772051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509525,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32908,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338932,"statfs":{"total":96636764160,"available":96603066368,"internally_reserved":0,"allocated":1757184,"data_stored":770403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509525,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32912,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338928,"statfs":{"total":96636764160,"available":96603062272,"internally_reserved":0,"allocated":1757184,"data_stored":770459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509525,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1757184,"data_stored":770403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542229,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33476,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338364,"statfs":{"total":96636764160,"available":96602484736,"internally_reserved":0,"allocated":2334720,"data_stored":1354437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:22.638 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:56:22.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 -- 172.21.15.78:0/2256661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b32d0 msgr2=0x7fd0ec0cf5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/2256661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b32d0 0x7fd0ec0cf5a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009bc0 tx=0x7fd0dc01cba0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 -- 172.21.15.78:0/2256661220 shutdown_connections 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/2256661220 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b5f10 0x7fd0ec15bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/2256661220 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b32d0 0x7fd0ec0cf5a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.822+0000 7fd0f1e00640 1 -- 172.21.15.78:0/2256661220 >> 172.21.15.78:0/2256661220 conn(0x7fd0ec0b1c50 msgr2=0x7fd0ec070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- 172.21.15.78:0/2256661220 shutdown_connections 2024-09-11T12:56:22.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- 172.21.15.78:0/2256661220 wait complete. 2024-09-11T12:56:22.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 Processor -- start 2024-09-11T12:56:22.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- start start 2024-09-11T12:56:22.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b32d0 0x7fd0ec148a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 0x7fd0ec14eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec152310 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd0ec152190 con 0x7fd0ec0b5f10 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd0ec152490 con 0x7fd0ec0b32d0 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43898/0 (socket says 172.21.15.78:43898) 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 -- 172.21.15.78:0/1957384578 learned_addr learned my addr 172.21.15.78:0/1957384578 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0eaa00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b32d0 0x7fd0ec148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0eb400640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 0x7fd0ec14eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 -- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b32d0 msgr2=0x7fd0ec148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b32d0 0x7fd0ec148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 -- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 msgr2=0x7fd0ec14eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 0x7fd0ec14eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 -- 172.21.15.78:0/1957384578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0ec149cb0 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0eb400640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 0x7fd0ec14eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0ea000640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c490 tx=0x7fd0d800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8016020 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0d8012940 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0ec149f90 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8012d90 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.826+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0ec148f90 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0ec0cf5a0 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fd0d8019020 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0e3400640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 0x7fd0cc074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:22.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fd0d80928a0 con 0x7fd0ec0b36b0 2024-09-11T12:56:22.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0eaa00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 0x7fd0cc074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:22.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.830+0000 7fd0eaa00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 0x7fd0cc074170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0af000 tx=0x7fd0dc01b3e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:22.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:22.834+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd0d8020020 con 0x7fd0ec0b36b0 2024-09-11T12:56:23.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.058+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd0ec15d000 con 0x7fd0ec0b36b0 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0e3400640 1 -- 172.21.15.78:0/1957384578 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7fd0d805dee0 con 0x7fd0ec0b36b0 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:56:23.065 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:56:23.066 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:56:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 msgr2=0x7fd0cc074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 0x7fd0cc074170 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0af000 tx=0x7fd0dc01b3e0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 msgr2=0x7fd0ec1491f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800c490 tx=0x7fd0d800c950 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 shutdown_connections 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd0cc071cb0 0x7fd0cc074170 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd0ec0b5f10 0x7fd0ec14eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd0ec0b36b0 0x7fd0ec1491f0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 --2- 172.21.15.78:0/1957384578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd0ec0b32d0 0x7fd0ec148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 >> 172.21.15.78:0/1957384578 conn(0x7fd0ec0b1c50 msgr2=0x7fd0ec06ff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 shutdown_connections 2024-09-11T12:56:23.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.062+0000 7fd0f1e00640 1 -- 172.21.15.78:0/1957384578 wait complete. 2024-09-11T12:56:23.087 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 -- 172.21.15.78:0/2349757528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc06fdf0 msgr2=0x7f3afc0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 --2- 172.21.15.78:0/2349757528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc06fdf0 0x7f3afc0a9d00 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f3ae8009840 tx=0x7f3ae801c820 comp rx=0 tx=0).stop 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 -- 172.21.15.78:0/2349757528 shutdown_connections 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 --2- 172.21.15.78:0/2349757528 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc0d0bf0 0x7f3afc0d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 --2- 172.21.15.78:0/2349757528 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc06fdf0 0x7f3afc0a9d00 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 -- 172.21.15.78:0/2349757528 >> 172.21.15.78:0/2349757528 conn(0x7f3afc074b90 msgr2=0x7f3afc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 -- 172.21.15.78:0/2349757528 shutdown_connections 2024-09-11T12:56:23.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 -- 172.21.15.78:0/2349757528 wait complete. 2024-09-11T12:56:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.226+0000 7f3b04400640 1 Processor -- start 2024-09-11T12:56:23.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- start start 2024-09-11T12:56:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc06fdf0 0x7f3afc06c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc0d0bf0 0x7f3afc0afe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc168e60 con 0x7f3afc0b14b0 2024-09-11T12:56:23.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3afc168ce0 con 0x7f3afc0d0bf0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3afc168fe0 con 0x7f3afc06fdf0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:43904/0 (socket says 172.21.15.78:43904) 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 -- 172.21.15.78:0/3774908699 learned_addr learned my addr 172.21.15.78:0/3774908699 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01a00640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc06fdf0 0x7f3afc06c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b02400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc0d0bf0 0x7f3afc0afe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 -- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc06fdf0 msgr2=0x7f3afc06c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc06fdf0 0x7f3afc06c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 -- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc0d0bf0 msgr2=0x7f3afc0afe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc0d0bf0 0x7f3afc0afe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 -- 172.21.15.78:0/3774908699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3afc06cb90 con 0x7f3afc0b14b0 2024-09-11T12:56:23.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b01000640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3ae801aed0 tx=0x7f3ae80a6780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8004140 con 0x7f3afc0b14b0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ae80043d0 con 0x7f3afc0b14b0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8005280 con 0x7f3afc0b14b0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3afc15ba30 con 0x7f3afc0b14b0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3afc15b750 con 0x7f3afc0b14b0 2024-09-11T12:56:23.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.230+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3afc0b2330 con 0x7f3afc0b14b0 2024-09-11T12:56:23.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.234+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f3ae80c2020 con 0x7f3afc0b14b0 2024-09-11T12:56:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.234+0000 7f3af9600640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 0x7f3adc074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.234+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f3ae812d630 con 0x7f3afc0b14b0 2024-09-11T12:56:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.234+0000 7f3b01a00640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 0x7f3adc074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:23.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.234+0000 7f3b01a00640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 0x7f3adc074170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3af0007fd0 tx=0x7f3af0007ea0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:23.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.238+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3ae8016410 con 0x7f3afc0b14b0 2024-09-11T12:56:23.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.438+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3afc069070 con 0x7f3afc0b14b0 2024-09-11T12:56:23.443 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.438+0000 7f3af9600640 1 -- 172.21.15.78:0/3774908699 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) ==== 95+0+5229 (secure 0 0 0) 0x7f3ae80f83c0 con 0x7f3afc0b14b0 2024-09-11T12:56:23.443 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:23.443 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":101,"btime":"2024-09-11T12:56:21:163184+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:23.443 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 101 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 msgr2=0x7f3adc074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 0x7f3adc074170 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3af0007fd0 tx=0x7f3af0007ea0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 msgr2=0x7f3afc13e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f3ae801aed0 tx=0x7f3ae80a6780 comp rx=0 tx=0).stop 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 shutdown_connections 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3adc071cb0 0x7f3adc074170 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3afc0d0bf0 0x7f3afc0afe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3afc0b14b0 0x7f3afc13e4f0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 --2- 172.21.15.78:0/3774908699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3afc06fdf0 0x7f3afc06c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:23.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 >> 172.21.15.78:0/3774908699 conn(0x7f3afc074b90 msgr2=0x7f3afc1502a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:23.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 shutdown_connections 2024-09-11T12:56:23.447 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:23.442+0000 7f3b04400640 1 -- 172.21.15.78:0/3774908699 wait complete. 2024-09-11T12:56:23.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:56:17.530187+0000', 'modified': '2024-09-11T12:56:20.551586+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8274}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8274': {'gid': 8274, 'name': 'c', 'rank': 0, 'incarnation': 97, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/835134630', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6834', 'nonce': 835134630}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 835134630}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8274, 'qdb_cluster': [8274]} 2024-09-11T12:56:23.868 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:56:23.868 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- 172.21.15.78:0/216490621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 msgr2=0x7f5138003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 --2- 172.21.15.78:0/216490621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5138003e30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f5144009ec0 tx=0x7f514401d5a0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f5153e00640 1 -- 172.21.15.78:0/216490621 reap_dead start 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- 172.21.15.78:0/216490621 shutdown_connections 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- 172.21.15.78:0/216490621 >> 172.21.15.78:0/216490621 conn(0x7f515406ca80 msgr2=0x7f51540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- 172.21.15.78:0/216490621 shutdown_connections 2024-09-11T12:56:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- 172.21.15.78:0/216490621 wait complete. 2024-09-11T12:56:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 Processor -- start 2024-09-11T12:56:24.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.010+0000 7f515a600640 1 -- start start 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 0x7f51540b13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51540b1930 0x7f51541592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f515406ecc0 con 0x7f5138003a10 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f515406eb40 con 0x7f51540c5300 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f515406ee40 con 0x7f51540b1930 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48534/0 (socket says 172.21.15.78:48534) 2024-09-11T12:56:24.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 -- 172.21.15.78:0/1358197352 learned_addr learned my addr 172.21.15.78:0/1358197352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 -- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51540b1930 msgr2=0x7f51541592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153e00640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51540b1930 0x7f51541592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5152a00640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 0x7f51540b13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51540b1930 0x7f51541592b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 -- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 msgr2=0x7f51540b13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 0x7f51540b13f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 -- 172.21.15.78:0/1358197352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51540c5080 con 0x7f5138003a10 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5152a00640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 0x7f51540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f5153400640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f51440af040 tx=0x7f5144009a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f514b400640 1 -- 172.21.15.78:0/1358197352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440b1070 con 0x7f5138003a10 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f514b400640 1 -- 172.21.15.78:0/1358197352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51440b2040 con 0x7f5138003a10 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51541fb2b0 con 0x7f5138003a10 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f514b400640 1 -- 172.21.15.78:0/1358197352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51440a8070 con 0x7f5138003a10 2024-09-11T12:56:24.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51541fb870 con 0x7f5138003a10 2024-09-11T12:56:24.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.014+0000 7f514b400640 1 -- 172.21.15.78:0/1358197352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f5144005040 con 0x7f5138003a10 2024-09-11T12:56:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f514b400640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 0x7f5130074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f514b400640 1 -- 172.21.15.78:0/1358197352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f514412e180 con 0x7f5138003a10 2024-09-11T12:56:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 0x7f5130074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 Processor -- start 2024-09-11T12:56:24.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- start start 2024-09-11T12:56:24.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 0x7f5130074170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f513c001870 tx=0x7f513c000ee0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a01b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a0165720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e29ffc2270 con 0x55e2a01b23c0 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e29ffb1770 con 0x55e2a01b1aa0 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55e29ffb57b0 con 0x55e2a0165340 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48536/0 (socket says 172.21.15.78:48536) 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 -- 172.21.15.78:0/2511908588 learned_addr learned my addr 172.21.15.78:0/2511908588 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5153400640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a01b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5153e00640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a0165720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 -- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 msgr2=0x55e2a0165720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a0165720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 -- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 msgr2=0x55e2a01b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a01b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 -- 172.21.15.78:0/2511908588 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29ff50f90 con 0x55e2a01b23c0 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5153400640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a01b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:24.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5152a00640 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f513c004440 tx=0x7f513c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a18f2734f60ade server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5143e00640 1 -- 172.21.15.78:0/2511908588 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c025070 con 0x55e2a01b23c0 2024-09-11T12:56:24.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5143e00640 1 -- 172.21.15.78:0/2511908588 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f513c00fd30 con 0x55e2a01b23c0 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f5143e00640 1 -- 172.21.15.78:0/2511908588 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c003de0 con 0x55e2a01b23c0 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- 172.21.15.78:0/2511908588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 msgr2=0x55e2a0164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f513c004440 tx=0x7f513c00d000 comp rx=0 tx=0).stop 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 -- 172.21.15.78:0/2511908588 shutdown_connections 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a0165720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164e00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.018+0000 7f515da24000 1 --2- 172.21.15.78:0/2511908588 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a01b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/2511908588 >> 172.21.15.78:0/2511908588 conn(0x55e2a0192b60 msgr2=0x55e2a0194f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/2511908588 shutdown_connections 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/2511908588 wait complete. 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- start start 2024-09-11T12:56:24.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 Processor -- start 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a0165340 0x55e2a0172d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a01b23c0 0x55e2a01737d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e2a0081290 con 0x55e2a01b1aa0 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e29ff5dab0 con 0x55e2a0165340 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55e29ff5b170 con 0x55e2a01b23c0 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 10 client.0 _ll_register_callbacks cb 0x55e2a0152830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48542/0 (socket says 172.21.15.78:48542) 2024-09-11T12:56:24.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 -- 172.21.15.78:0/819772826 learned_addr learned my addr 172.21.15.78:0/819772826 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5153400640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a0165340 0x55e2a0172d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5153e00640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a01b23c0 0x55e2a01737d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 -- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a01b23c0 msgr2=0x55e2a01737d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a01b23c0 0x55e2a01737d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 -- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a0165340 msgr2=0x55e2a0172d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a0165340 0x55e2a0172d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 -- 172.21.15.78:0/819772826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2a00366f0 con 0x55e2a01b1aa0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5152a00640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f513c005fd0 tx=0x7f513c004070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=163b2edb36163668 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c030060 con 0x55e2a01b1aa0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f513c02b670 con 0x55e2a01b1aa0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e29ff4af10 con 0x55e2a01b1aa0 2024-09-11T12:56:24.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f513c036050 con 0x55e2a01b1aa0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 10 client.8376 fetch_fsmap learned FSMap version 101 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55e2a000dc40 con 0x55e2a01b1aa0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55e29ff4af10 con 0x55e2a01b1aa0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7f513c039020 con 0x55e2a01b1aa0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 10 client.8376 fetch_fsmap finished waiting for FSMap version 101 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 10 client.8376 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 20 client.8376 populate_metadata read hostname 'smithi078' 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01741e0 0x55e2a01c3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 4 client.8376 mds_command: new command op to 8274 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.022+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 --> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e29ffc41c0 con 0x55e2a01741e0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5153400640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01741e0 0x55e2a01c3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5140c00640 10 client.8376 ms_handle_connect on v2:172.21.15.78:6834/835134630 2024-09-11T12:56:24.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5153400640 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01741e0 0x55e2a01c3b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff16eb8a773c36e server_cookie=12c408c23d2664a0 in_seq=0 out_seq=0 2024-09-11T12:56:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5140c00640 1 -- 172.21.15.78:0/819772826 <== mds.0 v2:172.21.15.78:6834/835134630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e29ffc41c0 con 0x55e2a01741e0 2024-09-11T12:56:24.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5140c00640 10 client.8376 handle_command_reply: tid=0 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 client.8376 shutdown 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 msgr2=0x55e2a0173290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f513c005fd0 tx=0x7f513c004070 comp rx=0 tx=0).stop 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 shutdown_connections 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01741e0 0x55e2a01c3b50 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a01b23c0 0x55e2a01737d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a0173290 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f5140c00640 0 client.8376 ms_handle_reset on v2:172.21.15.78:6834/835134630 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 --2- 172.21.15.78:0/819772826 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a0165340 0x55e2a0172d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 >> 172.21.15.78:0/819772826 conn(0x55e2a0192b60 msgr2=0x55e2a0164500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 shutdown_connections 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 1 -- 172.21.15.78:0/819772826 wait complete. 2024-09-11T12:56:24.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.026+0000 7f515da24000 20 client.8376 trim_cache size 0 max 16384 2024-09-11T12:56:24.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.042+0000 7f515da24000 1 Processor -- start 2024-09-11T12:56:24.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- start start 2024-09-11T12:56:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a0163ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5154203810 con 0x55e2a01b23c0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f51541f8400 con 0x55e2a01b1aa0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f51541f8a50 con 0x55e2a0165340 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48558/0 (socket says 172.21.15.78:48558) 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 -- 172.21.15.78:0/1059984462 learned_addr learned my addr 172.21.15.78:0/1059984462 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5152a00640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a0163ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153400640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 -- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 msgr2=0x55e2a01c73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 -- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 msgr2=0x55e2a0163ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a0163ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 -- 172.21.15.78:0/1059984462 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e29ff66e70 con 0x55e2a01b23c0 2024-09-11T12:56:24.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5152a00640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a0163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153400640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5153e00640 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f515406cf30 tx=0x7f514c00ae60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ade22b29000c67 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5143400640 1 -- 172.21.15.78:0/1059984462 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c00ed00 con 0x55e2a01b23c0 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5143400640 1 -- 172.21.15.78:0/1059984462 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f514c004030 con 0x55e2a01b23c0 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f5143400640 1 -- 172.21.15.78:0/1059984462 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514c004460 con 0x55e2a01b23c0 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- 172.21.15.78:0/1059984462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 msgr2=0x55e2a0164220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f515406cf30 tx=0x7f514c00ae60 comp rx=0 tx=0).stop 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- 172.21.15.78:0/1059984462 shutdown_connections 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b23c0 0x55e2a0164220 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b1aa0 0x55e2a0163ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 --2- 172.21.15.78:0/1059984462 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- 172.21.15.78:0/1059984462 >> 172.21.15.78:0/1059984462 conn(0x55e2a0192b60 msgr2=0x55e2a01635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- 172.21.15.78:0/1059984462 shutdown_connections 2024-09-11T12:56:24.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- 172.21.15.78:0/1059984462 wait complete. 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 -- start start 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.046+0000 7f515da24000 1 Processor -- start 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b23c0 0x55e2a01e1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55e29ffb9de0 con 0x55e2a01b1aa0 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55e29ffb4ac0 con 0x55e2a01b23c0 2024-09-11T12:56:24.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55e29ffc4ea0 con 0x55e2a0165340 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 10 client.0 _ll_register_callbacks cb 0x7f513c023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48566/0 (socket says 172.21.15.78:48566) 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 -- 172.21.15.78:0/2380205957 learned_addr learned my addr 172.21.15.78:0/2380205957 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5153e00640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b23c0 0x55e2a01e1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5153400640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 -- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 msgr2=0x55e2a01c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 -- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b23c0 msgr2=0x55e2a01e1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b23c0 0x55e2a01e1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 -- 172.21.15.78:0/2380205957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e29ff3bbf0 con 0x55e2a01b1aa0 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5153400640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:24.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5152a00640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f513c02be00 tx=0x7f513c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322a16e489e2424e server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c03f060 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f513c012880 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f514c004030 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) ==== 24+0+0 (secure 0 0 0) 0x7f513c045050 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 10 client.8382 fetch_fsmap learned FSMap version 101 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) -- 0x55e29ff57c60 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f514c004030 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 101) ==== 3502+0+0 (secure 0 0 0) 0x7f513c048020 con 0x55e2a01b1aa0 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 10 client.8382 fetch_fsmap finished waiting for FSMap version 101 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 10 client.8382 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 20 client.8382 populate_metadata read hostname 'smithi078' 2024-09-11T12:56:24.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01e2800 0x55e2a0225690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 4 client.8382 mds_command: new command op to 8274 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 --> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] -- command(tid 0: {"prefix": "status"}) -- 0x55e29ffbfc60 con 0x55e2a01e2800 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5153400640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01e2800 0x55e2a0225690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5153400640 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01e2800 0x55e2a0225690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=950b5c0c6565658a server_cookie=8fa165749b94500c in_seq=0 out_seq=0 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 10 client.8382 ms_handle_connect on v2:172.21.15.78:6834/835134630 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 1 -- 172.21.15.78:0/2380205957 <== mds.0 v2:172.21.15.78:6834/835134630 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x55e29ffbfc60 con 0x55e2a01e2800 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 10 client.8382 handle_command_reply: tid=0 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 client.8382 shutdown 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 msgr2=0x55e2a01c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f513c02be00 tx=0x7f513c02b8a0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 shutdown_connections 2024-09-11T12:56:24.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:6834/835134630,v1:172.21.15.78:6835/835134630] conn(0x55e2a01e2800 0x55e2a0225690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55e2a01b23c0 0x55e2a01e1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55e2a01b1aa0 0x55e2a01c68e0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 --2- 172.21.15.78:0/2380205957 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55e2a0165340 0x55e2a01c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f5140c00640 0 client.8382 ms_handle_reset on v2:172.21.15.78:6834/835134630 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 >> 172.21.15.78:0/2380205957 conn(0x55e2a0192b60 msgr2=0x55e2a0163df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 shutdown_connections 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 1 -- 172.21.15.78:0/2380205957 wait complete. 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.050+0000 7f515da24000 20 client.8382 trim_cache size 0 max 16384 2024-09-11T12:56:24.060 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8274, 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9170741060000003, 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 101, 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 140, 2024-09-11T12:56:24.061 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 140, 2024-09-11T12:56:24.062 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.403634137999999 2024-09-11T12:56:24.062 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:56:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 msgr2=0x7f5130074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 0x7f5130074170 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f513c001870 tx=0x7f513c000ee0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 msgr2=0x7f5154154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f51440af040 tx=0x7f5144009a90 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 shutdown_connections 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51540b1930 0x7f51541592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5130071cb0 0x7f5130074170 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51540c5300 0x7f51540b13f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 --2- 172.21.15.78:0/1358197352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5138003a10 0x7f5154154ab0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 >> 172.21.15.78:0/1358197352 conn(0x7f515406ca80 msgr2=0x7f5154156ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 shutdown_connections 2024-09-11T12:56:24.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.054+0000 7f515a600640 1 -- 172.21.15.78:0/1358197352 wait complete. 2024-09-11T12:56:24.075 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:56:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 -- 172.21.15.78:0/445632740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1108159fc0 msgr2=0x7f11080706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 --2- 172.21.15.78:0/445632740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1108159fc0 0x7f11080706f0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f110806cf90 tx=0x7f10f801dc70 comp rx=0 tx=0).stop 2024-09-11T12:56:24.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 -- 172.21.15.78:0/445632740 shutdown_connections 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 --2- 172.21.15.78:0/445632740 >> v2:172.21.15.157:3300/0 conn(0x7f11080cde50 0x7f11080b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 --2- 172.21.15.78:0/445632740 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108070c30 0x7f11080cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 --2- 172.21.15.78:0/445632740 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1108159fc0 0x7f11080706f0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 -- 172.21.15.78:0/445632740 >> 172.21.15.78:0/445632740 conn(0x7f110806ca80 msgr2=0x7f11080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 -- 172.21.15.78:0/445632740 shutdown_connections 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.214+0000 7f110ec00640 1 -- 172.21.15.78:0/445632740 wait complete. 2024-09-11T12:56:24.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 Processor -- start 2024-09-11T12:56:24.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- start start 2024-09-11T12:56:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108159fc0 0x7f110814ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11080ae880 0x7f11080aed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f110806f9a0 con 0x7f11080cde50 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f110806f820 con 0x7f11080ae880 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f110806fb20 con 0x7f1108159fc0 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48572/0 (socket says 172.21.15.78:48572) 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 -- 172.21.15.78:0/1716093315 learned_addr learned my addr 172.21.15.78:0/1716093315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110cc00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11080ae880 0x7f11080aed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107400640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108159fc0 0x7f110814ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 -- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108159fc0 msgr2=0x7f110814ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108159fc0 0x7f110814ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 -- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11080ae880 msgr2=0x7f11080aed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11080ae880 0x7f11080aed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 -- 172.21.15.78:0/1716093315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110814d800 con 0x7f11080cde50 2024-09-11T12:56:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f1107e00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000f90 tx=0x7f10f800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80b3040 con 0x7f11080cde50 2024-09-11T12:56:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10f8004d40 con 0x7f11080cde50 2024-09-11T12:56:24.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8004260 con 0x7f11080cde50 2024-09-11T12:56:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11080b0860 con 0x7f11080cde50 2024-09-11T12:56:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.218+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110814d520 con 0x7f11080cde50 2024-09-11T12:56:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110814d060 con 0x7f11080cde50 2024-09-11T12:56:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f10f80bf060 con 0x7f11080cde50 2024-09-11T12:56:24.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f10ffe00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 0x7f10e8074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f10f812c9a0 con 0x7f11080cde50 2024-09-11T12:56:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f1107400640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 0x7f10e8074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:24.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.222+0000 7f1107400640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 0x7f10e8074170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10f4005fd0 tx=0x7f10f4005e60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:24.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.226+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f110814d060 con 0x7f11080cde50 2024-09-11T12:56:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.430+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f11080b4db0 con 0x7f11080cde50 2024-09-11T12:56:24.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.430+0000 7f10ffe00640 1 -- 172.21.15.78:0/1716093315 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7f110814d800 con 0x7f11080cde50 2024-09-11T12:56:24.434 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 msgr2=0x7f10e8074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 0x7f10e8074170 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f10f4005fd0 tx=0x7f10f4005e60 comp rx=0 tx=0).stop 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 msgr2=0x7f1108145d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f10f8000f90 tx=0x7f10f800a470 comp rx=0 tx=0).stop 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 shutdown_connections 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f10e8071cb0 0x7f10e8074170 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f11080ae880 0x7f11080aed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1108159fc0 0x7f110814ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 --2- 172.21.15.78:0/1716093315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f11080cde50 0x7f1108145d80 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 >> 172.21.15.78:0/1716093315 conn(0x7f110806ca80 msgr2=0x7f1108151b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 shutdown_connections 2024-09-11T12:56:24.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24.434+0000 7f110ec00640 1 -- 172.21.15.78:0/1716093315 wait complete. 2024-09-11T12:56:24.454 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:56:24.454 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:56:24.455 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:56:24.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.455 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42050sec preferred_lft 42050sec 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:56:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.467 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:56:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:56:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:56:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:24.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.468 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:56:24.524 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:56:24.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.527 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:56:24.582 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:56:24.582 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:24.583 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:56:24.583 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:56:24.584 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:56:24.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:56:24.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:56:24.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:56:24.584 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:56:24.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:56:24.711 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:56:24.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.717 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:56:24.767 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:56:24.767 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:56:24.768 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:56:24.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.768 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:56:24.834 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:56:24.834 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:24.887 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:56:24.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:56:24.887 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:24.942 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:56:24.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:24.943 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:25.035 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:25.035 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:56:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:25.048 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:56:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:56:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:56:25.049 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:56:25.052 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:56:25.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.052 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:25.072 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:56:25.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:56:25.173 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:56:25.180 INFO:teuthology.orchestra.run.smithi078.stdout:client8391 2024-09-11T12:56:25.180 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:25.180 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:25.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:56:25.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 -- 172.21.15.78:0/213254227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4b60 msgr2=0x7fc46c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:25.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 --2- 172.21.15.78:0/213254227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4b60 0x7fc46c15ba90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fc464009c80 tx=0x7fc46401cd60 comp rx=0 tx=0).stop 2024-09-11T12:56:25.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 -- 172.21.15.78:0/213254227 shutdown_connections 2024-09-11T12:56:25.371 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 --2- 172.21.15.78:0/213254227 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4b60 0x7fc46c15ba90 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 --2- 172.21.15.78:0/213254227 >> v2:172.21.15.157:3300/0 conn(0x7fc46c0ccb20 0x7fc46c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 --2- 172.21.15.78:0/213254227 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0b4780 0x7fc46c073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 -- 172.21.15.78:0/213254227 >> 172.21.15.78:0/213254227 conn(0x7fc46c06ca80 msgr2=0x7fc46c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 -- 172.21.15.78:0/213254227 shutdown_connections 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.366+0000 7fc471400640 1 -- 172.21.15.78:0/213254227 wait complete. 2024-09-11T12:56:25.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 Processor -- start 2024-09-11T12:56:25.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- start start 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 0x7fc46c1504c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 0x7fc46c150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c06f190 con 0x7fc46c0b4780 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc46c06f010 con 0x7fc46c0b4b60 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc46c06f310 con 0x7fc46c0ccb20 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48600/0 (socket says 172.21.15.78:48600) 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 -- 172.21.15.78:0/790608615 learned_addr learned my addr 172.21.15.78:0/790608615 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc469e00640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 0x7fc46c1504c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:25.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46b200640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 0x7fc46c150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 -- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 msgr2=0x7fc46c150a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 0x7fc46c150a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 -- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 msgr2=0x7fc46c1504c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 0x7fc46c1504c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 -- 172.21.15.78:0/790608615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46c1578f0 con 0x7fc46c0b4780 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc469e00640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 0x7fc46c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46b200640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 0x7fc46c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc46a800640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9f0 tx=0x7fc45c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c00ccc0 con 0x7fc46c0b4780 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc45c00ce60 con 0x7fc46c0b4780 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c005280 con 0x7fc46c0b4780 2024-09-11T12:56:25.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46c1f7600 con 0x7fc46c0b4780 2024-09-11T12:56:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.370+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc46c1f72d0 con 0x7fc46c0b4780 2024-09-11T12:56:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc46c073f70 con 0x7fc46c0b4780 2024-09-11T12:56:25.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc45c005420 con 0x7fc46c0b4780 2024-09-11T12:56:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc462a00640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 0x7fc44c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc45c093550 con 0x7fc46c0b4780 2024-09-11T12:56:25.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc469e00640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 0x7fc44c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:25.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.374+0000 7fc469e00640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 0x7fc44c074170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1551b0 tx=0x7fc4580091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:25.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.378+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc45c05e970 con 0x7fc46c0b4780 2024-09-11T12:56:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc46c0720e0 con 0x7fc46c0b4780 2024-09-11T12:56:25.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc462a00640 1 -- 172.21.15.78:0/790608615 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) ==== 53+0+73 (secure 0 0 0) 0x7fc45c05e270 con 0x7fc46c0b4780 2024-09-11T12:56:25.584 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:56:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 msgr2=0x7fc44c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 0x7fc44c074170 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc46c1551b0 tx=0x7fc4580091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 msgr2=0x7fc46c14d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:25.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00e9f0 tx=0x7fc45c00eeb0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 shutdown_connections 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc46c0ccb20 0x7fc46c150a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc44c071cb0 0x7fc44c074170 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc46c0b4b60 0x7fc46c1504c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 --2- 172.21.15.78:0/790608615 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc46c0b4780 0x7fc46c14d780 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 >> 172.21.15.78:0/790608615 conn(0x7fc46c06ca80 msgr2=0x7fc46c15b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 shutdown_connections 2024-09-11T12:56:25.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:25.582+0000 7fc471400640 1 -- 172.21.15.78:0/790608615 wait complete. 2024-09-11T12:56:25.606 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:56:25.607 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:56:25.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.607 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:56:25.617 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.618 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42058sec preferred_lft 42058sec 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.619 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:56:25.620 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:56:25.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.621 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:56:25.682 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:56:25.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.684 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:56:25.742 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:56:25.743 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:56:25.744 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:56:25.842 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:56:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:56:25.885 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:56:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:56:25.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.892 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:56:25.901 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:56:25.901 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:56:25.902 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:56:25.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:25.902 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:56:25.962 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:56:25.962 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:26.014 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:56:26.014 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:56:26.014 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:26.066 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:56:26.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:26.066 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:26.156 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:26.156 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:56:26.170 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:56:26.170 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:56:26.170 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:56:26.170 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:56:26.170 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:56:26.175 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:56:26.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:26.175 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:26.195 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:56:26.288 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:56:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:56:26.291 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:56:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:56:26.297 INFO:teuthology.orchestra.run.smithi157.stdout:client6980 2024-09-11T12:56:26.298 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:26.298 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:26.306 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:56:26.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 -- 172.21.15.78:0/2454306716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040b1370 msgr2=0x7f51040b2e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:26.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 --2- 172.21.15.78:0/2454306716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040b1370 0x7f51040b2e60 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f50ec009c80 tx=0x7f50ec01f5a0 comp rx=0 tx=0).stop 2024-09-11T12:56:26.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 -- 172.21.15.78:0/2454306716 shutdown_connections 2024-09-11T12:56:26.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 --2- 172.21.15.78:0/2454306716 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040b1370 0x7f51040b2e60 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:26.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 --2- 172.21.15.78:0/2454306716 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b0f90 0x7f510415c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:26.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 -- 172.21.15.78:0/2454306716 >> 172.21.15.78:0/2454306716 conn(0x7f51040b2a50 msgr2=0x7f5104070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:26.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 -- 172.21.15.78:0/2454306716 shutdown_connections 2024-09-11T12:56:26.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.450+0000 7f5109600640 1 -- 172.21.15.78:0/2454306716 wait complete. 2024-09-11T12:56:26.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 Processor -- start 2024-09-11T12:56:26.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- start start 2024-09-11T12:56:26.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51040b0f90 0x7f5104155bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b1370 0x7f5104148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f51041528b0 con 0x7f51040cfa30 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5104152730 con 0x7f51040b0f90 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5104152a30 con 0x7f51040b1370 2024-09-11T12:56:26.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:26.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48626/0 (socket says 172.21.15.78:48626) 2024-09-11T12:56:26.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 -- 172.21.15.78:0/1281951904 learned_addr learned my addr 172.21.15.78:0/1281951904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5102000640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b1370 0x7f5104148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5102a00640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51040b0f90 0x7f5104155bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 -- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b1370 msgr2=0x7f5104148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b1370 0x7f5104148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 -- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51040b0f90 msgr2=0x7f5104155bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51040b0f90 0x7f5104155bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 -- 172.21.15.78:0/1281951904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51041493f0 con 0x7f51040cfa30 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5103400640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c3c0 tx=0x7f50fc00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc017070 con 0x7f51040cfa30 2024-09-11T12:56:26.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50fc003600 con 0x7f51040cfa30 2024-09-11T12:56:26.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003a90 con 0x7f51040cfa30 2024-09-11T12:56:26.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510406e050 con 0x7f51040cfa30 2024-09-11T12:56:26.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.454+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5104153f30 con 0x7f51040cfa30 2024-09-11T12:56:26.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510415c950 con 0x7f51040cfa30 2024-09-11T12:56:26.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f50fc01a070 con 0x7f51040cfa30 2024-09-11T12:56:26.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f50faa00640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 0x7f50e0074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:26.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f50fc091c70 con 0x7f51040cfa30 2024-09-11T12:56:26.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f5102a00640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 0x7f50e0074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:26.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.458+0000 7f5102a00640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 0x7f50e0074170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009be0 tx=0x7f50f40091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:26.467 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.462+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f51041493f0 con 0x7f51040cfa30 2024-09-11T12:56:26.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:26.662+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f510406e220 con 0x7f51040cfa30 2024-09-11T12:56:27.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f50faa00640 1 -- 172.21.15.78:0/1281951904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v102) ==== 136+0+0 (secure 0 0 0) 0x7f50fc05cb10 con 0x7f51040cfa30 2024-09-11T12:56:27.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 msgr2=0x7f50e0074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 0x7f50e0074170 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009be0 tx=0x7f50f40091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 msgr2=0x7f5104154730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c3c0 tx=0x7f50fc00c7c0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 shutdown_connections 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51040cfa30 0x7f5104154730 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51040b1370 0x7f5104148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e0071cb0 0x7f50e0074170 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 --2- 172.21.15.78:0/1281951904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51040b0f90 0x7f5104155bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 >> 172.21.15.78:0/1281951904 conn(0x7f51040b2a50 msgr2=0x7f51040aa740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 shutdown_connections 2024-09-11T12:56:27.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.186+0000 7f5109600640 1 -- 172.21.15.78:0/1281951904 wait complete. 2024-09-11T12:56:27.213 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:56:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 -- 172.21.15.78:0/3273788823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 msgr2=0x7f1038004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 --2- 172.21.15.78:0/3273788823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1038004c90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f103c009b40 tx=0x7f103c01d250 comp rx=0 tx=0).stop 2024-09-11T12:56:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 -- 172.21.15.78:0/3273788823 shutdown_connections 2024-09-11T12:56:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 --2- 172.21.15.78:0/3273788823 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1038004c90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 -- 172.21.15.78:0/3273788823 >> 172.21.15.78:0/3273788823 conn(0x7f105406ca80 msgr2=0x7f10540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 -- 172.21.15.78:0/3273788823 shutdown_connections 2024-09-11T12:56:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.354+0000 7f1059e00640 1 -- 172.21.15.78:0/3273788823 wait complete. 2024-09-11T12:56:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 Processor -- start 2024-09-11T12:56:27.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- start start 2024-09-11T12:56:27.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f105414f6b0 0x7f1054071d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 0x7f10540722a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f10540747a0 con 0x7f1038004810 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1054074620 con 0x7f105414fb30 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f105406fc50 con 0x7f105414f6b0 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48652/0 (socket says 172.21.15.78:48652) 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 -- 172.21.15.78:0/1863546951 learned_addr learned my addr 172.21.15.78:0/1863546951 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052000640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f105414f6b0 0x7f1054071d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1053400640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 0x7f10540722a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 -- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f105414f6b0 msgr2=0x7f1054071d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f105414f6b0 0x7f1054071d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 -- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 msgr2=0x7f10540722a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 0x7f10540722a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 -- 172.21.15.78:0/1863546951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f105414de10 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1053400640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 0x7f10540722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1052a00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f104400e310 tx=0x7f104400e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044003b90 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1044017020 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044006370 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f105414e0f0 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f105414e6b0 con 0x7f1038004810 2024-09-11T12:56:27.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.358+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f105415e5b0 con 0x7f1038004810 2024-09-11T12:56:27.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.362+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f104400d070 con 0x7f1038004810 2024-09-11T12:56:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.362+0000 7f104b400640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 0x7f1030074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.362+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f1044096520 con 0x7f1038004810 2024-09-11T12:56:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.362+0000 7f1052000640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 0x7f1030074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.362+0000 7f1052000640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 0x7f1030074170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f103c0ad000 tx=0x7f103c019ff0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:27.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.366+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1044021020 con 0x7f1038004810 2024-09-11T12:56:27.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f10540aa110 con 0x7f1030071cb0 2024-09-11T12:56:27.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f104b400640 1 -- 172.21.15.78:0/1863546951 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f10540aa110 con 0x7f1030071cb0 2024-09-11T12:56:27.557 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:56:27.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:56:27.560 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:27.560 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 msgr2=0x7f1030074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 0x7f1030074170 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f103c0ad000 tx=0x7f103c019ff0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 msgr2=0x7f1054148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f104400e310 tx=0x7f104400e7d0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 shutdown_connections 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1030071cb0 0x7f1030074170 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f105414fb30 0x7f10540722a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f105414f6b0 0x7f1054071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 --2- 172.21.15.78:0/1863546951 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1038004810 0x7f1054148730 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 >> 172.21.15.78:0/1863546951 conn(0x7f105406ca80 msgr2=0x7f105406f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:27.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 shutdown_connections 2024-09-11T12:56:27.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.554+0000 7f1059e00640 1 -- 172.21.15.78:0/1863546951 wait complete. 2024-09-11T12:56:27.579 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.718+0000 7f373da00640 1 -- 172.21.15.78:0/2885712746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 msgr2=0x7f37380b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.718+0000 7f373da00640 1 --2- 172.21.15.78:0/2885712746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b1a10 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f37380af400 tx=0x7f372801c5e0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- 172.21.15.78:0/2885712746 shutdown_connections 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- 172.21.15.78:0/2885712746 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 0x7f373815e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- 172.21.15.78:0/2885712746 >> v2:172.21.15.157:3300/0 conn(0x7f37380b1f50 0x7f37380cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- 172.21.15.78:0/2885712746 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b1a10 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- 172.21.15.78:0/2885712746 >> 172.21.15.78:0/2885712746 conn(0x7f3738071dd0 msgr2=0x7f37380721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- 172.21.15.78:0/2885712746 shutdown_connections 2024-09-11T12:56:27.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- 172.21.15.78:0/2885712746 wait complete. 2024-09-11T12:56:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 Processor -- start 2024-09-11T12:56:27.725 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- start start 2024-09-11T12:56:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37380b1f50 0x7f3738147610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 0x7f37380b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.727 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f37380700d0 con 0x7f37381518b0 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f373806ff50 con 0x7f37380b1f50 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f373da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3738070250 con 0x7f37380ce150 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48670/0 (socket says 172.21.15.78:48670) 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 -- 172.21.15.78:0/3586323789 learned_addr learned my addr 172.21.15.78:0/3586323789 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3736000640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 0x7f37380b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3736a00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37380b1f50 0x7f3738147610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 -- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 msgr2=0x7f37380b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 0x7f37380b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.728 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 -- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37380b1f50 msgr2=0x7f3738147610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37380b1f50 0x7f3738147610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 -- 172.21.15.78:0/3586323789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3738153460 con 0x7f37381518b0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f3737400640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f373000e990 tx=0x7f373000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730004d30 con 0x7f37381518b0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.722+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f373000cb00 con 0x7f37381518b0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3738153740 con 0x7f37381518b0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3730005020 con 0x7f37381518b0 2024-09-11T12:56:27.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3738157000 con 0x7f37381518b0 2024-09-11T12:56:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f373815e4d0 con 0x7f37381518b0 2024-09-11T12:56:27.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f37300051c0 con 0x7f37381518b0 2024-09-11T12:56:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f372ea00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 0x7f37180741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f37300931b0 con 0x7f37381518b0 2024-09-11T12:56:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.726+0000 7f3736a00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 0x7f37180741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:27.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.730+0000 7f3736a00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 0x7f37180741d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f37280af000 tx=0x7f37280b4040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:27.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.734+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3730016030 con 0x7f37381518b0 2024-09-11T12:56:27.902 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.898+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3738153910 con 0x7f37381518b0 2024-09-11T12:56:27.903 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.898+0000 7f372ea00640 1 -- 172.21.15.78:0/3586323789 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f373005df80 con 0x7f37381518b0 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 msgr2=0x7f37180741d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 0x7f37180741d0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f37280af000 tx=0x7f37280b4040 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 msgr2=0x7f37380b0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f373000e990 tx=0x7f373000ee50 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 shutdown_connections 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37381518b0 0x7f37380b0f80 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37380ce150 0x7f37380b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3718071d10 0x7f37180741d0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 --2- 172.21.15.78:0/3586323789 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37380b1f50 0x7f3738147610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:27.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 >> 172.21.15.78:0/3586323789 conn(0x7f3738071dd0 msgr2=0x7f373815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 shutdown_connections 2024-09-11T12:56:27.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:27.902+0000 7f373da00640 1 -- 172.21.15.78:0/3586323789 wait complete. 2024-09-11T12:56:27.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:27.924 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol11340) 2024-09-11T12:56:27.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:27.951 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp71280/subvol60711) 2024-09-11T12:56:28.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol11340 2024-09-11T12:56:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- 172.21.15.78:0/3438462778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 msgr2=0x7f66740a9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 --2- 172.21.15.78:0/3438462778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66740a9460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6664009840 tx=0x7f666401c7f0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- 172.21.15.78:0/3438462778 shutdown_connections 2024-09-11T12:56:28.201 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 --2- 172.21.15.78:0/3438462778 >> v2:172.21.15.157:3300/0 conn(0x7f66740702a0 0x7f6674070700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 --2- 172.21.15.78:0/3438462778 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66740a9460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- 172.21.15.78:0/3438462778 >> 172.21.15.78:0/3438462778 conn(0x7f6674075610 msgr2=0x7f6674075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- 172.21.15.78:0/3438462778 shutdown_connections 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- 172.21.15.78:0/3438462778 wait complete. 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 Processor -- start 2024-09-11T12:56:28.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.198+0000 7f667aa00640 1 -- start start 2024-09-11T12:56:28.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66741514f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667415c990 0x7f6674151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.204 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6674168a80 con 0x7f66740702a0 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6674168900 con 0x7f667415c990 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6674168c00 con 0x7f66740b1400 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48672/0 (socket says 172.21.15.78:48672) 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 -- 172.21.15.78:0/3554852328 learned_addr learned my addr 172.21.15.78:0/3554852328 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f666be00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66741514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6678a00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667415c990 0x7f6674151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 -- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 msgr2=0x7f66741514f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66741514f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 -- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667415c990 msgr2=0x7f6674151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667415c990 0x7f6674151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 -- 172.21.15.78:0/3554852328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f667406c980 con 0x7f66740702a0 2024-09-11T12:56:28.205 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6673e00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6664009790 tx=0x7f66640a5800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:28.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640bb070 con 0x7f66740702a0 2024-09-11T12:56:28.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6664003600 con 0x7f66740702a0 2024-09-11T12:56:28.206 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f667406cc60 con 0x7f66740702a0 2024-09-11T12:56:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66640b2040 con 0x7f66740702a0 2024-09-11T12:56:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f667413ea50 con 0x7f66740702a0 2024-09-11T12:56:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6674070700 con 0x7f66740702a0 2024-09-11T12:56:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.202+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f66640be070 con 0x7f66740702a0 2024-09-11T12:56:28.207 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.206+0000 7f6670c00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 0x7f6650074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.206+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f666412c230 con 0x7f66740702a0 2024-09-11T12:56:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.206+0000 7f666be00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 0x7f6650074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.206+0000 7f666be00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 0x7f6650074170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66740d1540 tx=0x7f665c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:28.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.210+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6664016410 con 0x7f66740702a0 2024-09-11T12:56:28.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.402+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol11340", "target": ["mon-mgr", ""]}) -- 0x7f66741651f0 con 0x7f6650071cb0 2024-09-11T12:56:28.411 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:28.406+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:28.411 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:28.406+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:28.411 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:28.406+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:28.411 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:28.406+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:28.411 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:28.406+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:28.438 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.434+0000 7f6670c00640 1 -- 172.21.15.78:0/3554852328 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7f66741651f0 con 0x7f6650071cb0 2024-09-11T12:56:28.439 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol11340 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 msgr2=0x7f6650074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 0x7f6650074170 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f66740d1540 tx=0x7f665c009040 comp rx=0 tx=0).stop 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 msgr2=0x7f667413e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f6664009790 tx=0x7f66640a5800 comp rx=0 tx=0).stop 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 shutdown_connections 2024-09-11T12:56:28.441 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6650071cb0 0x7f6650074170 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f667415c990 0x7f6674151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f66740b1400 0x7f66741514f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 --2- 172.21.15.78:0/3554852328 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f66740702a0 0x7f667413e510 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 >> 172.21.15.78:0/3554852328 conn(0x7f6674075610 msgr2=0x7f66740b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 shutdown_connections 2024-09-11T12:56:28.442 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.438+0000 7f667aa00640 1 -- 172.21.15.78:0/3554852328 wait complete. 2024-09-11T12:56:28.460 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol60711 subvol_grp71280 2024-09-11T12:56:28.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- 172.21.15.78:0/2770982153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c071150 msgr2=0x7f5c6c073670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 --2- 172.21.15.78:0/2770982153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c071150 0x7f5c6c073670 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009840 tx=0x7f5c5801c820 comp rx=0 tx=0).stop 2024-09-11T12:56:28.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- 172.21.15.78:0/2770982153 shutdown_connections 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 --2- 172.21.15.78:0/2770982153 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 0x7f5c6c0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 --2- 172.21.15.78:0/2770982153 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c071150 0x7f5c6c073670 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- 172.21.15.78:0/2770982153 >> 172.21.15.78:0/2770982153 conn(0x7f5c6c0cd190 msgr2=0x7f5c6c0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- 172.21.15.78:0/2770982153 shutdown_connections 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- 172.21.15.78:0/2770982153 wait complete. 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 Processor -- start 2024-09-11T12:56:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.602+0000 7f5c71400640 1 -- start start 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c071150 0x7f5c6c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 0x7f5c6c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1689c0 con 0x7f5c6c15b910 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c0c6ee0 con 0x7f5c6c071150 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5c6c168b40 con 0x7f5c6c073bb0 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48730/0 (socket says 172.21.15.78:48730) 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 -- 172.21.15.78:0/6530965 learned_addr learned my addr 172.21.15.78:0/6530965 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6a800640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c071150 0x7f5c6c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c69e00640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 0x7f5c6c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 -- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 msgr2=0x7f5c6c1514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 0x7f5c6c1514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 -- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c071150 msgr2=0x7f5c6c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c071150 0x7f5c6c151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 -- 172.21.15.78:0/6530965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c0caa20 con 0x7f5c6c15b910 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c6b200640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d980 tx=0x7f5c6400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64017030 con 0x7f5c6c15b910 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c64009a30 con 0x7f5c6c15b910 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c0cad00 con 0x7f5c6c15b910 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004020 con 0x7f5c6c15b910 2024-09-11T12:56:28.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c0cb2c0 con 0x7f5c6c15b910 2024-09-11T12:56:28.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c6c073670 con 0x7f5c6c15b910 2024-09-11T12:56:28.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.606+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7f5c6401a070 con 0x7f5c6c15b910 2024-09-11T12:56:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.610+0000 7f5c62a00640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 0x7f5c4c074170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.610+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f5c64092bd0 con 0x7f5c6c15b910 2024-09-11T12:56:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.610+0000 7f5c6a800640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 0x7f5c4c074170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.612 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.610+0000 7f5c6a800640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 0x7f5c4c074170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c002100 tx=0x7f5c5c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:28.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.614+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5c6405e040 con 0x7f5c6c15b910 2024-09-11T12:56:28.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.806+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol60711", "group_name": "subvol_grp71280", "target": ["mon-mgr", ""]}) -- 0x7f5c6c0cd8b0 con 0x7f5c4c071cb0 2024-09-11T12:56:28.826 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.822+0000 7f5c62a00640 1 -- 172.21.15.78:0/6530965 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5c6c0cd8b0 con 0x7f5c4c071cb0 2024-09-11T12:56:28.826 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp71280/subvol60711 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 msgr2=0x7f5c4c074170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 0x7f5c4c074170 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c002100 tx=0x7f5c5c009040 comp rx=0 tx=0).stop 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 msgr2=0x7f5c6c164f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d980 tx=0x7f5c6400de40 comp rx=0 tx=0).stop 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 shutdown_connections 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5c6c15b910 0x7f5c6c164f20 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5c6c073bb0 0x7f5c6c1514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5c4c071cb0 0x7f5c4c074170 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 --2- 172.21.15.78:0/6530965 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5c6c071150 0x7f5c6c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 >> 172.21.15.78:0/6530965 conn(0x7f5c6c0cd190 msgr2=0x7f5c6c165f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 shutdown_connections 2024-09-11T12:56:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.826+0000 7f5c71400640 1 -- 172.21.15.78:0/6530965 wait complete. 2024-09-11T12:56:28.848 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol11340 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- 172.21.15.78:0/539122986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40cd770 msgr2=0x7fc3d40cdba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- 172.21.15.78:0/539122986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40cd770 0x7fc3d40cdba0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7fc3d406d1a0 tx=0x7fc3cc01c8c0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- 172.21.15.78:0/539122986 shutdown_connections 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- 172.21.15.78:0/539122986 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40cd770 0x7fc3d40cdba0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- 172.21.15.78:0/539122986 >> v2:172.21.15.157:3300/0 conn(0x7fc3d40b3040 0x7fc3d40cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- 172.21.15.78:0/539122986 >> 172.21.15.78:0/539122986 conn(0x7fc3d4071dd0 msgr2=0x7fc3d40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:28.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- 172.21.15.78:0/539122986 shutdown_connections 2024-09-11T12:56:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- 172.21.15.78:0/539122986 wait complete. 2024-09-11T12:56:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 Processor -- start 2024-09-11T12:56:28.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 -- start start 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3d40b3420 0x7fc3d41f7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.990+0000 7fc3da600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3d40cd770 0x7fc3d41404a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d40af0a0 con 0x7fc3d40b3040 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc3d40aef20 con 0x7fc3d40b3420 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc3d40af220 con 0x7fc3d40cd770 2024-09-11T12:56:28.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48752/0 (socket says 172.21.15.78:48752) 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 -- 172.21.15.78:0/2159609331 learned_addr learned my addr 172.21.15.78:0/2159609331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d2a00640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3d40b3420 0x7fc3d41f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3e00640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3d40cd770 0x7fc3d41404a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 -- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3d40cd770 msgr2=0x7fc3d41404a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3d40cd770 0x7fc3d41404a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 -- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3d40b3420 msgr2=0x7fc3d41f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3d40b3420 0x7fc3d41f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 -- 172.21.15.78:0/2159609331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3d40aa3a0 con 0x7fc3d40b3040 2024-09-11T12:56:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3d3400640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009fd0 tx=0x7fc3c400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011470 con 0x7fc3d40b3040 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc3c4011700 con 0x7fc3d40b3040 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4011b30 con 0x7fc3d40b3040 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3d40aa680 con 0x7fc3d40b3040 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3d40aac40 con 0x7fc3d40b3040 2024-09-11T12:56:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3d40cf6e0 con 0x7fc3d40b3040 2024-09-11T12:56:28.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.994+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 21) ==== 95493+0+0 (secure 0 0 0) 0x7fc3c4011cd0 con 0x7fc3d40b3040 2024-09-11T12:56:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.998+0000 7fc3cb400640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 0x7fc3b40741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.998+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc3c40935d0 con 0x7fc3d40b3040 2024-09-11T12:56:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.998+0000 7fc3d2a00640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 0x7fc3b40741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:29.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:28.998+0000 7fc3d2a00640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 0x7fc3b40741a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4151720 tx=0x7fc3c0007e40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.002+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc3c4016030 con 0x7fc3d40b3040 2024-09-11T12:56:29.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.190+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol11340", "target": ["mon-mgr", ""]}) -- 0x7fc3d40722e0 con 0x7fc3b4071ce0 2024-09-11T12:56:29.200 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3cb400640 1 -- 172.21.15.78:0/2159609331 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fc3d40722e0 con 0x7fc3b4071ce0 2024-09-11T12:56:29.200 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol11340 2024-09-11T12:56:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 msgr2=0x7fc3b40741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 0x7fc3b40741a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc3d4151720 tx=0x7fc3c0007e40 comp rx=0 tx=0).stop 2024-09-11T12:56:29.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 msgr2=0x7fc3d4142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4009fd0 tx=0x7fc3c400c270 comp rx=0 tx=0).stop 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 shutdown_connections 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc3d40cd770 0x7fc3d41404a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc3b4071ce0 0x7fc3b40741a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc3d40b3420 0x7fc3d41f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 --2- 172.21.15.78:0/2159609331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc3d40b3040 0x7fc3d4142bf0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 >> 172.21.15.78:0/2159609331 conn(0x7fc3d4071dd0 msgr2=0x7fc3d41433f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 shutdown_connections 2024-09-11T12:56:29.203 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:29.198+0000 7fc3da600640 1 -- 172.21.15.78:0/2159609331 wait complete. 2024-09-11T12:56:29.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:29.221 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/cc62378de3b0884d030c00895e2284ae.meta) 2024-09-11T12:56:33.664 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:56:33.664 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:56:33.665 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:56:33.665 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol11340 2024-09-11T12:56:33.665 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:56:33.665 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:33.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol60711 subvol_grp71280 2024-09-11T12:56:33.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 -- 172.21.15.78:0/2941189261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 msgr2=0x7fa4700d0e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:33.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 --2- 172.21.15.78:0/2941189261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa4700d0e00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7fa460009c80 tx=0x7fa46001cd10 comp rx=0 tx=0).stop 2024-09-11T12:56:33.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 -- 172.21.15.78:0/2941189261 shutdown_connections 2024-09-11T12:56:33.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 --2- 172.21.15.78:0/2941189261 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 0x7fa4700d1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:33.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 --2- 172.21.15.78:0/2941189261 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa4700d0e00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:33.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 -- 172.21.15.78:0/2941189261 >> 172.21.15.78:0/2941189261 conn(0x7fa470072e30 msgr2=0x7fa4700ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:33.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 -- 172.21.15.78:0/2941189261 shutdown_connections 2024-09-11T12:56:33.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.818+0000 7fa475a00640 1 -- 172.21.15.78:0/2941189261 wait complete. 2024-09-11T12:56:33.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 Processor -- start 2024-09-11T12:56:33.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- start start 2024-09-11T12:56:33.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:33.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4700b4520 0x7fa470155fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:33.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 0x7fa47015ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:33.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa470168b20 con 0x7fa470070b10 2024-09-11T12:56:33.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa4701689a0 con 0x7fa4700b4520 2024-09-11T12:56:33.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa470168ca0 con 0x7fa4700b8080 2024-09-11T12:56:33.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:33.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48766/0 (socket says 172.21.15.78:48766) 2024-09-11T12:56:33.830 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 -- 172.21.15.78:0/1992136208 learned_addr learned my addr 172.21.15.78:0/1992136208 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46e000640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4700b4520 0x7fa470155fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46f400640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 0x7fa47015ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 -- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 msgr2=0x7fa47015ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 0x7fa47015ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 -- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4700b4520 msgr2=0x7fa470155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:33.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4700b4520 0x7fa470155fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:33.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 -- 172.21.15.78:0/1992136208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa470154a90 con 0x7fa470070b10 2024-09-11T12:56:33.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa46ea00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa460009bd0 tx=0x7fa4600a5af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:33.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600b2040 con 0x7fa470070b10 2024-09-11T12:56:33.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4600042c0 con 0x7fa470070b10 2024-09-11T12:56:33.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa470154d70 con 0x7fa470070b10 2024-09-11T12:56:33.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4600046f0 con 0x7fa470070b10 2024-09-11T12:56:33.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa470155330 con 0x7fa470070b10 2024-09-11T12:56:33.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.822+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4700b7970 con 0x7fa470070b10 2024-09-11T12:56:33.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.826+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7fa4600ad440 con 0x7fa470070b10 2024-09-11T12:56:33.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.826+0000 7fa466a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 0x7fa450074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:33.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.826+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fa46012d2a0 con 0x7fa470070b10 2024-09-11T12:56:33.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.826+0000 7fa46e000640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 0x7fa450074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:33.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.826+0000 7fa46e000640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 0x7fa450074200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4701faaa0 tx=0x7fa45c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:33.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:33.830+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa460016990 con 0x7fa470070b10 2024-09-11T12:56:34.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.018+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol60711", "group_name": "subvol_grp71280", "target": ["mon-mgr", ""]}) -- 0x7fa4700d1570 con 0x7fa450071d40 2024-09-11T12:56:34.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa466a00640 1 -- 172.21.15.78:0/1992136208 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa4700d1570 con 0x7fa450071d40 2024-09-11T12:56:34.027 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp71280/subvol60711 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 msgr2=0x7fa450074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 0x7fa450074200 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa4701faaa0 tx=0x7fa45c009040 comp rx=0 tx=0).stop 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 msgr2=0x7fa47014dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7fa460009bd0 tx=0x7fa4600a5af0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 shutdown_connections 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa4700b8080 0x7fa47015ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa450071d40 0x7fa450074200 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa4700b4520 0x7fa470155fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 --2- 172.21.15.78:0/1992136208 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa470070b10 0x7fa47014dd80 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 >> 172.21.15.78:0/1992136208 conn(0x7fa470072e30 msgr2=0x7fa4700ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:34.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 shutdown_connections 2024-09-11T12:56:34.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.026+0000 7fa475a00640 1 -- 172.21.15.78:0/1992136208 wait complete. 2024-09-11T12:56:34.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:34.049 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/aed84df36d2b178fc004c12021748ccc.meta) 2024-09-11T12:56:34.073 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:56:34.073 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:56:34.074 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:56:34.074 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/subvol_grp71280/subvol60711 2024-09-11T12:56:34.074 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:56:34.074 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:34.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol11340 2024-09-11T12:56:34.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- 172.21.15.78:0/1434849099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3420 msgr2=0x7f608415a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 --2- 172.21.15.78:0/1434849099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3420 0x7f608415a700 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f608406d1a0 tx=0x7f607001c890 comp rx=0 tx=0).stop 2024-09-11T12:56:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- 172.21.15.78:0/1434849099 shutdown_connections 2024-09-11T12:56:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 --2- 172.21.15.78:0/1434849099 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3420 0x7f608415a700 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 --2- 172.21.15.78:0/1434849099 >> v2:172.21.15.157:3300/0 conn(0x7f60840b3040 0x7f60840cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- 172.21.15.78:0/1434849099 >> 172.21.15.78:0/1434849099 conn(0x7f6084071dd0 msgr2=0x7f60840721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:34.271 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- 172.21.15.78:0/1434849099 shutdown_connections 2024-09-11T12:56:34.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- 172.21.15.78:0/1434849099 wait complete. 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 Processor -- start 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.266+0000 7f608c200640 1 -- start start 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60840b3420 0x7f60841410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60840cd770 0x7f60840ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60840af0f0 con 0x7f60840b3040 2024-09-11T12:56:34.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60840aef70 con 0x7f60840b3420 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60840af270 con 0x7f60840cd770 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55286/0 (socket says 172.21.15.78:55286) 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 -- 172.21.15.78:0/3043130998 learned_addr learned my addr 172.21.15.78:0/3043130998 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6088e00640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60840b3420 0x7f60841410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608a200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60840cd770 0x7f60840ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 -- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60840cd770 msgr2=0x7f60840ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60840cd770 0x7f60840ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 -- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60840b3420 msgr2=0x7f60841410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60840b3420 0x7f60841410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 -- 172.21.15.78:0/3043130998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60840cc140 con 0x7f60840b3040 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6089800640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f607800b580 tx=0x7f607800ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:34.275 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078014070 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6078010980 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078010dd0 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6084071450 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6084153f70 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.270+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60840cf6e0 con 0x7f60840b3040 2024-09-11T12:56:34.276 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.274+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7f6078017070 con 0x7f60840b3040 2024-09-11T12:56:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.274+0000 7f6081600640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 0x7f6064074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.274+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f6078092860 con 0x7f60840b3040 2024-09-11T12:56:34.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.274+0000 7f6088e00640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 0x7f6064074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.274+0000 7f6088e00640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 0x7f6064074200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f60700af000 tx=0x7f607001b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:34.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.278+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f607805d630 con 0x7f60840b3040 2024-09-11T12:56:34.470 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.466+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol11340", "target": ["mon-mgr", ""]}) -- 0x7f608415cfe0 con 0x7f6064071d40 2024-09-11T12:56:34.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f6081600640 1 -- 172.21.15.78:0/3043130998 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f608415cfe0 con 0x7f6064071d40 2024-09-11T12:56:34.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 msgr2=0x7f6064074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 0x7f6064074200 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f60700af000 tx=0x7f607001b0d0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 msgr2=0x7f6084154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f607800b580 tx=0x7f607800ba40 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 shutdown_connections 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60840cd770 0x7f60840ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6064071d40 0x7f6064074200 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60840b3420 0x7f60841410f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 --2- 172.21.15.78:0/3043130998 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60840b3040 0x7f6084154250 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 >> 172.21.15.78:0/3043130998 conn(0x7f6084071dd0 msgr2=0x7f608406f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 shutdown_connections 2024-09-11T12:56:34.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.494+0000 7f608c200640 1 -- 172.21.15.78:0/3043130998 wait complete. 2024-09-11T12:56:34.504 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.502+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.504 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.502+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.504 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.502+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.504 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.502+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.504 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.502+0000 7f1ed0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol60711 subvol_grp71280 2024-09-11T12:56:34.522 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.518+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.523 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.518+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.523 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.518+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.523 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.518+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.523 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:56:34.518+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- 172.21.15.78:0/2224502361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 msgr2=0x7f39dc071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- 172.21.15.78:0/2224502361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc071e10 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009900 tx=0x7f39c801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- 172.21.15.78:0/2224502361 shutdown_connections 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- 172.21.15.78:0/2224502361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc072350 0x7f39dc06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- 172.21.15.78:0/2224502361 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc071e10 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- 172.21.15.78:0/2224502361 >> 172.21.15.78:0/2224502361 conn(0x7f39dc0b3050 msgr2=0x7f39dc0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- 172.21.15.78:0/2224502361 shutdown_connections 2024-09-11T12:56:34.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- 172.21.15.78:0/2224502361 wait complete. 2024-09-11T12:56:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 Processor -- start 2024-09-11T12:56:34.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- start start 2024-09-11T12:56:34.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc072350 0x7f39dc1fb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc151050 0x7f39dc147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.670+0000 7f39e1000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc070430 con 0x7f39dc074060 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39e1000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f39dc0702b0 con 0x7f39dc072350 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39e1000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f39dc0705b0 con 0x7f39dc151050 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55366/0 (socket says 172.21.15.78:55366) 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 -- 172.21.15.78:0/1397934327 learned_addr learned my addr 172.21.15.78:0/1397934327 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9e00640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc072350 0x7f39dc1fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39da800640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc151050 0x7f39dc147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 -- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc151050 msgr2=0x7f39dc147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc151050 0x7f39dc147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 -- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc072350 msgr2=0x7f39dc1fb850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc072350 0x7f39dc1fb850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 -- 172.21.15.78:0/1397934327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39dc1480b0 con 0x7f39dc074060 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d9400640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009850 tx=0x7f39c8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c80b2040 con 0x7f39dc074060 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39c8005030 con 0x7f39dc074060 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39dc14cce0 con 0x7f39dc074060 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39c8005440 con 0x7f39dc074060 2024-09-11T12:56:34.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39dc14b170 con 0x7f39dc074060 2024-09-11T12:56:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39dc06f5d0 con 0x7f39dc074060 2024-09-11T12:56:34.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.674+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7f39c80046d0 con 0x7f39dc074060 2024-09-11T12:56:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.678+0000 7f39d2000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 0x7f39bc074230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.678+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f39c812bc10 con 0x7f39dc074060 2024-09-11T12:56:34.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.678+0000 7f39d9e00640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 0x7f39bc074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:34.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.678+0000 7f39d9e00640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 0x7f39bc074230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007fd0 tx=0x7f39cc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:34.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.682+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f39c80bc060 con 0x7f39dc074060 2024-09-11T12:56:34.877 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.874+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol60711", "group_name": "subvol_grp71280", "target": ["mon-mgr", ""]}) -- 0x7f39dc0cf910 con 0x7f39bc071d70 2024-09-11T12:56:34.908 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39d2000640 1 -- 172.21.15.78:0/1397934327 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f39dc0cf910 con 0x7f39bc071d70 2024-09-11T12:56:34.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 msgr2=0x7f39bc074230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 0x7f39bc074230 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f39cc007fd0 tx=0x7f39cc009040 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 msgr2=0x7f39dc14bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f39c8009850 tx=0x7f39c8004040 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 shutdown_connections 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f39dc151050 0x7f39dc147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f39dc074060 0x7f39dc14bc50 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f39bc071d70 0x7f39bc074230 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 --2- 172.21.15.78:0/1397934327 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f39dc072350 0x7f39dc1fb850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 >> 172.21.15.78:0/1397934327 conn(0x7f39dc0b3050 msgr2=0x7f39dc0b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 shutdown_connections 2024-09-11T12:56:34.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34.906+0000 7f39e1000640 1 -- 172.21.15.78:0/1397934327 wait complete. 2024-09-11T12:56:34.932 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:56:34.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:56:34.986 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:56:34.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:56:34.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp71280 2024-09-11T12:56:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- 172.21.15.78:0/91739203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86180ce320 msgr2=0x7f86181655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 --2- 172.21.15.78:0/91739203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86180ce320 0x7f86181655c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f8610009870 tx=0x7f861001c820 comp rx=0 tx=0).stop 2024-09-11T12:56:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- 172.21.15.78:0/91739203 shutdown_connections 2024-09-11T12:56:35.137 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 --2- 172.21.15.78:0/91739203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86180ce320 0x7f86181655c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- 172.21.15.78:0/91739203 >> 172.21.15.78:0/91739203 conn(0x7f8618071dd0 msgr2=0x7f86180721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- 172.21.15.78:0/91739203 shutdown_connections 2024-09-11T12:56:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- 172.21.15.78:0/91739203 wait complete. 2024-09-11T12:56:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 Processor -- start 2024-09-11T12:56:35.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.134+0000 7f8620600640 1 -- start start 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b2120 0x7f861815c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 0x7f861814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8618168ef0 con 0x7f86181518b0 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8618168d70 con 0x7f86180b2120 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8618169070 con 0x7f86180ce320 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55374/0 (socket says 172.21.15.78:55374) 2024-09-11T12:56:35.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 -- 172.21.15.78:0/1565670231 learned_addr learned my addr 172.21.15.78:0/1565670231 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861dc00640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b2120 0x7f861815c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861d200640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 0x7f861814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 -- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 msgr2=0x7f861814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 0x7f861814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 -- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b2120 msgr2=0x7f861815c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b2120 0x7f861815c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 -- 172.21.15.78:0/1565670231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86181fb180 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861d200640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 0x7f861814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f861e600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f861001aed0 tx=0x7f86100a67b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100bc020 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8610005030 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f861815d170 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86100044f0 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86181fb700 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86180cdc10 con 0x7f86181518b0 2024-09-11T12:56:35.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.138+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7f86100bf020 con 0x7f86181518b0 2024-09-11T12:56:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.142+0000 7f8615600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 0x7f85f8074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.142+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f861012d950 con 0x7f86181518b0 2024-09-11T12:56:35.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.142+0000 7f861dc00640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 0x7f85f8074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.142+0000 7f861dc00640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 0x7f85f8074200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f860c009f20 tx=0x7f860c0094d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:35.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.146+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f86100c6030 con 0x7f86181518b0 2024-09-11T12:56:35.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.334+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp71280", "target": ["mon-mgr", ""]}) -- 0x7f86180cd4f0 con 0x7f85f8071d40 2024-09-11T12:56:35.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8615600640 1 -- 172.21.15.78:0/1565670231 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f86180cd4f0 con 0x7f85f8071d40 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 msgr2=0x7f85f8074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 0x7f85f8074200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f860c009f20 tx=0x7f860c0094d0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 msgr2=0x7f861814d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f861001aed0 tx=0x7f86100a67b0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 shutdown_connections 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86181518b0 0x7f861814d170 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86180ce320 0x7f861814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f85f8071d40 0x7f85f8074200 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 --2- 172.21.15.78:0/1565670231 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86180b2120 0x7f861815c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 >> 172.21.15.78:0/1565670231 conn(0x7f8618071dd0 msgr2=0x7f861814e720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:35.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 shutdown_connections 2024-09-11T12:56:35.356 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.350+0000 7f8620600640 1 -- 172.21.15.78:0/1565670231 wait complete. 2024-09-11T12:56:35.375 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:56:35.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:35.486 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:35.577 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:56:35.577 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:35.577 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:35.577 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:35.587 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:35.587 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:35.639 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 -- 172.21.15.78:0/3881104342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f800aed20 msgr2=0x7f9f800cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 --2- 172.21.15.78:0/3881104342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f800aed20 0x7f9f800cf280 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9f800af2d0 tx=0x7f9f7001caa0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 -- 172.21.15.78:0/3881104342 shutdown_connections 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 --2- 172.21.15.78:0/3881104342 >> v2:172.21.15.157:3300/0 conn(0x7f9f800cf7c0 0x7f9f8015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 --2- 172.21.15.78:0/3881104342 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f800aed20 0x7f9f800cf280 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.827 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 -- 172.21.15.78:0/3881104342 >> 172.21.15.78:0/3881104342 conn(0x7f9f8006ca80 msgr2=0x7f9f800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:35.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 -- 172.21.15.78:0/3881104342 shutdown_connections 2024-09-11T12:56:35.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.822+0000 7f9f84a00640 1 -- 172.21.15.78:0/3881104342 wait complete. 2024-09-11T12:56:35.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 Processor -- start 2024-09-11T12:56:35.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- start start 2024-09-11T12:56:35.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f800aed20 0x7f9f801fabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f800cf7c0 0x7f9f80146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f800cd0d0 con 0x7f9f8015d010 2024-09-11T12:56:35.831 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f80152da0 con 0x7f9f800aed20 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f800cd250 con 0x7f9f800cf7c0 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55392/0 (socket says 172.21.15.78:55392) 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 -- 172.21.15.78:0/1593837811 learned_addr learned my addr 172.21.15.78:0/1593837811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7de00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f800aed20 0x7f9f801fabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7d400640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f800cf7c0 0x7f9f80146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 -- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f800cf7c0 msgr2=0x7f9f80146f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f800cf7c0 0x7f9f80146f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 -- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f800aed20 msgr2=0x7f9f801fabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:35.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f800aed20 0x7f9f801fabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 -- 172.21.15.78:0/1593837811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f801fb0f0 con 0x7f9f8015d010 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f7e800640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c3c0 tx=0x7f9f7800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78011790 con 0x7f9f8015d010 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f78017070 con 0x7f9f8015d010 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f8014de80 con 0x7f9f8015d010 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f78011ce0 con 0x7f9f8015d010 2024-09-11T12:56:35.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.826+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f800721e0 con 0x7f9f8015d010 2024-09-11T12:56:35.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f800cf280 con 0x7f9f8015d010 2024-09-11T12:56:35.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7f9f7801c460 con 0x7f9f8015d010 2024-09-11T12:56:35.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f76000640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 0x7f9f60074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:35.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f9f78092550 con 0x7f9f8015d010 2024-09-11T12:56:35.834 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f7de00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 0x7f9f60074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:35.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.830+0000 7f9f7de00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 0x7f9f60074200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f700af000 tx=0x7f9f7001b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:35.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:35.834+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9f801fb0f0 con 0x7f9f8015d010 2024-09-11T12:56:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f8014b050 con 0x7f9f8015d010 2024-09-11T12:56:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f76000640 1 -- 172.21.15.78:0/1593837811 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7f9f7805d4a0 con 0x7f9f8015d010 2024-09-11T12:56:36.016 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 msgr2=0x7f9f60074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 0x7f9f60074200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9f700af000 tx=0x7f9f7001b2e0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 msgr2=0x7f9f800ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c3c0 tx=0x7f9f7800c880 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 shutdown_connections 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f8015d010 0x7f9f800ad870 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f800cf7c0 0x7f9f80146f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f60071d40 0x7f9f60074200 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 --2- 172.21.15.78:0/1593837811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f800aed20 0x7f9f801fabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 >> 172.21.15.78:0/1593837811 conn(0x7f9f8006ca80 msgr2=0x7f9f80151320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:36.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 shutdown_connections 2024-09-11T12:56:36.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.014+0000 7f9f84a00640 1 -- 172.21.15.78:0/1593837811 wait complete. 2024-09-11T12:56:36.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:56:36.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:36.039 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:56:36.066 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:56:36.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:36.066 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:56:36.119 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:36.120 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:36.120 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:36.129 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:36.129 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:36.178 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- 172.21.15.78:0/3451936830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f080aed20 msgr2=0x7f2f080cf280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- 172.21.15.78:0/3451936830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f080aed20 0x7f2f080cf280 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f2f080b2f50 tx=0x7f2efc01c890 comp rx=0 tx=0).stop 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- 172.21.15.78:0/3451936830 shutdown_connections 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- 172.21.15.78:0/3451936830 >> v2:172.21.15.157:3300/0 conn(0x7f2f0815d010 0x7f2f080b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- 172.21.15.78:0/3451936830 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080cf7c0 0x7f2f0815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- 172.21.15.78:0/3451936830 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f080aed20 0x7f2f080cf280 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- 172.21.15.78:0/3451936830 >> 172.21.15.78:0/3451936830 conn(0x7f2f0806ca80 msgr2=0x7f2f080b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:36.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- 172.21.15.78:0/3451936830 shutdown_connections 2024-09-11T12:56:36.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- 172.21.15.78:0/3451936830 wait complete. 2024-09-11T12:56:36.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 Processor -- start 2024-09-11T12:56:36.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- start start 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080aed20 0x7f2f0813f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080cf7c0 0x7f2f0814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080cd460 con 0x7f2f0815d010 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f080cd2e0 con 0x7f2f080cf7c0 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.318+0000 7f2f10000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f080cd5e0 con 0x7f2f080aed20 2024-09-11T12:56:36.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0d600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080aed20 0x7f2f0813f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080cf7c0 0x7f2f0814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55412/0 (socket says 172.21.15.78:55412) 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 -- 172.21.15.78:0/3541012133 learned_addr learned my addr 172.21.15.78:0/3541012133 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 -- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080aed20 msgr2=0x7f2f0813f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080aed20 0x7f2f0813f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 -- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080cf7c0 msgr2=0x7f2f0814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080cf7c0 0x7f2f0814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 -- 172.21.15.78:0/3541012133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08140e30 con 0x7f2f0815d010 2024-09-11T12:56:36.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f0e000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9a0 tx=0x7f2f0000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:36.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00004d30 con 0x7f2f0815d010 2024-09-11T12:56:36.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f0000cb20 con 0x7f2f0815d010 2024-09-11T12:56:36.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08141110 con 0x7f2f0815d010 2024-09-11T12:56:36.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00005020 con 0x7f2f0815d010 2024-09-11T12:56:36.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f08153d70 con 0x7f2f0815d010 2024-09-11T12:56:36.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f08141350 con 0x7f2f0815d010 2024-09-11T12:56:36.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.322+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 22) ==== 95544+0+0 (secure 0 0 0) 0x7f2f000051c0 con 0x7f2f0815d010 2024-09-11T12:56:36.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.326+0000 7f2f05600640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 0x7f2ee8074200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.326+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f2f000933a0 con 0x7f2f0815d010 2024-09-11T12:56:36.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.326+0000 7f2f0d600640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 0x7f2ee8074200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.326+0000 7f2f0d600640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 0x7f2ee8074200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0af000 tx=0x7f2efc01b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:36.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.326+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2f08141350 con 0x7f2f0815d010 2024-09-11T12:56:36.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.506+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2f080cdb40 con 0x7f2f0815d010 2024-09-11T12:56:36.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.506+0000 7f2f05600640 1 -- 172.21.15.78:0/3541012133 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) ==== 80+0+0 (secure 0 0 0) 0x7f2f00016030 con 0x7f2f0815d010 2024-09-11T12:56:36.510 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 msgr2=0x7f2ee8074200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 0x7f2ee8074200 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0af000 tx=0x7f2efc01b0d0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 msgr2=0x7f2f0814f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9a0 tx=0x7f2f0000ee60 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 shutdown_connections 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f0815d010 0x7f2f0814f1d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ee8071d40 0x7f2ee8074200 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f080cf7c0 0x7f2f0814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 --2- 172.21.15.78:0/3541012133 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f080aed20 0x7f2f0813f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 >> 172.21.15.78:0/3541012133 conn(0x7f2f0806ca80 msgr2=0x7f2f0814b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 shutdown_connections 2024-09-11T12:56:36.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.510+0000 7f2f10000640 1 -- 172.21.15.78:0/3541012133 wait complete. 2024-09-11T12:56:36.532 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:56:36.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:36.532 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:56:36.560 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:56:36.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:56:36.561 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:56:36.612 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:36.612 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:36.612 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:36.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:36.621 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:56:36.621 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:36.621 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:36.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:36.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/2935263035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00b4780 msgr2=0x7f64c0073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- 172.21.15.78:0/2935263035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00b4780 0x7f64c0073f70 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009930 tx=0x7f64b001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/2935263035 shutdown_connections 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- 172.21.15.78:0/2935263035 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 0x7f64c006ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- 172.21.15.78:0/2935263035 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00b4780 0x7f64c0073f70 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/2935263035 >> 172.21.15.78:0/2935263035 conn(0x7f64c006ca80 msgr2=0x7f64c00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/2935263035 shutdown_connections 2024-09-11T12:56:36.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/2935263035 wait complete. 2024-09-11T12:56:36.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 Processor -- start 2024-09-11T12:56:36.809 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- start start 2024-09-11T12:56:36.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c00b4780 0x7f64c0161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.811 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 0x7f64c014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0168be0 con 0x7f64c00ccb20 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f64c0168a60 con 0x7f64c00b4780 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f64c0168d60 con 0x7f64c015bc60 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55426/0 (socket says 172.21.15.78:55426) 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 -- 172.21.15.78:0/4293053540 learned_addr learned my addr 172.21.15.78:0/4293053540 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bde00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c00b4780 0x7f64c0161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64be800640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 0x7f64c014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 -- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 msgr2=0x7f64c014c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 0x7f64c014c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 -- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c00b4780 msgr2=0x7f64c0161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c00b4780 0x7f64c0161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 -- 172.21.15.78:0/4293053540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64c015a410 con 0x7f64c00ccb20 2024-09-11T12:56:36.812 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64bd400640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c4b0 tx=0x7f64ac00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac0052d0 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64ac005470 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.806+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64c015a6f0 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac012870 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64c015acb0 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64c015aff0 con 0x7f64c00ccb20 2024-09-11T12:56:36.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f64ac003600 con 0x7f64c00ccb20 2024-09-11T12:56:36.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64b6000640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 0x7f64a0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:36.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f64ac092ce0 con 0x7f64c00ccb20 2024-09-11T12:56:36.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64bde00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 0x7f64a0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:36.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.810+0000 7f64bde00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 0x7f64a0074320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64b00af000 tx=0x7f64b001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:36.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:36.814+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f64c015aff0 con 0x7f64c00ccb20 2024-09-11T12:56:37.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.018+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64c015ea40 con 0x7f64c00ccb20 2024-09-11T12:56:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.018+0000 7f64b6000640 1 -- 172.21.15.78:0/4293053540 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7f64ac05db30 con 0x7f64c00ccb20 2024-09-11T12:56:37.023 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:37.023 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":102,"btime":"2024-09-11T12:56:27:184560+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:37.023 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 102 2024-09-11T12:56:37.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 msgr2=0x7f64a0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 0x7f64a0074320 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f64b00af000 tx=0x7f64b001b120 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 msgr2=0x7f64c0159950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00c4b0 tx=0x7f64ac00c970 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 shutdown_connections 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f64c015bc60 0x7f64c014c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f64c00ccb20 0x7f64c0159950 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f64a0071e60 0x7f64a0074320 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 --2- 172.21.15.78:0/4293053540 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f64c00b4780 0x7f64c0161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 >> 172.21.15.78:0/4293053540 conn(0x7f64c006ca80 msgr2=0x7f64c015c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 shutdown_connections 2024-09-11T12:56:37.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.022+0000 7f64c4e00640 1 -- 172.21.15.78:0/4293053540 wait complete. 2024-09-11T12:56:37.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- 172.21.15.78:0/2170382778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 msgr2=0x7fa0900051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 --2- 172.21.15.78:0/2170382778 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0900051b0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7fa0a400c640 tx=0x7fa0a401d240 comp rx=0 tx=0).stop 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b1200640 1 -- 172.21.15.78:0/2170382778 reap_dead start 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- 172.21.15.78:0/2170382778 shutdown_connections 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- 172.21.15.78:0/2170382778 >> 172.21.15.78:0/2170382778 conn(0x7fa0ac0ceab0 msgr2=0x7fa0ac0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- 172.21.15.78:0/2170382778 shutdown_connections 2024-09-11T12:56:37.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- 172.21.15.78:0/2170382778 wait complete. 2024-09-11T12:56:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 Processor -- start 2024-09-11T12:56:37.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- start start 2024-09-11T12:56:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 0x7fa0ac1f4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0ac14c650 0x7fa0ac1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac15cb50 con 0x7fa090004d50 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa0ac15c9d0 con 0x7fa0ac14d240 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0b3200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa0ac15ccd0 con 0x7fa0ac14c650 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.186+0000 7fa0abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55442/0 (socket says 172.21.15.78:55442) 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 -- 172.21.15.78:0/1457640529 learned_addr learned my addr 172.21.15.78:0/1457640529 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:37.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0ab400640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 0x7fa0ac1f4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0b1200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0ac14c650 0x7fa0ac1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 -- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0ac14c650 msgr2=0x7fa0ac1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0ac14c650 0x7fa0ac1f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 -- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 msgr2=0x7fa0ac1f4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 0x7fa0ac1f4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 -- 172.21.15.78:0/1457640529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0ac1f58b0 con 0x7fa090004d50 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0ab400640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 0x7fa0ac1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0abe00640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011ae0 tx=0x7fa09c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c01e070 con 0x7fa090004d50 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa09c00fcd0 con 0x7fa090004d50 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c019670 con 0x7fa090004d50 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0ac1f26a0 con 0x7fa090004d50 2024-09-11T12:56:37.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0ac1f2c60 con 0x7fa090004d50 2024-09-11T12:56:37.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0ac152930 con 0x7fa090004d50 2024-09-11T12:56:37.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fa09c021070 con 0x7fa090004d50 2024-09-11T12:56:37.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 0x7fa08c074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.190+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fa09c0939a0 con 0x7fa090004d50 2024-09-11T12:56:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.194+0000 7fa0ab400640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 0x7fa08c074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.194+0000 7fa0ab400640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 0x7fa08c074320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac14d640 tx=0x7fa098006c10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.198+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa09c05e120 con 0x7fa090004d50 2024-09-11T12:56:37.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.394+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0ac070480 con 0x7fa090004d50 2024-09-11T12:56:37.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.394+0000 7fa0a3e00640 1 -- 172.21.15.78:0/1457640529 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7fa09c062670 con 0x7fa090004d50 2024-09-11T12:56:37.398 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:37.398 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":102,"btime":"2024-09-11T12:56:27:184560+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:37.398 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 102 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 msgr2=0x7fa08c074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 0x7fa08c074320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac14d640 tx=0x7fa098006c10 comp rx=0 tx=0).stop 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 msgr2=0x7fa0ac0d0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fa09c011ae0 tx=0x7fa09c009ee0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 shutdown_connections 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa0ac14c650 0x7fa0ac1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa08c071e60 0x7fa08c074320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa0ac14d240 0x7fa0ac1f4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 --2- 172.21.15.78:0/1457640529 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa090004d50 0x7fa0ac0d0110 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 >> 172.21.15.78:0/1457640529 conn(0x7fa0ac0ceab0 msgr2=0x7fa0ac0d0d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 shutdown_connections 2024-09-11T12:56:37.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.398+0000 7fa0b3200640 1 -- 172.21.15.78:0/1457640529 wait complete. 2024-09-11T12:56:37.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:37.560 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- 172.21.15.78:0/2393874200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0073710 msgr2=0x7ff5d015ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- 172.21.15.78:0/2393874200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0073710 0x7ff5d015ae20 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00b41c0 tx=0x7ff5bc01c800 comp rx=0 tx=0).stop 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- 172.21.15.78:0/2393874200 shutdown_connections 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- 172.21.15.78:0/2393874200 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0073710 0x7ff5d015ae20 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- 172.21.15.78:0/2393874200 >> v2:172.21.15.157:3300/0 conn(0x7ff5d0157a50 0x7ff5d0070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- 172.21.15.78:0/2393874200 >> 172.21.15.78:0/2393874200 conn(0x7ff5d006ca80 msgr2=0x7ff5d00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- 172.21.15.78:0/2393874200 shutdown_connections 2024-09-11T12:56:37.561 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- 172.21.15.78:0/2393874200 wait complete. 2024-09-11T12:56:37.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 Processor -- start 2024-09-11T12:56:37.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- start start 2024-09-11T12:56:37.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d00b0ab0 0x7ff5d0142f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 0x7ff5d00ae690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d006eed0 con 0x7ff5d0157a50 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff5d006ed50 con 0x7ff5d00b0ab0 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d7800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff5d006f050 con 0x7ff5d00ae200 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d4e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55464/0 (socket says 172.21.15.78:55464) 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d4e00640 1 -- 172.21.15.78:0/3975631285 learned_addr learned my addr 172.21.15.78:0/3975631285 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5cfe00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d00b0ab0 0x7ff5d0142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.558+0000 7ff5d5800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 0x7ff5d00ae690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 -- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 msgr2=0x7ff5d00ae690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 0x7ff5d00ae690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 -- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d00b0ab0 msgr2=0x7ff5d0142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d00b0ab0 0x7ff5d0142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 -- 172.21.15.78:0/3975631285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5d0154c80 con 0x7ff5d0157a50 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d5800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 0x7ff5d00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d4e00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000ddc0 tx=0x7ff5c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c001b070 con 0x7ff5d0157a50 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5c000e740 con 0x7ff5d0157a50 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5d01fb580 con 0x7ff5d0157a50 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c000eb90 con 0x7ff5d0157a50 2024-09-11T12:56:37.566 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5d00b1250 con 0x7ff5d0157a50 2024-09-11T12:56:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5d015ae20 con 0x7ff5d0157a50 2024-09-11T12:56:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.562+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7ff5c001e070 con 0x7ff5d0157a50 2024-09-11T12:56:37.567 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.566+0000 7ff5ccc00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 0x7ff5b0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.566+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7ff5c00930a0 con 0x7ff5d0157a50 2024-09-11T12:56:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.566+0000 7ff5cfe00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 0x7ff5b0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.568 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.566+0000 7ff5cfe00640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 0x7ff5b0074320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0af000 tx=0x7ff5bc01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.570+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff5c005dcf0 con 0x7ff5d0157a50 2024-09-11T12:56:37.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.738+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff5d0143860 con 0x7ff5d0157a50 2024-09-11T12:56:37.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.738+0000 7ff5ccc00640 1 -- 172.21.15.78:0/3975631285 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16731 (secure 0 0 0) 0x7ff5c0062240 con 0x7ff5d0157a50 2024-09-11T12:56:37.742 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:37.742 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":140,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:19.149907+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:15.230647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:16.498978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:37.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 msgr2=0x7ff5b0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 0x7ff5b0074320 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc0af000 tx=0x7ff5bc01b040 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 msgr2=0x7ff5d01fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000ddc0 tx=0x7ff5c000a620 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 shutdown_connections 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff5d00ae200 0x7ff5d00ae690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff5b0071e60 0x7ff5b0074320 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff5d00b0ab0 0x7ff5d0142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 --2- 172.21.15.78:0/3975631285 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff5d0157a50 0x7ff5d01fb040 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 >> 172.21.15.78:0/3975631285 conn(0x7ff5d006ca80 msgr2=0x7ff5d0071d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 shutdown_connections 2024-09-11T12:56:37.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.742+0000 7ff5d7800640 1 -- 172.21.15.78:0/3975631285 wait complete. 2024-09-11T12:56:37.764 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:56:37.764 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:37.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- 172.21.15.78:0/2382123753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 msgr2=0x7fbeac0cdd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/2382123753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0cdd80 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fbeac0b2f50 tx=0x7fbe9801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- 172.21.15.78:0/2382123753 shutdown_connections 2024-09-11T12:56:37.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/2382123753 >> v2:172.21.15.157:3300/0 conn(0x7fbeac0b4db0 0x7fbeac0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.905 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/2382123753 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0cdd80 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- 172.21.15.78:0/2382123753 >> 172.21.15.78:0/2382123753 conn(0x7fbeac06ca80 msgr2=0x7fbeac0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- 172.21.15.78:0/2382123753 shutdown_connections 2024-09-11T12:56:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- 172.21.15.78:0/2382123753 wait complete. 2024-09-11T12:56:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 Processor -- start 2024-09-11T12:56:37.906 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 -- start start 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.902+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 0x7fbeac1449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeac159fc0 0x7fbeac0b34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0b49f0 con 0x7fbeac0cd910 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbeac0b4870 con 0x7fbeac0b4db0 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbeac0b4b70 con 0x7fbeac159fc0 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.909 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55482/0 (socket says 172.21.15.78:55482) 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 -- 172.21.15.78:0/1202433906 learned_addr learned my addr 172.21.15.78:0/1202433906 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 -- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeac159fc0 msgr2=0x7fbeac0b34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeac159fc0 0x7fbeac0b34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 -- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 msgr2=0x7fbeac1449b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9e00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 0x7fbeac1449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 0x7fbeac1449b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 -- 172.21.15.78:0/1202433906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeac15abf0 con 0x7fbeac0cd910 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9e00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 0x7fbeac1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea9400640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801af70 tx=0x7fbe980a6ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980b2040 con 0x7fbeac0cd910 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe98004320 con 0x7fbeac0cd910 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeac15aed0 con 0x7fbeac0cd910 2024-09-11T12:56:37.910 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98005280 con 0x7fbeac0cd910 2024-09-11T12:56:37.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeac15b490 con 0x7fbeac0cd910 2024-09-11T12:56:37.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbeac15b660 con 0x7fbeac0cd910 2024-09-11T12:56:37.911 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.906+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fbe98005420 con 0x7fbeac0cd910 2024-09-11T12:56:37.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.910+0000 7fbea2000640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 0x7fbe8c074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:37.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.910+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fbe9812c660 con 0x7fbeac0cd910 2024-09-11T12:56:37.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.910+0000 7fbea9e00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 0x7fbe8c074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:37.912 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.910+0000 7fbea9e00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 0x7fbe8c074320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007eb0 tx=0x7fbe9c0073c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:37.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:37.914+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbeac15b660 con 0x7fbeac0cd910 2024-09-11T12:56:38.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbeac075b00 con 0x7fbeac0cd910 2024-09-11T12:56:38.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbea2000640 1 -- 172.21.15.78:0/1202433906 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7fbe980bc060 con 0x7fbeac0cd910 2024-09-11T12:56:38.116 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:38.116 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":102,"btime":"2024-09-11T12:56:27:184560+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:38.117 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 102 2024-09-11T12:56:38.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 msgr2=0x7fbe8c074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 0x7fbe8c074320 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c007eb0 tx=0x7fbe9c0073c0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 msgr2=0x7fbeac0b2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fbe9801af70 tx=0x7fbe980a6ce0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 shutdown_connections 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbeac159fc0 0x7fbeac0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbeac0cd910 0x7fbeac0b2fa0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe8c071e60 0x7fbe8c074320 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 --2- 172.21.15.78:0/1202433906 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbeac0b4db0 0x7fbeac1449b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 >> 172.21.15.78:0/1202433906 conn(0x7fbeac06ca80 msgr2=0x7fbeac153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 shutdown_connections 2024-09-11T12:56:38.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.114+0000 7fbeb0c00640 1 -- 172.21.15.78:0/1202433906 wait complete. 2024-09-11T12:56:38.137 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- 172.21.15.78:0/3201760465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7864003ab0 msgr2=0x7f7864003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 --2- 172.21.15.78:0/3201760465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7864003ab0 0x7f7864003ed0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f787000c610 tx=0x7f787001d260 comp rx=0 tx=0).stop 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- 172.21.15.78:0/3201760465 shutdown_connections 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 --2- 172.21.15.78:0/3201760465 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7864003ab0 0x7f7864003ed0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- 172.21.15.78:0/3201760465 >> 172.21.15.78:0/3201760465 conn(0x7f788006ca80 msgr2=0x7f78800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- 172.21.15.78:0/3201760465 shutdown_connections 2024-09-11T12:56:38.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- 172.21.15.78:0/3201760465 wait complete. 2024-09-11T12:56:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 Processor -- start 2024-09-11T12:56:38.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.278+0000 7f7886a00640 1 -- start start 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7864003ab0 0x7f7880159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801589a0 0x7f78800ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f788006f0a0 con 0x7f7880154760 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f788006ef20 con 0x7f78801589a0 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f788006f220 con 0x7f7864003ab0 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55492/0 (socket says 172.21.15.78:55492) 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 -- 172.21.15.78:0/984421861 learned_addr learned my addr 172.21.15.78:0/984421861 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:38.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7884a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801589a0 0x7f78800ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787fe00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7864003ab0 0x7f7880159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 -- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7864003ab0 msgr2=0x7f7880159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7864003ab0 0x7f7880159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 -- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801589a0 msgr2=0x7f78800ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801589a0 0x7f78800ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 -- 172.21.15.78:0/984421861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f788015cfe0 con 0x7f7880154760 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f787f400640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f786c0117b0 tx=0x7f786c011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c01f070 con 0x7f7880154760 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f786c00fde0 con 0x7f7880154760 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f786c0126b0 con 0x7f7880154760 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f788015b970 con 0x7f7880154760 2024-09-11T12:56:38.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f788015c7e0 con 0x7f7880154760 2024-09-11T12:56:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7880158db0 con 0x7f7880154760 2024-09-11T12:56:38.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.282+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f786c022070 con 0x7f7880154760 2024-09-11T12:56:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.286+0000 7f7877e00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 0x7f7860074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.286+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f786c093450 con 0x7f7880154760 2024-09-11T12:56:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.286+0000 7f787fe00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 0x7f7860074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.286+0000 7f787fe00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 0x7f7860074350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f78700ac000 tx=0x7f787001a0a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:38.291 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.290+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7880158db0 con 0x7f7880154760 2024-09-11T12:56:38.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.490+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f788015e460 con 0x7f7880154760 2024-09-11T12:56:38.495 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.490+0000 7f7877e00640 1 -- 172.21.15.78:0/984421861 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5229 (secure 0 0 0) 0x7f786c011e60 con 0x7f7880154760 2024-09-11T12:56:38.495 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:38.495 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":102,"btime":"2024-09-11T12:56:27:184560+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:17.530187+0000","modified":"2024-09-11T12:56:20.551586+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8274},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8274":{"gid":8274,"name":"c","rank":0,"incarnation":97,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/835134630","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":835134630},{"type":"v1","addr":"172.21.15.78:6835","nonce":835134630}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8274,"qdb_cluster":[8274]},"id":11}]} 2024-09-11T12:56:38.495 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 102 2024-09-11T12:56:38.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 msgr2=0x7f7860074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 0x7f7860074350 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f78700ac000 tx=0x7f787001a0a0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 msgr2=0x7f7880158460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f786c0117b0 tx=0x7f786c011c70 comp rx=0 tx=0).stop 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 shutdown_connections 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7860071e90 0x7f7860074350 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f78801589a0 0x7f78800ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7880154760 0x7f7880158460 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 --2- 172.21.15.78:0/984421861 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7864003ab0 0x7f7880159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 >> 172.21.15.78:0/984421861 conn(0x7f788006ca80 msgr2=0x7f7880155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 shutdown_connections 2024-09-11T12:56:38.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.494+0000 7f7886a00640 1 -- 172.21.15.78:0/984421861 wait complete. 2024-09-11T12:56:38.516 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 -- 172.21.15.78:0/3236708361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc584151650 msgr2=0x7fc5840cfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 --2- 172.21.15.78:0/3236708361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc584151650 0x7fc5840cfb20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc574009870 tx=0x7fc57401d230 comp rx=0 tx=0).stop 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 -- 172.21.15.78:0/3236708361 shutdown_connections 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 --2- 172.21.15.78:0/3236708361 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc584151650 0x7fc5840cfb20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 -- 172.21.15.78:0/3236708361 >> 172.21.15.78:0/3236708361 conn(0x7fc58406d870 msgr2=0x7fc58406dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:38.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 -- 172.21.15.78:0/3236708361 shutdown_connections 2024-09-11T12:56:38.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.658+0000 7fc583e00640 1 -- 172.21.15.78:0/3236708361 wait complete. 2024-09-11T12:56:38.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 Processor -- start 2024-09-11T12:56:38.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- start start 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc584151650 0x7fc5841539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc58415ab00 0x7fc584153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc584168b90 con 0x7fc584073970 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc584168a10 con 0x7fc584151650 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc584168d10 con 0x7fc58415ab00 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55508/0 (socket says 172.21.15.78:55508) 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 -- 172.21.15.78:0/2023663248 learned_addr learned my addr 172.21.15.78:0/2023663248 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583400640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc58415ab00 0x7fc584153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582000640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc584151650 0x7fc5841539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 -- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc58415ab00 msgr2=0x7fc584153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc58415ab00 0x7fc584153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 -- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc584151650 msgr2=0x7fc5841539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc584151650 0x7fc5841539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 -- 172.21.15.78:0/2023663248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5840cc920 con 0x7fc584073970 2024-09-11T12:56:38.667 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc582a00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc574009790 tx=0x7fc574005d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5740b2040 con 0x7fc584073970 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc574004240 con 0x7fc584073970 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc574005020 con 0x7fc584073970 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5840ccc00 con 0x7fc584073970 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.662+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5840cd1c0 con 0x7fc584073970 2024-09-11T12:56:38.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5840cfb20 con 0x7fc584073970 2024-09-11T12:56:38.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fc5740051e0 con 0x7fc584073970 2024-09-11T12:56:38.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc57aa00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 0x7fc564074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:38.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7fc57412c670 con 0x7fc584073970 2024-09-11T12:56:38.670 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc582000640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 0x7fc564074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:38.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.666+0000 7fc582000640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 0x7fc564074320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc584155a90 tx=0x7fc570009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:38.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.670+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc574016450 con 0x7fc584073970 2024-09-11T12:56:38.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.838+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc58415cbc0 con 0x7fc584073970 2024-09-11T12:56:38.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.838+0000 7fc57aa00640 1 -- 172.21.15.78:0/2023663248 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) ==== 74+0+16731 (secure 0 0 0) 0x7fc5740f79a0 con 0x7fc584073970 2024-09-11T12:56:38.842 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:38.842 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":140,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:19.149907+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:15.230647+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:16.498978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 msgr2=0x7fc564074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 0x7fc564074320 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fc584155a90 tx=0x7fc570009040 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 msgr2=0x7fc58414cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fc574009790 tx=0x7fc574005d40 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 shutdown_connections 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc58415ab00 0x7fc584153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc564071e60 0x7fc564074320 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc584151650 0x7fc5841539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 --2- 172.21.15.78:0/2023663248 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc584073970 0x7fc58414cef0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 >> 172.21.15.78:0/2023663248 conn(0x7fc58406d870 msgr2=0x7fc5840b30e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 shutdown_connections 2024-09-11T12:56:38.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:38.842+0000 7fc583e00640 1 -- 172.21.15.78:0/2023663248 wait complete. 2024-09-11T12:56:38.863 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 -- 172.21.15.78:0/3795260858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080af080 msgr2=0x7f51080ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 --2- 172.21.15.78:0/3795260858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080af080 0x7f51080ae470 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009900 tx=0x7f50f401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 -- 172.21.15.78:0/3795260858 shutdown_connections 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 --2- 172.21.15.78:0/3795260858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080ae9b0 0x7f51080cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 --2- 172.21.15.78:0/3795260858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080af080 0x7f51080ae470 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 -- 172.21.15.78:0/3795260858 >> 172.21.15.78:0/3795260858 conn(0x7f51080725a0 msgr2=0x7f5108070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 -- 172.21.15.78:0/3795260858 shutdown_connections 2024-09-11T12:56:39.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.002+0000 7f5107e00640 1 -- 172.21.15.78:0/3795260858 wait complete. 2024-09-11T12:56:39.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 Processor -- start 2024-09-11T12:56:39.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- start start 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080ae9b0 0x7f51080b09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080af080 0x7f51081fab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f510816a7a0 con 0x7f51081548d0 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f510816a620 con 0x7f51080ae9b0 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f510816a920 con 0x7f51080af080 2024-09-11T12:56:39.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55510/0 (socket says 172.21.15.78:55510) 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 -- 172.21.15.78:0/2387206628 learned_addr learned my addr 172.21.15.78:0/2387206628 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5106000640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080af080 0x7f51081fab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5106a00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080ae9b0 0x7f51080b09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 -- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080af080 msgr2=0x7f51081fab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080af080 0x7f51081fab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 -- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080ae9b0 msgr2=0x7f51080b09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080ae9b0 0x7f51080b09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 -- 172.21.15.78:0/2387206628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5108152730 con 0x7f51081548d0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107400640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f510000e8c0 tx=0x7f510000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f510000ccc0 con 0x7f51081548d0 2024-09-11T12:56:39.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f510000ce60 con 0x7f51081548d0 2024-09-11T12:56:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5100005280 con 0x7f51081548d0 2024-09-11T12:56:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51080b3590 con 0x7f51081548d0 2024-09-11T12:56:39.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.006+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510814ed60 con 0x7f51081548d0 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51080cffb0 con 0x7f51081548d0 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f5100005420 con 0x7f51081548d0 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f50fea00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 0x7f50e8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 5709+0+0 (secure 0 0 0) 0x7f5100093890 con 0x7f51081548d0 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f5106a00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 0x7f50e8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.010+0000 7f5106a00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 0x7f50e8074350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001e40 tx=0x7f50f8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:39.017 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.014+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f510005e150 con 0x7f51081548d0 2024-09-11T12:56:39.217 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.214+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f5108154020 con 0x7f51081548d0 2024-09-11T12:56:39.219 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:56:39.214+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:56:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.218+0000 7f50fea00640 1 -- 172.21.15.78:0/2387206628 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) ==== 161+0+0 (secure 0 0 0) 0x7f51000626a0 con 0x7f51081548d0 2024-09-11T12:56:39.221 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 msgr2=0x7f50e8074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 0x7f50e8074350 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f50f8001e40 tx=0x7f50f8009040 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 msgr2=0x7f510813afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f510000e8c0 tx=0x7f510000ed80 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 shutdown_connections 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f51081548d0 0x7f510813afb0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51080af080 0x7f51081fab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50e8071e90 0x7f50e8074350 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 --2- 172.21.15.78:0/2387206628 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51080ae9b0 0x7f51080b09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 >> 172.21.15.78:0/2387206628 conn(0x7f51080725a0 msgr2=0x7f5108152f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:39.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 shutdown_connections 2024-09-11T12:56:39.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.222+0000 7f5107e00640 1 -- 172.21.15.78:0/2387206628 wait complete. 2024-09-11T12:56:39.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:56:39.365 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:56:39.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 -- 172.21.15.78:0/1171034179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842015e990 msgr2=0x7f842015edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 --2- 172.21.15.78:0/1171034179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842015e990 0x7f842015edd0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f8418009c80 tx=0x7f841801cda0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 -- 172.21.15.78:0/1171034179 shutdown_connections 2024-09-11T12:56:39.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 --2- 172.21.15.78:0/1171034179 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f842015e990 0x7f842015edd0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 --2- 172.21.15.78:0/1171034179 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8420151650 0x7f84200709a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 -- 172.21.15.78:0/1171034179 >> 172.21.15.78:0/1171034179 conn(0x7f8420075610 msgr2=0x7f8420075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 -- 172.21.15.78:0/1171034179 shutdown_connections 2024-09-11T12:56:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.506+0000 7f8426a00640 1 -- 172.21.15.78:0/1171034179 wait complete. 2024-09-11T12:56:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 Processor -- start 2024-09-11T12:56:39.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- start start 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8420151650 0x7f84201f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842015e990 0x7f842014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f84200b6370 con 0x7f8420151a30 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f84200b61f0 con 0x7f8420151650 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f84200b64f0 con 0x7f842015e990 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55542/0 (socket says 172.21.15.78:55542) 2024-09-11T12:56:39.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 -- 172.21.15.78:0/1317439504 learned_addr learned my addr 172.21.15.78:0/1317439504 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 -- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842015e990 msgr2=0x7f842014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842015e990 0x7f842014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 -- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8420151650 msgr2=0x7f84201f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8420151650 0x7f84201f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:39.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 -- 172.21.15.78:0/1317439504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842014b030 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f841f400640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f840c00c530 tx=0x7f840c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c017070 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f840c0053c0 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c012890 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f842014b310 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.510+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f842014b8d0 con 0x7f8420151a30 2024-09-11T12:56:39.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f842015edd0 con 0x7f8420151a30 2024-09-11T12:56:39.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f840c01a070 con 0x7f8420151a30 2024-09-11T12:56:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f8417e00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 0x7f8400074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5795+0+0 (secure 0 0 0) 0x7f840c0924a0 con 0x7f8420151a30 2024-09-11T12:56:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f841fe00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 0x7f8400074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:39.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.514+0000 7f841fe00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 0x7f8400074320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f841000a9b0 tx=0x7f8410005c80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:39.521 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.518+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f840c05d290 con 0x7f8420151a30 2024-09-11T12:56:39.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:39.726+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f8420070780 con 0x7f8420151a30 2024-09-11T12:56:40.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.218+0000 7f8417e00640 1 -- 172.21.15.78:0/1317439504 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) ==== 104+0+0 (secure 0 0 0) 0x7f840c05cb90 con 0x7f8420151a30 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 msgr2=0x7f8400074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 0x7f8400074320 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f841000a9b0 tx=0x7f8410005c80 comp rx=0 tx=0).stop 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 msgr2=0x7f8420072e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f840c00c530 tx=0x7f840c00c9f0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 shutdown_connections 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f842015e990 0x7f842014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8420151a30 0x7f8420072e30 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8400071e60 0x7f8400074320 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 --2- 172.21.15.78:0/1317439504 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8420151650 0x7f84201f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 >> 172.21.15.78:0/1317439504 conn(0x7f8420075610 msgr2=0x7f84200759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 shutdown_connections 2024-09-11T12:56:40.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.222+0000 7f8426a00640 1 -- 172.21.15.78:0/1317439504 wait complete. 2024-09-11T12:56:40.248 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:56:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- 172.21.15.78:0/810725247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c0738d0 msgr2=0x7f230c15af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- 172.21.15.78:0/810725247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c0738d0 0x7f230c15af50 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f22fc009c80 tx=0x7f22fc01cd70 comp rx=0 tx=0).stop 2024-09-11T12:56:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- 172.21.15.78:0/810725247 shutdown_connections 2024-09-11T12:56:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- 172.21.15.78:0/810725247 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0c64e0 0x7f230c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- 172.21.15.78:0/810725247 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c0738d0 0x7f230c15af50 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- 172.21.15.78:0/810725247 >> 172.21.15.78:0/810725247 conn(0x7f230c0cd190 msgr2=0x7f230c0cd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:40.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- 172.21.15.78:0/810725247 shutdown_connections 2024-09-11T12:56:40.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- 172.21.15.78:0/810725247 wait complete. 2024-09-11T12:56:40.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 Processor -- start 2024-09-11T12:56:40.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 -- start start 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0738d0 0x7f230c13b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.390+0000 7f2310a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c0c64e0 0x7f230c0c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f230c1676b0 con 0x7f230c073cb0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f230c15def0 con 0x7f230c0c64e0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f230c167830 con 0x7f230c0738d0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55562/0 (socket says 172.21.15.78:55562) 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 -- 172.21.15.78:0/3539511993 learned_addr learned my addr 172.21.15.78:0/3539511993 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f230a800640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c0c64e0 0x7f230c0c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309e00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0738d0 0x7f230c13b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:40.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 -- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0738d0 msgr2=0x7f230c13b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0738d0 0x7f230c13b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 -- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c0c64e0 msgr2=0x7f230c0c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c0c64e0 0x7f230c0c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 -- 172.21.15.78:0/3539511993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f230c139290 con 0x7f230c073cb0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2309400640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c4b0 tx=0x7f22f800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f80052d0 con 0x7f230c073cb0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f8005470 con 0x7f230c073cb0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8012870 con 0x7f230c073cb0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f230c139570 con 0x7f230c073cb0 2024-09-11T12:56:40.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f230c139b30 con 0x7f230c073cb0 2024-09-11T12:56:40.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f230c0c5dd0 con 0x7f230c073cb0 2024-09-11T12:56:40.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.394+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f22f8003600 con 0x7f230c073cb0 2024-09-11T12:56:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.398+0000 7f2302000640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 0x7f22ec074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.398+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 5795+0+0 (secure 0 0 0) 0x7f22f8092b20 con 0x7f230c073cb0 2024-09-11T12:56:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.398+0000 7f2309e00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 0x7f22ec074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:40.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.398+0000 7f2309e00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 0x7f22ec074350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0af000 tx=0x7f22fc01b5b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:40.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.402+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f22f805d2c0 con 0x7f230c073cb0 2024-09-11T12:56:40.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:40.570+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f230c13d2d0 con 0x7f230c073cb0 2024-09-11T12:56:41.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.222+0000 7f2302000640 1 -- 172.21.15.78:0/3539511993 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) ==== 181+0+0 (secure 0 0 0) 0x7f22f8061810 con 0x7f230c073cb0 2024-09-11T12:56:41.225 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:56:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 msgr2=0x7f22ec074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 0x7f22ec074350 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0af000 tx=0x7f22fc01b5b0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 msgr2=0x7f230c13b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c4b0 tx=0x7f22f800c970 comp rx=0 tx=0).stop 2024-09-11T12:56:41.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 shutdown_connections 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f22ec071e90 0x7f22ec074350 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f230c0c64e0 0x7f230c0c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f230c073cb0 0x7f230c13b900 secure :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f22f800c4b0 tx=0x7f22f800c970 comp rx=0 tx=0).stop 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 --2- 172.21.15.78:0/3539511993 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f230c0738d0 0x7f230c13b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 >> 172.21.15.78:0/3539511993 conn(0x7f230c0cd190 msgr2=0x7f230c142fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 shutdown_connections 2024-09-11T12:56:41.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.226+0000 7f2310a00640 1 -- 172.21.15.78:0/3539511993 wait complete. 2024-09-11T12:56:41.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- 172.21.15.78:0/3853910307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 msgr2=0x7f2fe00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 --2- 172.21.15.78:0/3853910307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2fe00051d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2ff0009b40 tx=0x7f2ff001d250 comp rx=0 tx=0).stop 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffe600640 1 -- 172.21.15.78:0/3853910307 reap_dead start 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- 172.21.15.78:0/3853910307 shutdown_connections 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- 172.21.15.78:0/3853910307 >> 172.21.15.78:0/3853910307 conn(0x7f2ff80b3050 msgr2=0x7f2ff80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- 172.21.15.78:0/3853910307 shutdown_connections 2024-09-11T12:56:41.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- 172.21.15.78:0/3853910307 wait complete. 2024-09-11T12:56:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 Processor -- start 2024-09-11T12:56:41.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- start start 2024-09-11T12:56:41.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:41.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ff815a700 0x7f2ff8147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:41.404 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 0x7f2ff80ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff80b58c0 con 0x7f2fe0004d50 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff80b5740 con 0x7f2ff81462a0 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f3000600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2ff80b5a40 con 0x7f2ff815a700 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffdc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffdc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55574/0 (socket says 172.21.15.78:55574) 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffdc00640 1 -- 172.21.15.78:0/2596464780 learned_addr learned my addr 172.21.15.78:0/2596464780 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffe600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 0x7f2ff80ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.398+0000 7f2ffd200640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ff815a700 0x7f2ff8147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 -- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ff815a700 msgr2=0x7f2ff8147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:41.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ff815a700 0x7f2ff8147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 -- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 msgr2=0x7f2ff80ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 0x7f2ff80ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 -- 172.21.15.78:0/2596464780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff8148510 con 0x7f2fe0004d50 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffe600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 0x7f2ff80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffdc00640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e980 tx=0x7f2fec00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec015070 con 0x7f2fe0004d50 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fec00cda0 con 0x7f2fe0004d50 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff815acc0 con 0x7f2fe0004d50 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0116c0 con 0x7f2fe0004d50 2024-09-11T12:56:41.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff815b280 con 0x7f2fe0004d50 2024-09-11T12:56:41.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff8071e10 con 0x7f2fe0004d50 2024-09-11T12:56:41.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f2fec018070 con 0x7f2fe0004d50 2024-09-11T12:56:41.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 0x7f2fd8074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:41.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5357+0+0 (secure 0 0 0) 0x7f2fec093750 con 0x7f2fe0004d50 2024-09-11T12:56:41.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.402+0000 7f2ffd200640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 0x7f2fd8074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:41.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.406+0000 7f2ffd200640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 0x7f2fd8074350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8150470 tx=0x7f2fe40091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:41.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.406+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2ff8148510 con 0x7f2fe0004d50 2024-09-11T12:56:41.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:41.574+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2ff8070ce0 con 0x7f2fe0004d50 2024-09-11T12:56:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.230+0000 7f2ff5600640 1 -- 172.21.15.78:0/2596464780 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) ==== 169+0+0 (secure 0 0 0) 0x7f2fec05e5f0 con 0x7f2fe0004d50 2024-09-11T12:56:42.234 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 msgr2=0x7f2fd8074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 0x7f2fd8074350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8150470 tx=0x7f2fe40091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 msgr2=0x7f2ff81f4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f2fec00e980 tx=0x7f2fec00ee40 comp rx=0 tx=0).stop 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 shutdown_connections 2024-09-11T12:56:42.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2fd8071e90 0x7f2fd8074350 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2ff81462a0 0x7f2ff80ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2ff815a700 0x7f2ff8147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 --2- 172.21.15.78:0/2596464780 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2fe0004d50 0x7f2ff81f4f50 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 >> 172.21.15.78:0/2596464780 conn(0x7f2ff80b3050 msgr2=0x7f2ff8148810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 shutdown_connections 2024-09-11T12:56:42.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.234+0000 7f3000600640 1 -- 172.21.15.78:0/2596464780 wait complete. 2024-09-11T12:56:42.259 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- 172.21.15.78:0/501388722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3740051d0 msgr2=0x7fb37400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 --2- 172.21.15.78:0/501388722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3740051d0 0x7fb37400b9d0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fb38800c310 tx=0x7fb38801cf60 comp rx=0 tx=0).stop 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- 172.21.15.78:0/501388722 shutdown_connections 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 --2- 172.21.15.78:0/501388722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3740051d0 0x7fb37400b9d0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- 172.21.15.78:0/501388722 >> 172.21.15.78:0/501388722 conn(0x7fb39006ca80 msgr2=0x7fb3900b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- 172.21.15.78:0/501388722 shutdown_connections 2024-09-11T12:56:42.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- 172.21.15.78:0/501388722 wait complete. 2024-09-11T12:56:42.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 Processor -- start 2024-09-11T12:56:42.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.410+0000 7fb395200640 1 -- start start 2024-09-11T12:56:42.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3740051d0 0x7fb3900ad570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3901f1a20 0x7fb39015a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb3900727f0 con 0x7fb3901f57f0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb390072670 con 0x7fb3740051d0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb39015e510 con 0x7fb3901f1a20 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55578/0 (socket says 172.21.15.78:55578) 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 -- 172.21.15.78:0/19105639 learned_addr learned my addr 172.21.15.78:0/19105639 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38de00640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3740051d0 0x7fb3900ad570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38d400640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3901f1a20 0x7fb39015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 -- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3901f1a20 msgr2=0x7fb39015a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3901f1a20 0x7fb39015a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 -- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3740051d0 msgr2=0x7fb3900ad570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3740051d0 0x7fb3900ad570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 -- 172.21.15.78:0/19105639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3900adab0 con 0x7fb3901f57f0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb38e800640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb38801d800 tx=0x7fb388006c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:42.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388001500 con 0x7fb3901f57f0 2024-09-11T12:56:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb388003600 con 0x7fb3901f57f0 2024-09-11T12:56:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb388003a30 con 0x7fb3901f57f0 2024-09-11T12:56:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3900a9fa0 con 0x7fb3901f57f0 2024-09-11T12:56:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3900aa560 con 0x7fb3901f57f0 2024-09-11T12:56:42.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.414+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb390070910 con 0x7fb3901f57f0 2024-09-11T12:56:42.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.418+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fb3880b8030 con 0x7fb3901f57f0 2024-09-11T12:56:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.418+0000 7fb386000640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 0x7fb370074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.418+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7fb38812c6c0 con 0x7fb3901f57f0 2024-09-11T12:56:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.418+0000 7fb38de00640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 0x7fb370074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.418+0000 7fb38de00640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 0x7fb370074320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb380009370 tx=0x7fb380009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:42.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.422+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb388016d10 con 0x7fb3901f57f0 2024-09-11T12:56:42.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.586+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fb3900aa760 con 0x7fb3901f57f0 2024-09-11T12:56:42.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb386000640 1 -- 172.21.15.78:0/19105639 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7fb3880fe0d0 con 0x7fb3901f57f0 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 msgr2=0x7fb370074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 0x7fb370074320 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb380009370 tx=0x7fb380009500 comp rx=0 tx=0).stop 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 msgr2=0x7fb3901f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fb38801d800 tx=0x7fb388006c80 comp rx=0 tx=0).stop 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 shutdown_connections 2024-09-11T12:56:42.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb3901f57f0 0x7fb3901f5c70 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb3901f1a20 0x7fb39015a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb370071e60 0x7fb370074320 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 --2- 172.21.15.78:0/19105639 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb3740051d0 0x7fb3900ad570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 >> 172.21.15.78:0/19105639 conn(0x7fb39006ca80 msgr2=0x7fb39015b860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 shutdown_connections 2024-09-11T12:56:42.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.594+0000 7fb395200640 1 -- 172.21.15.78:0/19105639 wait complete. 2024-09-11T12:56:42.617 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-11T12:56:42.731 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:56:42.734 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:56:42.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- 172.21.15.78:0/3481923620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e8c004d50 msgr2=0x7f7e8c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 --2- 172.21.15.78:0/3481923620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e8c004d50 0x7f7e8c0051d0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009ee0 tx=0x7f7ea001d630 comp rx=0 tx=0).stop 2024-09-11T12:56:42.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ea7200640 1 -- 172.21.15.78:0/3481923620 reap_dead start 2024-09-11T12:56:42.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- 172.21.15.78:0/3481923620 shutdown_connections 2024-09-11T12:56:42.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- 172.21.15.78:0/3481923620 >> 172.21.15.78:0/3481923620 conn(0x7f7ea8074b90 msgr2=0x7f7ea80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:42.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- 172.21.15.78:0/3481923620 shutdown_connections 2024-09-11T12:56:42.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- 172.21.15.78:0/3481923620 wait complete. 2024-09-11T12:56:42.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 Processor -- start 2024-09-11T12:56:42.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- start start 2024-09-11T12:56:42.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e8c004d50 0x7f7ea814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea80ce360 0x7f7ea80ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80aa1d0 con 0x7f7ea81e6da0 2024-09-11T12:56:42.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea80aa050 con 0x7f7e8c004d50 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ead400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7ea80aa350 con 0x7f7ea80ce360 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7e9fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7e9fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55592/0 (socket says 172.21.15.78:55592) 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7e9fe00640 1 -- 172.21.15.78:0/1774935681 learned_addr learned my addr 172.21.15.78:0/1774935681 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ea6800640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e8c004d50 0x7f7ea814c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.758+0000 7f7ea7200640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea80ce360 0x7f7ea80ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 -- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea80ce360 msgr2=0x7f7ea80ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea80ce360 0x7f7ea80ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 -- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e8c004d50 msgr2=0x7f7ea814c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e8c004d50 0x7f7ea814c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 -- 172.21.15.78:0/1774935681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea81f28b0 con 0x7f7ea81e6da0 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9fe00640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7e90011ab0 tx=0x7f7e90011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:42.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9001e070 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e9000fe70 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90004270 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea81f2b90 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81f3150 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea80ced00 con 0x7f7ea81e6da0 2024-09-11T12:56:42.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f7e90021070 con 0x7f7ea81e6da0 2024-09-11T12:56:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 0x7f7e84074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7f7e900938a0 con 0x7f7ea81e6da0 2024-09-11T12:56:42.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.762+0000 7f7ea6800640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 0x7f7e84074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:42.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.766+0000 7f7ea6800640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 0x7f7e84074320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e98001250 tx=0x7f7e980011e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:42.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.766+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7e9005e290 con 0x7f7ea81e6da0 2024-09-11T12:56:42.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:42.994+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f7ea815c990 con 0x7f7ea81e6da0 2024-09-11T12:56:43.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.234+0000 7f7e9ea00640 1 -- 172.21.15.78:0/1774935681 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v394) ==== 149+0+0 (secure 0 0 0) 0x7f7e900627e0 con 0x7f7ea81e6da0 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 msgr2=0x7f7e84074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 0x7f7e84074320 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7e98001250 tx=0x7f7e980011e0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 msgr2=0x7f7ea814e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7e90011ab0 tx=0x7f7e90011f70 comp rx=0 tx=0).stop 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 shutdown_connections 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7ea80ce360 0x7f7ea80ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7ea81e6da0 0x7f7ea814e860 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e84071e60 0x7f7e84074320 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 --2- 172.21.15.78:0/1774935681 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e8c004d50 0x7f7ea814c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 >> 172.21.15.78:0/1774935681 conn(0x7f7ea8074b90 msgr2=0x7f7ea80b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:43.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 shutdown_connections 2024-09-11T12:56:43.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.238+0000 7f7ead400640 1 -- 172.21.15.78:0/1774935681 wait complete. 2024-09-11T12:56:43.262 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-11T12:56:43.263 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-11T12:56:43.263 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:56:43.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- 172.21.15.78:0/2490544710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b015aec0 msgr2=0x7f47b0070830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- 172.21.15.78:0/2490544710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b015aec0 0x7f47b0070830 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f479c009900 tx=0x7f479c01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- 172.21.15.78:0/2490544710 shutdown_connections 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- 172.21.15.78:0/2490544710 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b015aec0 0x7f47b0070830 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- 172.21.15.78:0/2490544710 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b00cf760 0x7f47b0074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- 172.21.15.78:0/2490544710 >> 172.21.15.78:0/2490544710 conn(0x7f47b006ca80 msgr2=0x7f47b00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:43.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- 172.21.15.78:0/2490544710 shutdown_connections 2024-09-11T12:56:43.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- 172.21.15.78:0/2490544710 wait complete. 2024-09-11T12:56:43.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 Processor -- start 2024-09-11T12:56:43.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- start start 2024-09-11T12:56:43.408 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00cf760 0x7f47b01471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b015aec0 0x7f47b01476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0151820 con 0x7f47b006fc50 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47b00cfd50 con 0x7f47b00cf760 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.402+0000 7f47b7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47b01519a0 con 0x7f47b015aec0 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55620/0 (socket says 172.21.15.78:55620) 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 -- 172.21.15.78:0/1040740104 learned_addr learned my addr 172.21.15.78:0/1040740104 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47afe00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00cf760 0x7f47b01471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b5600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b015aec0 0x7f47b01476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 -- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b015aec0 msgr2=0x7f47b01476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b015aec0 0x7f47b01476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 -- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00cf760 msgr2=0x7f47b01471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00cf760 0x7f47b01471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 -- 172.21.15.78:0/1040740104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b0153510 con 0x7f47b006fc50 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b4c00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d950 tx=0x7f47a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a4017070 con 0x7f47b006fc50 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47a4009d20 con 0x7f47b006fc50 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b01537f0 con 0x7f47b006fc50 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40106c0 con 0x7f47b006fc50 2024-09-11T12:56:43.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b014d1e0 con 0x7f47b006fc50 2024-09-11T12:56:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b0153a30 con 0x7f47b006fc50 2024-09-11T12:56:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.406+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f47a4010860 con 0x7f47b006fc50 2024-09-11T12:56:43.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.410+0000 7f47acc00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 0x7f4790074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.410+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7f47a4090a40 con 0x7f47b006fc50 2024-09-11T12:56:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.410+0000 7f47afe00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 0x7f4790074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.410+0000 7f47afe00640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 0x7f4790074350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f479c0af000 tx=0x7f479c01b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:43.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.414+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f47b0153a30 con 0x7f47b006fc50 2024-09-11T12:56:43.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47b0074240 con 0x7f47b006fc50 2024-09-11T12:56:43.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47acc00640 1 -- 172.21.15.78:0/1040740104 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) ==== 74+0+13040 (secure 0 0 0) 0x7f47a405db50 con 0x7f47b006fc50 2024-09-11T12:56:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:43.585 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":143,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:42.234001+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/835134630":"2024-09-12T12:56:39.221083+0000","172.21.15.78:6834/835134630":"2024-09-12T12:56:39.221083+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 msgr2=0x7f4790074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 0x7f4790074350 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f479c0af000 tx=0x7f479c01b0f0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 msgr2=0x7f47b014cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f47a400d950 tx=0x7f47a400de10 comp rx=0 tx=0).stop 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 shutdown_connections 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47b015aec0 0x7f47b01476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4790071e90 0x7f4790074350 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47b00cf760 0x7f47b01471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 --2- 172.21.15.78:0/1040740104 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47b006fc50 0x7f47b014cca0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 >> 172.21.15.78:0/1040740104 conn(0x7f47b006ca80 msgr2=0x7f47b00b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:43.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 shutdown_connections 2024-09-11T12:56:43.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.582+0000 7f47b7600640 1 -- 172.21.15.78:0/1040740104 wait complete. 2024-09-11T12:56:43.606 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:56:43.606 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- 172.21.15.78:0/1306457187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 msgr2=0x7feab80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- 172.21.15.78:0/1306457187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab80b1a10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009840 tx=0x7feaa801c820 comp rx=0 tx=0).stop 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- 172.21.15.78:0/1306457187 shutdown_connections 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- 172.21.15.78:0/1306457187 >> v2:172.21.15.157:3300/0 conn(0x7feab80b1f50 0x7feab80cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- 172.21.15.78:0/1306457187 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab80b1a10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- 172.21.15.78:0/1306457187 >> 172.21.15.78:0/1306457187 conn(0x7feab8071dd0 msgr2=0x7feab80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:43.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- 172.21.15.78:0/1306457187 shutdown_connections 2024-09-11T12:56:43.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- 172.21.15.78:0/1306457187 wait complete. 2024-09-11T12:56:43.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 Processor -- start 2024-09-11T12:56:43.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- start start 2024-09-11T12:56:43.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 0x7feab81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab81f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7feab8167960 con 0x7feab80b1f50 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7feab81677e0 con 0x7feab80ce150 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabf800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7feab8167ae0 con 0x7feab81518b0 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:55634/0 (socket says 172.21.15.78:55634) 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 -- 172.21.15.78:0/976727429 learned_addr learned my addr 172.21.15.78:0/976727429 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feab7e00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 0x7feab81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabd800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab81f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 -- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 msgr2=0x7feab81f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab81f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 -- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 msgr2=0x7feab81555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 0x7feab81555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feabce00640 1 -- 172.21.15.78:0/976727429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab81475c0 con 0x7feab80b1f50 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.750+0000 7feab7e00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 0x7feab81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feabce00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009790 tx=0x7feaa80a6500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa80bd020 con 0x7feab80b1f50 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feaa80a66f0 con 0x7feab80b1f50 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab81478a0 con 0x7feab80b1f50 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaa8003600 con 0x7feab80b1f50 2024-09-11T12:56:43.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab8158fa0 con 0x7feab80b1f50 2024-09-11T12:56:43.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab80cdc10 con 0x7feab80b1f50 2024-09-11T12:56:43.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7feaa80c0020 con 0x7feab80b1f50 2024-09-11T12:56:43.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 0x7fea98074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:43.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7feaa812da10 con 0x7feab80b1f50 2024-09-11T12:56:43.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.754+0000 7feab7e00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 0x7fea98074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:43.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.758+0000 7feab7e00640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 0x7fea98074320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feab806e940 tx=0x7feaa40091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:43.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.758+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7feaa80c7030 con 0x7feab80b1f50 2024-09-11T12:56:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7feab815cac0 con 0x7feab80b1f50 2024-09-11T12:56:43.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feab4c00640 1 -- 172.21.15.78:0/976727429 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v143) ==== 93+0+10 (secure 0 0 0) 0x7feaa8016410 con 0x7feab80b1f50 2024-09-11T12:56:43.929 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 msgr2=0x7fea98074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 0x7fea98074320 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7feab806e940 tx=0x7feaa40091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 msgr2=0x7feab8146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7feaa8009790 tx=0x7feaa80a6500 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 shutdown_connections 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7feab81518b0 0x7feab81f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea98071e60 0x7fea98074320 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7feab80ce150 0x7feab81555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 --2- 172.21.15.78:0/976727429 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7feab80b1f50 0x7feab8146390 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:43.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 >> 172.21.15.78:0/976727429 conn(0x7feab8071dd0 msgr2=0x7feab80b3880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 shutdown_connections 2024-09-11T12:56:43.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:43.926+0000 7feabf800640 1 -- 172.21.15.78:0/976727429 wait complete. 2024-09-11T12:56:43.951 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-11T12:56:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- 172.21.15.78:0/4232774107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 msgr2=0x7f586815af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 --2- 172.21.15.78:0/4232774107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f586815af90 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f58680b58b0 tx=0x7f585001c810 comp rx=0 tx=0).stop 2024-09-11T12:56:44.093 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- 172.21.15.78:0/4232774107 shutdown_connections 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 --2- 172.21.15.78:0/4232774107 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f586815af90 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 --2- 172.21.15.78:0/4232774107 >> v2:172.21.15.157:3300/0 conn(0x7f58680b4db0 0x7f58680b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 --2- 172.21.15.78:0/4232774107 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 0x7f5868070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- 172.21.15.78:0/4232774107 >> 172.21.15.78:0/4232774107 conn(0x7f586806ca80 msgr2=0x7f58680b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- 172.21.15.78:0/4232774107 shutdown_connections 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- 172.21.15.78:0/4232774107 wait complete. 2024-09-11T12:56:44.094 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 Processor -- start 2024-09-11T12:56:44.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.090+0000 7f586d400640 1 -- start start 2024-09-11T12:56:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:44.096 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4db0 0x7f586814c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 0x7f5868154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f586806f0c0 con 0x7f58680736d0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f586806ef40 con 0x7f58680b4db0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f586806f240 con 0x7f5868157ae0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48898/0 (socket says 172.21.15.78:48898) 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 -- 172.21.15.78:0/3980761397 learned_addr learned my addr 172.21.15.78:0/3980761397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f585fe00640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4db0 0x7f586814c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5867200640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 0x7f5868154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 -- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 msgr2=0x7f5868154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 0x7f5868154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 -- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4db0 msgr2=0x7f586814c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4db0 0x7f586814c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:44.097 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 -- 172.21.15.78:0/3980761397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5868150e90 con 0x7f58680736d0 2024-09-11T12:56:44.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f5866800640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f585800d750 tx=0x7f585800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:44.098 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858017070 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5858009cc0 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5858005280 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5868151170 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5868151730 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680b51f0 con 0x7f58680736d0 2024-09-11T12:56:44.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.094+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f5858005420 con 0x7f58680736d0 2024-09-11T12:56:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.098+0000 7f585ea00640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 0x7f5844074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.098+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7f5858092ed0 con 0x7f58680736d0 2024-09-11T12:56:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.098+0000 7f585fe00640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 0x7f5844074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:44.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.098+0000 7f585fe00640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 0x7f5844074320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f58500ae000 tx=0x7f585001b050 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:44.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.102+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f585800de60 con 0x7f58680736d0 2024-09-11T12:56:44.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:44.326+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f58680b2020 con 0x7f58680736d0 2024-09-11T12:56:45.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.242+0000 7f585ea00640 1 -- 172.21.15.78:0/3980761397 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v396) ==== 148+0+0 (secure 0 0 0) 0x7f585805df10 con 0x7f58680736d0 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 msgr2=0x7f5844074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 0x7f5844074320 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f58500ae000 tx=0x7f585001b050 comp rx=0 tx=0).stop 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 msgr2=0x7f58681468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f585800d750 tx=0x7f585800dc10 comp rx=0 tx=0).stop 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 shutdown_connections 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5868157ae0 0x7f5868154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5844071e60 0x7f5844074320 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f58680b4db0 0x7f586814c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 --2- 172.21.15.78:0/3980761397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f58680736d0 0x7f58681468b0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 >> 172.21.15.78:0/3980761397 conn(0x7f586806ca80 msgr2=0x7f58680cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 shutdown_connections 2024-09-11T12:56:45.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.246+0000 7f586d400640 1 -- 172.21.15.78:0/3980761397 wait complete. 2024-09-11T12:56:45.273 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:56:45.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 -- 172.21.15.78:0/3495682151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d00d0b30 msgr2=0x7f82d00d0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 --2- 172.21.15.78:0/3495682151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d00d0b30 0x7f82d00d0fe0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f82d00d2d30 tx=0x7f82c801cd80 comp rx=0 tx=0).stop 2024-09-11T12:56:45.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 -- 172.21.15.78:0/3495682151 shutdown_connections 2024-09-11T12:56:45.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 --2- 172.21.15.78:0/3495682151 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d00d0b30 0x7f82d00d0fe0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 --2- 172.21.15.78:0/3495682151 >> v2:172.21.15.157:3300/0 conn(0x7f82d015f7b0 0x7f82d015fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 --2- 172.21.15.78:0/3495682151 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 0x7f82d015d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 -- 172.21.15.78:0/3495682151 >> 172.21.15.78:0/3495682151 conn(0x7f82d0074b90 msgr2=0x7f82d00d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 -- 172.21.15.78:0/3495682151 shutdown_connections 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.414+0000 7f82d6a00640 1 -- 172.21.15.78:0/3495682151 wait complete. 2024-09-11T12:56:45.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 Processor -- start 2024-09-11T12:56:45.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- start start 2024-09-11T12:56:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 0x7f82d00b2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 0x7f82d01faab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82d015c550 con 0x7f82d015f7b0 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f82d015c3d0 con 0x7f82d00d0b30 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f82d015c6d0 con 0x7f82d0140bd0 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 0x7f82d00b2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82cfe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 0x7f82d00b2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:52638/0 (socket says 172.21.15.78:52638) 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82c7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 0x7f82d01faab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82cfe00640 1 -- 172.21.15.78:0/1425289108 learned_addr learned my addr 172.21.15.78:0/1425289108 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48912/0 (socket says 172.21.15.78:48912) 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 -- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 msgr2=0x7f82d01faab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 0x7f82d01faab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 -- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 msgr2=0x7f82d00b2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 0x7f82d00b2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 -- 172.21.15.78:0/1425289108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d01fb320 con 0x7f82d015f7b0 2024-09-11T12:56:45.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d4a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ac000 tx=0x7f82c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80bb070 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82c8004b40 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c80b2040 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d01fb8e0 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d01faff0 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.418+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d00d0fe0 con 0x7f82d015f7b0 2024-09-11T12:56:45.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.422+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f82c8004460 con 0x7f82d015f7b0 2024-09-11T12:56:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.422+0000 7f82ccc00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 0x7f82ac074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.422+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7f82c812c9b0 con 0x7f82d015f7b0 2024-09-11T12:56:45.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.422+0000 7f82cfe00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 0x7f82ac074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.422+0000 7f82cfe00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 0x7f82ac074320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bc40 tx=0x7f82c000b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:45.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.426+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f82c80f1da0 con 0x7f82d015f7b0 2024-09-11T12:56:45.597 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.594+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f82d015f320 con 0x7f82d015f7b0 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.594+0000 7f82ccc00640 1 -- 172.21.15.78:0/1425289108 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+1998 (secure 0 0 0) 0x7f82c8016970 con 0x7f82d015f7b0 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:56:45.598 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:56:45.599 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:56:45.600 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:56:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 msgr2=0x7f82ac074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 0x7f82ac074320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f82c000bc40 tx=0x7f82c000b390 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 msgr2=0x7f82d01f7d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f82c80ac000 tx=0x7f82c8004270 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 shutdown_connections 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82d015f7b0 0x7f82d01f7d10 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82d0140bd0 0x7f82d01faab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82ac071e60 0x7f82ac074320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 --2- 172.21.15.78:0/1425289108 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82d00d0b30 0x7f82d00b2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 >> 172.21.15.78:0/1425289108 conn(0x7f82d0074b90 msgr2=0x7f82d0159d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 shutdown_connections 2024-09-11T12:56:45.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.598+0000 7f82d6a00640 1 -- 172.21.15.78:0/1425289108 wait complete. 2024-09-11T12:56:45.619 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:56:45.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- 172.21.15.78:0/3706295249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64005220 msgr2=0x7fdc6400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3706295249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64005220 0x7fdc6400b9d0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fdc78001f50 tx=0x7fdc7801cc40 comp rx=0 tx=0).stop 2024-09-11T12:56:45.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- 172.21.15.78:0/3706295249 shutdown_connections 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3706295249 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc64005220 0x7fdc6400b9d0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- 172.21.15.78:0/3706295249 >> 172.21.15.78:0/3706295249 conn(0x7fdc80071dd0 msgr2=0x7fdc800721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- 172.21.15.78:0/3706295249 shutdown_connections 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- 172.21.15.78:0/3706295249 wait complete. 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 Processor -- start 2024-09-11T12:56:45.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.762+0000 7fdc85c00640 1 -- start start 2024-09-11T12:56:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc64005220 0x7fdc801f0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 0x7fdc8015af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800cdd90 con 0x7fdc80144120 2024-09-11T12:56:45.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdc800cdc10 con 0x7fdc801485c0 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdc800cdf10 con 0x7fdc64005220 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48930/0 (socket says 172.21.15.78:48930) 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 -- 172.21.15.78:0/3142745051 learned_addr learned my addr 172.21.15.78:0/3142745051 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7ea00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc64005220 0x7fdc801f0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7e000640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 0x7fdc8015af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 -- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc64005220 msgr2=0x7fdc801f0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc64005220 0x7fdc801f0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 -- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 msgr2=0x7fdc8015af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 0x7fdc8015af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 -- 172.21.15.78:0/3142745051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc80144ba0 con 0x7fdc80144120 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7e000640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 0x7fdc8015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc7f400640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdc7801d4e0 tx=0x7fdc78004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:45.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc780a5af0 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc78006210 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc801f1550 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78006640 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc801f1b10 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.766+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc800b1c40 con 0x7fdc80144120 2024-09-11T12:56:45.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.770+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fdc780bb020 con 0x7fdc80144120 2024-09-11T12:56:45.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.770+0000 7fdc76a00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 0x7fdc60074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:45.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.770+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7fdc7812c910 con 0x7fdc80144120 2024-09-11T12:56:45.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.770+0000 7fdc7ea00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 0x7fdc60074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:45.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.770+0000 7fdc7ea00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 0x7fdc60074320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc700017f0 tx=0x7fdc70001120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:45.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.774+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdc780c2020 con 0x7fdc80144120 2024-09-11T12:56:45.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.938+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fdc80147f60 con 0x7fdc80144120 2024-09-11T12:56:45.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.942+0000 7fdc76a00640 1 -- 172.21.15.78:0/3142745051 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fdc780169f0 con 0x7fdc80144120 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 msgr2=0x7fdc60074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 0x7fdc60074320 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fdc700017f0 tx=0x7fdc70001120 comp rx=0 tx=0).stop 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 msgr2=0x7fdc801461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fdc7801d4e0 tx=0x7fdc78004ff0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 shutdown_connections 2024-09-11T12:56:45.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdc80144120 0x7fdc801461b0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc60071e60 0x7fdc60074320 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdc801485c0 0x7fdc8015af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 --2- 172.21.15.78:0/3142745051 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdc64005220 0x7fdc801f0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 >> 172.21.15.78:0/3142745051 conn(0x7fdc80071dd0 msgr2=0x7fdc8006fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 shutdown_connections 2024-09-11T12:56:45.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:45.946+0000 7fdc85c00640 1 -- 172.21.15.78:0/3142745051 wait complete. 2024-09-11T12:56:45.967 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:56:45.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:46.046 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:56:46.149 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:56:46.149 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:46.149 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:46.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:46.158 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:46.158 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:46.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:46.168 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:56:46.168 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:46.207 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:56:46.207 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:56:46.207 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:56:46.218 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:56:46.218 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:56:46.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- 172.21.15.78:0/1074481167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 msgr2=0x7f513415af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- 172.21.15.78:0/1074481167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f513415af10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f5124009bc0 tx=0x7f512401cb10 comp rx=0 tx=0).stop 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- 172.21.15.78:0/1074481167 shutdown_connections 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- 172.21.15.78:0/1074481167 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340707f0 0x7f513415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- 172.21.15.78:0/1074481167 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f513415af10 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- 172.21.15.78:0/1074481167 >> 172.21.15.78:0/1074481167 conn(0x7f513406ca80 msgr2=0x7f51340b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- 172.21.15.78:0/1074481167 shutdown_connections 2024-09-11T12:56:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- 172.21.15.78:0/1074481167 wait complete. 2024-09-11T12:56:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 Processor -- start 2024-09-11T12:56:46.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- start start 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 0x7f51341f8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340b4db0 0x7f5134153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5134166e70 con 0x7f5134073ed0 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5134166cf0 con 0x7f51340707f0 2024-09-11T12:56:46.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f513b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5134166ff0 con 0x7f51340b4db0 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48938/0 (socket says 172.21.15.78:48938) 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 -- 172.21.15.78:0/1776993503 learned_addr learned my addr 172.21.15.78:0/1776993503 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5138e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 0x7f51341f8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5139800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340b4db0 0x7f5134153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 -- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340b4db0 msgr2=0x7f5134153140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340b4db0 0x7f5134153140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 -- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 msgr2=0x7f51341f8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 0x7f51341f8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5133e00640 1 -- 172.21.15.78:0/1776993503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5134151730 con 0x7f5134073ed0 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.390+0000 7f5138e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 0x7f51341f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:46.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f5133e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5120005d10 tx=0x7f512000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:46.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512001b070 con 0x7f5134073ed0 2024-09-11T12:56:46.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f512000e450 con 0x7f5134073ed0 2024-09-11T12:56:46.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512000e8a0 con 0x7f5134073ed0 2024-09-11T12:56:46.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5134151a10 con 0x7f5134073ed0 2024-09-11T12:56:46.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5134151fd0 con 0x7f5134073ed0 2024-09-11T12:56:46.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f513415db60 con 0x7f5134073ed0 2024-09-11T12:56:46.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.394+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f512001e070 con 0x7f5134073ed0 2024-09-11T12:56:46.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.398+0000 7f5130c00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 0x7f5114074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.398+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7f5120090b00 con 0x7f5134073ed0 2024-09-11T12:56:46.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.398+0000 7f5138e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 0x7f5114074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.398+0000 7f5138e00640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 0x7f5114074320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f51240af000 tx=0x7f512401b3b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:46.403 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.402+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f512005dc10 con 0x7f5134073ed0 2024-09-11T12:56:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.598+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51340695e0 con 0x7f5134073ed0 2024-09-11T12:56:46.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f5130c00640 1 -- 172.21.15.78:0/1776993503 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) ==== 95+0+3916 (secure 0 0 0) 0x7f5120062160 con 0x7f5134073ed0 2024-09-11T12:56:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:46.604 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":104,"btime":"2024-09-11T12:56:40:223564+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1770039168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1770039168},{"type":"v1","addr":"172.21.15.157:6801","nonce":1770039168}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7762,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/4250158681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":4250158681},{"type":"v1","addr":"172.21.15.157:6835","nonce":4250158681}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8265,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6833/441618508","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":441618508},{"type":"v1","addr":"172.21.15.78:6833","nonce":441618508}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":95},{"gid":8541,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/2021702866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6834","nonce":2021702866},{"type":"v1","addr":"172.21.15.78:6835","nonce":2021702866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104}],"filesystems":[]} 2024-09-11T12:56:46.604 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 104 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 msgr2=0x7f5114074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 0x7f5114074320 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f51240af000 tx=0x7f512401b3b0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 msgr2=0x7f5134143810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f5120005d10 tx=0x7f512000a620 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 shutdown_connections 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f51340b4db0 0x7f5134153140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5134073ed0 0x7f5134143810 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5114071e60 0x7f5114074320 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 --2- 172.21.15.78:0/1776993503 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f51340707f0 0x7f51341f8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 >> 172.21.15.78:0/1776993503 conn(0x7f513406ca80 msgr2=0x7f513414fa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 shutdown_connections 2024-09-11T12:56:46.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.602+0000 7f513b800640 1 -- 172.21.15.78:0/1776993503 wait complete. 2024-09-11T12:56:46.626 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:56:46.626 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:56:46.626 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:56:46.626 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:46.627 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:56:46.627 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:56:46.627 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:56:46.627 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:46.627 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:56:46.627 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:56:46.627 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:56:46.628 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:46.628 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:56:46.628 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:56:46.628 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:56:46.628 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:56:46.638 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:56:46.638 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:56:46.640 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:56:46.640 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:56:46.645 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:56:46.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:56:46.646 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:56:46.646 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:56:46.685 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:56:46.686 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:46.687 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:56:46.688 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:46.693 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:56:46.693 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:46.694 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:56:46.695 DEBUG:teuthology.parallel:result is None 2024-09-11T12:56:46.695 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:56:46.772 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:56:46.773 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:56:46.784 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:56:46.785 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 -- 172.21.15.78:0/816076146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad80b4c30 msgr2=0x7fdad815cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 --2- 172.21.15.78:0/816076146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad80b4c30 0x7fdad815cf00 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009900 tx=0x7fdac401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 -- 172.21.15.78:0/816076146 shutdown_connections 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 --2- 172.21.15.78:0/816076146 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad80b4c30 0x7fdad815cf00 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 -- 172.21.15.78:0/816076146 >> 172.21.15.78:0/816076146 conn(0x7fdad806ca80 msgr2=0x7fdad80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 -- 172.21.15.78:0/816076146 shutdown_connections 2024-09-11T12:56:46.846 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.842+0000 7fdadf800640 1 -- 172.21.15.78:0/816076146 wait complete. 2024-09-11T12:56:46.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 Processor -- start 2024-09-11T12:56:46.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- start start 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 0x7fdad81f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdad815eda0 0x7fdad8149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdad8169a50 con 0x7fdad815bee0 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdad81698d0 con 0x7fdad80b4c30 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdad8169bd0 con 0x7fdad815eda0 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:48984/0 (socket says 172.21.15.78:48984) 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 -- 172.21.15.78:0/1455903194 learned_addr learned my addr 172.21.15.78:0/1455903194 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadd800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdad815eda0 0x7fdad8149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadce00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 0x7fdad81f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 -- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdad815eda0 msgr2=0x7fdad8149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdad815eda0 0x7fdad8149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 -- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 msgr2=0x7fdad81f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 0x7fdad81f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 -- 172.21.15.78:0/1455903194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad80b3da0 con 0x7fdad815bee0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadce00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 0x7fdad81f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad7e00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009850 tx=0x7fdac40a58d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40b2040 con 0x7fdad815bee0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac4005310 con 0x7fdad815bee0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad81485b0 con 0x7fdad815bee0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4003700 con 0x7fdad815bee0 2024-09-11T12:56:46.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad8148e50 con 0x7fdad815bee0 2024-09-11T12:56:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.846+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad81489a0 con 0x7fdad815bee0 2024-09-11T12:56:46.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.850+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fdac40038a0 con 0x7fdad815bee0 2024-09-11T12:56:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.850+0000 7fdad4c00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 0x7fdab8074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.850+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 4943+0+0 (secure 0 0 0) 0x7fdac412ce50 con 0x7fdad815bee0 2024-09-11T12:56:46.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.850+0000 7fdadce00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 0x7fdab8074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:46.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.850+0000 7fdadce00640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 0x7fdab8074320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdad81f2e60 tx=0x7fdacc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:46.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:46.854+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdac40164d0 con 0x7fdad815bee0 2024-09-11T12:56:47.035 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.030+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fdad815cf00 con 0x7fdad815bee0 2024-09-11T12:56:47.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.258+0000 7fdad4c00640 1 -- 172.21.15.78:0/1455903194 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v145) ==== 97+0+0 (secure 0 0 0) 0x7fdac40f84d0 con 0x7fdad815bee0 2024-09-11T12:56:47.261 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:56:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 msgr2=0x7fdab8074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 0x7fdab8074320 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdad81f2e60 tx=0x7fdacc009040 comp rx=0 tx=0).stop 2024-09-11T12:56:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 msgr2=0x7fdad8145180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009850 tx=0x7fdac40a58d0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 shutdown_connections 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdad815eda0 0x7fdad8149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdad815bee0 0x7fdad8145180 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdab8071e60 0x7fdab8074320 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 --2- 172.21.15.78:0/1455903194 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdad80b4c30 0x7fdad81f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 >> 172.21.15.78:0/1455903194 conn(0x7fdad806ca80 msgr2=0x7fdad814e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 shutdown_connections 2024-09-11T12:56:47.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.262+0000 7fdadf800640 1 -- 172.21.15.78:0/1455903194 wait complete. 2024-09-11T12:56:47.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:56:47.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- 172.21.15.78:0/3437713119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 msgr2=0x7fdca0070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 --2- 172.21.15.78:0/3437713119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca0070a00 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fdca015dad0 tx=0x7fdc9001de20 comp rx=0 tx=0).stop 2024-09-11T12:56:47.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- 172.21.15.78:0/3437713119 shutdown_connections 2024-09-11T12:56:47.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 --2- 172.21.15.78:0/3437713119 >> v2:172.21.15.157:3300/0 conn(0x7fdca00b4db0 0x7fdca00b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 --2- 172.21.15.78:0/3437713119 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca0070a00 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- 172.21.15.78:0/3437713119 >> 172.21.15.78:0/3437713119 conn(0x7fdca006ca80 msgr2=0x7fdca00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- 172.21.15.78:0/3437713119 shutdown_connections 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- 172.21.15.78:0/3437713119 wait complete. 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 Processor -- start 2024-09-11T12:56:47.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.426+0000 7fdca7a00640 1 -- start start 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca00736d0 0x7fdca015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca00b4db0 0x7fdca0154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdca006f200 con 0x7fdca0157ae0 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdca006f080 con 0x7fdca00b4db0 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdca006f380 con 0x7fdca00736d0 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49006/0 (socket says 172.21.15.78:49006) 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 -- 172.21.15.78:0/2829552576 learned_addr learned my addr 172.21.15.78:0/2829552576 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:47.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5000640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca00736d0 0x7fdca015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdc9fe00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca00b4db0 0x7fdca0154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 -- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca00736d0 msgr2=0x7fdca015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca00736d0 0x7fdca015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 -- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca00b4db0 msgr2=0x7fdca0154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca00b4db0 0x7fdca0154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 -- 172.21.15.78:0/2829552576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdca01546c0 con 0x7fdca0157ae0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca5a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d750 tx=0x7fdc9800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc98017070 con 0x7fdca0157ae0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc98009d20 con 0x7fdca0157ae0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc9800de60 con 0x7fdca0157ae0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdca0155020 con 0x7fdca0157ae0 2024-09-11T12:56:47.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdca00ab3e0 con 0x7fdca0157ae0 2024-09-11T12:56:47.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdca0073b00 con 0x7fdca0157ae0 2024-09-11T12:56:47.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.430+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fdc98005280 con 0x7fdca0157ae0 2024-09-11T12:56:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.434+0000 7fdc9cc00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 0x7fdc80074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.434+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4857+0+0 (secure 0 0 0) 0x7fdc980907d0 con 0x7fdca0157ae0 2024-09-11T12:56:47.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.434+0000 7fdca5000640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 0x7fdc80074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.434+0000 7fdca5000640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 0x7fdc80074320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdc900b0000 tx=0x7fdc9001c660 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:47.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.438+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdca0155020 con 0x7fdca0157ae0 2024-09-11T12:56:47.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.662+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdca0159310 con 0x7fdca0157ae0 2024-09-11T12:56:47.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.662+0000 7fdc9cc00640 1 -- 172.21.15.78:0/2829552576 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) ==== 80+0+5304 (secure 0 0 0) 0x7fdc9805df30 con 0x7fdca0157ae0 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:56:47.668 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.669 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:56:47.672 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.673 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:56:47.675 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:56:47.676 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:56:47.677 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:56:47.678 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.679 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:56:47.680 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:56:47.680 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.680 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:56:47.680 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:56:47.680 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.681 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:56:47.682 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:56:47.683 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:56:47.684 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:56:47.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 msgr2=0x7fdc80074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 0x7fdc80074320 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdc900b0000 tx=0x7fdc9001c660 comp rx=0 tx=0).stop 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 msgr2=0x7fdca00ce0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fdc9800d750 tx=0x7fdc9800dc10 comp rx=0 tx=0).stop 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 shutdown_connections 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdca0157ae0 0x7fdca00ce0c0 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdc80071e60 0x7fdc80074320 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdca00b4db0 0x7fdca0154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 --2- 172.21.15.78:0/2829552576 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdca00736d0 0x7fdca015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 >> 172.21.15.78:0/2829552576 conn(0x7fdca006ca80 msgr2=0x7fdca00cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 shutdown_connections 2024-09-11T12:56:47.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.666+0000 7fdca7a00640 1 -- 172.21.15.78:0/2829552576 wait complete. 2024-09-11T12:56:47.688 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:56:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3318794589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c0039c0 msgr2=0x7fbe4c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.832 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3318794589 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe4c0039c0 0x7fbe4c003de0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fbe58009ee0 tx=0x7fbe5801d5c0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 -- 172.21.15.78:0/3318794589 reap_dead start 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3318794589 shutdown_connections 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3318794589 >> 172.21.15.78:0/3318794589 conn(0x7fbe6806ca80 msgr2=0x7fbe680b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3318794589 shutdown_connections 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3318794589 wait complete. 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 Processor -- start 2024-09-11T12:56:47.833 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- start start 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 0x7fbe681fb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe681f76a0 0x7fbe68152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680cd460 con 0x7fbe68153af0 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe680cd2e0 con 0x7fbe4c0039c0 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe680cd5e0 con 0x7fbe681f76a0 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49024/0 (socket says 172.21.15.78:49024) 2024-09-11T12:56:47.835 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 -- 172.21.15.78:0/3025416481 learned_addr learned my addr 172.21.15.78:0/3025416481 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe67e00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 0x7fbe681fb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe67400640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe681f76a0 0x7fbe68152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 -- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe681f76a0 msgr2=0x7fbe68152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe681f76a0 0x7fbe68152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 -- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 msgr2=0x7fbe681fb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 0x7fbe681fb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe6ce00640 1 -- 172.21.15.78:0/3025416481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe681fbbe0 con 0x7fbe68153af0 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.830+0000 7fbe67e00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 0x7fbe681fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:47.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe6ce00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbe680b2680 tx=0x7fbe60011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60004230 con 0x7fbe68153af0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe600043d0 con 0x7fbe68153af0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6000fb50 con 0x7fbe68153af0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe681f7bb0 con 0x7fbe68153af0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe681f8170 con 0x7fbe68153af0 2024-09-11T12:56:47.837 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe680cdb70 con 0x7fbe68153af0 2024-09-11T12:56:47.838 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fbe6000fcf0 con 0x7fbe68153af0 2024-09-11T12:56:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 0x7fbe48074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4857+0+0 (secure 0 0 0) 0x7fbe60090940 con 0x7fbe68153af0 2024-09-11T12:56:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.834+0000 7fbe67e00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 0x7fbe48074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:47.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.838+0000 7fbe67e00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 0x7fbe48074350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe580b0000 tx=0x7fbe5801a360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:47.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:47.838+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbe60020030 con 0x7fbe68153af0 2024-09-11T12:56:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.006+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbe68153420 con 0x7fbe68153af0 2024-09-11T12:56:48.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.006+0000 7fbe5fe00640 1 -- 172.21.15.78:0/3025416481 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v147) ==== 84+0+65 (secure 0 0 0) 0x7fbe60094090 con 0x7fbe68153af0 2024-09-11T12:56:48.011 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:56:48.012 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:56:48.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 msgr2=0x7fbe48074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 0x7fbe48074350 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbe580b0000 tx=0x7fbe5801a360 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 msgr2=0x7fbe68073350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fbe680b2680 tx=0x7fbe60011ba0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 shutdown_connections 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe68153af0 0x7fbe68073350 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe681f76a0 0x7fbe68152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe48071e90 0x7fbe48074350 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 --2- 172.21.15.78:0/3025416481 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe4c0039c0 0x7fbe681fb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 >> 172.21.15.78:0/3025416481 conn(0x7fbe6806ca80 msgr2=0x7fbe6815cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 shutdown_connections 2024-09-11T12:56:48.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.010+0000 7fbe6ee00640 1 -- 172.21.15.78:0/3025416481 wait complete. 2024-09-11T12:56:48.034 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:56:48.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:56:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 -- 172.21.15.78:0/1176130065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c4004810 msgr2=0x7f12c4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 --2- 172.21.15.78:0/1176130065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12c4004810 0x7f12c4004c90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009b60 tx=0x7f12cc01d270 comp rx=0 tx=0).stop 2024-09-11T12:56:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e4e00640 1 -- 172.21.15.78:0/1176130065 reap_dead start 2024-09-11T12:56:48.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 -- 172.21.15.78:0/1176130065 shutdown_connections 2024-09-11T12:56:48.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 -- 172.21.15.78:0/1176130065 >> 172.21.15.78:0/1176130065 conn(0x7f12e00b3050 msgr2=0x7f12e00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:48.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 -- 172.21.15.78:0/1176130065 shutdown_connections 2024-09-11T12:56:48.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.178+0000 7f12e6e00640 1 -- 172.21.15.78:0/1176130065 wait complete. 2024-09-11T12:56:48.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 Processor -- start 2024-09-11T12:56:48.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- start start 2024-09-11T12:56:48.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 0x7f12e015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12e015dbb0 0x7f12e015e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f12e00cde70 con 0x7f12e0149120 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f12e00b59c0 con 0x7f12c4004810 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f12e015c520 con 0x7f12e015dbb0 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49044/0 (socket says 172.21.15.78:49044) 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 -- 172.21.15.78:0/277274203 learned_addr learned my addr 172.21.15.78:0/277274203 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12dfe00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 0x7f12e015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e4e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12e015dbb0 0x7f12e015e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:48.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 -- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12e015dbb0 msgr2=0x7f12e015e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12e015dbb0 0x7f12e015e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 -- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 msgr2=0x7f12e015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 0x7f12e015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 -- 172.21.15.78:0/277274203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e015e980 con 0x7f12e0149120 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12dfe00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 0x7f12e015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12df400640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0af040 tx=0x7f12cc0acd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0bc030 con 0x7f12e0149120 2024-09-11T12:56:48.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12cc0ac400 con 0x7f12e0149120 2024-09-11T12:56:48.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0b1040 con 0x7f12e0149120 2024-09-11T12:56:48.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e015ec60 con 0x7f12e0149120 2024-09-11T12:56:48.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e013ff30 con 0x7f12e0149120 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.182+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e00cf960 con 0x7f12e0149120 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.186+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f12cc0bf070 con 0x7f12e0149120 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.186+0000 7f12d7e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 0x7f12c0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.186+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 4857+0+0 (secure 0 0 0) 0x7f12cc12d990 con 0x7f12e0149120 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.186+0000 7f12dfe00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 0x7f12c0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:48.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.186+0000 7f12dfe00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 0x7f12c0074320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009a90 tx=0x7f12d00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:48.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.190+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f12cc016fc0 con 0x7f12e0149120 2024-09-11T12:56:48.366 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:48.362+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f12e00cd950 con 0x7f12e0149120 2024-09-11T12:56:49.274 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.270+0000 7f12d7e00640 1 -- 172.21.15.78:0/277274203 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v148) ==== 138+0+0 (secure 0 0 0) 0x7f12cc101100 con 0x7f12e0149120 2024-09-11T12:56:49.274 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:56:49.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 msgr2=0x7f12c0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:49.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 0x7f12c0074320 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f12d0009a90 tx=0x7f12d00091e0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 msgr2=0x7f12e013f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:49.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f12cc0af040 tx=0x7f12cc0acd20 comp rx=0 tx=0).stop 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 shutdown_connections 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f12e015dbb0 0x7f12e015e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f12e0149120 0x7f12e013f9f0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f12c0071e60 0x7f12c0074320 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 --2- 172.21.15.78:0/277274203 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f12c4004810 0x7f12e015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 >> 172.21.15.78:0/277274203 conn(0x7f12e00b3050 msgr2=0x7f12e0153ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 shutdown_connections 2024-09-11T12:56:49.278 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.274+0000 7f12e6e00640 1 -- 172.21.15.78:0/277274203 wait complete. 2024-09-11T12:56:49.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:56:49.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 -- 172.21.15.78:0/3813058268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 msgr2=0x7fb2ec15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 --2- 172.21.15.78:0/3813058268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec15c440 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc009840 tx=0x7fb2dc01c740 comp rx=0 tx=0).stop 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 -- 172.21.15.78:0/3813058268 shutdown_connections 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 --2- 172.21.15.78:0/3813058268 >> v2:172.21.15.157:3300/0 conn(0x7fb2ec072e70 0x7fb2ec15c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 --2- 172.21.15.78:0/3813058268 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec15c440 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 -- 172.21.15.78:0/3813058268 >> 172.21.15.78:0/3813058268 conn(0x7fb2ec0b2ec0 msgr2=0x7fb2ec0b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.454+0000 7fb2f4800640 1 -- 172.21.15.78:0/3813058268 shutdown_connections 2024-09-11T12:56:49.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 -- 172.21.15.78:0/3813058268 wait complete. 2024-09-11T12:56:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 Processor -- start 2024-09-11T12:56:49.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 -- start start 2024-09-11T12:56:49.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec148760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ec15dfc0 0x7fb2ec1490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec168250 con 0x7fb2ec072e70 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb2ec1680d0 con 0x7fb2ec15dfc0 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb2ec1683d0 con 0x7fb2ec0701b0 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49060/0 (socket says 172.21.15.78:49060) 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 -- 172.21.15.78:0/1780085299 learned_addr learned my addr 172.21.15.78:0/1780085299 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1e00640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec148760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f2800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ec15dfc0 0x7fb2ec1490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 -- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 msgr2=0x7fb2ec148760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec148760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 -- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ec15dfc0 msgr2=0x7fb2ec1490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ec15dfc0 0x7fb2ec1490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 -- 172.21.15.78:0/1780085299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2ec1faf30 con 0x7fb2ec072e70 2024-09-11T12:56:49.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.458+0000 7fb2f1400640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc01cc20 tx=0x7fb2dc005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0ac070 con 0x7fb2ec072e70 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2dc004320 con 0x7fb2ec072e70 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc0048f0 con 0x7fb2ec072e70 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2ec1fb210 con 0x7fb2ec072e70 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2ec1fb7d0 con 0x7fb2ec072e70 2024-09-11T12:56:49.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2ec06faa0 con 0x7fb2ec072e70 2024-09-11T12:56:49.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.462+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fb2dc0af070 con 0x7fb2ec072e70 2024-09-11T12:56:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.466+0000 7fb2e6000640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 0x7fb2d0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.466+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 5227+0+0 (secure 0 0 0) 0x7fb2dc12cba0 con 0x7fb2ec072e70 2024-09-11T12:56:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.466+0000 7fb2f1e00640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 0x7fb2d0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:49.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.466+0000 7fb2f1e00640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 0x7fb2d0074320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec151980 tx=0x7fb2e0001da0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:49.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.470+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb2dc0c1030 con 0x7fb2ec072e70 2024-09-11T12:56:49.642 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:49.638+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fb2ec0695e0 con 0x7fb2ec072e70 2024-09-11T12:56:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2e6000640 1 -- 172.21.15.78:0/1780085299 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v149) ==== 170+0+0 (secure 0 0 0) 0x7fb2dc016330 con 0x7fb2ec072e70 2024-09-11T12:56:50.277 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:56:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 msgr2=0x7fb2d0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 0x7fb2d0074320 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec151980 tx=0x7fb2e0001da0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 msgr2=0x7fb2ec147b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc01cc20 tx=0x7fb2dc005750 comp rx=0 tx=0).stop 2024-09-11T12:56:50.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 shutdown_connections 2024-09-11T12:56:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb2d0071e60 0x7fb2d0074320 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb2ec15dfc0 0x7fb2ec1490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb2ec072e70 0x7fb2ec147b80 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 --2- 172.21.15.78:0/1780085299 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb2ec0701b0 0x7fb2ec148760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 >> 172.21.15.78:0/1780085299 conn(0x7fb2ec0b2ec0 msgr2=0x7fb2ec152070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:50.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.273+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 shutdown_connections 2024-09-11T12:56:50.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.277+0000 7fb2f4800640 1 -- 172.21.15.78:0/1780085299 wait complete. 2024-09-11T12:56:50.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:56:50.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 -- 172.21.15.78:0/124589325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d00d0ef0 msgr2=0x7f47d015f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:50.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 --2- 172.21.15.78:0/124589325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d00d0ef0 0x7f47d015f500 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f47d006f910 tx=0x7f47c801ca90 comp rx=0 tx=0).stop 2024-09-11T12:56:50.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 -- 172.21.15.78:0/124589325 shutdown_connections 2024-09-11T12:56:50.458 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 --2- 172.21.15.78:0/124589325 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d00d0ef0 0x7f47d015f500 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 --2- 172.21.15.78:0/124589325 >> v2:172.21.15.157:3300/0 conn(0x7f47d0070750 0x7f47d00d09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 --2- 172.21.15.78:0/124589325 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 0x7f47d0070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 -- 172.21.15.78:0/124589325 >> 172.21.15.78:0/124589325 conn(0x7f47d0074480 msgr2=0x7f47d0074890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 -- 172.21.15.78:0/124589325 shutdown_connections 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 -- 172.21.15.78:0/124589325 wait complete. 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.453+0000 7f47d7e00640 1 Processor -- start 2024-09-11T12:56:50.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- start start 2024-09-11T12:56:50.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d00d0ef0 0x7f47d013ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 0x7f47d013f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00a9b50 con 0x7f47d0070750 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f47d00a99d0 con 0x7f47d00d0ef0 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f47d00a9cd0 con 0x7f47d013e7f0 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49072/0 (socket says 172.21.15.78:49072) 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 -- 172.21.15.78:0/3924442033 learned_addr learned my addr 172.21.15.78:0/3924442033 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 0x7f47d013f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d4a00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d00d0ef0 0x7f47d013ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 -- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 msgr2=0x7f47d013f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 0x7f47d013f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 -- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d00d0ef0 msgr2=0x7f47d013ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:50.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d00d0ef0 0x7f47d013ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 -- 172.21.15.78:0/3924442033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d0150de0 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d5400640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d720 tx=0x7f47c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4017070 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47c4009d20 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d01510c0 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4005280 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d013b1a0 con 0x7f47d0070750 2024-09-11T12:56:50.464 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.457+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d015f500 con 0x7f47d0070750 2024-09-11T12:56:50.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.461+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f47c4005420 con 0x7f47d0070750 2024-09-11T12:56:50.465 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.461+0000 7f47cd600640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 0x7f47b0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:50.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.461+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 5609+0+0 (secure 0 0 0) 0x7f47c4093050 con 0x7f47d0070750 2024-09-11T12:56:50.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.461+0000 7f47d4a00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 0x7f47b0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:50.466 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.461+0000 7f47d4a00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 0x7f47b0074320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005fd0 tx=0x7f47bc005e60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:50.468 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.465+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f47c405e3d0 con 0x7f47d0070750 2024-09-11T12:56:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:50.669+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f47d013f420 con 0x7f47d0070750 2024-09-11T12:56:51.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.305+0000 7f47cd600640 1 -- 172.21.15.78:0/3924442033 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v107) ==== 174+0+0 (secure 0 0 0) 0x7f47c405dcd0 con 0x7f47d0070750 2024-09-11T12:56:51.312 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 24 and data pool 25 2024-09-11T12:56:51.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 msgr2=0x7f47b0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 0x7f47b0074320 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f47bc005fd0 tx=0x7f47bc005e60 comp rx=0 tx=0).stop 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 msgr2=0x7f47d01399e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f47c400d720 tx=0x7f47c400dbe0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 shutdown_connections 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f47d013e7f0 0x7f47d013f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f47b0071e60 0x7f47b0074320 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f47d00d0ef0 0x7f47d013ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 --2- 172.21.15.78:0/3924442033 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f47d0070750 0x7f47d01399e0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.309+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 >> 172.21.15.78:0/3924442033 conn(0x7f47d0074480 msgr2=0x7f47d0074860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.313+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 shutdown_connections 2024-09-11T12:56:51.316 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.313+0000 7f47d7e00640 1 -- 172.21.15.78:0/3924442033 wait complete. 2024-09-11T12:56:51.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- 172.21.15.78:0/2632047747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09940cf760 msgr2=0x7f0994074260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 --2- 172.21.15.78:0/2632047747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09940cf760 0x7f0994074260 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f0984009c80 tx=0x7f098401cd70 comp rx=0 tx=0).stop 2024-09-11T12:56:51.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- 172.21.15.78:0/2632047747 shutdown_connections 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 --2- 172.21.15.78:0/2632047747 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09940cf760 0x7f0994074260 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- 172.21.15.78:0/2632047747 >> 172.21.15.78:0/2632047747 conn(0x7f099406ca80 msgr2=0x7f09940b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- 172.21.15.78:0/2632047747 shutdown_connections 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- 172.21.15.78:0/2632047747 wait complete. 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 Processor -- start 2024-09-11T12:56:51.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.493+0000 7f0999200640 1 -- start start 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f099406fc50 0x7f0994131670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 0x7f0994131bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0994151bb0 con 0x7f099415aec0 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0994151a30 con 0x7f09940cf760 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0994151d30 con 0x7f099406fc50 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49086/0 (socket says 172.21.15.78:49086) 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 -- 172.21.15.78:0/2808505742 learned_addr learned my addr 172.21.15.78:0/2808505742 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0991e00640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f099406fc50 0x7f0994131670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0991400640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 0x7f0994131bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 -- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f099406fc50 msgr2=0x7f0994131670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f099406fc50 0x7f0994131670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 -- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 msgr2=0x7f0994131bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 0x7f0994131bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 -- 172.21.15.78:0/2808505742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0994145c00 con 0x7f099415aec0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0991400640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 0x7f0994131bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0992800640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c3d0 tx=0x7f098c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c017070 con 0x7f099415aec0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f098c003600 con 0x7f099415aec0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f098c003a50 con 0x7f099415aec0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0994145640 con 0x7f099415aec0 2024-09-11T12:56:51.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f099412f710 con 0x7f099415aec0 2024-09-11T12:56:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0994074260 con 0x7f099415aec0 2024-09-11T12:56:51.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.497+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f098c01a070 con 0x7f099415aec0 2024-09-11T12:56:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.501+0000 7f098a000640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 0x7f0974074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.501+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5709+0+0 (secure 0 0 0) 0x7f098c092260 con 0x7f099415aec0 2024-09-11T12:56:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.501+0000 7f0991e00640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 0x7f0974074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.501+0000 7f0991e00640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 0x7f0974074350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09840af000 tx=0x7f098401b5b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:51.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.505+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f098c05caf0 con 0x7f099415aec0 2024-09-11T12:56:51.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f099414e200 con 0x7f099415aec0 2024-09-11T12:56:51.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f098a000640 1 -- 172.21.15.78:0/2808505742 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16752 (secure 0 0 0) 0x7f098c065380 con 0x7f099415aec0 2024-09-11T12:56:51.682 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:51.682 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":150,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:50.675125+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:48.378165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:49.654186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:51.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 msgr2=0x7f0974074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 0x7f0974074350 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f09840af000 tx=0x7f098401b5b0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 msgr2=0x7f09941f8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f098c00c3d0 tx=0x7f098c00c890 comp rx=0 tx=0).stop 2024-09-11T12:56:51.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 shutdown_connections 2024-09-11T12:56:51.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f099415aec0 0x7f09941f8f20 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0974071e90 0x7f0974074350 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09940cf760 0x7f0994131bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 --2- 172.21.15.78:0/2808505742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f099406fc50 0x7f0994131670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.677+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 >> 172.21.15.78:0/2808505742 conn(0x7f099406ca80 msgr2=0x7f09940b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:51.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.681+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 shutdown_connections 2024-09-11T12:56:51.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.681+0000 7f0999200640 1 -- 172.21.15.78:0/2808505742 wait complete. 2024-09-11T12:56:51.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 -- 172.21.15.78:0/1635252193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c78003a60 msgr2=0x7f7c78003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 --2- 172.21.15.78:0/1635252193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c78003a60 0x7f7c78003e80 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009b40 tx=0x7f7c8401d220 comp rx=0 tx=0).stop 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 -- 172.21.15.78:0/1635252193 shutdown_connections 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 --2- 172.21.15.78:0/1635252193 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c78003a60 0x7f7c78003e80 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 -- 172.21.15.78:0/1635252193 >> 172.21.15.78:0/1635252193 conn(0x7f7c940ceab0 msgr2=0x7f7c940ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 -- 172.21.15.78:0/1635252193 shutdown_connections 2024-09-11T12:56:51.847 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.841+0000 7f7c9b000640 1 -- 172.21.15.78:0/1635252193 wait complete. 2024-09-11T12:56:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 Processor -- start 2024-09-11T12:56:51.848 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- start start 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c78003a60 0x7f7c941494b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c941fb170 0x7f7c941e83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94152c50 con 0x7f7c941e8930 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c94152ad0 con 0x7f7c941fb170 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c94152dd0 con 0x7f7c78003a60 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49108/0 (socket says 172.21.15.78:49108) 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 -- 172.21.15.78:0/3544827667 learned_addr learned my addr 172.21.15.78:0/3544827667 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:51.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c8be00640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c941fb170 0x7f7c941e83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c93e00640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c78003a60 0x7f7c941494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 -- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c78003a60 msgr2=0x7f7c941494b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c78003a60 0x7f7c941494b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 -- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c941fb170 msgr2=0x7f7c941e83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c941fb170 0x7f7c941e83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 -- 172.21.15.78:0/3544827667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c94149da0 con 0x7f7c941e8930 2024-09-11T12:56:51.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c99000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7c9415c990 tx=0x7f7c8c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c010890 con 0x7f7c941e8930 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c8c017070 con 0x7f7c941e8930 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c941fb710 con 0x7f7c941e8930 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c010da0 con 0x7f7c941e8930 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.845+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c941fbcd0 con 0x7f7c941e8930 2024-09-11T12:56:51.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c9415f8f0 con 0x7f7c941e8930 2024-09-11T12:56:51.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f7c8c01c460 con 0x7f7c941e8930 2024-09-11T12:56:51.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c90c00640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 0x7f7c70074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:51.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5709+0+0 (secure 0 0 0) 0x7f7c8c090a00 con 0x7f7c941e8930 2024-09-11T12:56:51.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c93e00640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 0x7f7c70074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:51.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.849+0000 7f7c93e00640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 0x7f7c70074350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c840ad000 tx=0x7f7c84019fc0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:51.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:51.853+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7c8c05e160 con 0x7f7c941e8930 2024-09-11T12:56:52.023 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c9406ff00 con 0x7f7c941e8930 2024-09-11T12:56:52.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c90c00640 1 -- 172.21.15.78:0/3544827667 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) ==== 74+0+16752 (secure 0 0 0) 0x7f7c8c0663a0 con 0x7f7c941e8930 2024-09-11T12:56:52.025 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:52.025 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":150,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:50.675125+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:48.378165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:49.654186+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 msgr2=0x7f7c70074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 0x7f7c70074350 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7c840ad000 tx=0x7f7c84019fc0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 msgr2=0x7f7c941e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f7c9415c990 tx=0x7f7c8c00eae0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 shutdown_connections 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c941e8930 0x7f7c941e8db0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c70071e90 0x7f7c70074350 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c941fb170 0x7f7c941e83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 --2- 172.21.15.78:0/3544827667 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c78003a60 0x7f7c941494b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 >> 172.21.15.78:0/3544827667 conn(0x7f7c940ceab0 msgr2=0x7f7c94160e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:52.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 shutdown_connections 2024-09-11T12:56:52.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.021+0000 7f7c9b000640 1 -- 172.21.15.78:0/3544827667 wait complete. 2024-09-11T12:56:52.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:56:52.191 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 -- 172.21.15.78:0/1411524967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e000052f0 msgr2=0x7f3e0000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 --2- 172.21.15.78:0/1411524967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e000052f0 0x7f3e0000b9d0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009b10 tx=0x7f3e1001d270 comp rx=0 tx=0).stop 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 -- 172.21.15.78:0/1411524967 shutdown_connections 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 --2- 172.21.15.78:0/1411524967 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e000052f0 0x7f3e0000b9d0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 -- 172.21.15.78:0/1411524967 >> 172.21.15.78:0/1411524967 conn(0x7f3e18071dd0 msgr2=0x7f3e180721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 -- 172.21.15.78:0/1411524967 shutdown_connections 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.185+0000 7f3e20400640 1 -- 172.21.15.78:0/1411524967 wait complete. 2024-09-11T12:56:52.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 Processor -- start 2024-09-11T12:56:52.193 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- start start 2024-09-11T12:56:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000052f0 0x7f3e181345c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:52.194 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e181324a0 0x7f3e18132920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1815ab40 con 0x7f3e181e3ec0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3e1815a9c0 con 0x7f3e181324a0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1815acc0 con 0x7f3e000052f0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49132/0 (socket says 172.21.15.78:49132) 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 -- 172.21.15.78:0/1976362331 learned_addr learned my addr 172.21.15.78:0/1976362331 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1da00640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000052f0 0x7f3e181345c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1e400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e181324a0 0x7f3e18132920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 -- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000052f0 msgr2=0x7f3e181345c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000052f0 0x7f3e181345c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 -- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e181324a0 msgr2=0x7f3e18132920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e181324a0 0x7f3e18132920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:52.195 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 -- 172.21.15.78:0/1976362331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e18131400 con 0x7f3e181e3ec0 2024-09-11T12:56:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e1d000640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3e04001770 tx=0x7f3e0400e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e04012850 con 0x7f3e181e3ec0 2024-09-11T12:56:52.196 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e04012ae0 con 0x7f3e181e3ec0 2024-09-11T12:56:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0401a400 con 0x7f3e181e3ec0 2024-09-11T12:56:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e181316e0 con 0x7f3e181e3ec0 2024-09-11T12:56:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.189+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e18131ca0 con 0x7f3e181e3ec0 2024-09-11T12:56:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e180b33c0 con 0x7f3e181e3ec0 2024-09-11T12:56:52.197 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f3e0400d030 con 0x7f3e181e3ec0 2024-09-11T12:56:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e15600640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 0x7f3df8074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(150..150 src has 1..150) ==== 5709+0+0 (secure 0 0 0) 0x7f3e040937f0 con 0x7f3e181e3ec0 2024-09-11T12:56:52.198 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e1da00640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 0x7f3df8074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:52.199 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.193+0000 7f3e1da00640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 0x7f3df8074320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f90 tx=0x7f3e0c006f00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:52.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.197+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3e04016030 con 0x7f3e181e3ec0 2024-09-11T12:56:52.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:52.405+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3e180cdd70 con 0x7f3e181e3ec0 2024-09-11T12:56:53.273 INFO:tasks.ceph.mds.c.smithi078.stderr:2024-09-11T12:56:53.273+0000 7fa307600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:56:53.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.317+0000 7f3e15600640 1 -- 172.21.15.78:0/1976362331 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v110) ==== 118+0+0 (secure 0 0 0) 0x7f3e04060f50 con 0x7f3e181e3ec0 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 msgr2=0x7f3df8074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 0x7f3df8074320 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c000f90 tx=0x7f3e0c006f00 comp rx=0 tx=0).stop 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 msgr2=0x7f3e181e4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f3e04001770 tx=0x7f3e0400e6b0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 shutdown_connections 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3df8071e60 0x7f3df8074320 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3e181324a0 0x7f3e18132920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3e181e3ec0 0x7f3e181e4340 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 --2- 172.21.15.78:0/1976362331 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3e000052f0 0x7f3e181345c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 >> 172.21.15.78:0/1976362331 conn(0x7f3e18071dd0 msgr2=0x7f3e180cd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 shutdown_connections 2024-09-11T12:56:53.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.321+0000 7f3e20400640 1 -- 172.21.15.78:0/1976362331 wait complete. 2024-09-11T12:56:53.347 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:56:53.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:56:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- 172.21.15.78:0/1191057878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 msgr2=0x7fc5100aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 --2- 172.21.15.78:0/1191057878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5100aa660 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7fc51015be90 tx=0x7fc4fc01c830 comp rx=0 tx=0).stop 2024-09-11T12:56:53.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- 172.21.15.78:0/1191057878 shutdown_connections 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 --2- 172.21.15.78:0/1191057878 >> v2:172.21.15.157:3300/0 conn(0x7fc5100d1740 0x7fc5100aaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 --2- 172.21.15.78:0/1191057878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5100aa660 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- 172.21.15.78:0/1191057878 >> 172.21.15.78:0/1191057878 conn(0x7fc510075610 msgr2=0x7fc510075a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- 172.21.15.78:0/1191057878 shutdown_connections 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- 172.21.15.78:0/1191057878 wait complete. 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 Processor -- start 2024-09-11T12:56:53.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.493+0000 7fc515e00640 1 -- start start 2024-09-11T12:56:53.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 0x7fc51014b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 0x7fc5101f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc51015d470 con 0x7fc51015f030 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc51015d2f0 con 0x7fc51015ec50 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc51015d5f0 con 0x7fc5100d1740 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:49142/0 (socket says 172.21.15.78:49142) 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 -- 172.21.15.78:0/221339766 learned_addr learned my addr 172.21.15.78:0/221339766 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50e000640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 0x7fc5101f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50ea00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 0x7fc51014b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 -- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 msgr2=0x7fc51014b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:53.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 0x7fc51014b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 -- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 msgr2=0x7fc5101f9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 0x7fc5101f9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 -- 172.21.15.78:0/221339766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5101649c0 con 0x7fc51015f030 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50e000640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 0x7fc5101f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50ea00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 0x7fc51014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc50f400640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc50800d750 tx=0x7fc50800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc508017070 con 0x7fc51015f030 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc508009d20 con 0x7fc51015f030 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5101fabe0 con 0x7fc51015f030 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50800de60 con 0x7fc51015f030 2024-09-11T12:56:53.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc510165cb0 con 0x7fc51015f030 2024-09-11T12:56:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51015c8c0 con 0x7fc51015f030 2024-09-11T12:56:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.497+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fc508005280 con 0x7fc51015f030 2024-09-11T12:56:53.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.501+0000 7fc507400640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 0x7fc4f0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.501+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fc508093230 con 0x7fc51015f030 2024-09-11T12:56:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.501+0000 7fc50ea00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 0x7fc4f0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:53.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.501+0000 7fc50ea00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 0x7fc4f0074350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc5000012f0 tx=0x7fc500009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:53.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.505+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc50805df70 con 0x7fc51015f030 2024-09-11T12:56:53.708 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:53.705+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fc5100707f0 con 0x7fc51015f030 2024-09-11T12:56:54.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.321+0000 7fc507400640 1 -- 172.21.15.78:0/221339766 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v111) ==== 122+0+0 (secure 0 0 0) 0x7fc5080661b0 con 0x7fc51015f030 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 msgr2=0x7fc4f0074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 0x7fc4f0074350 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc5000012f0 tx=0x7fc500009040 comp rx=0 tx=0).stop 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 msgr2=0x7fc5101fa3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fc50800d750 tx=0x7fc50800dc10 comp rx=0 tx=0).stop 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 shutdown_connections 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc51015f030 0x7fc5101fa3a0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4f0071e90 0x7fc4f0074350 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc51015ec50 0x7fc5101f9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 --2- 172.21.15.78:0/221339766 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5100d1740 0x7fc51014b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 >> 172.21.15.78:0/221339766 conn(0x7fc510075610 msgr2=0x7fc51015fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 shutdown_connections 2024-09-11T12:56:54.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.325+0000 7fc515e00640 1 -- 172.21.15.78:0/221339766 wait complete. 2024-09-11T12:56:54.349 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:56:54.350 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:56:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- 172.21.15.78:0/353805718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc15d010 msgr2=0x7f7bfc0b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- 172.21.15.78:0/353805718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc15d010 0x7f7bfc0b4520 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc0b2980 tx=0x7f7bf401c890 comp rx=0 tx=0).stop 2024-09-11T12:56:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- 172.21.15.78:0/353805718 shutdown_connections 2024-09-11T12:56:54.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- 172.21.15.78:0/353805718 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc15d010 0x7f7bfc0b4520 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- 172.21.15.78:0/353805718 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0cf7c0 0x7f7bfc15cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- 172.21.15.78:0/353805718 >> v2:172.21.15.157:3300/0 conn(0x7f7bfc0aed20 0x7f7bfc0cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- 172.21.15.78:0/353805718 >> 172.21.15.78:0/353805718 conn(0x7f7bfc06ca80 msgr2=0x7f7bfc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- 172.21.15.78:0/353805718 shutdown_connections 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- 172.21.15.78:0/353805718 wait complete. 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 Processor -- start 2024-09-11T12:56:54.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 -- start start 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0aed20 0x7f7bfc14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.489+0000 7f7c01a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bfc15d010 0x7f7bfc148520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0cd460 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7bfc0cd2e0 con 0x7f7bfc15d010 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7bfc0cd5e0 con 0x7f7bfc0aed20 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46240/0 (socket says 172.21.15.78:46240) 2024-09-11T12:56:54.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 -- 172.21.15.78:0/4268729013 learned_addr learned my addr 172.21.15.78:0/4268729013 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfaa00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0aed20 0x7f7bfc14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfb400640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bfc15d010 0x7f7bfc148520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 -- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0aed20 msgr2=0x7f7bfc14ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0aed20 0x7f7bfc14ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 -- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bfc15d010 msgr2=0x7f7bfc148520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bfc15d010 0x7f7bfc148520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 -- 172.21.15.78:0/4268729013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bfc1406a0 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bfa000640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c9a0 tx=0x7f7be800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8007c70 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7be800f450 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bfc146110 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be800f8a0 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bfc14d5c0 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.498 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bfc0cdb00 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.499 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.493+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f7be800fa40 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.497+0000 7f7bf2a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 0x7f7bdc074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.497+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f7be8090e20 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.497+0000 7f7bfaa00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 0x7f7bdc074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.500 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.497+0000 7f7bfaa00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 0x7f7bdc074320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005cd0 tx=0x7f7bec00a580 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:54.503 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.501+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7bfc1406a0 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.697+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bfc0b0f60 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7bf2a00640 1 -- 172.21.15.78:0/4268729013 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5236 (secure 0 0 0) 0x7f7be8061490 con 0x7f7bfc0cf7c0 2024-09-11T12:56:54.704 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:54.704 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":111,"btime":"2024-09-11T12:56:54:322400+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:56:54.704 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 111 2024-09-11T12:56:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 msgr2=0x7f7bdc074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 0x7f7bdc074320 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f7bec005cd0 tx=0x7f7bec00a580 comp rx=0 tx=0).stop 2024-09-11T12:56:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 msgr2=0x7f7bfc140160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c9a0 tx=0x7f7be800ce60 comp rx=0 tx=0).stop 2024-09-11T12:56:54.706 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 shutdown_connections 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7bdc071e60 0x7f7bdc074320 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7bfc15d010 0x7f7bfc148520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7bfc0cf7c0 0x7f7bfc140160 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 --2- 172.21.15.78:0/4268729013 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7bfc0aed20 0x7f7bfc14ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 >> 172.21.15.78:0/4268729013 conn(0x7f7bfc06ca80 msgr2=0x7f7bfc141fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 shutdown_connections 2024-09-11T12:56:54.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.701+0000 7f7c01a00640 1 -- 172.21.15.78:0/4268729013 wait complete. 2024-09-11T12:56:54.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:56:54.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 -- 172.21.15.78:0/3151876172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 msgr2=0x7fb71c071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 --2- 172.21.15.78:0/3151876172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c071d60 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb718009870 tx=0x7fb71801c750 comp rx=0 tx=0).stop 2024-09-11T12:56:54.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 -- 172.21.15.78:0/3151876172 shutdown_connections 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 --2- 172.21.15.78:0/3151876172 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c071d60 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 -- 172.21.15.78:0/3151876172 >> 172.21.15.78:0/3151876172 conn(0x7fb71c06ca80 msgr2=0x7fb71c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 -- 172.21.15.78:0/3151876172 shutdown_connections 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.861+0000 7fb724800640 1 -- 172.21.15.78:0/3151876172 wait complete. 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 Processor -- start 2024-09-11T12:56:54.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- start start 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c15ba80 0x7fb71c15d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c168ae0 con 0x7fb71c074240 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb71c168960 con 0x7fb71c15ba80 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb71c168c60 con 0x7fb71c070360 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46256/0 (socket says 172.21.15.78:46256) 2024-09-11T12:56:54.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 -- 172.21.15.78:0/945937945 learned_addr learned my addr 172.21.15.78:0/945937945 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:54.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721e00640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb722800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c15ba80 0x7fb71c15d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 -- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 msgr2=0x7fb71c144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c144d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 -- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c15ba80 msgr2=0x7fb71c15d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c15ba80 0x7fb71c15d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 -- 172.21.15.78:0/945937945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb71c1483d0 con 0x7fb71c074240 2024-09-11T12:56:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb721400640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c700 tx=0x7fb70c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:54.872 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c005270 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb70c005410 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb71c1486b0 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb70c00a880 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.865+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb71c148c70 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71c071d60 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fb70c00aa20 con 0x7fb71c074240 2024-09-11T12:56:54.873 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb716000640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 0x7fb700074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fb70c092cc0 con 0x7fb71c074240 2024-09-11T12:56:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb721e00640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 0x7fb700074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:54.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.869+0000 7fb721e00640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 0x7fb700074320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb710001f20 tx=0x7fb710009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:54.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:54.873+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fb70c05d500 con 0x7fb71c074240 2024-09-11T12:56:55.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.037+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb71c0695e0 con 0x7fb71c074240 2024-09-11T12:56:55.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb716000640 1 -- 172.21.15.78:0/945937945 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16739 (secure 0 0 0) 0x7fb70c061a50 con 0x7fb71c074240 2024-09-11T12:56:55.044 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:55.044 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":151,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:52.296488+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:48.378165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:49.654186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:56:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 msgr2=0x7fb700074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 0x7fb700074320 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb710001f20 tx=0x7fb710009040 comp rx=0 tx=0).stop 2024-09-11T12:56:55.046 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 msgr2=0x7fb71c15d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7fb70c00c700 tx=0x7fb70c00cbc0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 shutdown_connections 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb700071e60 0x7fb700074320 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb71c15ba80 0x7fb71c15d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c074240 0x7fb71c15d0a0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 --2- 172.21.15.78:0/945937945 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb71c070360 0x7fb71c144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 >> 172.21.15.78:0/945937945 conn(0x7fb71c06ca80 msgr2=0x7fb71c0ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 shutdown_connections 2024-09-11T12:56:55.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.041+0000 7fb724800640 1 -- 172.21.15.78:0/945937945 wait complete. 2024-09-11T12:56:55.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:56:55.066 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- 172.21.15.78:0/3091973730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970159fc0 msgr2=0x7f99700706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- 172.21.15.78:0/3091973730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970159fc0 0x7f99700706f0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f9960009840 tx=0x7f996001c820 comp rx=0 tx=0).stop 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- 172.21.15.78:0/3091973730 shutdown_connections 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- 172.21.15.78:0/3091973730 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 0x7f997006f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- 172.21.15.78:0/3091973730 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9970159fc0 0x7f99700706f0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.208 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- 172.21.15.78:0/3091973730 >> 172.21.15.78:0/3091973730 conn(0x7f997006ca80 msgr2=0x7f99700b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- 172.21.15.78:0/3091973730 shutdown_connections 2024-09-11T12:56:55.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- 172.21.15.78:0/3091973730 wait complete. 2024-09-11T12:56:55.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 Processor -- start 2024-09-11T12:56:55.209 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- start start 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 0x7f9970149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 0x7f99700ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f99700b5280 con 0x7f99700addb0 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f99700b5100 con 0x7f9970159fc0 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f9975000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9970167e30 con 0x7f997006f200 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.211 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46278/0 (socket says 172.21.15.78:46278) 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 -- 172.21.15.78:0/2498747835 learned_addr learned my addr 172.21.15.78:0/2498747835 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996de00640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 0x7f9970149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996d400640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 0x7f99700ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 -- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 msgr2=0x7f9970149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 0x7f9970149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 -- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 msgr2=0x7f99700ad870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 0x7f99700ad870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996e800640 1 -- 172.21.15.78:0/2498747835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99701fafd0 con 0x7f99700addb0 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.205+0000 7f996d400640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 0x7f99700ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:55.212 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f996e800640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f996800b970 tx=0x7f996800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968019070 con 0x7f99700addb0 2024-09-11T12:56:55.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f996800cbf0 con 0x7f99700addb0 2024-09-11T12:56:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9968014440 con 0x7f99700addb0 2024-09-11T12:56:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99701fb2b0 con 0x7f99700addb0 2024-09-11T12:56:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99701f74e0 con 0x7f99700addb0 2024-09-11T12:56:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99701f7740 con 0x7f99700addb0 2024-09-11T12:56:55.214 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.209+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f996801c070 con 0x7f99700addb0 2024-09-11T12:56:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.213+0000 7f9966000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 0x7f9950074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.213+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f9968092cd0 con 0x7f99700addb0 2024-09-11T12:56:55.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.213+0000 7f996de00640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 0x7f9950074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.213+0000 7f996de00640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 0x7f9950074350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99600af000 tx=0x7f996001b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.217+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f996805d560 con 0x7f99700addb0 2024-09-11T12:56:55.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.385+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99700706f0 con 0x7f9950071e90 2024-09-11T12:56:55.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.389+0000 7f9966000640 1 -- 172.21.15.78:0/2498747835 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7f99700706f0 con 0x7f9950071e90 2024-09-11T12:56:55.395 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:55.396 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 msgr2=0x7f9950074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 0x7f9950074350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f99600af000 tx=0x7f996001b060 comp rx=0 tx=0).stop 2024-09-11T12:56:55.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 msgr2=0x7f99701554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f996800b970 tx=0x7f996800be30 comp rx=0 tx=0).stop 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 shutdown_connections 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f99700addb0 0x7f99701554f0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9950071e90 0x7f9950074350 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9970159fc0 0x7f99700ad870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 --2- 172.21.15.78:0/2498747835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f997006f200 0x7f9970149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 >> 172.21.15.78:0/2498747835 conn(0x7f997006ca80 msgr2=0x7f997015cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 shutdown_connections 2024-09-11T12:56:55.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.393+0000 7f9975000640 1 -- 172.21.15.78:0/2498747835 wait complete. 2024-09-11T12:56:55.415 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T12:56:54.508980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281896,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692824,"statfs":{"total":773094113280,"available":772805451776,"internally_reserved":0,"allocated":16101376,"data_stored":7713313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.230353"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296293+0000","last_change":"2024-09-11T12:56:50.296293+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296293+0000","last_undegraded":"2024-09-11T12:56:50.296293+0000","last_fullsized":"2024-09-11T12:56:50.296293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292499+0000","last_change":"2024-09-11T12:56:50.292499+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292499+0000","last_undegraded":"2024-09-11T12:56:50.292499+0000","last_fullsized":"2024-09-11T12:56:50.292499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.303507+0000","last_change":"2024-09-11T12:56:50.303507+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.303507+0000","last_undegraded":"2024-09-11T12:56:50.303507+0000","last_fullsized":"2024-09-11T12:56:50.303507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292365+0000","last_change":"2024-09-11T12:56:50.292365+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292365+0000","last_undegraded":"2024-09-11T12:56:50.292365+0000","last_fullsized":"2024-09-11T12:56:50.292365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292501+0000","last_change":"2024-09-11T12:56:50.292501+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292501+0000","last_undegraded":"2024-09-11T12:56:50.292501+0000","last_fullsized":"2024-09-11T12:56:50.292501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293414+0000","last_change":"2024-09-11T12:56:50.293414+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293414+0000","last_undegraded":"2024-09-11T12:56:50.293414+0000","last_fullsized":"2024-09-11T12:56:50.293414+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284696+0000","last_change":"2024-09-11T12:56:50.284696+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284696+0000","last_undegraded":"2024-09-11T12:56:50.284696+0000","last_fullsized":"2024-09-11T12:56:50.284696+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290692+0000","last_change":"2024-09-11T12:56:50.290692+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290692+0000","last_undegraded":"2024-09-11T12:56:50.290692+0000","last_fullsized":"2024-09-11T12:56:50.290692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284457+0000","last_change":"2024-09-11T12:56:50.284457+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284457+0000","last_undegraded":"2024-09-11T12:56:50.284457+0000","last_fullsized":"2024-09-11T12:56:50.284457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294783+0000","last_change":"2024-09-11T12:56:50.294783+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294783+0000","last_undegraded":"2024-09-11T12:56:50.294783+0000","last_fullsized":"2024-09-11T12:56:50.294783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285384+0000","last_change":"2024-09-11T12:56:50.285384+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285384+0000","last_undegraded":"2024-09-11T12:56:50.285384+0000","last_fullsized":"2024-09-11T12:56:50.285384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291364+0000","last_change":"2024-09-11T12:56:50.291364+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291364+0000","last_undegraded":"2024-09-11T12:56:50.291364+0000","last_fullsized":"2024-09-11T12:56:50.291364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291751+0000","last_change":"2024-09-11T12:56:50.291751+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291751+0000","last_undegraded":"2024-09-11T12:56:50.291751+0000","last_fullsized":"2024-09-11T12:56:50.291751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285450+0000","last_change":"2024-09-11T12:56:50.285450+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285450+0000","last_undegraded":"2024-09-11T12:56:50.285450+0000","last_fullsized":"2024-09-11T12:56:50.285450+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293196+0000","last_change":"2024-09-11T12:56:50.293196+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293196+0000","last_undegraded":"2024-09-11T12:56:50.293196+0000","last_fullsized":"2024-09-11T12:56:50.293196+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290239+0000","last_change":"2024-09-11T12:56:50.290239+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290239+0000","last_undegraded":"2024-09-11T12:56:50.290239+0000","last_fullsized":"2024-09-11T12:56:50.290239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293571+0000","last_change":"2024-09-11T12:56:50.293571+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293571+0000","last_undegraded":"2024-09-11T12:56:50.293571+0000","last_fullsized":"2024-09-11T12:56:50.293571+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296392+0000","last_change":"2024-09-11T12:56:50.296392+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296392+0000","last_undegraded":"2024-09-11T12:56:50.296392+0000","last_fullsized":"2024-09-11T12:56:50.296392+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294671+0000","last_change":"2024-09-11T12:56:50.294671+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294671+0000","last_undegraded":"2024-09-11T12:56:50.294671+0000","last_fullsized":"2024-09-11T12:56:50.294671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.298439+0000","last_change":"2024-09-11T12:56:50.298439+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.298439+0000","last_undegraded":"2024-09-11T12:56:50.298439+0000","last_fullsized":"2024-09-11T12:56:50.298439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291492+0000","last_change":"2024-09-11T12:56:50.291492+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291492+0000","last_undegraded":"2024-09-11T12:56:50.291492+0000","last_fullsized":"2024-09-11T12:56:50.291492+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294603+0000","last_change":"2024-09-11T12:56:50.294603+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294603+0000","last_undegraded":"2024-09-11T12:56:50.294603+0000","last_fullsized":"2024-09-11T12:56:50.294603+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293271+0000","last_change":"2024-09-11T12:56:50.293271+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293271+0000","last_undegraded":"2024-09-11T12:56:50.293271+0000","last_fullsized":"2024-09-11T12:56:50.293271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.289748+0000","last_change":"2024-09-11T12:56:50.289748+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.289748+0000","last_undegraded":"2024-09-11T12:56:50.289748+0000","last_fullsized":"2024-09-11T12:56:50.289748+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290465+0000","last_change":"2024-09-11T12:56:50.290465+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290465+0000","last_undegraded":"2024-09-11T12:56:50.290465+0000","last_fullsized":"2024-09-11T12:56:50.290465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287922+0000","last_change":"2024-09-11T12:56:50.287922+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287922+0000","last_undegraded":"2024-09-11T12:56:50.287922+0000","last_fullsized":"2024-09-11T12:56:50.287922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.301309+0000","last_change":"2024-09-11T12:56:50.301309+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.301309+0000","last_undegraded":"2024-09-11T12:56:50.301309+0000","last_fullsized":"2024-09-11T12:56:50.301309+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293978+0000","last_change":"2024-09-11T12:56:50.293978+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293978+0000","last_undegraded":"2024-09-11T12:56:50.293978+0000","last_fullsized":"2024-09-11T12:56:50.293978+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290638+0000","last_change":"2024-09-11T12:56:50.290638+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290638+0000","last_undegraded":"2024-09-11T12:56:50.290638+0000","last_fullsized":"2024-09-11T12:56:50.290638+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284031+0000","last_change":"2024-09-11T12:56:50.284031+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284031+0000","last_undegraded":"2024-09-11T12:56:50.284031+0000","last_fullsized":"2024-09-11T12:56:50.284031+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292818+0000","last_change":"2024-09-11T12:56:50.292818+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292818+0000","last_undegraded":"2024-09-11T12:56:50.292818+0000","last_fullsized":"2024-09-11T12:56:50.292818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291033+0000","last_change":"2024-09-11T12:56:50.291033+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291033+0000","last_undegraded":"2024-09-11T12:56:50.291033+0000","last_fullsized":"2024-09-11T12:56:50.291033+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292095+0000","last_change":"2024-09-11T12:56:50.292095+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292095+0000","last_undegraded":"2024-09-11T12:56:50.292095+0000","last_fullsized":"2024-09-11T12:56:50.292095+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302831+0000","last_change":"2024-09-11T12:56:50.290099+0000","last_active":"2024-09-11T12:56:50.302831+0000","last_peered":"2024-09-11T12:56:50.302831+0000","last_clean":"2024-09-11T12:56:50.302831+0000","last_became_active":"2024-09-11T12:56:50.289571+0000","last_became_peered":"2024-09-11T12:56:50.289571+0000","last_unstale":"2024-09-11T12:56:50.302831+0000","last_undegraded":"2024-09-11T12:56:50.302831+0000","last_fullsized":"2024-09-11T12:56:50.302831+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:44.844033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295530+0000","last_change":"2024-09-11T12:56:50.295530+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295530+0000","last_undegraded":"2024-09-11T12:56:50.295530+0000","last_fullsized":"2024-09-11T12:56:50.295530+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.306688+0000","last_change":"2024-09-11T12:56:50.305706+0000","last_active":"2024-09-11T12:56:50.306688+0000","last_peered":"2024-09-11T12:56:50.306688+0000","last_clean":"2024-09-11T12:56:50.306688+0000","last_became_active":"2024-09-11T12:56:50.303736+0000","last_became_peered":"2024-09-11T12:56:50.303736+0000","last_unstale":"2024-09-11T12:56:50.306688+0000","last_undegraded":"2024-09-11T12:56:50.306688+0000","last_fullsized":"2024-09-11T12:56:50.306688+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:09.904686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287500+0000","last_change":"2024-09-11T12:56:50.287500+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287500+0000","last_undegraded":"2024-09-11T12:56:50.287500+0000","last_fullsized":"2024-09-11T12:56:50.287500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309084+0000","last_change":"2024-09-11T12:56:50.306449+0000","last_active":"2024-09-11T12:56:50.309084+0000","last_peered":"2024-09-11T12:56:50.309084+0000","last_clean":"2024-09-11T12:56:50.309084+0000","last_became_active":"2024-09-11T12:56:50.306013+0000","last_became_peered":"2024-09-11T12:56:50.306013+0000","last_unstale":"2024-09-11T12:56:50.309084+0000","last_undegraded":"2024-09-11T12:56:50.309084+0000","last_fullsized":"2024-09-11T12:56:50.309084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:55.224946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283841+0000","last_change":"2024-09-11T12:56:50.283841+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283841+0000","last_undegraded":"2024-09-11T12:56:50.283841+0000","last_fullsized":"2024-09-11T12:56:50.283841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305283+0000","last_change":"2024-09-11T12:56:50.302084+0000","last_active":"2024-09-11T12:56:50.305283+0000","last_peered":"2024-09-11T12:56:50.305283+0000","last_clean":"2024-09-11T12:56:50.305283+0000","last_became_active":"2024-09-11T12:56:50.301802+0000","last_became_peered":"2024-09-11T12:56:50.301802+0000","last_unstale":"2024-09-11T12:56:50.305283+0000","last_undegraded":"2024-09-11T12:56:50.305283+0000","last_fullsized":"2024-09-11T12:56:50.305283+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:19.743245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292388+0000","last_change":"2024-09-11T12:56:50.292388+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292388+0000","last_undegraded":"2024-09-11T12:56:50.292388+0000","last_fullsized":"2024-09-11T12:56:50.292388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301964+0000","last_change":"2024-09-11T12:56:50.289506+0000","last_active":"2024-09-11T12:56:50.301964+0000","last_peered":"2024-09-11T12:56:50.301964+0000","last_clean":"2024-09-11T12:56:50.301964+0000","last_became_active":"2024-09-11T12:56:50.289233+0000","last_became_peered":"2024-09-11T12:56:50.289233+0000","last_unstale":"2024-09-11T12:56:50.301964+0000","last_undegraded":"2024-09-11T12:56:50.301964+0000","last_fullsized":"2024-09-11T12:56:50.301964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:05.921628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291264+0000","last_change":"2024-09-11T12:56:50.291264+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291264+0000","last_undegraded":"2024-09-11T12:56:50.291264+0000","last_fullsized":"2024-09-11T12:56:50.291264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310773+0000","last_change":"2024-09-11T12:56:50.305183+0000","last_active":"2024-09-11T12:56:50.310773+0000","last_peered":"2024-09-11T12:56:50.310773+0000","last_clean":"2024-09-11T12:56:50.310773+0000","last_became_active":"2024-09-11T12:56:50.304940+0000","last_became_peered":"2024-09-11T12:56:50.304940+0000","last_unstale":"2024-09-11T12:56:50.310773+0000","last_undegraded":"2024-09-11T12:56:50.310773+0000","last_fullsized":"2024-09-11T12:56:50.310773+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:34.597272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284148+0000","last_change":"2024-09-11T12:56:50.284148+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284148+0000","last_undegraded":"2024-09-11T12:56:50.284148+0000","last_fullsized":"2024-09-11T12:56:50.284148+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310117+0000","last_change":"2024-09-11T12:56:50.308680+0000","last_active":"2024-09-11T12:56:50.310117+0000","last_peered":"2024-09-11T12:56:50.310117+0000","last_clean":"2024-09-11T12:56:50.310117+0000","last_became_active":"2024-09-11T12:56:50.308390+0000","last_became_peered":"2024-09-11T12:56:50.308390+0000","last_unstale":"2024-09-11T12:56:50.310117+0000","last_undegraded":"2024-09-11T12:56:50.310117+0000","last_fullsized":"2024-09-11T12:56:50.310117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:49.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293772+0000","last_change":"2024-09-11T12:56:50.293772+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293772+0000","last_undegraded":"2024-09-11T12:56:50.293772+0000","last_fullsized":"2024-09-11T12:56:50.293772+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.294139+0000","last_change":"2024-09-11T12:56:50.292337+0000","last_active":"2024-09-11T12:56:50.294139+0000","last_peered":"2024-09-11T12:56:50.294139+0000","last_clean":"2024-09-11T12:56:50.294139+0000","last_became_active":"2024-09-11T12:56:50.292057+0000","last_became_peered":"2024-09-11T12:56:50.292057+0000","last_unstale":"2024-09-11T12:56:50.294139+0000","last_undegraded":"2024-09-11T12:56:50.294139+0000","last_fullsized":"2024-09-11T12:56:50.294139+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:21.706481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291443+0000","last_change":"2024-09-11T12:56:50.291443+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291443+0000","last_undegraded":"2024-09-11T12:56:50.291443+0000","last_fullsized":"2024-09-11T12:56:50.291443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305437+0000","last_change":"2024-09-11T12:56:50.300136+0000","last_active":"2024-09-11T12:56:50.305437+0000","last_peered":"2024-09-11T12:56:50.305437+0000","last_clean":"2024-09-11T12:56:50.305437+0000","last_became_active":"2024-09-11T12:56:50.299878+0000","last_became_peered":"2024-09-11T12:56:50.299878+0000","last_unstale":"2024-09-11T12:56:50.305437+0000","last_undegraded":"2024-09-11T12:56:50.305437+0000","last_fullsized":"2024-09-11T12:56:50.305437+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:33.035922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287074+0000","last_change":"2024-09-11T12:56:50.287074+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287074+0000","last_undegraded":"2024-09-11T12:56:50.287074+0000","last_fullsized":"2024-09-11T12:56:50.287074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305578+0000","last_change":"2024-09-11T12:56:50.302419+0000","last_active":"2024-09-11T12:56:50.305578+0000","last_peered":"2024-09-11T12:56:50.305578+0000","last_clean":"2024-09-11T12:56:50.305578+0000","last_became_active":"2024-09-11T12:56:50.302121+0000","last_became_peered":"2024-09-11T12:56:50.302121+0000","last_unstale":"2024-09-11T12:56:50.305578+0000","last_undegraded":"2024-09-11T12:56:50.305578+0000","last_fullsized":"2024-09-11T12:56:50.305578+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:28.653360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288808+0000","last_change":"2024-09-11T12:56:50.288808+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288808+0000","last_undegraded":"2024-09-11T12:56:50.288808+0000","last_fullsized":"2024-09-11T12:56:50.288808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308813+0000","last_change":"2024-09-11T12:56:50.306460+0000","last_active":"2024-09-11T12:56:50.308813+0000","last_peered":"2024-09-11T12:56:50.308813+0000","last_clean":"2024-09-11T12:56:50.308813+0000","last_became_active":"2024-09-11T12:56:50.306176+0000","last_became_peered":"2024-09-11T12:56:50.306176+0000","last_unstale":"2024-09-11T12:56:50.308813+0000","last_undegraded":"2024-09-11T12:56:50.308813+0000","last_fullsized":"2024-09-11T12:56:50.308813+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:22.085546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292960+0000","last_change":"2024-09-11T12:56:50.292960+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292960+0000","last_undegraded":"2024-09-11T12:56:50.292960+0000","last_fullsized":"2024-09-11T12:56:50.292960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310618+0000","last_change":"2024-09-11T12:56:50.309887+0000","last_active":"2024-09-11T12:56:50.310618+0000","last_peered":"2024-09-11T12:56:50.310618+0000","last_clean":"2024-09-11T12:56:50.310618+0000","last_became_active":"2024-09-11T12:56:50.309619+0000","last_became_peered":"2024-09-11T12:56:50.309619+0000","last_unstale":"2024-09-11T12:56:50.310618+0000","last_undegraded":"2024-09-11T12:56:50.310618+0000","last_fullsized":"2024-09-11T12:56:50.310618+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:18.393994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.286085+0000","last_change":"2024-09-11T12:56:50.286085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.286085+0000","last_undegraded":"2024-09-11T12:56:50.286085+0000","last_fullsized":"2024-09-11T12:56:50.286085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302268+0000","last_change":"2024-09-11T12:56:50.298086+0000","last_active":"2024-09-11T12:56:50.302268+0000","last_peered":"2024-09-11T12:56:50.302268+0000","last_clean":"2024-09-11T12:56:50.302268+0000","last_became_active":"2024-09-11T12:56:50.297806+0000","last_became_peered":"2024-09-11T12:56:50.297806+0000","last_unstale":"2024-09-11T12:56:50.302268+0000","last_undegraded":"2024-09-11T12:56:50.302268+0000","last_fullsized":"2024-09-11T12:56:50.302268+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:04.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284022+0000","last_change":"2024-09-11T12:56:50.284022+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284022+0000","last_undegraded":"2024-09-11T12:56:50.284022+0000","last_fullsized":"2024-09-11T12:56:50.284022+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304406+0000","last_change":"2024-09-11T12:56:50.299386+0000","last_active":"2024-09-11T12:56:50.304406+0000","last_peered":"2024-09-11T12:56:50.304406+0000","last_clean":"2024-09-11T12:56:50.304406+0000","last_became_active":"2024-09-11T12:56:50.299158+0000","last_became_peered":"2024-09-11T12:56:50.299158+0000","last_unstale":"2024-09-11T12:56:50.304406+0000","last_undegraded":"2024-09-11T12:56:50.304406+0000","last_fullsized":"2024-09-11T12:56:50.304406+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:36.222739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288673+0000","last_change":"2024-09-11T12:56:50.288673+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288673+0000","last_undegraded":"2024-09-11T12:56:50.288673+0000","last_fullsized":"2024-09-11T12:56:50.288673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310138+0000","last_change":"2024-09-11T12:56:50.308722+0000","last_active":"2024-09-11T12:56:50.310138+0000","last_peered":"2024-09-11T12:56:50.310138+0000","last_clean":"2024-09-11T12:56:50.310138+0000","last_became_active":"2024-09-11T12:56:50.308431+0000","last_became_peered":"2024-09-11T12:56:50.308431+0000","last_unstale":"2024-09-11T12:56:50.310138+0000","last_undegraded":"2024-09-11T12:56:50.310138+0000","last_fullsized":"2024-09-11T12:56:50.310138+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:17.859415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288526+0000","last_change":"2024-09-11T12:56:50.288526+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288526+0000","last_undegraded":"2024-09-11T12:56:50.288526+0000","last_fullsized":"2024-09-11T12:56:50.288526+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302811+0000","last_change":"2024-09-11T12:56:50.289863+0000","last_active":"2024-09-11T12:56:50.302811+0000","last_peered":"2024-09-11T12:56:50.302811+0000","last_clean":"2024-09-11T12:56:50.302811+0000","last_became_active":"2024-09-11T12:56:50.289217+0000","last_became_peered":"2024-09-11T12:56:50.289217+0000","last_unstale":"2024-09-11T12:56:50.302811+0000","last_undegraded":"2024-09-11T12:56:50.302811+0000","last_fullsized":"2024-09-11T12:56:50.302811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:06.556348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283848+0000","last_change":"2024-09-11T12:56:50.283848+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283848+0000","last_undegraded":"2024-09-11T12:56:50.283848+0000","last_fullsized":"2024-09-11T12:56:50.283848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309890+0000","last_change":"2024-09-11T12:56:50.302013+0000","last_active":"2024-09-11T12:56:50.309890+0000","last_peered":"2024-09-11T12:56:50.309890+0000","last_clean":"2024-09-11T12:56:50.309890+0000","last_became_active":"2024-09-11T12:56:50.299221+0000","last_became_peered":"2024-09-11T12:56:50.299221+0000","last_unstale":"2024-09-11T12:56:50.309890+0000","last_undegraded":"2024-09-11T12:56:50.309890+0000","last_fullsized":"2024-09-11T12:56:50.309890+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:50.911770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283957+0000","last_change":"2024-09-11T12:56:50.283957+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283957+0000","last_undegraded":"2024-09-11T12:56:50.283957+0000","last_fullsized":"2024-09-11T12:56:50.283957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.312133+0000","last_change":"2024-09-11T12:56:50.311439+0000","last_active":"2024-09-11T12:56:50.312133+0000","last_peered":"2024-09-11T12:56:50.312133+0000","last_clean":"2024-09-11T12:56:50.312133+0000","last_became_active":"2024-09-11T12:56:50.311186+0000","last_became_peered":"2024-09-11T12:56:50.311186+0000","last_unstale":"2024-09-11T12:56:50.312133+0000","last_undegraded":"2024-09-11T12:56:50.312133+0000","last_fullsized":"2024-09-11T12:56:50.312133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:28.445966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290959+0000","last_change":"2024-09-11T12:56:50.290959+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290959+0000","last_undegraded":"2024-09-11T12:56:50.290959+0000","last_fullsized":"2024-09-11T12:56:50.290959+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301466+0000","last_change":"2024-09-11T12:56:50.297243+0000","last_active":"2024-09-11T12:56:50.301466+0000","last_peered":"2024-09-11T12:56:50.301466+0000","last_clean":"2024-09-11T12:56:50.301466+0000","last_became_active":"2024-09-11T12:56:50.296921+0000","last_became_peered":"2024-09-11T12:56:50.296921+0000","last_unstale":"2024-09-11T12:56:50.301466+0000","last_undegraded":"2024-09-11T12:56:50.301466+0000","last_fullsized":"2024-09-11T12:56:50.301466+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288448+0000","last_change":"2024-09-11T12:56:50.288448+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288448+0000","last_undegraded":"2024-09-11T12:56:50.288448+0000","last_fullsized":"2024-09-11T12:56:50.288448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311866+0000","last_change":"2024-09-11T12:56:50.309614+0000","last_active":"2024-09-11T12:56:50.311866+0000","last_peered":"2024-09-11T12:56:50.311866+0000","last_clean":"2024-09-11T12:56:50.311866+0000","last_became_active":"2024-09-11T12:56:50.309309+0000","last_became_peered":"2024-09-11T12:56:50.309309+0000","last_unstale":"2024-09-11T12:56:50.311866+0000","last_undegraded":"2024-09-11T12:56:50.311866+0000","last_fullsized":"2024-09-11T12:56:50.311866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:12.510525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285640+0000","last_change":"2024-09-11T12:56:50.285640+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285640+0000","last_undegraded":"2024-09-11T12:56:50.285640+0000","last_fullsized":"2024-09-11T12:56:50.285640+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304458+0000","last_change":"2024-09-11T12:56:50.297571+0000","last_active":"2024-09-11T12:56:50.304458+0000","last_peered":"2024-09-11T12:56:50.304458+0000","last_clean":"2024-09-11T12:56:50.304458+0000","last_became_active":"2024-09-11T12:56:50.297307+0000","last_became_peered":"2024-09-11T12:56:50.297307+0000","last_unstale":"2024-09-11T12:56:50.304458+0000","last_undegraded":"2024-09-11T12:56:50.304458+0000","last_fullsized":"2024-09-11T12:56:50.304458+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:37.967545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292344+0000","last_change":"2024-09-11T12:56:50.292344+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292344+0000","last_undegraded":"2024-09-11T12:56:50.292344+0000","last_fullsized":"2024-09-11T12:56:50.292344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310522+0000","last_change":"2024-09-11T12:56:50.309534+0000","last_active":"2024-09-11T12:56:50.310522+0000","last_peered":"2024-09-11T12:56:50.310522+0000","last_clean":"2024-09-11T12:56:50.310522+0000","last_became_active":"2024-09-11T12:56:50.309270+0000","last_became_peered":"2024-09-11T12:56:50.309270+0000","last_unstale":"2024-09-11T12:56:50.310522+0000","last_undegraded":"2024-09-11T12:56:50.310522+0000","last_fullsized":"2024-09-11T12:56:50.310522+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:23.661282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291864+0000","last_change":"2024-09-11T12:56:50.291864+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291864+0000","last_undegraded":"2024-09-11T12:56:50.291864+0000","last_fullsized":"2024-09-11T12:56:50.291864+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301757+0000","last_change":"2024-09-11T12:56:50.295252+0000","last_active":"2024-09-11T12:56:50.301757+0000","last_peered":"2024-09-11T12:56:50.301757+0000","last_clean":"2024-09-11T12:56:50.301757+0000","last_became_active":"2024-09-11T12:56:50.294990+0000","last_became_peered":"2024-09-11T12:56:50.294990+0000","last_unstale":"2024-09-11T12:56:50.301757+0000","last_undegraded":"2024-09-11T12:56:50.301757+0000","last_fullsized":"2024-09-11T12:56:50.301757+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:28.110557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291247+0000","last_change":"2024-09-11T12:56:50.291247+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291247+0000","last_undegraded":"2024-09-11T12:56:50.291247+0000","last_fullsized":"2024-09-11T12:56:50.291247+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308452+0000","last_change":"2024-09-11T12:56:50.307695+0000","last_active":"2024-09-11T12:56:50.308452+0000","last_peered":"2024-09-11T12:56:50.308452+0000","last_clean":"2024-09-11T12:56:50.308452+0000","last_became_active":"2024-09-11T12:56:50.307465+0000","last_became_peered":"2024-09-11T12:56:50.307465+0000","last_unstale":"2024-09-11T12:56:50.308452+0000","last_undegraded":"2024-09-11T12:56:50.308452+0000","last_fullsized":"2024-09-11T12:56:50.308452+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:08.572106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290085+0000","last_change":"2024-09-11T12:56:50.290085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290085+0000","last_undegraded":"2024-09-11T12:56:50.290085+0000","last_fullsized":"2024-09-11T12:56:50.290085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309115+0000","last_change":"2024-09-11T12:56:50.303866+0000","last_active":"2024-09-11T12:56:50.309115+0000","last_peered":"2024-09-11T12:56:50.309115+0000","last_clean":"2024-09-11T12:56:50.309115+0000","last_became_active":"2024-09-11T12:56:50.303577+0000","last_became_peered":"2024-09-11T12:56:50.303577+0000","last_unstale":"2024-09-11T12:56:50.309115+0000","last_undegraded":"2024-09-11T12:56:50.309115+0000","last_fullsized":"2024-09-11T12:56:50.309115+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:49.212192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291910+0000","last_change":"2024-09-11T12:56:50.291910+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291910+0000","last_undegraded":"2024-09-11T12:56:50.291910+0000","last_fullsized":"2024-09-11T12:56:50.291910+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309981+0000","last_change":"2024-09-11T12:56:50.306743+0000","last_active":"2024-09-11T12:56:50.309981+0000","last_peered":"2024-09-11T12:56:50.309981+0000","last_clean":"2024-09-11T12:56:50.309981+0000","last_became_active":"2024-09-11T12:56:50.306472+0000","last_became_peered":"2024-09-11T12:56:50.306472+0000","last_unstale":"2024-09-11T12:56:50.309981+0000","last_undegraded":"2024-09-11T12:56:50.309981+0000","last_fullsized":"2024-09-11T12:56:50.309981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:13.220188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291840+0000","last_change":"2024-09-11T12:56:50.291840+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291840+0000","last_undegraded":"2024-09-11T12:56:50.291840+0000","last_fullsized":"2024-09-11T12:56:50.291840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309616+0000","last_change":"2024-09-11T12:56:50.297814+0000","last_active":"2024-09-11T12:56:50.309616+0000","last_peered":"2024-09-11T12:56:50.309616+0000","last_clean":"2024-09-11T12:56:50.309616+0000","last_became_active":"2024-09-11T12:56:50.296767+0000","last_became_peered":"2024-09-11T12:56:50.296767+0000","last_unstale":"2024-09-11T12:56:50.309616+0000","last_undegraded":"2024-09-11T12:56:50.309616+0000","last_fullsized":"2024-09-11T12:56:50.309616+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:51.352295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295313+0000","last_change":"2024-09-11T12:56:50.295313+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295313+0000","last_undegraded":"2024-09-11T12:56:50.295313+0000","last_fullsized":"2024-09-11T12:56:50.295313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301834+0000","last_change":"2024-09-11T12:56:50.296255+0000","last_active":"2024-09-11T12:56:50.301834+0000","last_peered":"2024-09-11T12:56:50.301834+0000","last_clean":"2024-09-11T12:56:50.301834+0000","last_became_active":"2024-09-11T12:56:50.296043+0000","last_became_peered":"2024-09-11T12:56:50.296043+0000","last_unstale":"2024-09-11T12:56:50.301834+0000","last_undegraded":"2024-09-11T12:56:50.301834+0000","last_fullsized":"2024-09-11T12:56:50.301834+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:32:03.915319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295714+0000","last_change":"2024-09-11T12:56:50.295714+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295714+0000","last_undegraded":"2024-09-11T12:56:50.295714+0000","last_fullsized":"2024-09-11T12:56:50.295714+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.291746+0000","last_change":"2024-09-11T12:56:50.289677+0000","last_active":"2024-09-11T12:56:50.291746+0000","last_peered":"2024-09-11T12:56:50.291746+0000","last_clean":"2024-09-11T12:56:50.291746+0000","last_became_active":"2024-09-11T12:56:50.289134+0000","last_became_peered":"2024-09-11T12:56:50.289134+0000","last_unstale":"2024-09-11T12:56:50.291746+0000","last_undegraded":"2024-09-11T12:56:50.291746+0000","last_fullsized":"2024-09-11T12:56:50.291746+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:55.883458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291338+0000","last_change":"2024-09-11T12:56:50.291338+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291338+0000","last_undegraded":"2024-09-11T12:56:50.291338+0000","last_fullsized":"2024-09-11T12:56:50.291338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304907+0000","last_change":"2024-09-11T12:56:50.301293+0000","last_active":"2024-09-11T12:56:50.304907+0000","last_peered":"2024-09-11T12:56:50.304907+0000","last_clean":"2024-09-11T12:56:50.304907+0000","last_became_active":"2024-09-11T12:56:50.300991+0000","last_became_peered":"2024-09-11T12:56:50.300991+0000","last_unstale":"2024-09-11T12:56:50.304907+0000","last_undegraded":"2024-09-11T12:56:50.304907+0000","last_fullsized":"2024-09-11T12:56:50.304907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291474+0000","last_change":"2024-09-11T12:56:50.291474+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291474+0000","last_undegraded":"2024-09-11T12:56:50.291474+0000","last_fullsized":"2024-09-11T12:56:50.291474+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311941+0000","last_change":"2024-09-11T12:56:50.311418+0000","last_active":"2024-09-11T12:56:50.311941+0000","last_peered":"2024-09-11T12:56:50.311941+0000","last_clean":"2024-09-11T12:56:50.311941+0000","last_became_active":"2024-09-11T12:56:50.311129+0000","last_became_peered":"2024-09-11T12:56:50.311129+0000","last_unstale":"2024-09-11T12:56:50.311941+0000","last_undegraded":"2024-09-11T12:56:50.311941+0000","last_fullsized":"2024-09-11T12:56:50.311941+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:21.576248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288558+0000","last_change":"2024-09-11T12:56:50.288558+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288558+0000","last_undegraded":"2024-09-11T12:56:50.288558+0000","last_fullsized":"2024-09-11T12:56:50.288558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305320+0000","last_change":"2024-09-11T12:56:50.303975+0000","last_active":"2024-09-11T12:56:50.305320+0000","last_peered":"2024-09-11T12:56:50.305320+0000","last_clean":"2024-09-11T12:56:50.305320+0000","last_became_active":"2024-09-11T12:56:50.303570+0000","last_became_peered":"2024-09-11T12:56:50.303570+0000","last_unstale":"2024-09-11T12:56:50.305320+0000","last_undegraded":"2024-09-11T12:56:50.305320+0000","last_fullsized":"2024-09-11T12:56:50.305320+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:11.499583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":368,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.297184+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:50.297184+0000","last_peered":"2024-09-11T12:56:50.297184+0000","last_clean":"2024-09-11T12:56:50.297184+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:50.297184+0000","last_undegraded":"2024-09-11T12:56:50.297184+0000","last_fullsized":"2024-09-11T12:56:50.297184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542235,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:55.420 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:55.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- 172.21.15.78:0/2319990921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 msgr2=0x7f8510073650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 --2- 172.21.15.78:0/2319990921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f8510073650 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009900 tx=0x7f84f801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- 172.21.15.78:0/2319990921 shutdown_connections 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 --2- 172.21.15.78:0/2319990921 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f8510073650 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 --2- 172.21.15.78:0/2319990921 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8510152800 0x7f85100a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- 172.21.15.78:0/2319990921 >> 172.21.15.78:0/2319990921 conn(0x7f851006d870 msgr2=0x7f851006dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- 172.21.15.78:0/2319990921 shutdown_connections 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- 172.21.15.78:0/2319990921 wait complete. 2024-09-11T12:56:55.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 Processor -- start 2024-09-11T12:56:55.603 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.597+0000 7f8514c00640 1 -- start start 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8510071db0 0x7f8510148c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8510152800 0x7f85101550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f85100b36a0 con 0x7f85100731f0 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f85100a9cf0 con 0x7f8510152800 2024-09-11T12:56:55.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8510168c90 con 0x7f8510071db0 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46300/0 (socket says 172.21.15.78:46300) 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 -- 172.21.15.78:0/37961175 learned_addr learned my addr 172.21.15.78:0/37961175 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 -- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8510071db0 msgr2=0x7f8510148c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8510071db0 0x7f8510148c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 -- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8510152800 msgr2=0x7f85101550b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8510152800 0x7f85101550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 -- 172.21.15.78:0/37961175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85101f8bc0 con 0x7f85100731f0 2024-09-11T12:56:55.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507e00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f80040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f80b2040 con 0x7f85100731f0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84f8005030 con 0x7f85100731f0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f8005460 con 0x7f85100731f0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85101f8ea0 con 0x7f85100731f0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85101f9460 con 0x7f85100731f0 2024-09-11T12:56:55.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8510073650 con 0x7f85100731f0 2024-09-11T12:56:55.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.601+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f84f80046d0 con 0x7f85100731f0 2024-09-11T12:56:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.605+0000 7f8507400640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 0x7f84ec074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.605+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f84f812a3d0 con 0x7f85100731f0 2024-09-11T12:56:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.605+0000 7f850f400640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 0x7f84ec074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.608 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.605+0000 7f850f400640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 0x7f84ec074320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85000058d0 tx=0x7f8500005860 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.609+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f84f80164d0 con 0x7f85100731f0 2024-09-11T12:56:55.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.773+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f85100af7f0 con 0x7f84ec071e60 2024-09-11T12:56:55.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.777+0000 7f8507400640 1 -- 172.21.15.78:0/37961175 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7f85100af7f0 con 0x7f84ec071e60 2024-09-11T12:56:55.783 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:55.785 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 msgr2=0x7f84ec074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 0x7f84ec074320 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f85000058d0 tx=0x7f8500005860 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 msgr2=0x7f85100aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009850 tx=0x7f84f80040c0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 shutdown_connections 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f84ec071e60 0x7f84ec074320 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8510152800 0x7f85101550b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f85100731f0 0x7f85100aee80 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 --2- 172.21.15.78:0/37961175 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8510071db0 0x7f8510148c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 >> 172.21.15.78:0/37961175 conn(0x7f851006d870 msgr2=0x7f851015f240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 shutdown_connections 2024-09-11T12:56:55.787 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.781+0000 7f8514c00640 1 -- 172.21.15.78:0/37961175 wait complete. 2024-09-11T12:56:55.803 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T12:56:54.508980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281896,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692824,"statfs":{"total":773094113280,"available":772805451776,"internally_reserved":0,"allocated":16101376,"data_stored":7713313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.230353"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296293+0000","last_change":"2024-09-11T12:56:50.296293+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296293+0000","last_undegraded":"2024-09-11T12:56:50.296293+0000","last_fullsized":"2024-09-11T12:56:50.296293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292499+0000","last_change":"2024-09-11T12:56:50.292499+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292499+0000","last_undegraded":"2024-09-11T12:56:50.292499+0000","last_fullsized":"2024-09-11T12:56:50.292499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.303507+0000","last_change":"2024-09-11T12:56:50.303507+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.303507+0000","last_undegraded":"2024-09-11T12:56:50.303507+0000","last_fullsized":"2024-09-11T12:56:50.303507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292365+0000","last_change":"2024-09-11T12:56:50.292365+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292365+0000","last_undegraded":"2024-09-11T12:56:50.292365+0000","last_fullsized":"2024-09-11T12:56:50.292365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292501+0000","last_change":"2024-09-11T12:56:50.292501+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292501+0000","last_undegraded":"2024-09-11T12:56:50.292501+0000","last_fullsized":"2024-09-11T12:56:50.292501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293414+0000","last_change":"2024-09-11T12:56:50.293414+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293414+0000","last_undegraded":"2024-09-11T12:56:50.293414+0000","last_fullsized":"2024-09-11T12:56:50.293414+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284696+0000","last_change":"2024-09-11T12:56:50.284696+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284696+0000","last_undegraded":"2024-09-11T12:56:50.284696+0000","last_fullsized":"2024-09-11T12:56:50.284696+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290692+0000","last_change":"2024-09-11T12:56:50.290692+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290692+0000","last_undegraded":"2024-09-11T12:56:50.290692+0000","last_fullsized":"2024-09-11T12:56:50.290692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284457+0000","last_change":"2024-09-11T12:56:50.284457+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284457+0000","last_undegraded":"2024-09-11T12:56:50.284457+0000","last_fullsized":"2024-09-11T12:56:50.284457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294783+0000","last_change":"2024-09-11T12:56:50.294783+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294783+0000","last_undegraded":"2024-09-11T12:56:50.294783+0000","last_fullsized":"2024-09-11T12:56:50.294783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285384+0000","last_change":"2024-09-11T12:56:50.285384+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285384+0000","last_undegraded":"2024-09-11T12:56:50.285384+0000","last_fullsized":"2024-09-11T12:56:50.285384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291364+0000","last_change":"2024-09-11T12:56:50.291364+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291364+0000","last_undegraded":"2024-09-11T12:56:50.291364+0000","last_fullsized":"2024-09-11T12:56:50.291364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291751+0000","last_change":"2024-09-11T12:56:50.291751+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291751+0000","last_undegraded":"2024-09-11T12:56:50.291751+0000","last_fullsized":"2024-09-11T12:56:50.291751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285450+0000","last_change":"2024-09-11T12:56:50.285450+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285450+0000","last_undegraded":"2024-09-11T12:56:50.285450+0000","last_fullsized":"2024-09-11T12:56:50.285450+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293196+0000","last_change":"2024-09-11T12:56:50.293196+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293196+0000","last_undegraded":"2024-09-11T12:56:50.293196+0000","last_fullsized":"2024-09-11T12:56:50.293196+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290239+0000","last_change":"2024-09-11T12:56:50.290239+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290239+0000","last_undegraded":"2024-09-11T12:56:50.290239+0000","last_fullsized":"2024-09-11T12:56:50.290239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293571+0000","last_change":"2024-09-11T12:56:50.293571+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293571+0000","last_undegraded":"2024-09-11T12:56:50.293571+0000","last_fullsized":"2024-09-11T12:56:50.293571+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296392+0000","last_change":"2024-09-11T12:56:50.296392+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296392+0000","last_undegraded":"2024-09-11T12:56:50.296392+0000","last_fullsized":"2024-09-11T12:56:50.296392+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294671+0000","last_change":"2024-09-11T12:56:50.294671+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294671+0000","last_undegraded":"2024-09-11T12:56:50.294671+0000","last_fullsized":"2024-09-11T12:56:50.294671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.298439+0000","last_change":"2024-09-11T12:56:50.298439+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.298439+0000","last_undegraded":"2024-09-11T12:56:50.298439+0000","last_fullsized":"2024-09-11T12:56:50.298439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291492+0000","last_change":"2024-09-11T12:56:50.291492+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291492+0000","last_undegraded":"2024-09-11T12:56:50.291492+0000","last_fullsized":"2024-09-11T12:56:50.291492+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294603+0000","last_change":"2024-09-11T12:56:50.294603+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294603+0000","last_undegraded":"2024-09-11T12:56:50.294603+0000","last_fullsized":"2024-09-11T12:56:50.294603+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293271+0000","last_change":"2024-09-11T12:56:50.293271+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293271+0000","last_undegraded":"2024-09-11T12:56:50.293271+0000","last_fullsized":"2024-09-11T12:56:50.293271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.289748+0000","last_change":"2024-09-11T12:56:50.289748+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.289748+0000","last_undegraded":"2024-09-11T12:56:50.289748+0000","last_fullsized":"2024-09-11T12:56:50.289748+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290465+0000","last_change":"2024-09-11T12:56:50.290465+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290465+0000","last_undegraded":"2024-09-11T12:56:50.290465+0000","last_fullsized":"2024-09-11T12:56:50.290465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287922+0000","last_change":"2024-09-11T12:56:50.287922+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287922+0000","last_undegraded":"2024-09-11T12:56:50.287922+0000","last_fullsized":"2024-09-11T12:56:50.287922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.301309+0000","last_change":"2024-09-11T12:56:50.301309+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.301309+0000","last_undegraded":"2024-09-11T12:56:50.301309+0000","last_fullsized":"2024-09-11T12:56:50.301309+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293978+0000","last_change":"2024-09-11T12:56:50.293978+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293978+0000","last_undegraded":"2024-09-11T12:56:50.293978+0000","last_fullsized":"2024-09-11T12:56:50.293978+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290638+0000","last_change":"2024-09-11T12:56:50.290638+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290638+0000","last_undegraded":"2024-09-11T12:56:50.290638+0000","last_fullsized":"2024-09-11T12:56:50.290638+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284031+0000","last_change":"2024-09-11T12:56:50.284031+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284031+0000","last_undegraded":"2024-09-11T12:56:50.284031+0000","last_fullsized":"2024-09-11T12:56:50.284031+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292818+0000","last_change":"2024-09-11T12:56:50.292818+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292818+0000","last_undegraded":"2024-09-11T12:56:50.292818+0000","last_fullsized":"2024-09-11T12:56:50.292818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291033+0000","last_change":"2024-09-11T12:56:50.291033+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291033+0000","last_undegraded":"2024-09-11T12:56:50.291033+0000","last_fullsized":"2024-09-11T12:56:50.291033+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292095+0000","last_change":"2024-09-11T12:56:50.292095+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292095+0000","last_undegraded":"2024-09-11T12:56:50.292095+0000","last_fullsized":"2024-09-11T12:56:50.292095+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302831+0000","last_change":"2024-09-11T12:56:50.290099+0000","last_active":"2024-09-11T12:56:50.302831+0000","last_peered":"2024-09-11T12:56:50.302831+0000","last_clean":"2024-09-11T12:56:50.302831+0000","last_became_active":"2024-09-11T12:56:50.289571+0000","last_became_peered":"2024-09-11T12:56:50.289571+0000","last_unstale":"2024-09-11T12:56:50.302831+0000","last_undegraded":"2024-09-11T12:56:50.302831+0000","last_fullsized":"2024-09-11T12:56:50.302831+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:44.844033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295530+0000","last_change":"2024-09-11T12:56:50.295530+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295530+0000","last_undegraded":"2024-09-11T12:56:50.295530+0000","last_fullsized":"2024-09-11T12:56:50.295530+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.306688+0000","last_change":"2024-09-11T12:56:50.305706+0000","last_active":"2024-09-11T12:56:50.306688+0000","last_peered":"2024-09-11T12:56:50.306688+0000","last_clean":"2024-09-11T12:56:50.306688+0000","last_became_active":"2024-09-11T12:56:50.303736+0000","last_became_peered":"2024-09-11T12:56:50.303736+0000","last_unstale":"2024-09-11T12:56:50.306688+0000","last_undegraded":"2024-09-11T12:56:50.306688+0000","last_fullsized":"2024-09-11T12:56:50.306688+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:09.904686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287500+0000","last_change":"2024-09-11T12:56:50.287500+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287500+0000","last_undegraded":"2024-09-11T12:56:50.287500+0000","last_fullsized":"2024-09-11T12:56:50.287500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309084+0000","last_change":"2024-09-11T12:56:50.306449+0000","last_active":"2024-09-11T12:56:50.309084+0000","last_peered":"2024-09-11T12:56:50.309084+0000","last_clean":"2024-09-11T12:56:50.309084+0000","last_became_active":"2024-09-11T12:56:50.306013+0000","last_became_peered":"2024-09-11T12:56:50.306013+0000","last_unstale":"2024-09-11T12:56:50.309084+0000","last_undegraded":"2024-09-11T12:56:50.309084+0000","last_fullsized":"2024-09-11T12:56:50.309084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:55.224946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283841+0000","last_change":"2024-09-11T12:56:50.283841+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283841+0000","last_undegraded":"2024-09-11T12:56:50.283841+0000","last_fullsized":"2024-09-11T12:56:50.283841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305283+0000","last_change":"2024-09-11T12:56:50.302084+0000","last_active":"2024-09-11T12:56:50.305283+0000","last_peered":"2024-09-11T12:56:50.305283+0000","last_clean":"2024-09-11T12:56:50.305283+0000","last_became_active":"2024-09-11T12:56:50.301802+0000","last_became_peered":"2024-09-11T12:56:50.301802+0000","last_unstale":"2024-09-11T12:56:50.305283+0000","last_undegraded":"2024-09-11T12:56:50.305283+0000","last_fullsized":"2024-09-11T12:56:50.305283+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:19.743245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292388+0000","last_change":"2024-09-11T12:56:50.292388+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292388+0000","last_undegraded":"2024-09-11T12:56:50.292388+0000","last_fullsized":"2024-09-11T12:56:50.292388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301964+0000","last_change":"2024-09-11T12:56:50.289506+0000","last_active":"2024-09-11T12:56:50.301964+0000","last_peered":"2024-09-11T12:56:50.301964+0000","last_clean":"2024-09-11T12:56:50.301964+0000","last_became_active":"2024-09-11T12:56:50.289233+0000","last_became_peered":"2024-09-11T12:56:50.289233+0000","last_unstale":"2024-09-11T12:56:50.301964+0000","last_undegraded":"2024-09-11T12:56:50.301964+0000","last_fullsized":"2024-09-11T12:56:50.301964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:05.921628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291264+0000","last_change":"2024-09-11T12:56:50.291264+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291264+0000","last_undegraded":"2024-09-11T12:56:50.291264+0000","last_fullsized":"2024-09-11T12:56:50.291264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310773+0000","last_change":"2024-09-11T12:56:50.305183+0000","last_active":"2024-09-11T12:56:50.310773+0000","last_peered":"2024-09-11T12:56:50.310773+0000","last_clean":"2024-09-11T12:56:50.310773+0000","last_became_active":"2024-09-11T12:56:50.304940+0000","last_became_peered":"2024-09-11T12:56:50.304940+0000","last_unstale":"2024-09-11T12:56:50.310773+0000","last_undegraded":"2024-09-11T12:56:50.310773+0000","last_fullsized":"2024-09-11T12:56:50.310773+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:34.597272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284148+0000","last_change":"2024-09-11T12:56:50.284148+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284148+0000","last_undegraded":"2024-09-11T12:56:50.284148+0000","last_fullsized":"2024-09-11T12:56:50.284148+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310117+0000","last_change":"2024-09-11T12:56:50.308680+0000","last_active":"2024-09-11T12:56:50.310117+0000","last_peered":"2024-09-11T12:56:50.310117+0000","last_clean":"2024-09-11T12:56:50.310117+0000","last_became_active":"2024-09-11T12:56:50.308390+0000","last_became_peered":"2024-09-11T12:56:50.308390+0000","last_unstale":"2024-09-11T12:56:50.310117+0000","last_undegraded":"2024-09-11T12:56:50.310117+0000","last_fullsized":"2024-09-11T12:56:50.310117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:49.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293772+0000","last_change":"2024-09-11T12:56:50.293772+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293772+0000","last_undegraded":"2024-09-11T12:56:50.293772+0000","last_fullsized":"2024-09-11T12:56:50.293772+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.294139+0000","last_change":"2024-09-11T12:56:50.292337+0000","last_active":"2024-09-11T12:56:50.294139+0000","last_peered":"2024-09-11T12:56:50.294139+0000","last_clean":"2024-09-11T12:56:50.294139+0000","last_became_active":"2024-09-11T12:56:50.292057+0000","last_became_peered":"2024-09-11T12:56:50.292057+0000","last_unstale":"2024-09-11T12:56:50.294139+0000","last_undegraded":"2024-09-11T12:56:50.294139+0000","last_fullsized":"2024-09-11T12:56:50.294139+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:21.706481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291443+0000","last_change":"2024-09-11T12:56:50.291443+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291443+0000","last_undegraded":"2024-09-11T12:56:50.291443+0000","last_fullsized":"2024-09-11T12:56:50.291443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305437+0000","last_change":"2024-09-11T12:56:50.300136+0000","last_active":"2024-09-11T12:56:50.305437+0000","last_peered":"2024-09-11T12:56:50.305437+0000","last_clean":"2024-09-11T12:56:50.305437+0000","last_became_active":"2024-09-11T12:56:50.299878+0000","last_became_peered":"2024-09-11T12:56:50.299878+0000","last_unstale":"2024-09-11T12:56:50.305437+0000","last_undegraded":"2024-09-11T12:56:50.305437+0000","last_fullsized":"2024-09-11T12:56:50.305437+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:33.035922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287074+0000","last_change":"2024-09-11T12:56:50.287074+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287074+0000","last_undegraded":"2024-09-11T12:56:50.287074+0000","last_fullsized":"2024-09-11T12:56:50.287074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305578+0000","last_change":"2024-09-11T12:56:50.302419+0000","last_active":"2024-09-11T12:56:50.305578+0000","last_peered":"2024-09-11T12:56:50.305578+0000","last_clean":"2024-09-11T12:56:50.305578+0000","last_became_active":"2024-09-11T12:56:50.302121+0000","last_became_peered":"2024-09-11T12:56:50.302121+0000","last_unstale":"2024-09-11T12:56:50.305578+0000","last_undegraded":"2024-09-11T12:56:50.305578+0000","last_fullsized":"2024-09-11T12:56:50.305578+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:28.653360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288808+0000","last_change":"2024-09-11T12:56:50.288808+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288808+0000","last_undegraded":"2024-09-11T12:56:50.288808+0000","last_fullsized":"2024-09-11T12:56:50.288808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308813+0000","last_change":"2024-09-11T12:56:50.306460+0000","last_active":"2024-09-11T12:56:50.308813+0000","last_peered":"2024-09-11T12:56:50.308813+0000","last_clean":"2024-09-11T12:56:50.308813+0000","last_became_active":"2024-09-11T12:56:50.306176+0000","last_became_peered":"2024-09-11T12:56:50.306176+0000","last_unstale":"2024-09-11T12:56:50.308813+0000","last_undegraded":"2024-09-11T12:56:50.308813+0000","last_fullsized":"2024-09-11T12:56:50.308813+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:22.085546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292960+0000","last_change":"2024-09-11T12:56:50.292960+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292960+0000","last_undegraded":"2024-09-11T12:56:50.292960+0000","last_fullsized":"2024-09-11T12:56:50.292960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310618+0000","last_change":"2024-09-11T12:56:50.309887+0000","last_active":"2024-09-11T12:56:50.310618+0000","last_peered":"2024-09-11T12:56:50.310618+0000","last_clean":"2024-09-11T12:56:50.310618+0000","last_became_active":"2024-09-11T12:56:50.309619+0000","last_became_peered":"2024-09-11T12:56:50.309619+0000","last_unstale":"2024-09-11T12:56:50.310618+0000","last_undegraded":"2024-09-11T12:56:50.310618+0000","last_fullsized":"2024-09-11T12:56:50.310618+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:18.393994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.286085+0000","last_change":"2024-09-11T12:56:50.286085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.286085+0000","last_undegraded":"2024-09-11T12:56:50.286085+0000","last_fullsized":"2024-09-11T12:56:50.286085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302268+0000","last_change":"2024-09-11T12:56:50.298086+0000","last_active":"2024-09-11T12:56:50.302268+0000","last_peered":"2024-09-11T12:56:50.302268+0000","last_clean":"2024-09-11T12:56:50.302268+0000","last_became_active":"2024-09-11T12:56:50.297806+0000","last_became_peered":"2024-09-11T12:56:50.297806+0000","last_unstale":"2024-09-11T12:56:50.302268+0000","last_undegraded":"2024-09-11T12:56:50.302268+0000","last_fullsized":"2024-09-11T12:56:50.302268+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:04.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284022+0000","last_change":"2024-09-11T12:56:50.284022+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284022+0000","last_undegraded":"2024-09-11T12:56:50.284022+0000","last_fullsized":"2024-09-11T12:56:50.284022+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304406+0000","last_change":"2024-09-11T12:56:50.299386+0000","last_active":"2024-09-11T12:56:50.304406+0000","last_peered":"2024-09-11T12:56:50.304406+0000","last_clean":"2024-09-11T12:56:50.304406+0000","last_became_active":"2024-09-11T12:56:50.299158+0000","last_became_peered":"2024-09-11T12:56:50.299158+0000","last_unstale":"2024-09-11T12:56:50.304406+0000","last_undegraded":"2024-09-11T12:56:50.304406+0000","last_fullsized":"2024-09-11T12:56:50.304406+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:36.222739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288673+0000","last_change":"2024-09-11T12:56:50.288673+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288673+0000","last_undegraded":"2024-09-11T12:56:50.288673+0000","last_fullsized":"2024-09-11T12:56:50.288673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310138+0000","last_change":"2024-09-11T12:56:50.308722+0000","last_active":"2024-09-11T12:56:50.310138+0000","last_peered":"2024-09-11T12:56:50.310138+0000","last_clean":"2024-09-11T12:56:50.310138+0000","last_became_active":"2024-09-11T12:56:50.308431+0000","last_became_peered":"2024-09-11T12:56:50.308431+0000","last_unstale":"2024-09-11T12:56:50.310138+0000","last_undegraded":"2024-09-11T12:56:50.310138+0000","last_fullsized":"2024-09-11T12:56:50.310138+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:17.859415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288526+0000","last_change":"2024-09-11T12:56:50.288526+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288526+0000","last_undegraded":"2024-09-11T12:56:50.288526+0000","last_fullsized":"2024-09-11T12:56:50.288526+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302811+0000","last_change":"2024-09-11T12:56:50.289863+0000","last_active":"2024-09-11T12:56:50.302811+0000","last_peered":"2024-09-11T12:56:50.302811+0000","last_clean":"2024-09-11T12:56:50.302811+0000","last_became_active":"2024-09-11T12:56:50.289217+0000","last_became_peered":"2024-09-11T12:56:50.289217+0000","last_unstale":"2024-09-11T12:56:50.302811+0000","last_undegraded":"2024-09-11T12:56:50.302811+0000","last_fullsized":"2024-09-11T12:56:50.302811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:06.556348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283848+0000","last_change":"2024-09-11T12:56:50.283848+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283848+0000","last_undegraded":"2024-09-11T12:56:50.283848+0000","last_fullsized":"2024-09-11T12:56:50.283848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309890+0000","last_change":"2024-09-11T12:56:50.302013+0000","last_active":"2024-09-11T12:56:50.309890+0000","last_peered":"2024-09-11T12:56:50.309890+0000","last_clean":"2024-09-11T12:56:50.309890+0000","last_became_active":"2024-09-11T12:56:50.299221+0000","last_became_peered":"2024-09-11T12:56:50.299221+0000","last_unstale":"2024-09-11T12:56:50.309890+0000","last_undegraded":"2024-09-11T12:56:50.309890+0000","last_fullsized":"2024-09-11T12:56:50.309890+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:50.911770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283957+0000","last_change":"2024-09-11T12:56:50.283957+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283957+0000","last_undegraded":"2024-09-11T12:56:50.283957+0000","last_fullsized":"2024-09-11T12:56:50.283957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.312133+0000","last_change":"2024-09-11T12:56:50.311439+0000","last_active":"2024-09-11T12:56:50.312133+0000","last_peered":"2024-09-11T12:56:50.312133+0000","last_clean":"2024-09-11T12:56:50.312133+0000","last_became_active":"2024-09-11T12:56:50.311186+0000","last_became_peered":"2024-09-11T12:56:50.311186+0000","last_unstale":"2024-09-11T12:56:50.312133+0000","last_undegraded":"2024-09-11T12:56:50.312133+0000","last_fullsized":"2024-09-11T12:56:50.312133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:28.445966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290959+0000","last_change":"2024-09-11T12:56:50.290959+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290959+0000","last_undegraded":"2024-09-11T12:56:50.290959+0000","last_fullsized":"2024-09-11T12:56:50.290959+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301466+0000","last_change":"2024-09-11T12:56:50.297243+0000","last_active":"2024-09-11T12:56:50.301466+0000","last_peered":"2024-09-11T12:56:50.301466+0000","last_clean":"2024-09-11T12:56:50.301466+0000","last_became_active":"2024-09-11T12:56:50.296921+0000","last_became_peered":"2024-09-11T12:56:50.296921+0000","last_unstale":"2024-09-11T12:56:50.301466+0000","last_undegraded":"2024-09-11T12:56:50.301466+0000","last_fullsized":"2024-09-11T12:56:50.301466+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288448+0000","last_change":"2024-09-11T12:56:50.288448+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288448+0000","last_undegraded":"2024-09-11T12:56:50.288448+0000","last_fullsized":"2024-09-11T12:56:50.288448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311866+0000","last_change":"2024-09-11T12:56:50.309614+0000","last_active":"2024-09-11T12:56:50.311866+0000","last_peered":"2024-09-11T12:56:50.311866+0000","last_clean":"2024-09-11T12:56:50.311866+0000","last_became_active":"2024-09-11T12:56:50.309309+0000","last_became_peered":"2024-09-11T12:56:50.309309+0000","last_unstale":"2024-09-11T12:56:50.311866+0000","last_undegraded":"2024-09-11T12:56:50.311866+0000","last_fullsized":"2024-09-11T12:56:50.311866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:12.510525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285640+0000","last_change":"2024-09-11T12:56:50.285640+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285640+0000","last_undegraded":"2024-09-11T12:56:50.285640+0000","last_fullsized":"2024-09-11T12:56:50.285640+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304458+0000","last_change":"2024-09-11T12:56:50.297571+0000","last_active":"2024-09-11T12:56:50.304458+0000","last_peered":"2024-09-11T12:56:50.304458+0000","last_clean":"2024-09-11T12:56:50.304458+0000","last_became_active":"2024-09-11T12:56:50.297307+0000","last_became_peered":"2024-09-11T12:56:50.297307+0000","last_unstale":"2024-09-11T12:56:50.304458+0000","last_undegraded":"2024-09-11T12:56:50.304458+0000","last_fullsized":"2024-09-11T12:56:50.304458+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:37.967545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292344+0000","last_change":"2024-09-11T12:56:50.292344+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292344+0000","last_undegraded":"2024-09-11T12:56:50.292344+0000","last_fullsized":"2024-09-11T12:56:50.292344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310522+0000","last_change":"2024-09-11T12:56:50.309534+0000","last_active":"2024-09-11T12:56:50.310522+0000","last_peered":"2024-09-11T12:56:50.310522+0000","last_clean":"2024-09-11T12:56:50.310522+0000","last_became_active":"2024-09-11T12:56:50.309270+0000","last_became_peered":"2024-09-11T12:56:50.309270+0000","last_unstale":"2024-09-11T12:56:50.310522+0000","last_undegraded":"2024-09-11T12:56:50.310522+0000","last_fullsized":"2024-09-11T12:56:50.310522+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:23.661282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291864+0000","last_change":"2024-09-11T12:56:50.291864+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291864+0000","last_undegraded":"2024-09-11T12:56:50.291864+0000","last_fullsized":"2024-09-11T12:56:50.291864+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301757+0000","last_change":"2024-09-11T12:56:50.295252+0000","last_active":"2024-09-11T12:56:50.301757+0000","last_peered":"2024-09-11T12:56:50.301757+0000","last_clean":"2024-09-11T12:56:50.301757+0000","last_became_active":"2024-09-11T12:56:50.294990+0000","last_became_peered":"2024-09-11T12:56:50.294990+0000","last_unstale":"2024-09-11T12:56:50.301757+0000","last_undegraded":"2024-09-11T12:56:50.301757+0000","last_fullsized":"2024-09-11T12:56:50.301757+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:28.110557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291247+0000","last_change":"2024-09-11T12:56:50.291247+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291247+0000","last_undegraded":"2024-09-11T12:56:50.291247+0000","last_fullsized":"2024-09-11T12:56:50.291247+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308452+0000","last_change":"2024-09-11T12:56:50.307695+0000","last_active":"2024-09-11T12:56:50.308452+0000","last_peered":"2024-09-11T12:56:50.308452+0000","last_clean":"2024-09-11T12:56:50.308452+0000","last_became_active":"2024-09-11T12:56:50.307465+0000","last_became_peered":"2024-09-11T12:56:50.307465+0000","last_unstale":"2024-09-11T12:56:50.308452+0000","last_undegraded":"2024-09-11T12:56:50.308452+0000","last_fullsized":"2024-09-11T12:56:50.308452+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:08.572106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290085+0000","last_change":"2024-09-11T12:56:50.290085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290085+0000","last_undegraded":"2024-09-11T12:56:50.290085+0000","last_fullsized":"2024-09-11T12:56:50.290085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309115+0000","last_change":"2024-09-11T12:56:50.303866+0000","last_active":"2024-09-11T12:56:50.309115+0000","last_peered":"2024-09-11T12:56:50.309115+0000","last_clean":"2024-09-11T12:56:50.309115+0000","last_became_active":"2024-09-11T12:56:50.303577+0000","last_became_peered":"2024-09-11T12:56:50.303577+0000","last_unstale":"2024-09-11T12:56:50.309115+0000","last_undegraded":"2024-09-11T12:56:50.309115+0000","last_fullsized":"2024-09-11T12:56:50.309115+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:49.212192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291910+0000","last_change":"2024-09-11T12:56:50.291910+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291910+0000","last_undegraded":"2024-09-11T12:56:50.291910+0000","last_fullsized":"2024-09-11T12:56:50.291910+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309981+0000","last_change":"2024-09-11T12:56:50.306743+0000","last_active":"2024-09-11T12:56:50.309981+0000","last_peered":"2024-09-11T12:56:50.309981+0000","last_clean":"2024-09-11T12:56:50.309981+0000","last_became_active":"2024-09-11T12:56:50.306472+0000","last_became_peered":"2024-09-11T12:56:50.306472+0000","last_unstale":"2024-09-11T12:56:50.309981+0000","last_undegraded":"2024-09-11T12:56:50.309981+0000","last_fullsized":"2024-09-11T12:56:50.309981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:13.220188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291840+0000","last_change":"2024-09-11T12:56:50.291840+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291840+0000","last_undegraded":"2024-09-11T12:56:50.291840+0000","last_fullsized":"2024-09-11T12:56:50.291840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309616+0000","last_change":"2024-09-11T12:56:50.297814+0000","last_active":"2024-09-11T12:56:50.309616+0000","last_peered":"2024-09-11T12:56:50.309616+0000","last_clean":"2024-09-11T12:56:50.309616+0000","last_became_active":"2024-09-11T12:56:50.296767+0000","last_became_peered":"2024-09-11T12:56:50.296767+0000","last_unstale":"2024-09-11T12:56:50.309616+0000","last_undegraded":"2024-09-11T12:56:50.309616+0000","last_fullsized":"2024-09-11T12:56:50.309616+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:51.352295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295313+0000","last_change":"2024-09-11T12:56:50.295313+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295313+0000","last_undegraded":"2024-09-11T12:56:50.295313+0000","last_fullsized":"2024-09-11T12:56:50.295313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301834+0000","last_change":"2024-09-11T12:56:50.296255+0000","last_active":"2024-09-11T12:56:50.301834+0000","last_peered":"2024-09-11T12:56:50.301834+0000","last_clean":"2024-09-11T12:56:50.301834+0000","last_became_active":"2024-09-11T12:56:50.296043+0000","last_became_peered":"2024-09-11T12:56:50.296043+0000","last_unstale":"2024-09-11T12:56:50.301834+0000","last_undegraded":"2024-09-11T12:56:50.301834+0000","last_fullsized":"2024-09-11T12:56:50.301834+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:32:03.915319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295714+0000","last_change":"2024-09-11T12:56:50.295714+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295714+0000","last_undegraded":"2024-09-11T12:56:50.295714+0000","last_fullsized":"2024-09-11T12:56:50.295714+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.291746+0000","last_change":"2024-09-11T12:56:50.289677+0000","last_active":"2024-09-11T12:56:50.291746+0000","last_peered":"2024-09-11T12:56:50.291746+0000","last_clean":"2024-09-11T12:56:50.291746+0000","last_became_active":"2024-09-11T12:56:50.289134+0000","last_became_peered":"2024-09-11T12:56:50.289134+0000","last_unstale":"2024-09-11T12:56:50.291746+0000","last_undegraded":"2024-09-11T12:56:50.291746+0000","last_fullsized":"2024-09-11T12:56:50.291746+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:55.883458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291338+0000","last_change":"2024-09-11T12:56:50.291338+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291338+0000","last_undegraded":"2024-09-11T12:56:50.291338+0000","last_fullsized":"2024-09-11T12:56:50.291338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304907+0000","last_change":"2024-09-11T12:56:50.301293+0000","last_active":"2024-09-11T12:56:50.304907+0000","last_peered":"2024-09-11T12:56:50.304907+0000","last_clean":"2024-09-11T12:56:50.304907+0000","last_became_active":"2024-09-11T12:56:50.300991+0000","last_became_peered":"2024-09-11T12:56:50.300991+0000","last_unstale":"2024-09-11T12:56:50.304907+0000","last_undegraded":"2024-09-11T12:56:50.304907+0000","last_fullsized":"2024-09-11T12:56:50.304907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291474+0000","last_change":"2024-09-11T12:56:50.291474+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291474+0000","last_undegraded":"2024-09-11T12:56:50.291474+0000","last_fullsized":"2024-09-11T12:56:50.291474+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311941+0000","last_change":"2024-09-11T12:56:50.311418+0000","last_active":"2024-09-11T12:56:50.311941+0000","last_peered":"2024-09-11T12:56:50.311941+0000","last_clean":"2024-09-11T12:56:50.311941+0000","last_became_active":"2024-09-11T12:56:50.311129+0000","last_became_peered":"2024-09-11T12:56:50.311129+0000","last_unstale":"2024-09-11T12:56:50.311941+0000","last_undegraded":"2024-09-11T12:56:50.311941+0000","last_fullsized":"2024-09-11T12:56:50.311941+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:21.576248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288558+0000","last_change":"2024-09-11T12:56:50.288558+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288558+0000","last_undegraded":"2024-09-11T12:56:50.288558+0000","last_fullsized":"2024-09-11T12:56:50.288558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305320+0000","last_change":"2024-09-11T12:56:50.303975+0000","last_active":"2024-09-11T12:56:50.305320+0000","last_peered":"2024-09-11T12:56:50.305320+0000","last_clean":"2024-09-11T12:56:50.305320+0000","last_became_active":"2024-09-11T12:56:50.303570+0000","last_became_peered":"2024-09-11T12:56:50.303570+0000","last_unstale":"2024-09-11T12:56:50.305320+0000","last_undegraded":"2024-09-11T12:56:50.305320+0000","last_fullsized":"2024-09-11T12:56:50.305320+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:11.499583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":368,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.297184+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:50.297184+0000","last_peered":"2024-09-11T12:56:50.297184+0000","last_clean":"2024-09-11T12:56:50.297184+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:50.297184+0000","last_undegraded":"2024-09-11T12:56:50.297184+0000","last_fullsized":"2024-09-11T12:56:50.297184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542235,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:55.806 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- 172.21.15.78:0/1129446751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 msgr2=0x7f95b815ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- 172.21.15.78:0/1129446751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b815ba80 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f95ac009900 tx=0x7f95ac01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- 172.21.15.78:0/1129446751 shutdown_connections 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- 172.21.15.78:0/1129446751 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 0x7f95b815ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- 172.21.15.78:0/1129446751 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b815ba80 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- 172.21.15.78:0/1129446751 >> 172.21.15.78:0/1129446751 conn(0x7f95b806ca80 msgr2=0x7f95b80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- 172.21.15.78:0/1129446751 shutdown_connections 2024-09-11T12:56:55.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- 172.21.15.78:0/1129446751 wait complete. 2024-09-11T12:56:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 Processor -- start 2024-09-11T12:56:55.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- start start 2024-09-11T12:56:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 0x7f95b81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 0x7f95b8153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f95b8167740 con 0x7f95b8074240 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f95b80728c0 con 0x7f95b815e5b0 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95c0600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f95b81678c0 con 0x7f95b8070360 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.985+0000 7f95bd200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46322/0 (socket says 172.21.15.78:46322) 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bdc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 0x7f95b81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 -- 172.21.15.78:0/4085053376 learned_addr learned my addr 172.21.15.78:0/4085053376 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95be600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 0x7f95b8153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 -- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 msgr2=0x7f95b81500b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 0x7f95b81500b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 -- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 msgr2=0x7f95b8153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:55.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 0x7f95b8153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 -- 172.21.15.78:0/4085053376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95b8153ff0 con 0x7f95b8074240 2024-09-11T12:56:55.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95be600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 0x7f95b8153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95bd200640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c3a0 tx=0x7f95a400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4017070 con 0x7f95b8074240 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95a40053e0 con 0x7f95b8074240 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95a4003880 con 0x7f95b8074240 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95b815f670 con 0x7f95b8074240 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95b815fc30 con 0x7f95b8074240 2024-09-11T12:56:55.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95b815fe30 con 0x7f95b8074240 2024-09-11T12:56:55.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.989+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f95a401a070 con 0x7f95b8074240 2024-09-11T12:56:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.993+0000 7f95b5600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 0x7f9598074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.993+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f95a4092370 con 0x7f95b8074240 2024-09-11T12:56:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.993+0000 7f95bdc00640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 0x7f9598074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:55.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.993+0000 7f95bdc00640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 0x7f9598074350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0af000 tx=0x7f95ac01b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:55.999 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:55.993+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f95b815fe30 con 0x7f95b8074240 2024-09-11T12:56:56.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.157+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f95b8164ba0 con 0x7f9598071e90 2024-09-11T12:56:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.161+0000 7f95b5600640 1 -- 172.21.15.78:0/4085053376 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247114 (secure 0 0 0) 0x7f95b8164ba0 con 0x7f9598071e90 2024-09-11T12:56:56.166 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 msgr2=0x7f9598074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 0x7f9598074350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f95ac0af000 tx=0x7f95ac01b0f0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 msgr2=0x7f95b8160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f95a400c3a0 tx=0x7f95a400c860 comp rx=0 tx=0).stop 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 shutdown_connections 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9598071e90 0x7f9598074350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f95b815e5b0 0x7f95b8153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f95b8074240 0x7f95b8160150 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 --2- 172.21.15.78:0/4085053376 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f95b8070360 0x7f95b81500b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 >> 172.21.15.78:0/4085053376 conn(0x7f95b806ca80 msgr2=0x7f95b80cefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 shutdown_connections 2024-09-11T12:56:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.165+0000 7f95c0600640 1 -- 172.21.15.78:0/4085053376 wait complete. 2024-09-11T12:56:56.187 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T12:56:54.508980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":23,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":281896,"kb_used_data":15724,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692824,"statfs":{"total":773094113280,"available":772805451776,"internally_reserved":0,"allocated":16101376,"data_stored":7713313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.230353"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296293+0000","last_change":"2024-09-11T12:56:50.296293+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296293+0000","last_undegraded":"2024-09-11T12:56:50.296293+0000","last_fullsized":"2024-09-11T12:56:50.296293+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292499+0000","last_change":"2024-09-11T12:56:50.292499+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292499+0000","last_undegraded":"2024-09-11T12:56:50.292499+0000","last_fullsized":"2024-09-11T12:56:50.292499+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.303507+0000","last_change":"2024-09-11T12:56:50.303507+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.303507+0000","last_undegraded":"2024-09-11T12:56:50.303507+0000","last_fullsized":"2024-09-11T12:56:50.303507+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292365+0000","last_change":"2024-09-11T12:56:50.292365+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292365+0000","last_undegraded":"2024-09-11T12:56:50.292365+0000","last_fullsized":"2024-09-11T12:56:50.292365+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292501+0000","last_change":"2024-09-11T12:56:50.292501+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292501+0000","last_undegraded":"2024-09-11T12:56:50.292501+0000","last_fullsized":"2024-09-11T12:56:50.292501+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293414+0000","last_change":"2024-09-11T12:56:50.293414+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293414+0000","last_undegraded":"2024-09-11T12:56:50.293414+0000","last_fullsized":"2024-09-11T12:56:50.293414+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284696+0000","last_change":"2024-09-11T12:56:50.284696+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284696+0000","last_undegraded":"2024-09-11T12:56:50.284696+0000","last_fullsized":"2024-09-11T12:56:50.284696+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290692+0000","last_change":"2024-09-11T12:56:50.290692+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290692+0000","last_undegraded":"2024-09-11T12:56:50.290692+0000","last_fullsized":"2024-09-11T12:56:50.290692+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284457+0000","last_change":"2024-09-11T12:56:50.284457+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284457+0000","last_undegraded":"2024-09-11T12:56:50.284457+0000","last_fullsized":"2024-09-11T12:56:50.284457+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294783+0000","last_change":"2024-09-11T12:56:50.294783+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294783+0000","last_undegraded":"2024-09-11T12:56:50.294783+0000","last_fullsized":"2024-09-11T12:56:50.294783+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285384+0000","last_change":"2024-09-11T12:56:50.285384+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285384+0000","last_undegraded":"2024-09-11T12:56:50.285384+0000","last_fullsized":"2024-09-11T12:56:50.285384+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291364+0000","last_change":"2024-09-11T12:56:50.291364+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291364+0000","last_undegraded":"2024-09-11T12:56:50.291364+0000","last_fullsized":"2024-09-11T12:56:50.291364+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291751+0000","last_change":"2024-09-11T12:56:50.291751+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291751+0000","last_undegraded":"2024-09-11T12:56:50.291751+0000","last_fullsized":"2024-09-11T12:56:50.291751+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285450+0000","last_change":"2024-09-11T12:56:50.285450+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285450+0000","last_undegraded":"2024-09-11T12:56:50.285450+0000","last_fullsized":"2024-09-11T12:56:50.285450+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293196+0000","last_change":"2024-09-11T12:56:50.293196+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293196+0000","last_undegraded":"2024-09-11T12:56:50.293196+0000","last_fullsized":"2024-09-11T12:56:50.293196+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290239+0000","last_change":"2024-09-11T12:56:50.290239+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290239+0000","last_undegraded":"2024-09-11T12:56:50.290239+0000","last_fullsized":"2024-09-11T12:56:50.290239+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293571+0000","last_change":"2024-09-11T12:56:50.293571+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293571+0000","last_undegraded":"2024-09-11T12:56:50.293571+0000","last_fullsized":"2024-09-11T12:56:50.293571+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.296392+0000","last_change":"2024-09-11T12:56:50.296392+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.296392+0000","last_undegraded":"2024-09-11T12:56:50.296392+0000","last_fullsized":"2024-09-11T12:56:50.296392+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294671+0000","last_change":"2024-09-11T12:56:50.294671+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294671+0000","last_undegraded":"2024-09-11T12:56:50.294671+0000","last_fullsized":"2024-09-11T12:56:50.294671+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.298439+0000","last_change":"2024-09-11T12:56:50.298439+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.298439+0000","last_undegraded":"2024-09-11T12:56:50.298439+0000","last_fullsized":"2024-09-11T12:56:50.298439+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291492+0000","last_change":"2024-09-11T12:56:50.291492+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291492+0000","last_undegraded":"2024-09-11T12:56:50.291492+0000","last_fullsized":"2024-09-11T12:56:50.291492+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.294603+0000","last_change":"2024-09-11T12:56:50.294603+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.294603+0000","last_undegraded":"2024-09-11T12:56:50.294603+0000","last_fullsized":"2024-09-11T12:56:50.294603+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293271+0000","last_change":"2024-09-11T12:56:50.293271+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293271+0000","last_undegraded":"2024-09-11T12:56:50.293271+0000","last_fullsized":"2024-09-11T12:56:50.293271+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.289748+0000","last_change":"2024-09-11T12:56:50.289748+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.289748+0000","last_undegraded":"2024-09-11T12:56:50.289748+0000","last_fullsized":"2024-09-11T12:56:50.289748+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290465+0000","last_change":"2024-09-11T12:56:50.290465+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290465+0000","last_undegraded":"2024-09-11T12:56:50.290465+0000","last_fullsized":"2024-09-11T12:56:50.290465+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287922+0000","last_change":"2024-09-11T12:56:50.287922+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287922+0000","last_undegraded":"2024-09-11T12:56:50.287922+0000","last_fullsized":"2024-09-11T12:56:50.287922+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.301309+0000","last_change":"2024-09-11T12:56:50.301309+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.301309+0000","last_undegraded":"2024-09-11T12:56:50.301309+0000","last_fullsized":"2024-09-11T12:56:50.301309+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293978+0000","last_change":"2024-09-11T12:56:50.293978+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293978+0000","last_undegraded":"2024-09-11T12:56:50.293978+0000","last_fullsized":"2024-09-11T12:56:50.293978+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290638+0000","last_change":"2024-09-11T12:56:50.290638+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290638+0000","last_undegraded":"2024-09-11T12:56:50.290638+0000","last_fullsized":"2024-09-11T12:56:50.290638+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284031+0000","last_change":"2024-09-11T12:56:50.284031+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284031+0000","last_undegraded":"2024-09-11T12:56:50.284031+0000","last_fullsized":"2024-09-11T12:56:50.284031+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292818+0000","last_change":"2024-09-11T12:56:50.292818+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292818+0000","last_undegraded":"2024-09-11T12:56:50.292818+0000","last_fullsized":"2024-09-11T12:56:50.292818+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291033+0000","last_change":"2024-09-11T12:56:50.291033+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291033+0000","last_undegraded":"2024-09-11T12:56:50.291033+0000","last_fullsized":"2024-09-11T12:56:50.291033+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292095+0000","last_change":"2024-09-11T12:56:50.292095+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292095+0000","last_undegraded":"2024-09-11T12:56:50.292095+0000","last_fullsized":"2024-09-11T12:56:50.292095+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302831+0000","last_change":"2024-09-11T12:56:50.290099+0000","last_active":"2024-09-11T12:56:50.302831+0000","last_peered":"2024-09-11T12:56:50.302831+0000","last_clean":"2024-09-11T12:56:50.302831+0000","last_became_active":"2024-09-11T12:56:50.289571+0000","last_became_peered":"2024-09-11T12:56:50.289571+0000","last_unstale":"2024-09-11T12:56:50.302831+0000","last_undegraded":"2024-09-11T12:56:50.302831+0000","last_fullsized":"2024-09-11T12:56:50.302831+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:12:44.844033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295530+0000","last_change":"2024-09-11T12:56:50.295530+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295530+0000","last_undegraded":"2024-09-11T12:56:50.295530+0000","last_fullsized":"2024-09-11T12:56:50.295530+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.306688+0000","last_change":"2024-09-11T12:56:50.305706+0000","last_active":"2024-09-11T12:56:50.306688+0000","last_peered":"2024-09-11T12:56:50.306688+0000","last_clean":"2024-09-11T12:56:50.306688+0000","last_became_active":"2024-09-11T12:56:50.303736+0000","last_became_peered":"2024-09-11T12:56:50.303736+0000","last_unstale":"2024-09-11T12:56:50.306688+0000","last_undegraded":"2024-09-11T12:56:50.306688+0000","last_fullsized":"2024-09-11T12:56:50.306688+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:52:09.904686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287500+0000","last_change":"2024-09-11T12:56:50.287500+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287500+0000","last_undegraded":"2024-09-11T12:56:50.287500+0000","last_fullsized":"2024-09-11T12:56:50.287500+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309084+0000","last_change":"2024-09-11T12:56:50.306449+0000","last_active":"2024-09-11T12:56:50.309084+0000","last_peered":"2024-09-11T12:56:50.309084+0000","last_clean":"2024-09-11T12:56:50.309084+0000","last_became_active":"2024-09-11T12:56:50.306013+0000","last_became_peered":"2024-09-11T12:56:50.306013+0000","last_unstale":"2024-09-11T12:56:50.309084+0000","last_undegraded":"2024-09-11T12:56:50.309084+0000","last_fullsized":"2024-09-11T12:56:50.309084+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:55.224946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283841+0000","last_change":"2024-09-11T12:56:50.283841+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283841+0000","last_undegraded":"2024-09-11T12:56:50.283841+0000","last_fullsized":"2024-09-11T12:56:50.283841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305283+0000","last_change":"2024-09-11T12:56:50.302084+0000","last_active":"2024-09-11T12:56:50.305283+0000","last_peered":"2024-09-11T12:56:50.305283+0000","last_clean":"2024-09-11T12:56:50.305283+0000","last_became_active":"2024-09-11T12:56:50.301802+0000","last_became_peered":"2024-09-11T12:56:50.301802+0000","last_unstale":"2024-09-11T12:56:50.305283+0000","last_undegraded":"2024-09-11T12:56:50.305283+0000","last_fullsized":"2024-09-11T12:56:50.305283+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:19.743245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292388+0000","last_change":"2024-09-11T12:56:50.292388+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292388+0000","last_undegraded":"2024-09-11T12:56:50.292388+0000","last_fullsized":"2024-09-11T12:56:50.292388+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301964+0000","last_change":"2024-09-11T12:56:50.289506+0000","last_active":"2024-09-11T12:56:50.301964+0000","last_peered":"2024-09-11T12:56:50.301964+0000","last_clean":"2024-09-11T12:56:50.301964+0000","last_became_active":"2024-09-11T12:56:50.289233+0000","last_became_peered":"2024-09-11T12:56:50.289233+0000","last_unstale":"2024-09-11T12:56:50.301964+0000","last_undegraded":"2024-09-11T12:56:50.301964+0000","last_fullsized":"2024-09-11T12:56:50.301964+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:04:05.921628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291264+0000","last_change":"2024-09-11T12:56:50.291264+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291264+0000","last_undegraded":"2024-09-11T12:56:50.291264+0000","last_fullsized":"2024-09-11T12:56:50.291264+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310773+0000","last_change":"2024-09-11T12:56:50.305183+0000","last_active":"2024-09-11T12:56:50.310773+0000","last_peered":"2024-09-11T12:56:50.310773+0000","last_clean":"2024-09-11T12:56:50.310773+0000","last_became_active":"2024-09-11T12:56:50.304940+0000","last_became_peered":"2024-09-11T12:56:50.304940+0000","last_unstale":"2024-09-11T12:56:50.310773+0000","last_undegraded":"2024-09-11T12:56:50.310773+0000","last_fullsized":"2024-09-11T12:56:50.310773+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:34.597272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284148+0000","last_change":"2024-09-11T12:56:50.284148+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284148+0000","last_undegraded":"2024-09-11T12:56:50.284148+0000","last_fullsized":"2024-09-11T12:56:50.284148+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310117+0000","last_change":"2024-09-11T12:56:50.308680+0000","last_active":"2024-09-11T12:56:50.310117+0000","last_peered":"2024-09-11T12:56:50.310117+0000","last_clean":"2024-09-11T12:56:50.310117+0000","last_became_active":"2024-09-11T12:56:50.308390+0000","last_became_peered":"2024-09-11T12:56:50.308390+0000","last_unstale":"2024-09-11T12:56:50.310117+0000","last_undegraded":"2024-09-11T12:56:50.310117+0000","last_fullsized":"2024-09-11T12:56:50.310117+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:49.212446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.293772+0000","last_change":"2024-09-11T12:56:50.293772+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.293772+0000","last_undegraded":"2024-09-11T12:56:50.293772+0000","last_fullsized":"2024-09-11T12:56:50.293772+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.294139+0000","last_change":"2024-09-11T12:56:50.292337+0000","last_active":"2024-09-11T12:56:50.294139+0000","last_peered":"2024-09-11T12:56:50.294139+0000","last_clean":"2024-09-11T12:56:50.294139+0000","last_became_active":"2024-09-11T12:56:50.292057+0000","last_became_peered":"2024-09-11T12:56:50.292057+0000","last_unstale":"2024-09-11T12:56:50.294139+0000","last_undegraded":"2024-09-11T12:56:50.294139+0000","last_fullsized":"2024-09-11T12:56:50.294139+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:21.706481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291443+0000","last_change":"2024-09-11T12:56:50.291443+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291443+0000","last_undegraded":"2024-09-11T12:56:50.291443+0000","last_fullsized":"2024-09-11T12:56:50.291443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305437+0000","last_change":"2024-09-11T12:56:50.300136+0000","last_active":"2024-09-11T12:56:50.305437+0000","last_peered":"2024-09-11T12:56:50.305437+0000","last_clean":"2024-09-11T12:56:50.305437+0000","last_became_active":"2024-09-11T12:56:50.299878+0000","last_became_peered":"2024-09-11T12:56:50.299878+0000","last_unstale":"2024-09-11T12:56:50.305437+0000","last_undegraded":"2024-09-11T12:56:50.305437+0000","last_fullsized":"2024-09-11T12:56:50.305437+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:02:33.035922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.287074+0000","last_change":"2024-09-11T12:56:50.287074+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.287074+0000","last_undegraded":"2024-09-11T12:56:50.287074+0000","last_fullsized":"2024-09-11T12:56:50.287074+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305578+0000","last_change":"2024-09-11T12:56:50.302419+0000","last_active":"2024-09-11T12:56:50.305578+0000","last_peered":"2024-09-11T12:56:50.305578+0000","last_clean":"2024-09-11T12:56:50.305578+0000","last_became_active":"2024-09-11T12:56:50.302121+0000","last_became_peered":"2024-09-11T12:56:50.302121+0000","last_unstale":"2024-09-11T12:56:50.305578+0000","last_undegraded":"2024-09-11T12:56:50.305578+0000","last_fullsized":"2024-09-11T12:56:50.305578+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:28:28.653360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288808+0000","last_change":"2024-09-11T12:56:50.288808+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288808+0000","last_undegraded":"2024-09-11T12:56:50.288808+0000","last_fullsized":"2024-09-11T12:56:50.288808+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308813+0000","last_change":"2024-09-11T12:56:50.306460+0000","last_active":"2024-09-11T12:56:50.308813+0000","last_peered":"2024-09-11T12:56:50.308813+0000","last_clean":"2024-09-11T12:56:50.308813+0000","last_became_active":"2024-09-11T12:56:50.306176+0000","last_became_peered":"2024-09-11T12:56:50.306176+0000","last_unstale":"2024-09-11T12:56:50.308813+0000","last_undegraded":"2024-09-11T12:56:50.308813+0000","last_fullsized":"2024-09-11T12:56:50.308813+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:00:22.085546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292960+0000","last_change":"2024-09-11T12:56:50.292960+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292960+0000","last_undegraded":"2024-09-11T12:56:50.292960+0000","last_fullsized":"2024-09-11T12:56:50.292960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310618+0000","last_change":"2024-09-11T12:56:50.309887+0000","last_active":"2024-09-11T12:56:50.310618+0000","last_peered":"2024-09-11T12:56:50.310618+0000","last_clean":"2024-09-11T12:56:50.310618+0000","last_became_active":"2024-09-11T12:56:50.309619+0000","last_became_peered":"2024-09-11T12:56:50.309619+0000","last_unstale":"2024-09-11T12:56:50.310618+0000","last_undegraded":"2024-09-11T12:56:50.310618+0000","last_fullsized":"2024-09-11T12:56:50.310618+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:18.393994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.286085+0000","last_change":"2024-09-11T12:56:50.286085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.286085+0000","last_undegraded":"2024-09-11T12:56:50.286085+0000","last_fullsized":"2024-09-11T12:56:50.286085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302268+0000","last_change":"2024-09-11T12:56:50.298086+0000","last_active":"2024-09-11T12:56:50.302268+0000","last_peered":"2024-09-11T12:56:50.302268+0000","last_clean":"2024-09-11T12:56:50.302268+0000","last_became_active":"2024-09-11T12:56:50.297806+0000","last_became_peered":"2024-09-11T12:56:50.297806+0000","last_unstale":"2024-09-11T12:56:50.302268+0000","last_undegraded":"2024-09-11T12:56:50.302268+0000","last_fullsized":"2024-09-11T12:56:50.302268+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:17:04.599515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.284022+0000","last_change":"2024-09-11T12:56:50.284022+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.284022+0000","last_undegraded":"2024-09-11T12:56:50.284022+0000","last_fullsized":"2024-09-11T12:56:50.284022+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304406+0000","last_change":"2024-09-11T12:56:50.299386+0000","last_active":"2024-09-11T12:56:50.304406+0000","last_peered":"2024-09-11T12:56:50.304406+0000","last_clean":"2024-09-11T12:56:50.304406+0000","last_became_active":"2024-09-11T12:56:50.299158+0000","last_became_peered":"2024-09-11T12:56:50.299158+0000","last_unstale":"2024-09-11T12:56:50.304406+0000","last_undegraded":"2024-09-11T12:56:50.304406+0000","last_fullsized":"2024-09-11T12:56:50.304406+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:36.222739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288673+0000","last_change":"2024-09-11T12:56:50.288673+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288673+0000","last_undegraded":"2024-09-11T12:56:50.288673+0000","last_fullsized":"2024-09-11T12:56:50.288673+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310138+0000","last_change":"2024-09-11T12:56:50.308722+0000","last_active":"2024-09-11T12:56:50.310138+0000","last_peered":"2024-09-11T12:56:50.310138+0000","last_clean":"2024-09-11T12:56:50.310138+0000","last_became_active":"2024-09-11T12:56:50.308431+0000","last_became_peered":"2024-09-11T12:56:50.308431+0000","last_unstale":"2024-09-11T12:56:50.310138+0000","last_undegraded":"2024-09-11T12:56:50.310138+0000","last_fullsized":"2024-09-11T12:56:50.310138+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:17.859415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288526+0000","last_change":"2024-09-11T12:56:50.288526+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288526+0000","last_undegraded":"2024-09-11T12:56:50.288526+0000","last_fullsized":"2024-09-11T12:56:50.288526+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.302811+0000","last_change":"2024-09-11T12:56:50.289863+0000","last_active":"2024-09-11T12:56:50.302811+0000","last_peered":"2024-09-11T12:56:50.302811+0000","last_clean":"2024-09-11T12:56:50.302811+0000","last_became_active":"2024-09-11T12:56:50.289217+0000","last_became_peered":"2024-09-11T12:56:50.289217+0000","last_unstale":"2024-09-11T12:56:50.302811+0000","last_undegraded":"2024-09-11T12:56:50.302811+0000","last_fullsized":"2024-09-11T12:56:50.302811+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:06.556348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283848+0000","last_change":"2024-09-11T12:56:50.283848+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283848+0000","last_undegraded":"2024-09-11T12:56:50.283848+0000","last_fullsized":"2024-09-11T12:56:50.283848+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309890+0000","last_change":"2024-09-11T12:56:50.302013+0000","last_active":"2024-09-11T12:56:50.309890+0000","last_peered":"2024-09-11T12:56:50.309890+0000","last_clean":"2024-09-11T12:56:50.309890+0000","last_became_active":"2024-09-11T12:56:50.299221+0000","last_became_peered":"2024-09-11T12:56:50.299221+0000","last_unstale":"2024-09-11T12:56:50.309890+0000","last_undegraded":"2024-09-11T12:56:50.309890+0000","last_fullsized":"2024-09-11T12:56:50.309890+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:50.911770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.283957+0000","last_change":"2024-09-11T12:56:50.283957+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.283957+0000","last_undegraded":"2024-09-11T12:56:50.283957+0000","last_fullsized":"2024-09-11T12:56:50.283957+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.312133+0000","last_change":"2024-09-11T12:56:50.311439+0000","last_active":"2024-09-11T12:56:50.312133+0000","last_peered":"2024-09-11T12:56:50.312133+0000","last_clean":"2024-09-11T12:56:50.312133+0000","last_became_active":"2024-09-11T12:56:50.311186+0000","last_became_peered":"2024-09-11T12:56:50.311186+0000","last_unstale":"2024-09-11T12:56:50.312133+0000","last_undegraded":"2024-09-11T12:56:50.312133+0000","last_fullsized":"2024-09-11T12:56:50.312133+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:28.445966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290959+0000","last_change":"2024-09-11T12:56:50.290959+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290959+0000","last_undegraded":"2024-09-11T12:56:50.290959+0000","last_fullsized":"2024-09-11T12:56:50.290959+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301466+0000","last_change":"2024-09-11T12:56:50.297243+0000","last_active":"2024-09-11T12:56:50.301466+0000","last_peered":"2024-09-11T12:56:50.301466+0000","last_clean":"2024-09-11T12:56:50.301466+0000","last_became_active":"2024-09-11T12:56:50.296921+0000","last_became_peered":"2024-09-11T12:56:50.296921+0000","last_unstale":"2024-09-11T12:56:50.301466+0000","last_undegraded":"2024-09-11T12:56:50.301466+0000","last_fullsized":"2024-09-11T12:56:50.301466+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288448+0000","last_change":"2024-09-11T12:56:50.288448+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288448+0000","last_undegraded":"2024-09-11T12:56:50.288448+0000","last_fullsized":"2024-09-11T12:56:50.288448+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311866+0000","last_change":"2024-09-11T12:56:50.309614+0000","last_active":"2024-09-11T12:56:50.311866+0000","last_peered":"2024-09-11T12:56:50.311866+0000","last_clean":"2024-09-11T12:56:50.311866+0000","last_became_active":"2024-09-11T12:56:50.309309+0000","last_became_peered":"2024-09-11T12:56:50.309309+0000","last_unstale":"2024-09-11T12:56:50.311866+0000","last_undegraded":"2024-09-11T12:56:50.311866+0000","last_fullsized":"2024-09-11T12:56:50.311866+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:12.510525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.285640+0000","last_change":"2024-09-11T12:56:50.285640+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.285640+0000","last_undegraded":"2024-09-11T12:56:50.285640+0000","last_fullsized":"2024-09-11T12:56:50.285640+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304458+0000","last_change":"2024-09-11T12:56:50.297571+0000","last_active":"2024-09-11T12:56:50.304458+0000","last_peered":"2024-09-11T12:56:50.304458+0000","last_clean":"2024-09-11T12:56:50.304458+0000","last_became_active":"2024-09-11T12:56:50.297307+0000","last_became_peered":"2024-09-11T12:56:50.297307+0000","last_unstale":"2024-09-11T12:56:50.304458+0000","last_undegraded":"2024-09-11T12:56:50.304458+0000","last_fullsized":"2024-09-11T12:56:50.304458+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:12:37.967545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.292344+0000","last_change":"2024-09-11T12:56:50.292344+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.292344+0000","last_undegraded":"2024-09-11T12:56:50.292344+0000","last_fullsized":"2024-09-11T12:56:50.292344+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.310522+0000","last_change":"2024-09-11T12:56:50.309534+0000","last_active":"2024-09-11T12:56:50.310522+0000","last_peered":"2024-09-11T12:56:50.310522+0000","last_clean":"2024-09-11T12:56:50.310522+0000","last_became_active":"2024-09-11T12:56:50.309270+0000","last_became_peered":"2024-09-11T12:56:50.309270+0000","last_unstale":"2024-09-11T12:56:50.310522+0000","last_undegraded":"2024-09-11T12:56:50.310522+0000","last_fullsized":"2024-09-11T12:56:50.310522+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:23.661282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291864+0000","last_change":"2024-09-11T12:56:50.291864+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291864+0000","last_undegraded":"2024-09-11T12:56:50.291864+0000","last_fullsized":"2024-09-11T12:56:50.291864+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301757+0000","last_change":"2024-09-11T12:56:50.295252+0000","last_active":"2024-09-11T12:56:50.301757+0000","last_peered":"2024-09-11T12:56:50.301757+0000","last_clean":"2024-09-11T12:56:50.301757+0000","last_became_active":"2024-09-11T12:56:50.294990+0000","last_became_peered":"2024-09-11T12:56:50.294990+0000","last_unstale":"2024-09-11T12:56:50.301757+0000","last_undegraded":"2024-09-11T12:56:50.301757+0000","last_fullsized":"2024-09-11T12:56:50.301757+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:28.110557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291247+0000","last_change":"2024-09-11T12:56:50.291247+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291247+0000","last_undegraded":"2024-09-11T12:56:50.291247+0000","last_fullsized":"2024-09-11T12:56:50.291247+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.308452+0000","last_change":"2024-09-11T12:56:50.307695+0000","last_active":"2024-09-11T12:56:50.308452+0000","last_peered":"2024-09-11T12:56:50.308452+0000","last_clean":"2024-09-11T12:56:50.308452+0000","last_became_active":"2024-09-11T12:56:50.307465+0000","last_became_peered":"2024-09-11T12:56:50.307465+0000","last_unstale":"2024-09-11T12:56:50.308452+0000","last_undegraded":"2024-09-11T12:56:50.308452+0000","last_fullsized":"2024-09-11T12:56:50.308452+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:08.572106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.290085+0000","last_change":"2024-09-11T12:56:50.290085+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.290085+0000","last_undegraded":"2024-09-11T12:56:50.290085+0000","last_fullsized":"2024-09-11T12:56:50.290085+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309115+0000","last_change":"2024-09-11T12:56:50.303866+0000","last_active":"2024-09-11T12:56:50.309115+0000","last_peered":"2024-09-11T12:56:50.309115+0000","last_clean":"2024-09-11T12:56:50.309115+0000","last_became_active":"2024-09-11T12:56:50.303577+0000","last_became_peered":"2024-09-11T12:56:50.303577+0000","last_unstale":"2024-09-11T12:56:50.309115+0000","last_undegraded":"2024-09-11T12:56:50.309115+0000","last_fullsized":"2024-09-11T12:56:50.309115+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:22:49.212192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291910+0000","last_change":"2024-09-11T12:56:50.291910+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291910+0000","last_undegraded":"2024-09-11T12:56:50.291910+0000","last_fullsized":"2024-09-11T12:56:50.291910+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309981+0000","last_change":"2024-09-11T12:56:50.306743+0000","last_active":"2024-09-11T12:56:50.309981+0000","last_peered":"2024-09-11T12:56:50.309981+0000","last_clean":"2024-09-11T12:56:50.309981+0000","last_became_active":"2024-09-11T12:56:50.306472+0000","last_became_peered":"2024-09-11T12:56:50.306472+0000","last_unstale":"2024-09-11T12:56:50.309981+0000","last_undegraded":"2024-09-11T12:56:50.309981+0000","last_fullsized":"2024-09-11T12:56:50.309981+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:26:13.220188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291840+0000","last_change":"2024-09-11T12:56:50.291840+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291840+0000","last_undegraded":"2024-09-11T12:56:50.291840+0000","last_fullsized":"2024-09-11T12:56:50.291840+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.309616+0000","last_change":"2024-09-11T12:56:50.297814+0000","last_active":"2024-09-11T12:56:50.309616+0000","last_peered":"2024-09-11T12:56:50.309616+0000","last_clean":"2024-09-11T12:56:50.309616+0000","last_became_active":"2024-09-11T12:56:50.296767+0000","last_became_peered":"2024-09-11T12:56:50.296767+0000","last_unstale":"2024-09-11T12:56:50.309616+0000","last_undegraded":"2024-09-11T12:56:50.309616+0000","last_fullsized":"2024-09-11T12:56:50.309616+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:51.352295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295313+0000","last_change":"2024-09-11T12:56:50.295313+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295313+0000","last_undegraded":"2024-09-11T12:56:50.295313+0000","last_fullsized":"2024-09-11T12:56:50.295313+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.301834+0000","last_change":"2024-09-11T12:56:50.296255+0000","last_active":"2024-09-11T12:56:50.301834+0000","last_peered":"2024-09-11T12:56:50.301834+0000","last_clean":"2024-09-11T12:56:50.301834+0000","last_became_active":"2024-09-11T12:56:50.296043+0000","last_became_peered":"2024-09-11T12:56:50.296043+0000","last_unstale":"2024-09-11T12:56:50.301834+0000","last_undegraded":"2024-09-11T12:56:50.301834+0000","last_fullsized":"2024-09-11T12:56:50.301834+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:32:03.915319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.295714+0000","last_change":"2024-09-11T12:56:50.295714+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.295714+0000","last_undegraded":"2024-09-11T12:56:50.295714+0000","last_fullsized":"2024-09-11T12:56:50.295714+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.291746+0000","last_change":"2024-09-11T12:56:50.289677+0000","last_active":"2024-09-11T12:56:50.291746+0000","last_peered":"2024-09-11T12:56:50.291746+0000","last_clean":"2024-09-11T12:56:50.291746+0000","last_became_active":"2024-09-11T12:56:50.289134+0000","last_became_peered":"2024-09-11T12:56:50.289134+0000","last_unstale":"2024-09-11T12:56:50.291746+0000","last_undegraded":"2024-09-11T12:56:50.291746+0000","last_fullsized":"2024-09-11T12:56:50.291746+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:26:55.883458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291338+0000","last_change":"2024-09-11T12:56:50.291338+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291338+0000","last_undegraded":"2024-09-11T12:56:50.291338+0000","last_fullsized":"2024-09-11T12:56:50.291338+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.304907+0000","last_change":"2024-09-11T12:56:50.301293+0000","last_active":"2024-09-11T12:56:50.304907+0000","last_peered":"2024-09-11T12:56:50.304907+0000","last_clean":"2024-09-11T12:56:50.304907+0000","last_became_active":"2024-09-11T12:56:50.300991+0000","last_became_peered":"2024-09-11T12:56:50.300991+0000","last_unstale":"2024-09-11T12:56:50.304907+0000","last_undegraded":"2024-09-11T12:56:50.304907+0000","last_fullsized":"2024-09-11T12:56:50.304907+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:42:01.979395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.291474+0000","last_change":"2024-09-11T12:56:50.291474+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.291474+0000","last_undegraded":"2024-09-11T12:56:50.291474+0000","last_fullsized":"2024-09-11T12:56:50.291474+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.311941+0000","last_change":"2024-09-11T12:56:50.311418+0000","last_active":"2024-09-11T12:56:50.311941+0000","last_peered":"2024-09-11T12:56:50.311941+0000","last_clean":"2024-09-11T12:56:50.311941+0000","last_became_active":"2024-09-11T12:56:50.311129+0000","last_became_peered":"2024-09-11T12:56:50.311129+0000","last_unstale":"2024-09-11T12:56:50.311941+0000","last_undegraded":"2024-09-11T12:56:50.311941+0000","last_fullsized":"2024-09-11T12:56:50.311941+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:50:21.576248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":1,"reported_epoch":149,"state":"creating+peering","last_fresh":"2024-09-11T12:56:50.288558+0000","last_change":"2024-09-11T12:56:50.288558+0000","last_active":"2024-09-11T12:56:50.275883+0000","last_peered":"2024-09-11T12:56:50.275883+0000","last_clean":"2024-09-11T12:56:50.275883+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T12:56:50.288558+0000","last_undegraded":"2024-09-11T12:56:50.288558+0000","last_fullsized":"2024-09-11T12:56:50.288558+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.305320+0000","last_change":"2024-09-11T12:56:50.303975+0000","last_active":"2024-09-11T12:56:50.305320+0000","last_peered":"2024-09-11T12:56:50.305320+0000","last_clean":"2024-09-11T12:56:50.305320+0000","last_became_active":"2024-09-11T12:56:50.303570+0000","last_became_peered":"2024-09-11T12:56:50.303570+0000","last_unstale":"2024-09-11T12:56:50.305320+0000","last_undegraded":"2024-09-11T12:56:50.305320+0000","last_fullsized":"2024-09-11T12:56:50.305320+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:11.499583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":368,"reported_epoch":149,"state":"active+clean","last_fresh":"2024-09-11T12:56:50.297184+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:50.297184+0000","last_peered":"2024-09-11T12:56:50.297184+0000","last_clean":"2024-09-11T12:56:50.297184+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:50.297184+0000","last_undegraded":"2024-09-11T12:56:50.297184+0000","last_fullsized":"2024-09-11T12:56:50.297184+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37744,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334096,"statfs":{"total":96636764160,"available":96598114304,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37172,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334668,"statfs":{"total":96636764160,"available":96598700032,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37176,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334664,"statfs":{"total":96636764160,"available":96598695936,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1871872,"data_stored":820494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509531,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33008,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338832,"statfs":{"total":96636764160,"available":96602963968,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509531,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33016,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338824,"statfs":{"total":96636764160,"available":96602955776,"internally_reserved":0,"allocated":1859584,"data_stored":814219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542235,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2453504,"data_stored":1404587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:56.191 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3f is not active+clean 2024-09-11T12:56:56.191 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.296293+0000', 'last_change': '2024-09-11T12:56:50.296293+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.296293+0000', 'last_undegraded': '2024-09-11T12:56:50.296293+0000', 'last_fullsized': '2024-09-11T12:56:50.296293+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3e is not active+clean 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292499+0000', 'last_change': '2024-09-11T12:56:50.292499+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292499+0000', 'last_undegraded': '2024-09-11T12:56:50.292499+0000', 'last_fullsized': '2024-09-11T12:56:50.292499+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3d is not active+clean 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.303507+0000', 'last_change': '2024-09-11T12:56:50.303507+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.303507+0000', 'last_undegraded': '2024-09-11T12:56:50.303507+0000', 'last_fullsized': '2024-09-11T12:56:50.303507+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3c is not active+clean 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292365+0000', 'last_change': '2024-09-11T12:56:50.292365+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292365+0000', 'last_undegraded': '2024-09-11T12:56:50.292365+0000', 'last_fullsized': '2024-09-11T12:56:50.292365+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3b is not active+clean 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292501+0000', 'last_change': '2024-09-11T12:56:50.292501+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292501+0000', 'last_undegraded': '2024-09-11T12:56:50.292501+0000', 'last_fullsized': '2024-09-11T12:56:50.292501+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3a is not active+clean 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293414+0000', 'last_change': '2024-09-11T12:56:50.293414+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293414+0000', 'last_undegraded': '2024-09-11T12:56:50.293414+0000', 'last_fullsized': '2024-09-11T12:56:50.293414+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.192 INFO:tasks.ceph.ceph_manager.ceph:PG 25.39 is not active+clean 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.284696+0000', 'last_change': '2024-09-11T12:56:50.284696+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.284696+0000', 'last_undegraded': '2024-09-11T12:56:50.284696+0000', 'last_fullsized': '2024-09-11T12:56:50.284696+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:PG 25.38 is not active+clean 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290692+0000', 'last_change': '2024-09-11T12:56:50.290692+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290692+0000', 'last_undegraded': '2024-09-11T12:56:50.290692+0000', 'last_fullsized': '2024-09-11T12:56:50.290692+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:PG 25.37 is not active+clean 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.284457+0000', 'last_change': '2024-09-11T12:56:50.284457+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.284457+0000', 'last_undegraded': '2024-09-11T12:56:50.284457+0000', 'last_fullsized': '2024-09-11T12:56:50.284457+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:PG 25.36 is not active+clean 2024-09-11T12:56:56.193 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.294783+0000', 'last_change': '2024-09-11T12:56:50.294783+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.294783+0000', 'last_undegraded': '2024-09-11T12:56:50.294783+0000', 'last_fullsized': '2024-09-11T12:56:50.294783+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:PG 25.35 is not active+clean 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.285384+0000', 'last_change': '2024-09-11T12:56:50.285384+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.285384+0000', 'last_undegraded': '2024-09-11T12:56:50.285384+0000', 'last_fullsized': '2024-09-11T12:56:50.285384+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:PG 25.34 is not active+clean 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291364+0000', 'last_change': '2024-09-11T12:56:50.291364+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291364+0000', 'last_undegraded': '2024-09-11T12:56:50.291364+0000', 'last_fullsized': '2024-09-11T12:56:50.291364+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:PG 25.33 is not active+clean 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291751+0000', 'last_change': '2024-09-11T12:56:50.291751+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291751+0000', 'last_undegraded': '2024-09-11T12:56:50.291751+0000', 'last_fullsized': '2024-09-11T12:56:50.291751+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:PG 25.32 is not active+clean 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.285450+0000', 'last_change': '2024-09-11T12:56:50.285450+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.285450+0000', 'last_undegraded': '2024-09-11T12:56:50.285450+0000', 'last_fullsized': '2024-09-11T12:56:50.285450+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:PG 25.31 is not active+clean 2024-09-11T12:56:56.194 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293196+0000', 'last_change': '2024-09-11T12:56:50.293196+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293196+0000', 'last_undegraded': '2024-09-11T12:56:50.293196+0000', 'last_fullsized': '2024-09-11T12:56:50.293196+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:PG 25.30 is not active+clean 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290239+0000', 'last_change': '2024-09-11T12:56:50.290239+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290239+0000', 'last_undegraded': '2024-09-11T12:56:50.290239+0000', 'last_fullsized': '2024-09-11T12:56:50.290239+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2f is not active+clean 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293571+0000', 'last_change': '2024-09-11T12:56:50.293571+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293571+0000', 'last_undegraded': '2024-09-11T12:56:50.293571+0000', 'last_fullsized': '2024-09-11T12:56:50.293571+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2e is not active+clean 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.296392+0000', 'last_change': '2024-09-11T12:56:50.296392+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.296392+0000', 'last_undegraded': '2024-09-11T12:56:50.296392+0000', 'last_fullsized': '2024-09-11T12:56:50.296392+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2d is not active+clean 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.294671+0000', 'last_change': '2024-09-11T12:56:50.294671+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.294671+0000', 'last_undegraded': '2024-09-11T12:56:50.294671+0000', 'last_fullsized': '2024-09-11T12:56:50.294671+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2c is not active+clean 2024-09-11T12:56:56.195 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.298439+0000', 'last_change': '2024-09-11T12:56:50.298439+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.298439+0000', 'last_undegraded': '2024-09-11T12:56:50.298439+0000', 'last_fullsized': '2024-09-11T12:56:50.298439+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2b is not active+clean 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291492+0000', 'last_change': '2024-09-11T12:56:50.291492+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291492+0000', 'last_undegraded': '2024-09-11T12:56:50.291492+0000', 'last_fullsized': '2024-09-11T12:56:50.291492+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2a is not active+clean 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.294603+0000', 'last_change': '2024-09-11T12:56:50.294603+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.294603+0000', 'last_undegraded': '2024-09-11T12:56:50.294603+0000', 'last_fullsized': '2024-09-11T12:56:50.294603+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.29 is not active+clean 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293271+0000', 'last_change': '2024-09-11T12:56:50.293271+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293271+0000', 'last_undegraded': '2024-09-11T12:56:50.293271+0000', 'last_fullsized': '2024-09-11T12:56:50.293271+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.28 is not active+clean 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.289748+0000', 'last_change': '2024-09-11T12:56:50.289748+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.289748+0000', 'last_undegraded': '2024-09-11T12:56:50.289748+0000', 'last_fullsized': '2024-09-11T12:56:50.289748+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.27 is not active+clean 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290465+0000', 'last_change': '2024-09-11T12:56:50.290465+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290465+0000', 'last_undegraded': '2024-09-11T12:56:50.290465+0000', 'last_fullsized': '2024-09-11T12:56:50.290465+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.196 INFO:tasks.ceph.ceph_manager.ceph:PG 25.26 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.287922+0000', 'last_change': '2024-09-11T12:56:50.287922+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.287922+0000', 'last_undegraded': '2024-09-11T12:56:50.287922+0000', 'last_fullsized': '2024-09-11T12:56:50.287922+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:PG 25.25 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.301309+0000', 'last_change': '2024-09-11T12:56:50.301309+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.301309+0000', 'last_undegraded': '2024-09-11T12:56:50.301309+0000', 'last_fullsized': '2024-09-11T12:56:50.301309+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:PG 25.24 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293978+0000', 'last_change': '2024-09-11T12:56:50.293978+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293978+0000', 'last_undegraded': '2024-09-11T12:56:50.293978+0000', 'last_fullsized': '2024-09-11T12:56:50.293978+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:PG 25.23 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290638+0000', 'last_change': '2024-09-11T12:56:50.290638+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290638+0000', 'last_undegraded': '2024-09-11T12:56:50.290638+0000', 'last_fullsized': '2024-09-11T12:56:50.290638+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:PG 25.22 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.284031+0000', 'last_change': '2024-09-11T12:56:50.284031+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.284031+0000', 'last_undegraded': '2024-09-11T12:56:50.284031+0000', 'last_fullsized': '2024-09-11T12:56:50.284031+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:PG 25.21 is not active+clean 2024-09-11T12:56:56.197 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292818+0000', 'last_change': '2024-09-11T12:56:50.292818+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292818+0000', 'last_undegraded': '2024-09-11T12:56:50.292818+0000', 'last_fullsized': '2024-09-11T12:56:50.292818+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:PG 25.20 is not active+clean 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291033+0000', 'last_change': '2024-09-11T12:56:50.291033+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291033+0000', 'last_undegraded': '2024-09-11T12:56:50.291033+0000', 'last_fullsized': '2024-09-11T12:56:50.291033+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1e is not active+clean 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292095+0000', 'last_change': '2024-09-11T12:56:50.292095+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292095+0000', 'last_undegraded': '2024-09-11T12:56:50.292095+0000', 'last_fullsized': '2024-09-11T12:56:50.292095+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1f is not active+clean 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.295530+0000', 'last_change': '2024-09-11T12:56:50.295530+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.295530+0000', 'last_undegraded': '2024-09-11T12:56:50.295530+0000', 'last_fullsized': '2024-09-11T12:56:50.295530+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1c is not active+clean 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.287500+0000', 'last_change': '2024-09-11T12:56:50.287500+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.287500+0000', 'last_undegraded': '2024-09-11T12:56:50.287500+0000', 'last_fullsized': '2024-09-11T12:56:50.287500+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1d is not active+clean 2024-09-11T12:56:56.198 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.283841+0000', 'last_change': '2024-09-11T12:56:50.283841+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.283841+0000', 'last_undegraded': '2024-09-11T12:56:50.283841+0000', 'last_fullsized': '2024-09-11T12:56:50.283841+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1a is not active+clean 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292388+0000', 'last_change': '2024-09-11T12:56:50.292388+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292388+0000', 'last_undegraded': '2024-09-11T12:56:50.292388+0000', 'last_fullsized': '2024-09-11T12:56:50.292388+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1b is not active+clean 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291264+0000', 'last_change': '2024-09-11T12:56:50.291264+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291264+0000', 'last_undegraded': '2024-09-11T12:56:50.291264+0000', 'last_fullsized': '2024-09-11T12:56:50.291264+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.19 is not active+clean 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.284148+0000', 'last_change': '2024-09-11T12:56:50.284148+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.284148+0000', 'last_undegraded': '2024-09-11T12:56:50.284148+0000', 'last_fullsized': '2024-09-11T12:56:50.284148+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.16 is not active+clean 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.293772+0000', 'last_change': '2024-09-11T12:56:50.293772+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.293772+0000', 'last_undegraded': '2024-09-11T12:56:50.293772+0000', 'last_fullsized': '2024-09-11T12:56:50.293772+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.17 is not active+clean 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291443+0000', 'last_change': '2024-09-11T12:56:50.291443+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291443+0000', 'last_undegraded': '2024-09-11T12:56:50.291443+0000', 'last_fullsized': '2024-09-11T12:56:50.291443+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.199 INFO:tasks.ceph.ceph_manager.ceph:PG 25.14 is not active+clean 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.287074+0000', 'last_change': '2024-09-11T12:56:50.287074+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.287074+0000', 'last_undegraded': '2024-09-11T12:56:50.287074+0000', 'last_fullsized': '2024-09-11T12:56:50.287074+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:PG 25.15 is not active+clean 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.288808+0000', 'last_change': '2024-09-11T12:56:50.288808+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.288808+0000', 'last_undegraded': '2024-09-11T12:56:50.288808+0000', 'last_fullsized': '2024-09-11T12:56:50.288808+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:PG 25.12 is not active+clean 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292960+0000', 'last_change': '2024-09-11T12:56:50.292960+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292960+0000', 'last_undegraded': '2024-09-11T12:56:50.292960+0000', 'last_fullsized': '2024-09-11T12:56:50.292960+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:PG 25.13 is not active+clean 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.286085+0000', 'last_change': '2024-09-11T12:56:50.286085+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.286085+0000', 'last_undegraded': '2024-09-11T12:56:50.286085+0000', 'last_fullsized': '2024-09-11T12:56:50.286085+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:PG 25.10 is not active+clean 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.284022+0000', 'last_change': '2024-09-11T12:56:50.284022+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.284022+0000', 'last_undegraded': '2024-09-11T12:56:50.284022+0000', 'last_fullsized': '2024-09-11T12:56:50.284022+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.200 INFO:tasks.ceph.ceph_manager.ceph:PG 25.11 is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.288673+0000', 'last_change': '2024-09-11T12:56:50.288673+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.288673+0000', 'last_undegraded': '2024-09-11T12:56:50.288673+0000', 'last_fullsized': '2024-09-11T12:56:50.288673+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:PG 25.e is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.288526+0000', 'last_change': '2024-09-11T12:56:50.288526+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.288526+0000', 'last_undegraded': '2024-09-11T12:56:50.288526+0000', 'last_fullsized': '2024-09-11T12:56:50.288526+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:PG 25.f is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.283848+0000', 'last_change': '2024-09-11T12:56:50.283848+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.283848+0000', 'last_undegraded': '2024-09-11T12:56:50.283848+0000', 'last_fullsized': '2024-09-11T12:56:50.283848+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:PG 25.c is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.283957+0000', 'last_change': '2024-09-11T12:56:50.283957+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.283957+0000', 'last_undegraded': '2024-09-11T12:56:50.283957+0000', 'last_fullsized': '2024-09-11T12:56:50.283957+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:PG 25.d is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290959+0000', 'last_change': '2024-09-11T12:56:50.290959+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290959+0000', 'last_undegraded': '2024-09-11T12:56:50.290959+0000', 'last_fullsized': '2024-09-11T12:56:50.290959+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:PG 25.a is not active+clean 2024-09-11T12:56:56.201 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.288448+0000', 'last_change': '2024-09-11T12:56:50.288448+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.288448+0000', 'last_undegraded': '2024-09-11T12:56:50.288448+0000', 'last_fullsized': '2024-09-11T12:56:50.288448+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.b is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.285640+0000', 'last_change': '2024-09-11T12:56:50.285640+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.285640+0000', 'last_undegraded': '2024-09-11T12:56:50.285640+0000', 'last_fullsized': '2024-09-11T12:56:50.285640+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.8 is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.292344+0000', 'last_change': '2024-09-11T12:56:50.292344+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.292344+0000', 'last_undegraded': '2024-09-11T12:56:50.292344+0000', 'last_fullsized': '2024-09-11T12:56:50.292344+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.9 is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291864+0000', 'last_change': '2024-09-11T12:56:50.291864+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291864+0000', 'last_undegraded': '2024-09-11T12:56:50.291864+0000', 'last_fullsized': '2024-09-11T12:56:50.291864+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.6 is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291247+0000', 'last_change': '2024-09-11T12:56:50.291247+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291247+0000', 'last_undegraded': '2024-09-11T12:56:50.291247+0000', 'last_fullsized': '2024-09-11T12:56:50.291247+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.7 is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.290085+0000', 'last_change': '2024-09-11T12:56:50.290085+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.290085+0000', 'last_undegraded': '2024-09-11T12:56:50.290085+0000', 'last_fullsized': '2024-09-11T12:56:50.290085+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:PG 25.4 is not active+clean 2024-09-11T12:56:56.202 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291910+0000', 'last_change': '2024-09-11T12:56:50.291910+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291910+0000', 'last_undegraded': '2024-09-11T12:56:50.291910+0000', 'last_fullsized': '2024-09-11T12:56:50.291910+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.5 is not active+clean 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291840+0000', 'last_change': '2024-09-11T12:56:50.291840+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291840+0000', 'last_undegraded': '2024-09-11T12:56:50.291840+0000', 'last_fullsized': '2024-09-11T12:56:50.291840+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.2 is not active+clean 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.295313+0000', 'last_change': '2024-09-11T12:56:50.295313+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.295313+0000', 'last_undegraded': '2024-09-11T12:56:50.295313+0000', 'last_fullsized': '2024-09-11T12:56:50.295313+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.3 is not active+clean 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.295714+0000', 'last_change': '2024-09-11T12:56:50.295714+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.295714+0000', 'last_undegraded': '2024-09-11T12:56:50.295714+0000', 'last_fullsized': '2024-09-11T12:56:50.295714+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.0 is not active+clean 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291338+0000', 'last_change': '2024-09-11T12:56:50.291338+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291338+0000', 'last_undegraded': '2024-09-11T12:56:50.291338+0000', 'last_fullsized': '2024-09-11T12:56:50.291338+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.1 is not active+clean 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.291474+0000', 'last_change': '2024-09-11T12:56:50.291474+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.291474+0000', 'last_undegraded': '2024-09-11T12:56:50.291474+0000', 'last_fullsized': '2024-09-11T12:56:50.291474+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T12:56:56.203 INFO:tasks.ceph.ceph_manager.ceph:PG 25.18 is not active+clean 2024-09-11T12:56:56.204 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '25.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 149, 'state': 'creating+peering', 'last_fresh': '2024-09-11T12:56:50.288558+0000', 'last_change': '2024-09-11T12:56:50.288558+0000', 'last_active': '2024-09-11T12:56:50.275883+0000', 'last_peered': '2024-09-11T12:56:50.275883+0000', 'last_clean': '2024-09-11T12:56:50.275883+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-11T12:56:50.288558+0000', 'last_undegraded': '2024-09-11T12:56:50.288558+0000', 'last_fullsized': '2024-09-11T12:56:50.288558+0000', 'mapping_epoch': 149, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 149, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'last_clean_scrub_stamp': '2024-09-11T12:56:50.275883+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T12:56:56.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- 172.21.15.78:0/3712413341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc15c1d0 msgr2=0x7f06dc06e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 --2- 172.21.15.78:0/3712413341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc15c1d0 0x7f06dc06e860 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009900 tx=0x7f06d001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- 172.21.15.78:0/3712413341 shutdown_connections 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 --2- 172.21.15.78:0/3712413341 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc0d15d0 0x7f06dc15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 --2- 172.21.15.78:0/3712413341 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc15c1d0 0x7f06dc06e860 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- 172.21.15.78:0/3712413341 >> 172.21.15.78:0/3712413341 conn(0x7f06dc0ceab0 msgr2=0x7f06dc0ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- 172.21.15.78:0/3712413341 shutdown_connections 2024-09-11T12:56:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- 172.21.15.78:0/3712413341 wait complete. 2024-09-11T12:56:56.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 Processor -- start 2024-09-11T12:56:56.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.373+0000 7f06e3800640 1 -- start start 2024-09-11T12:56:56.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc15c1d0 0x7f06dc13a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc15c5b0 0x7f06dc13c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc0a9d10 con 0x7f06dc0d15d0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f06dc0a9b90 con 0x7f06dc15c5b0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f06dc0a9e90 con 0x7f06dc15c1d0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46340/0 (socket says 172.21.15.78:46340) 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 -- 172.21.15.78:0/2964877759 learned_addr learned my addr 172.21.15.78:0/2964877759 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06dbe00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc15c1d0 0x7f06dc13a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e1800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc15c5b0 0x7f06dc13c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 -- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc15c1d0 msgr2=0x7f06dc13a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc15c1d0 0x7f06dc13a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 -- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc15c5b0 msgr2=0x7f06dc13c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc15c5b0 0x7f06dc13c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 -- 172.21.15.78:0/2964877759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06dc13c2d0 con 0x7f06dc0d15d0 2024-09-11T12:56:56.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e0e00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f06d00b0040 tx=0x7f06d0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:56.382 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00b2040 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06d00036f0 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06dc13ad50 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003b20 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06dc13b590 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.377+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06dc06e860 con 0x7f06dc0d15d0 2024-09-11T12:56:56.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.381+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f06d00bf020 con 0x7f06dc0d15d0 2024-09-11T12:56:56.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.381+0000 7f06d8c00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 0x7f06bc074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.381+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f06d012d340 con 0x7f06dc0d15d0 2024-09-11T12:56:56.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.381+0000 7f06dbe00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 0x7f06bc074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.381+0000 7f06dbe00640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 0x7f06bc074320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f06dc140bb0 tx=0x7f06c8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:56.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.385+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f06dc13ad50 con 0x7f06dc0d15d0 2024-09-11T12:56:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.609+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f06dc070480 con 0x7f06dc0d15d0 2024-09-11T12:56:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.609+0000 7f06d8c00640 1 -- 172.21.15.78:0/2964877759 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1500 (secure 0 0 0) 0x7f06d00164d0 con 0x7f06dc0d15d0 2024-09-11T12:56:56.616 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:56.617 INFO:teuthology.orchestra.run.smithi078.stdout:{"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":458,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":151,"num_osds":8,"num_up_osds":8,"osd_up_since":1726058972,"num_in_osds":8,"osd_in_since":1726058961,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":288661504,"bytes_avail":772805451776,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":111,"btime":"2024-09-11T12:56:54:322400+0000","id":12,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":12,"rank":0,"name":"c","status":"up:active","gid":8631}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-11T12:56:06.500082+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T12:56:56.617 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 msgr2=0x7f06bc074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 0x7f06bc074320 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f06dc140bb0 tx=0x7f06c8009040 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 msgr2=0x7f06dc13bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f06d00b0040 tx=0x7f06d0005780 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 shutdown_connections 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f06bc071e60 0x7f06bc074320 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f06dc15c5b0 0x7f06dc13c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f06dc15c1d0 0x7f06dc13a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 --2- 172.21.15.78:0/2964877759 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f06dc0d15d0 0x7f06dc13bd90 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 >> 172.21.15.78:0/2964877759 conn(0x7f06dc0ceab0 msgr2=0x7f06dc166de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 shutdown_connections 2024-09-11T12:56:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.613+0000 7f06e3800640 1 -- 172.21.15.78:0/2964877759 wait complete. 2024-09-11T12:56:56.636 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-11T12:56:56.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:56:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 -- 172.21.15.78:0/3090264512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c0159fc0 msgr2=0x7fa2c00706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 --2- 172.21.15.78:0/3090264512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c0159fc0 0x7fa2c00706f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa2b0009870 tx=0x7fa2b001c860 comp rx=0 tx=0).stop 2024-09-11T12:56:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 -- 172.21.15.78:0/3090264512 shutdown_connections 2024-09-11T12:56:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 --2- 172.21.15.78:0/3090264512 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c0159fc0 0x7fa2c00706f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 -- 172.21.15.78:0/3090264512 >> 172.21.15.78:0/3090264512 conn(0x7fa2c006ca80 msgr2=0x7fa2c00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 -- 172.21.15.78:0/3090264512 shutdown_connections 2024-09-11T12:56:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.773+0000 7fa2c6000640 1 -- 172.21.15.78:0/3090264512 wait complete. 2024-09-11T12:56:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 Processor -- start 2024-09-11T12:56:56.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- start start 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c006f200 0x7fa2c01f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0159fc0 0x7fa2c014f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c0168a20 con 0x7fa2c01f9680 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c01688a0 con 0x7fa2c0159fc0 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa2c0168ba0 con 0x7fa2c006f200 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bf400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c006f200 0x7fa2c01f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0159fc0 0x7fa2c014f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46356/0 (socket says 172.21.15.78:46356) 2024-09-11T12:56:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 -- 172.21.15.78:0/2711449388 learned_addr learned my addr 172.21.15.78:0/2711449388 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:56:56.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 -- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c006f200 msgr2=0x7fa2c01f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c006f200 0x7fa2c01f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 -- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0159fc0 msgr2=0x7fa2c014f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0159fc0 0x7fa2c014f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 -- 172.21.15.78:0/2711449388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c0152b00 con 0x7fa2c01f9680 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2bfe00640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c790 tx=0x7fa2b800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b8003b80 con 0x7fa2c01f9680 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c01f55c0 con 0x7fa2c01f9680 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2b8003d20 con 0x7fa2c01f9680 2024-09-11T12:56:56.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b800aab0 con 0x7fa2c01f9680 2024-09-11T12:56:56.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.777+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c01f5b80 con 0x7fa2c01f9680 2024-09-11T12:56:56.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c00706f0 con 0x7fa2c01f9680 2024-09-11T12:56:56.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fa2b800ac50 con 0x7fa2c01f9680 2024-09-11T12:56:56.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2b7400640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 0x7fa2a0074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:56:56.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fa2b8092f00 con 0x7fa2c01f9680 2024-09-11T12:56:56.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2bf400640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 0x7fa2a0074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:56:56.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.781+0000 7fa2bf400640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 0x7fa2a0074320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00af000 tx=0x7fa2b001b0a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:56:56.789 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.785+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa2b805d680 con 0x7fa2c01f9680 2024-09-11T12:56:56.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.949+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa2c0146d80 con 0x7fa2a0071e60 2024-09-11T12:56:56.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.953+0000 7fa2b7400640 1 -- 172.21.15.78:0/2711449388 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7fa2c0146d80 con 0x7fa2a0071e60 2024-09-11T12:56:56.961 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:56:56.963 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 msgr2=0x7fa2a0074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 0x7fa2a0074320 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fa2b00af000 tx=0x7fa2b001b0a0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 msgr2=0x7fa2c0146fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa2b800c790 tx=0x7fa2b800cc50 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 shutdown_connections 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa2c01f9680 0x7fa2c0146fa0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa2a0071e60 0x7fa2a0074320 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa2c0159fc0 0x7fa2c014f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 --2- 172.21.15.78:0/2711449388 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa2c006f200 0x7fa2c01f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 >> 172.21.15.78:0/2711449388 conn(0x7fa2c006ca80 msgr2=0x7fa2c015af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 shutdown_connections 2024-09-11T12:56:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:56:56.957+0000 7fa2c6000640 1 -- 172.21.15.78:0/2711449388 wait complete. 2024-09-11T12:56:56.983 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-09-11T12:56:56.509373+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282232,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692488,"statfs":{"total":773094113280,"available":772805107712,"internally_reserved":0,"allocated":16424960,"data_stored":7854108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.227837"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800594+0000","last_change":"2024-09-11T12:56:51.325800+0000","last_active":"2024-09-11T12:56:52.800594+0000","last_peered":"2024-09-11T12:56:52.800594+0000","last_clean":"2024-09-11T12:56:52.800594+0000","last_became_active":"2024-09-11T12:56:51.325534+0000","last_became_peered":"2024-09-11T12:56:51.325534+0000","last_unstale":"2024-09-11T12:56:52.800594+0000","last_undegraded":"2024-09-11T12:56:52.800594+0000","last_fullsized":"2024-09-11T12:56:52.800594+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:37.218816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799665+0000","last_change":"2024-09-11T12:56:51.323065+0000","last_active":"2024-09-11T12:56:52.799665+0000","last_peered":"2024-09-11T12:56:52.799665+0000","last_clean":"2024-09-11T12:56:52.799665+0000","last_became_active":"2024-09-11T12:56:51.322215+0000","last_became_peered":"2024-09-11T12:56:51.322215+0000","last_unstale":"2024-09-11T12:56:52.799665+0000","last_undegraded":"2024-09-11T12:56:52.799665+0000","last_fullsized":"2024-09-11T12:56:52.799665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:12.506659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323538+0000","last_change":"2024-09-11T12:56:51.312717+0000","last_active":"2024-09-11T12:56:52.323538+0000","last_peered":"2024-09-11T12:56:52.323538+0000","last_clean":"2024-09-11T12:56:52.323538+0000","last_became_active":"2024-09-11T12:56:51.312438+0000","last_became_peered":"2024-09-11T12:56:51.312438+0000","last_unstale":"2024-09-11T12:56:52.323538+0000","last_undegraded":"2024-09-11T12:56:52.323538+0000","last_fullsized":"2024-09-11T12:56:52.323538+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:18.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800400+0000","last_change":"2024-09-11T12:56:51.328683+0000","last_active":"2024-09-11T12:56:52.800400+0000","last_peered":"2024-09-11T12:56:52.800400+0000","last_clean":"2024-09-11T12:56:52.800400+0000","last_became_active":"2024-09-11T12:56:51.328383+0000","last_became_peered":"2024-09-11T12:56:51.328383+0000","last_unstale":"2024-09-11T12:56:52.800400+0000","last_undegraded":"2024-09-11T12:56:52.800400+0000","last_fullsized":"2024-09-11T12:56:52.800400+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798092+0000","last_change":"2024-09-11T12:56:51.314989+0000","last_active":"2024-09-11T12:56:52.798092+0000","last_peered":"2024-09-11T12:56:52.798092+0000","last_clean":"2024-09-11T12:56:52.798092+0000","last_became_active":"2024-09-11T12:56:51.314693+0000","last_became_peered":"2024-09-11T12:56:51.314693+0000","last_unstale":"2024-09-11T12:56:52.798092+0000","last_undegraded":"2024-09-11T12:56:52.798092+0000","last_fullsized":"2024-09-11T12:56:52.798092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:14.806808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323841+0000","last_change":"2024-09-11T12:56:51.322503+0000","last_active":"2024-09-11T12:56:52.323841+0000","last_peered":"2024-09-11T12:56:52.323841+0000","last_clean":"2024-09-11T12:56:52.323841+0000","last_became_active":"2024-09-11T12:56:51.322178+0000","last_became_peered":"2024-09-11T12:56:51.322178+0000","last_unstale":"2024-09-11T12:56:52.323841+0000","last_undegraded":"2024-09-11T12:56:52.323841+0000","last_fullsized":"2024-09-11T12:56:52.323841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:36.089725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302972+0000","last_change":"2024-09-11T12:56:51.314700+0000","last_active":"2024-09-11T12:56:52.302972+0000","last_peered":"2024-09-11T12:56:52.302972+0000","last_clean":"2024-09-11T12:56:52.302972+0000","last_became_active":"2024-09-11T12:56:51.314451+0000","last_became_peered":"2024-09-11T12:56:51.314451+0000","last_unstale":"2024-09-11T12:56:52.302972+0000","last_undegraded":"2024-09-11T12:56:52.302972+0000","last_fullsized":"2024-09-11T12:56:52.302972+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:55.290264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300452+0000","last_change":"2024-09-11T12:56:51.306750+0000","last_active":"2024-09-11T12:56:52.300452+0000","last_peered":"2024-09-11T12:56:52.300452+0000","last_clean":"2024-09-11T12:56:52.300452+0000","last_became_active":"2024-09-11T12:56:51.306324+0000","last_became_peered":"2024-09-11T12:56:51.306324+0000","last_unstale":"2024-09-11T12:56:52.300452+0000","last_undegraded":"2024-09-11T12:56:52.300452+0000","last_fullsized":"2024-09-11T12:56:52.300452+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304279+0000","last_change":"2024-09-11T12:56:51.323990+0000","last_active":"2024-09-11T12:56:52.304279+0000","last_peered":"2024-09-11T12:56:52.304279+0000","last_clean":"2024-09-11T12:56:52.304279+0000","last_became_active":"2024-09-11T12:56:51.323693+0000","last_became_peered":"2024-09-11T12:56:51.323693+0000","last_unstale":"2024-09-11T12:56:52.304279+0000","last_undegraded":"2024-09-11T12:56:52.304279+0000","last_fullsized":"2024-09-11T12:56:52.304279+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:38.455546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321464+0000","last_change":"2024-09-11T12:56:51.323632+0000","last_active":"2024-09-11T12:56:52.321464+0000","last_peered":"2024-09-11T12:56:52.321464+0000","last_clean":"2024-09-11T12:56:52.321464+0000","last_became_active":"2024-09-11T12:56:51.322881+0000","last_became_peered":"2024-09-11T12:56:51.322881+0000","last_unstale":"2024-09-11T12:56:52.321464+0000","last_undegraded":"2024-09-11T12:56:52.321464+0000","last_fullsized":"2024-09-11T12:56:52.321464+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:00.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303354+0000","last_change":"2024-09-11T12:56:51.318511+0000","last_active":"2024-09-11T12:56:52.303354+0000","last_peered":"2024-09-11T12:56:52.303354+0000","last_clean":"2024-09-11T12:56:52.303354+0000","last_became_active":"2024-09-11T12:56:51.318236+0000","last_became_peered":"2024-09-11T12:56:51.318236+0000","last_unstale":"2024-09-11T12:56:52.303354+0000","last_undegraded":"2024-09-11T12:56:52.303354+0000","last_fullsized":"2024-09-11T12:56:52.303354+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:13.058353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800128+0000","last_change":"2024-09-11T12:56:51.320498+0000","last_active":"2024-09-11T12:56:52.800128+0000","last_peered":"2024-09-11T12:56:52.800128+0000","last_clean":"2024-09-11T12:56:52.800128+0000","last_became_active":"2024-09-11T12:56:51.320113+0000","last_became_peered":"2024-09-11T12:56:51.320113+0000","last_unstale":"2024-09-11T12:56:52.800128+0000","last_undegraded":"2024-09-11T12:56:52.800128+0000","last_fullsized":"2024-09-11T12:56:52.800128+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:29.453042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799646+0000","last_change":"2024-09-11T12:56:51.331608+0000","last_active":"2024-09-11T12:56:52.799646+0000","last_peered":"2024-09-11T12:56:52.799646+0000","last_clean":"2024-09-11T12:56:52.799646+0000","last_became_active":"2024-09-11T12:56:51.331170+0000","last_became_peered":"2024-09-11T12:56:51.331170+0000","last_unstale":"2024-09-11T12:56:52.799646+0000","last_undegraded":"2024-09-11T12:56:52.799646+0000","last_fullsized":"2024-09-11T12:56:52.799646+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:19.401069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302918+0000","last_change":"2024-09-11T12:56:51.314000+0000","last_active":"2024-09-11T12:56:52.302918+0000","last_peered":"2024-09-11T12:56:52.302918+0000","last_clean":"2024-09-11T12:56:52.302918+0000","last_became_active":"2024-09-11T12:56:51.313716+0000","last_became_peered":"2024-09-11T12:56:51.313716+0000","last_unstale":"2024-09-11T12:56:52.302918+0000","last_undegraded":"2024-09-11T12:56:52.302918+0000","last_fullsized":"2024-09-11T12:56:52.302918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:00.097772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799766+0000","last_change":"2024-09-11T12:56:51.320626+0000","last_active":"2024-09-11T12:56:52.799766+0000","last_peered":"2024-09-11T12:56:52.799766+0000","last_clean":"2024-09-11T12:56:52.799766+0000","last_became_active":"2024-09-11T12:56:51.320268+0000","last_became_peered":"2024-09-11T12:56:51.320268+0000","last_unstale":"2024-09-11T12:56:52.799766+0000","last_undegraded":"2024-09-11T12:56:52.799766+0000","last_fullsized":"2024-09-11T12:56:52.799766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:09.579181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300490+0000","last_change":"2024-09-11T12:56:51.307274+0000","last_active":"2024-09-11T12:56:52.300490+0000","last_peered":"2024-09-11T12:56:52.300490+0000","last_clean":"2024-09-11T12:56:52.300490+0000","last_became_active":"2024-09-11T12:56:51.307078+0000","last_became_peered":"2024-09-11T12:56:51.307078+0000","last_unstale":"2024-09-11T12:56:52.300490+0000","last_undegraded":"2024-09-11T12:56:52.300490+0000","last_fullsized":"2024-09-11T12:56:52.300490+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:36.089725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797989+0000","last_change":"2024-09-11T12:56:51.321448+0000","last_active":"2024-09-11T12:56:52.797989+0000","last_peered":"2024-09-11T12:56:52.797989+0000","last_clean":"2024-09-11T12:56:52.797989+0000","last_became_active":"2024-09-11T12:56:51.321101+0000","last_became_peered":"2024-09-11T12:56:51.321101+0000","last_unstale":"2024-09-11T12:56:52.797989+0000","last_undegraded":"2024-09-11T12:56:52.797989+0000","last_fullsized":"2024-09-11T12:56:52.797989+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:54:06.506829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800345+0000","last_change":"2024-09-11T12:56:51.322295+0000","last_active":"2024-09-11T12:56:52.800345+0000","last_peered":"2024-09-11T12:56:52.800345+0000","last_clean":"2024-09-11T12:56:52.800345+0000","last_became_active":"2024-09-11T12:56:51.322029+0000","last_became_peered":"2024-09-11T12:56:51.322029+0000","last_unstale":"2024-09-11T12:56:52.800345+0000","last_undegraded":"2024-09-11T12:56:52.800345+0000","last_fullsized":"2024-09-11T12:56:52.800345+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:47.295276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321278+0000","last_change":"2024-09-11T12:56:51.323737+0000","last_active":"2024-09-11T12:56:52.321278+0000","last_peered":"2024-09-11T12:56:52.321278+0000","last_clean":"2024-09-11T12:56:52.321278+0000","last_became_active":"2024-09-11T12:56:51.323259+0000","last_became_peered":"2024-09-11T12:56:51.323259+0000","last_unstale":"2024-09-11T12:56:52.321278+0000","last_undegraded":"2024-09-11T12:56:52.321278+0000","last_fullsized":"2024-09-11T12:56:52.321278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:36.162519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797823+0000","last_change":"2024-09-11T12:56:51.322436+0000","last_active":"2024-09-11T12:56:52.797823+0000","last_peered":"2024-09-11T12:56:52.797823+0000","last_clean":"2024-09-11T12:56:52.797823+0000","last_became_active":"2024-09-11T12:56:51.322152+0000","last_became_peered":"2024-09-11T12:56:51.322152+0000","last_unstale":"2024-09-11T12:56:52.797823+0000","last_undegraded":"2024-09-11T12:56:52.797823+0000","last_fullsized":"2024-09-11T12:56:52.797823+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:27.249091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797752+0000","last_change":"2024-09-11T12:56:51.309729+0000","last_active":"2024-09-11T12:56:52.797752+0000","last_peered":"2024-09-11T12:56:52.797752+0000","last_clean":"2024-09-11T12:56:52.797752+0000","last_became_active":"2024-09-11T12:56:51.309458+0000","last_became_peered":"2024-09-11T12:56:51.309458+0000","last_unstale":"2024-09-11T12:56:52.797752+0000","last_undegraded":"2024-09-11T12:56:52.797752+0000","last_fullsized":"2024-09-11T12:56:52.797752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:57.142651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320856+0000","last_change":"2024-09-11T12:56:51.327363+0000","last_active":"2024-09-11T12:56:52.320856+0000","last_peered":"2024-09-11T12:56:52.320856+0000","last_clean":"2024-09-11T12:56:52.320856+0000","last_became_active":"2024-09-11T12:56:51.327039+0000","last_became_peered":"2024-09-11T12:56:51.327039+0000","last_unstale":"2024-09-11T12:56:52.320856+0000","last_undegraded":"2024-09-11T12:56:52.320856+0000","last_fullsized":"2024-09-11T12:56:52.320856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:39.941859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323443+0000","last_change":"2024-09-11T12:56:51.319865+0000","last_active":"2024-09-11T12:56:52.323443+0000","last_peered":"2024-09-11T12:56:52.323443+0000","last_clean":"2024-09-11T12:56:52.323443+0000","last_became_active":"2024-09-11T12:56:51.317358+0000","last_became_peered":"2024-09-11T12:56:51.317358+0000","last_unstale":"2024-09-11T12:56:52.323443+0000","last_undegraded":"2024-09-11T12:56:52.323443+0000","last_fullsized":"2024-09-11T12:56:52.323443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:09.579181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300631+0000","last_change":"2024-09-11T12:56:51.315972+0000","last_active":"2024-09-11T12:56:52.300631+0000","last_peered":"2024-09-11T12:56:52.300631+0000","last_clean":"2024-09-11T12:56:52.300631+0000","last_became_active":"2024-09-11T12:56:51.315276+0000","last_became_peered":"2024-09-11T12:56:51.315276+0000","last_unstale":"2024-09-11T12:56:52.300631+0000","last_undegraded":"2024-09-11T12:56:52.300631+0000","last_fullsized":"2024-09-11T12:56:52.300631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:45.659768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799878+0000","last_change":"2024-09-11T12:56:51.325966+0000","last_active":"2024-09-11T12:56:52.799878+0000","last_peered":"2024-09-11T12:56:52.799878+0000","last_clean":"2024-09-11T12:56:52.799878+0000","last_became_active":"2024-09-11T12:56:51.325626+0000","last_became_peered":"2024-09-11T12:56:51.325626+0000","last_unstale":"2024-09-11T12:56:52.799878+0000","last_undegraded":"2024-09-11T12:56:52.799878+0000","last_fullsized":"2024-09-11T12:56:52.799878+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:18.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303623+0000","last_change":"2024-09-11T12:56:51.329476+0000","last_active":"2024-09-11T12:56:52.303623+0000","last_peered":"2024-09-11T12:56:52.303623+0000","last_clean":"2024-09-11T12:56:52.303623+0000","last_became_active":"2024-09-11T12:56:51.327136+0000","last_became_peered":"2024-09-11T12:56:51.327136+0000","last_unstale":"2024-09-11T12:56:52.303623+0000","last_undegraded":"2024-09-11T12:56:52.303623+0000","last_fullsized":"2024-09-11T12:56:52.303623+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:23.047576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323708+0000","last_change":"2024-09-11T12:56:51.322218+0000","last_active":"2024-09-11T12:56:52.323708+0000","last_peered":"2024-09-11T12:56:52.323708+0000","last_clean":"2024-09-11T12:56:52.323708+0000","last_became_active":"2024-09-11T12:56:51.321925+0000","last_became_peered":"2024-09-11T12:56:51.321925+0000","last_unstale":"2024-09-11T12:56:52.323708+0000","last_undegraded":"2024-09-11T12:56:52.323708+0000","last_fullsized":"2024-09-11T12:56:52.323708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:29.453042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797910+0000","last_change":"2024-09-11T12:56:51.309714+0000","last_active":"2024-09-11T12:56:52.797910+0000","last_peered":"2024-09-11T12:56:52.797910+0000","last_clean":"2024-09-11T12:56:52.797910+0000","last_became_active":"2024-09-11T12:56:51.309402+0000","last_became_peered":"2024-09-11T12:56:51.309402+0000","last_unstale":"2024-09-11T12:56:52.797910+0000","last_undegraded":"2024-09-11T12:56:52.797910+0000","last_fullsized":"2024-09-11T12:56:52.797910+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:48.842364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799598+0000","last_change":"2024-09-11T12:56:51.323081+0000","last_active":"2024-09-11T12:56:52.799598+0000","last_peered":"2024-09-11T12:56:52.799598+0000","last_clean":"2024-09-11T12:56:52.799598+0000","last_became_active":"2024-09-11T12:56:51.322737+0000","last_became_peered":"2024-09-11T12:56:51.322737+0000","last_unstale":"2024-09-11T12:56:52.799598+0000","last_undegraded":"2024-09-11T12:56:52.799598+0000","last_fullsized":"2024-09-11T12:56:52.799598+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:19.076624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303123+0000","last_change":"2024-09-11T12:56:51.321697+0000","last_active":"2024-09-11T12:56:52.303123+0000","last_peered":"2024-09-11T12:56:52.303123+0000","last_clean":"2024-09-11T12:56:52.303123+0000","last_became_active":"2024-09-11T12:56:51.321385+0000","last_became_peered":"2024-09-11T12:56:51.321385+0000","last_unstale":"2024-09-11T12:56:52.303123+0000","last_undegraded":"2024-09-11T12:56:52.303123+0000","last_fullsized":"2024-09-11T12:56:52.303123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:49.339713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323504+0000","last_change":"2024-09-11T12:56:51.332411+0000","last_active":"2024-09-11T12:56:52.323504+0000","last_peered":"2024-09-11T12:56:52.323504+0000","last_clean":"2024-09-11T12:56:52.323504+0000","last_became_active":"2024-09-11T12:56:51.331877+0000","last_became_peered":"2024-09-11T12:56:51.331877+0000","last_unstale":"2024-09-11T12:56:52.323504+0000","last_undegraded":"2024-09-11T12:56:52.323504+0000","last_fullsized":"2024-09-11T12:56:52.323504+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:50.219522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797484+0000","last_change":"2024-09-11T12:56:51.310078+0000","last_active":"2024-09-11T12:56:52.797484+0000","last_peered":"2024-09-11T12:56:52.797484+0000","last_clean":"2024-09-11T12:56:52.797484+0000","last_became_active":"2024-09-11T12:56:51.309774+0000","last_became_peered":"2024-09-11T12:56:51.309774+0000","last_unstale":"2024-09-11T12:56:52.797484+0000","last_undegraded":"2024-09-11T12:56:52.797484+0000","last_fullsized":"2024-09-11T12:56:52.797484+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:09.595235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300955+0000","last_change":"2024-09-11T12:56:51.307445+0000","last_active":"2024-09-11T12:56:52.300955+0000","last_peered":"2024-09-11T12:56:52.300955+0000","last_clean":"2024-09-11T12:56:52.300955+0000","last_became_active":"2024-09-11T12:56:51.307116+0000","last_became_peered":"2024-09-11T12:56:51.307116+0000","last_unstale":"2024-09-11T12:56:52.300955+0000","last_undegraded":"2024-09-11T12:56:52.300955+0000","last_fullsized":"2024-09-11T12:56:52.300955+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:18.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"150'4","reported_seq":19,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797963+0000","last_change":"2024-09-11T12:56:50.290099+0000","last_active":"2024-09-11T12:56:52.797963+0000","last_peered":"2024-09-11T12:56:52.797963+0000","last_clean":"2024-09-11T12:56:52.797963+0000","last_became_active":"2024-09-11T12:56:50.289571+0000","last_became_peered":"2024-09-11T12:56:50.289571+0000","last_unstale":"2024-09-11T12:56:52.797963+0000","last_undegraded":"2024-09-11T12:56:52.797963+0000","last_fullsized":"2024-09-11T12:56:52.797963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:53.672036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321489+0000","last_change":"2024-09-11T12:56:51.321133+0000","last_active":"2024-09-11T12:56:52.321489+0000","last_peered":"2024-09-11T12:56:52.321489+0000","last_clean":"2024-09-11T12:56:52.321489+0000","last_became_active":"2024-09-11T12:56:51.319696+0000","last_became_peered":"2024-09-11T12:56:51.319696+0000","last_unstale":"2024-09-11T12:56:52.321489+0000","last_undegraded":"2024-09-11T12:56:52.321489+0000","last_fullsized":"2024-09-11T12:56:52.321489+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:32:47.236732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304187+0000","last_change":"2024-09-11T12:56:50.305706+0000","last_active":"2024-09-11T12:56:52.304187+0000","last_peered":"2024-09-11T12:56:52.304187+0000","last_clean":"2024-09-11T12:56:52.304187+0000","last_became_active":"2024-09-11T12:56:50.303736+0000","last_became_peered":"2024-09-11T12:56:50.303736+0000","last_unstale":"2024-09-11T12:56:52.304187+0000","last_undegraded":"2024-09-11T12:56:52.304187+0000","last_fullsized":"2024-09-11T12:56:52.304187+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:16:26.722834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304075+0000","last_change":"2024-09-11T12:56:51.315464+0000","last_active":"2024-09-11T12:56:52.304075+0000","last_peered":"2024-09-11T12:56:52.304075+0000","last_clean":"2024-09-11T12:56:52.304075+0000","last_became_active":"2024-09-11T12:56:51.315149+0000","last_became_peered":"2024-09-11T12:56:51.315149+0000","last_unstale":"2024-09-11T12:56:52.304075+0000","last_undegraded":"2024-09-11T12:56:52.304075+0000","last_fullsized":"2024-09-11T12:56:52.304075+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:12.506659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323786+0000","last_change":"2024-09-11T12:56:50.306449+0000","last_active":"2024-09-11T12:56:52.323786+0000","last_peered":"2024-09-11T12:56:52.323786+0000","last_clean":"2024-09-11T12:56:52.323786+0000","last_became_active":"2024-09-11T12:56:50.306013+0000","last_became_peered":"2024-09-11T12:56:50.306013+0000","last_unstale":"2024-09-11T12:56:52.323786+0000","last_undegraded":"2024-09-11T12:56:52.323786+0000","last_fullsized":"2024-09-11T12:56:52.323786+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:54.283189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304002+0000","last_change":"2024-09-11T12:56:51.309155+0000","last_active":"2024-09-11T12:56:52.304002+0000","last_peered":"2024-09-11T12:56:52.304002+0000","last_clean":"2024-09-11T12:56:52.304002+0000","last_became_active":"2024-09-11T12:56:51.308914+0000","last_became_peered":"2024-09-11T12:56:51.308914+0000","last_unstale":"2024-09-11T12:56:52.304002+0000","last_undegraded":"2024-09-11T12:56:52.304002+0000","last_fullsized":"2024-09-11T12:56:52.304002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:06.482630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800249+0000","last_change":"2024-09-11T12:56:50.302084+0000","last_active":"2024-09-11T12:56:52.800249+0000","last_peered":"2024-09-11T12:56:52.800249+0000","last_clean":"2024-09-11T12:56:52.800249+0000","last_became_active":"2024-09-11T12:56:50.301802+0000","last_became_peered":"2024-09-11T12:56:50.301802+0000","last_unstale":"2024-09-11T12:56:52.800249+0000","last_undegraded":"2024-09-11T12:56:52.800249+0000","last_fullsized":"2024-09-11T12:56:52.800249+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:23.661282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321008+0000","last_change":"2024-09-11T12:56:51.324291+0000","last_active":"2024-09-11T12:56:52.321008+0000","last_peered":"2024-09-11T12:56:52.321008+0000","last_clean":"2024-09-11T12:56:52.321008+0000","last_became_active":"2024-09-11T12:56:51.322861+0000","last_became_peered":"2024-09-11T12:56:51.322861+0000","last_unstale":"2024-09-11T12:56:52.321008+0000","last_undegraded":"2024-09-11T12:56:52.321008+0000","last_fullsized":"2024-09-11T12:56:52.321008+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:00.745036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797897+0000","last_change":"2024-09-11T12:56:50.289506+0000","last_active":"2024-09-11T12:56:52.797897+0000","last_peered":"2024-09-11T12:56:52.797897+0000","last_clean":"2024-09-11T12:56:52.797897+0000","last_became_active":"2024-09-11T12:56:50.289233+0000","last_became_peered":"2024-09-11T12:56:50.289233+0000","last_unstale":"2024-09-11T12:56:52.797897+0000","last_undegraded":"2024-09-11T12:56:52.797897+0000","last_fullsized":"2024-09-11T12:56:52.797897+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:10.824978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300697+0000","last_change":"2024-09-11T12:56:51.308409+0000","last_active":"2024-09-11T12:56:52.300697+0000","last_peered":"2024-09-11T12:56:52.300697+0000","last_clean":"2024-09-11T12:56:52.300697+0000","last_became_active":"2024-09-11T12:56:51.308123+0000","last_became_peered":"2024-09-11T12:56:51.308123+0000","last_unstale":"2024-09-11T12:56:52.300697+0000","last_undegraded":"2024-09-11T12:56:52.300697+0000","last_fullsized":"2024-09-11T12:56:52.300697+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:06.109961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300536+0000","last_change":"2024-09-11T12:56:50.305183+0000","last_active":"2024-09-11T12:56:52.300536+0000","last_peered":"2024-09-11T12:56:52.300536+0000","last_clean":"2024-09-11T12:56:52.300536+0000","last_became_active":"2024-09-11T12:56:50.304940+0000","last_became_peered":"2024-09-11T12:56:50.304940+0000","last_unstale":"2024-09-11T12:56:52.300536+0000","last_undegraded":"2024-09-11T12:56:52.300536+0000","last_fullsized":"2024-09-11T12:56:52.300536+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:22.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303185+0000","last_change":"2024-09-11T12:56:51.318619+0000","last_active":"2024-09-11T12:56:52.303185+0000","last_peered":"2024-09-11T12:56:52.303185+0000","last_clean":"2024-09-11T12:56:52.303185+0000","last_became_active":"2024-09-11T12:56:51.318318+0000","last_became_peered":"2024-09-11T12:56:51.318318+0000","last_unstale":"2024-09-11T12:56:52.303185+0000","last_undegraded":"2024-09-11T12:56:52.303185+0000","last_fullsized":"2024-09-11T12:56:52.303185+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:10.975605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799362+0000","last_change":"2024-09-11T12:56:50.308680+0000","last_active":"2024-09-11T12:56:52.799362+0000","last_peered":"2024-09-11T12:56:52.799362+0000","last_clean":"2024-09-11T12:56:52.799362+0000","last_became_active":"2024-09-11T12:56:50.308390+0000","last_became_peered":"2024-09-11T12:56:50.308390+0000","last_unstale":"2024-09-11T12:56:52.799362+0000","last_undegraded":"2024-09-11T12:56:52.799362+0000","last_fullsized":"2024-09-11T12:56:52.799362+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:16.923224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323799+0000","last_change":"2024-09-11T12:56:51.325404+0000","last_active":"2024-09-11T12:56:52.323799+0000","last_peered":"2024-09-11T12:56:52.323799+0000","last_clean":"2024-09-11T12:56:52.323799+0000","last_became_active":"2024-09-11T12:56:51.325127+0000","last_became_peered":"2024-09-11T12:56:51.325127+0000","last_unstale":"2024-09-11T12:56:52.323799+0000","last_undegraded":"2024-09-11T12:56:52.323799+0000","last_fullsized":"2024-09-11T12:56:52.323799+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303558+0000","last_change":"2024-09-11T12:56:50.292337+0000","last_active":"2024-09-11T12:56:52.303558+0000","last_peered":"2024-09-11T12:56:52.303558+0000","last_clean":"2024-09-11T12:56:52.303558+0000","last_became_active":"2024-09-11T12:56:50.292057+0000","last_became_peered":"2024-09-11T12:56:50.292057+0000","last_unstale":"2024-09-11T12:56:52.303558+0000","last_undegraded":"2024-09-11T12:56:52.303558+0000","last_fullsized":"2024-09-11T12:56:52.303558+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:19.743245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300862+0000","last_change":"2024-09-11T12:56:51.321692+0000","last_active":"2024-09-11T12:56:52.300862+0000","last_peered":"2024-09-11T12:56:52.300862+0000","last_clean":"2024-09-11T12:56:52.300862+0000","last_became_active":"2024-09-11T12:56:51.321415+0000","last_became_peered":"2024-09-11T12:56:51.321415+0000","last_unstale":"2024-09-11T12:56:52.300862+0000","last_undegraded":"2024-09-11T12:56:52.300862+0000","last_fullsized":"2024-09-11T12:56:52.300862+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:18.097421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798012+0000","last_change":"2024-09-11T12:56:50.300136+0000","last_active":"2024-09-11T12:56:52.798012+0000","last_peered":"2024-09-11T12:56:52.798012+0000","last_clean":"2024-09-11T12:56:52.798012+0000","last_became_active":"2024-09-11T12:56:50.299878+0000","last_became_peered":"2024-09-11T12:56:50.299878+0000","last_unstale":"2024-09-11T12:56:52.798012+0000","last_undegraded":"2024-09-11T12:56:52.798012+0000","last_fullsized":"2024-09-11T12:56:52.798012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:22:03.290036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303822+0000","last_change":"2024-09-11T12:56:51.308310+0000","last_active":"2024-09-11T12:56:52.303822+0000","last_peered":"2024-09-11T12:56:52.303822+0000","last_clean":"2024-09-11T12:56:52.303822+0000","last_became_active":"2024-09-11T12:56:51.307974+0000","last_became_peered":"2024-09-11T12:56:51.307974+0000","last_unstale":"2024-09-11T12:56:52.303822+0000","last_undegraded":"2024-09-11T12:56:52.303822+0000","last_fullsized":"2024-09-11T12:56:52.303822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:19.076624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"150'3","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797808+0000","last_change":"2024-09-11T12:56:50.302419+0000","last_active":"2024-09-11T12:56:52.797808+0000","last_peered":"2024-09-11T12:56:52.797808+0000","last_clean":"2024-09-11T12:56:52.797808+0000","last_became_active":"2024-09-11T12:56:50.302121+0000","last_became_peered":"2024-09-11T12:56:50.302121+0000","last_unstale":"2024-09-11T12:56:52.797808+0000","last_undegraded":"2024-09-11T12:56:52.797808+0000","last_fullsized":"2024-09-11T12:56:52.797808+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:42:04.424616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321351+0000","last_change":"2024-09-11T12:56:51.323861+0000","last_active":"2024-09-11T12:56:52.321351+0000","last_peered":"2024-09-11T12:56:52.321351+0000","last_clean":"2024-09-11T12:56:52.321351+0000","last_became_active":"2024-09-11T12:56:51.323610+0000","last_became_peered":"2024-09-11T12:56:51.323610+0000","last_unstale":"2024-09-11T12:56:52.321351+0000","last_undegraded":"2024-09-11T12:56:52.321351+0000","last_fullsized":"2024-09-11T12:56:52.321351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:48:39.091060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"150'6","reported_seq":21,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323678+0000","last_change":"2024-09-11T12:56:50.306460+0000","last_active":"2024-09-11T12:56:52.323678+0000","last_peered":"2024-09-11T12:56:52.323678+0000","last_clean":"2024-09-11T12:56:52.323678+0000","last_became_active":"2024-09-11T12:56:50.306176+0000","last_became_peered":"2024-09-11T12:56:50.306176+0000","last_unstale":"2024-09-11T12:56:52.323678+0000","last_undegraded":"2024-09-11T12:56:52.323678+0000","last_fullsized":"2024-09-11T12:56:52.323678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:34.597272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323539+0000","last_change":"2024-09-11T12:56:51.316949+0000","last_active":"2024-09-11T12:56:52.323539+0000","last_peered":"2024-09-11T12:56:52.323539+0000","last_clean":"2024-09-11T12:56:52.323539+0000","last_became_active":"2024-09-11T12:56:51.316608+0000","last_became_peered":"2024-09-11T12:56:51.316608+0000","last_unstale":"2024-09-11T12:56:52.323539+0000","last_undegraded":"2024-09-11T12:56:52.323539+0000","last_fullsized":"2024-09-11T12:56:52.323539+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:19.401069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800398+0000","last_change":"2024-09-11T12:56:50.309887+0000","last_active":"2024-09-11T12:56:52.800398+0000","last_peered":"2024-09-11T12:56:52.800398+0000","last_clean":"2024-09-11T12:56:52.800398+0000","last_became_active":"2024-09-11T12:56:50.309619+0000","last_became_peered":"2024-09-11T12:56:50.309619+0000","last_unstale":"2024-09-11T12:56:52.800398+0000","last_undegraded":"2024-09-11T12:56:52.800398+0000","last_fullsized":"2024-09-11T12:56:52.800398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:05.102886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303717+0000","last_change":"2024-09-11T12:56:51.315637+0000","last_active":"2024-09-11T12:56:52.303717+0000","last_peered":"2024-09-11T12:56:52.303717+0000","last_clean":"2024-09-11T12:56:52.303717+0000","last_became_active":"2024-09-11T12:56:51.313997+0000","last_became_peered":"2024-09-11T12:56:51.313997+0000","last_unstale":"2024-09-11T12:56:52.303717+0000","last_undegraded":"2024-09-11T12:56:52.303717+0000","last_fullsized":"2024-09-11T12:56:52.303717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:41.783318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321186+0000","last_change":"2024-09-11T12:56:50.298086+0000","last_active":"2024-09-11T12:56:52.321186+0000","last_peered":"2024-09-11T12:56:52.321186+0000","last_clean":"2024-09-11T12:56:52.321186+0000","last_became_active":"2024-09-11T12:56:50.297806+0000","last_became_peered":"2024-09-11T12:56:50.297806+0000","last_unstale":"2024-09-11T12:56:52.321186+0000","last_undegraded":"2024-09-11T12:56:52.321186+0000","last_fullsized":"2024-09-11T12:56:52.321186+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:10.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302960+0000","last_change":"2024-09-11T12:56:51.314477+0000","last_active":"2024-09-11T12:56:52.302960+0000","last_peered":"2024-09-11T12:56:52.302960+0000","last_clean":"2024-09-11T12:56:52.302960+0000","last_became_active":"2024-09-11T12:56:51.314217+0000","last_became_peered":"2024-09-11T12:56:51.314217+0000","last_unstale":"2024-09-11T12:56:52.302960+0000","last_undegraded":"2024-09-11T12:56:52.302960+0000","last_fullsized":"2024-09-11T12:56:52.302960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:29.509403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300348+0000","last_change":"2024-09-11T12:56:50.299386+0000","last_active":"2024-09-11T12:56:52.300348+0000","last_peered":"2024-09-11T12:56:52.300348+0000","last_clean":"2024-09-11T12:56:52.300348+0000","last_became_active":"2024-09-11T12:56:50.299158+0000","last_became_peered":"2024-09-11T12:56:50.299158+0000","last_unstale":"2024-09-11T12:56:52.300348+0000","last_undegraded":"2024-09-11T12:56:52.300348+0000","last_fullsized":"2024-09-11T12:56:52.300348+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:54.283189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320803+0000","last_change":"2024-09-11T12:56:51.326787+0000","last_active":"2024-09-11T12:56:52.320803+0000","last_peered":"2024-09-11T12:56:52.320803+0000","last_clean":"2024-09-11T12:56:52.320803+0000","last_became_active":"2024-09-11T12:56:51.325559+0000","last_became_peered":"2024-09-11T12:56:51.325559+0000","last_unstale":"2024-09-11T12:56:52.320803+0000","last_undegraded":"2024-09-11T12:56:52.320803+0000","last_fullsized":"2024-09-11T12:56:52.320803+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:05.908814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"150'8","reported_seq":23,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799371+0000","last_change":"2024-09-11T12:56:50.308722+0000","last_active":"2024-09-11T12:56:52.799371+0000","last_peered":"2024-09-11T12:56:52.799371+0000","last_clean":"2024-09-11T12:56:52.799371+0000","last_became_active":"2024-09-11T12:56:50.308431+0000","last_became_peered":"2024-09-11T12:56:50.308431+0000","last_unstale":"2024-09-11T12:56:52.799371+0000","last_undegraded":"2024-09-11T12:56:52.799371+0000","last_fullsized":"2024-09-11T12:56:52.799371+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:35.082649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321540+0000","last_change":"2024-09-11T12:56:51.312223+0000","last_active":"2024-09-11T12:56:52.321540+0000","last_peered":"2024-09-11T12:56:52.321540+0000","last_clean":"2024-09-11T12:56:52.321540+0000","last_became_active":"2024-09-11T12:56:51.311973+0000","last_became_peered":"2024-09-11T12:56:51.311973+0000","last_unstale":"2024-09-11T12:56:52.321540+0000","last_undegraded":"2024-09-11T12:56:52.321540+0000","last_fullsized":"2024-09-11T12:56:52.321540+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:39.565642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798055+0000","last_change":"2024-09-11T12:56:50.289863+0000","last_active":"2024-09-11T12:56:52.798055+0000","last_peered":"2024-09-11T12:56:52.798055+0000","last_clean":"2024-09-11T12:56:52.798055+0000","last_became_active":"2024-09-11T12:56:50.289217+0000","last_became_peered":"2024-09-11T12:56:50.289217+0000","last_unstale":"2024-09-11T12:56:52.798055+0000","last_undegraded":"2024-09-11T12:56:52.798055+0000","last_fullsized":"2024-09-11T12:56:52.798055+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:52.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304319+0000","last_change":"2024-09-11T12:56:51.327503+0000","last_active":"2024-09-11T12:56:52.304319+0000","last_peered":"2024-09-11T12:56:52.304319+0000","last_clean":"2024-09-11T12:56:52.304319+0000","last_became_active":"2024-09-11T12:56:51.327198+0000","last_became_peered":"2024-09-11T12:56:51.327198+0000","last_unstale":"2024-09-11T12:56:52.304319+0000","last_undegraded":"2024-09-11T12:56:52.304319+0000","last_fullsized":"2024-09-11T12:56:52.304319+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:48:05.531019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300963+0000","last_change":"2024-09-11T12:56:50.302013+0000","last_active":"2024-09-11T12:56:52.300963+0000","last_peered":"2024-09-11T12:56:52.300963+0000","last_clean":"2024-09-11T12:56:52.300963+0000","last_became_active":"2024-09-11T12:56:50.299221+0000","last_became_peered":"2024-09-11T12:56:50.299221+0000","last_unstale":"2024-09-11T12:56:52.300963+0000","last_undegraded":"2024-09-11T12:56:52.300963+0000","last_fullsized":"2024-09-11T12:56:52.300963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:12.051278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303663+0000","last_change":"2024-09-11T12:56:51.319142+0000","last_active":"2024-09-11T12:56:52.303663+0000","last_peered":"2024-09-11T12:56:52.303663+0000","last_clean":"2024-09-11T12:56:52.303663+0000","last_became_active":"2024-09-11T12:56:51.318865+0000","last_became_peered":"2024-09-11T12:56:51.318865+0000","last_unstale":"2024-09-11T12:56:52.303663+0000","last_undegraded":"2024-09-11T12:56:52.303663+0000","last_fullsized":"2024-09-11T12:56:52.303663+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:33:06.964539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800166+0000","last_change":"2024-09-11T12:56:50.311439+0000","last_active":"2024-09-11T12:56:52.800166+0000","last_peered":"2024-09-11T12:56:52.800166+0000","last_clean":"2024-09-11T12:56:52.800166+0000","last_became_active":"2024-09-11T12:56:50.311186+0000","last_became_peered":"2024-09-11T12:56:50.311186+0000","last_unstale":"2024-09-11T12:56:52.800166+0000","last_undegraded":"2024-09-11T12:56:52.800166+0000","last_fullsized":"2024-09-11T12:56:52.800166+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:17.090345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799791+0000","last_change":"2024-09-11T12:56:51.329352+0000","last_active":"2024-09-11T12:56:52.799791+0000","last_peered":"2024-09-11T12:56:52.799791+0000","last_clean":"2024-09-11T12:56:52.799791+0000","last_became_active":"2024-09-11T12:56:51.327853+0000","last_became_peered":"2024-09-11T12:56:51.327853+0000","last_unstale":"2024-09-11T12:56:52.799791+0000","last_undegraded":"2024-09-11T12:56:52.799791+0000","last_fullsized":"2024-09-11T12:56:52.799791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:50.219522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303923+0000","last_change":"2024-09-11T12:56:50.297243+0000","last_active":"2024-09-11T12:56:52.303923+0000","last_peered":"2024-09-11T12:56:52.303923+0000","last_clean":"2024-09-11T12:56:52.303923+0000","last_became_active":"2024-09-11T12:56:50.296921+0000","last_became_peered":"2024-09-11T12:56:50.296921+0000","last_unstale":"2024-09-11T12:56:52.303923+0000","last_undegraded":"2024-09-11T12:56:52.303923+0000","last_fullsized":"2024-09-11T12:56:52.303923+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:55.224946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321061+0000","last_change":"2024-09-11T12:56:51.310844+0000","last_active":"2024-09-11T12:56:52.321061+0000","last_peered":"2024-09-11T12:56:52.321061+0000","last_clean":"2024-09-11T12:56:52.321061+0000","last_became_active":"2024-09-11T12:56:51.310553+0000","last_became_peered":"2024-09-11T12:56:51.310553+0000","last_unstale":"2024-09-11T12:56:52.321061+0000","last_undegraded":"2024-09-11T12:56:52.321061+0000","last_fullsized":"2024-09-11T12:56:52.321061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:40.902725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323402+0000","last_change":"2024-09-11T12:56:50.309614+0000","last_active":"2024-09-11T12:56:52.323402+0000","last_peered":"2024-09-11T12:56:52.323402+0000","last_clean":"2024-09-11T12:56:52.323402+0000","last_became_active":"2024-09-11T12:56:50.309309+0000","last_became_peered":"2024-09-11T12:56:50.309309+0000","last_unstale":"2024-09-11T12:56:52.323402+0000","last_undegraded":"2024-09-11T12:56:52.323402+0000","last_fullsized":"2024-09-11T12:56:52.323402+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:51.352295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303899+0000","last_change":"2024-09-11T12:56:51.308171+0000","last_active":"2024-09-11T12:56:52.303899+0000","last_peered":"2024-09-11T12:56:52.303899+0000","last_clean":"2024-09-11T12:56:52.303899+0000","last_became_active":"2024-09-11T12:56:51.307772+0000","last_became_peered":"2024-09-11T12:56:51.307772+0000","last_unstale":"2024-09-11T12:56:52.303899+0000","last_undegraded":"2024-09-11T12:56:52.303899+0000","last_fullsized":"2024-09-11T12:56:52.303899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:42.264156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797642+0000","last_change":"2024-09-11T12:56:50.297571+0000","last_active":"2024-09-11T12:56:52.797642+0000","last_peered":"2024-09-11T12:56:52.797642+0000","last_clean":"2024-09-11T12:56:52.797642+0000","last_became_active":"2024-09-11T12:56:50.297307+0000","last_became_peered":"2024-09-11T12:56:50.297307+0000","last_unstale":"2024-09-11T12:56:52.797642+0000","last_undegraded":"2024-09-11T12:56:52.797642+0000","last_fullsized":"2024-09-11T12:56:52.797642+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:51.965420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799437+0000","last_change":"2024-09-11T12:56:51.323087+0000","last_active":"2024-09-11T12:56:52.799437+0000","last_peered":"2024-09-11T12:56:52.799437+0000","last_clean":"2024-09-11T12:56:52.799437+0000","last_became_active":"2024-09-11T12:56:51.322826+0000","last_became_peered":"2024-09-11T12:56:51.322826+0000","last_unstale":"2024-09-11T12:56:52.799437+0000","last_undegraded":"2024-09-11T12:56:52.799437+0000","last_fullsized":"2024-09-11T12:56:52.799437+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:41.783318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323586+0000","last_change":"2024-09-11T12:56:50.309534+0000","last_active":"2024-09-11T12:56:52.323586+0000","last_peered":"2024-09-11T12:56:52.323586+0000","last_clean":"2024-09-11T12:56:52.323586+0000","last_became_active":"2024-09-11T12:56:50.309270+0000","last_became_peered":"2024-09-11T12:56:50.309270+0000","last_unstale":"2024-09-11T12:56:52.323586+0000","last_undegraded":"2024-09-11T12:56:52.323586+0000","last_fullsized":"2024-09-11T12:56:52.323586+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:36.222739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797520+0000","last_change":"2024-09-11T12:56:51.330131+0000","last_active":"2024-09-11T12:56:52.797520+0000","last_peered":"2024-09-11T12:56:52.797520+0000","last_clean":"2024-09-11T12:56:52.797520+0000","last_became_active":"2024-09-11T12:56:51.329855+0000","last_became_peered":"2024-09-11T12:56:51.329855+0000","last_unstale":"2024-09-11T12:56:52.797520+0000","last_undegraded":"2024-09-11T12:56:52.797520+0000","last_fullsized":"2024-09-11T12:56:52.797520+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:21.554037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303092+0000","last_change":"2024-09-11T12:56:50.295252+0000","last_active":"2024-09-11T12:56:52.303092+0000","last_peered":"2024-09-11T12:56:52.303092+0000","last_clean":"2024-09-11T12:56:52.303092+0000","last_became_active":"2024-09-11T12:56:50.294990+0000","last_became_peered":"2024-09-11T12:56:50.294990+0000","last_unstale":"2024-09-11T12:56:52.303092+0000","last_undegraded":"2024-09-11T12:56:52.303092+0000","last_fullsized":"2024-09-11T12:56:52.303092+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:07.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797893+0000","last_change":"2024-09-11T12:56:51.319122+0000","last_active":"2024-09-11T12:56:52.797893+0000","last_peered":"2024-09-11T12:56:52.797893+0000","last_clean":"2024-09-11T12:56:52.797893+0000","last_became_active":"2024-09-11T12:56:51.318832+0000","last_became_peered":"2024-09-11T12:56:51.318832+0000","last_unstale":"2024-09-11T12:56:52.797893+0000","last_undegraded":"2024-09-11T12:56:52.797893+0000","last_fullsized":"2024-09-11T12:56:52.797893+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:39.391737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800669+0000","last_change":"2024-09-11T12:56:50.307695+0000","last_active":"2024-09-11T12:56:52.800669+0000","last_peered":"2024-09-11T12:56:52.800669+0000","last_clean":"2024-09-11T12:56:52.800669+0000","last_became_active":"2024-09-11T12:56:50.307465+0000","last_became_peered":"2024-09-11T12:56:50.307465+0000","last_unstale":"2024-09-11T12:56:52.800669+0000","last_undegraded":"2024-09-11T12:56:52.800669+0000","last_fullsized":"2024-09-11T12:56:52.800669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:50.337765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321645+0000","last_change":"2024-09-11T12:56:51.323587+0000","last_active":"2024-09-11T12:56:52.321645+0000","last_peered":"2024-09-11T12:56:52.321645+0000","last_clean":"2024-09-11T12:56:52.321645+0000","last_became_active":"2024-09-11T12:56:51.323303+0000","last_became_peered":"2024-09-11T12:56:51.323303+0000","last_unstale":"2024-09-11T12:56:52.321645+0000","last_undegraded":"2024-09-11T12:56:52.321645+0000","last_fullsized":"2024-09-11T12:56:52.321645+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:27:34.683781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"151'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321541+0000","last_change":"2024-09-11T12:56:50.303866+0000","last_active":"2024-09-11T12:56:52.321541+0000","last_peered":"2024-09-11T12:56:52.321541+0000","last_clean":"2024-09-11T12:56:52.321541+0000","last_became_active":"2024-09-11T12:56:50.303577+0000","last_became_peered":"2024-09-11T12:56:50.303577+0000","last_unstale":"2024-09-11T12:56:52.321541+0000","last_undegraded":"2024-09-11T12:56:52.321541+0000","last_fullsized":"2024-09-11T12:56:52.321541+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:05.285265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300734+0000","last_change":"2024-09-11T12:56:51.316304+0000","last_active":"2024-09-11T12:56:52.300734+0000","last_peered":"2024-09-11T12:56:52.300734+0000","last_clean":"2024-09-11T12:56:52.300734+0000","last_became_active":"2024-09-11T12:56:51.316048+0000","last_became_peered":"2024-09-11T12:56:51.316048+0000","last_unstale":"2024-09-11T12:56:52.300734+0000","last_undegraded":"2024-09-11T12:56:52.300734+0000","last_fullsized":"2024-09-11T12:56:52.300734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:51.344841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321336+0000","last_change":"2024-09-11T12:56:50.306743+0000","last_active":"2024-09-11T12:56:52.321336+0000","last_peered":"2024-09-11T12:56:52.321336+0000","last_clean":"2024-09-11T12:56:52.321336+0000","last_became_active":"2024-09-11T12:56:50.306472+0000","last_became_peered":"2024-09-11T12:56:50.306472+0000","last_unstale":"2024-09-11T12:56:52.321336+0000","last_undegraded":"2024-09-11T12:56:52.321336+0000","last_fullsized":"2024-09-11T12:56:52.321336+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:32.955887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.301024+0000","last_change":"2024-09-11T12:56:51.307726+0000","last_active":"2024-09-11T12:56:52.301024+0000","last_peered":"2024-09-11T12:56:52.301024+0000","last_clean":"2024-09-11T12:56:52.301024+0000","last_became_active":"2024-09-11T12:56:51.306896+0000","last_became_peered":"2024-09-11T12:56:51.306896+0000","last_unstale":"2024-09-11T12:56:52.301024+0000","last_undegraded":"2024-09-11T12:56:52.301024+0000","last_fullsized":"2024-09-11T12:56:52.301024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:21.819257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300730+0000","last_change":"2024-09-11T12:56:50.297814+0000","last_active":"2024-09-11T12:56:52.300730+0000","last_peered":"2024-09-11T12:56:52.300730+0000","last_clean":"2024-09-11T12:56:52.300730+0000","last_became_active":"2024-09-11T12:56:50.296767+0000","last_became_peered":"2024-09-11T12:56:50.296767+0000","last_unstale":"2024-09-11T12:56:52.300730+0000","last_undegraded":"2024-09-11T12:56:52.300730+0000","last_fullsized":"2024-09-11T12:56:52.300730+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:09.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800083+0000","last_change":"2024-09-11T12:56:51.325130+0000","last_active":"2024-09-11T12:56:52.800083+0000","last_peered":"2024-09-11T12:56:52.800083+0000","last_clean":"2024-09-11T12:56:52.800083+0000","last_became_active":"2024-09-11T12:56:51.324863+0000","last_became_peered":"2024-09-11T12:56:51.324863+0000","last_unstale":"2024-09-11T12:56:52.800083+0000","last_undegraded":"2024-09-11T12:56:52.800083+0000","last_fullsized":"2024-09-11T12:56:52.800083+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:56.211711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303888+0000","last_change":"2024-09-11T12:56:50.296255+0000","last_active":"2024-09-11T12:56:52.303888+0000","last_peered":"2024-09-11T12:56:52.303888+0000","last_clean":"2024-09-11T12:56:52.303888+0000","last_became_active":"2024-09-11T12:56:50.296043+0000","last_became_peered":"2024-09-11T12:56:50.296043+0000","last_unstale":"2024-09-11T12:56:52.303888+0000","last_undegraded":"2024-09-11T12:56:52.303888+0000","last_fullsized":"2024-09-11T12:56:52.303888+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:08.325587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800366+0000","last_change":"2024-09-11T12:56:51.328451+0000","last_active":"2024-09-11T12:56:52.800366+0000","last_peered":"2024-09-11T12:56:52.800366+0000","last_clean":"2024-09-11T12:56:52.800366+0000","last_became_active":"2024-09-11T12:56:51.326555+0000","last_became_peered":"2024-09-11T12:56:51.326555+0000","last_unstale":"2024-09-11T12:56:52.800366+0000","last_undegraded":"2024-09-11T12:56:52.800366+0000","last_fullsized":"2024-09-11T12:56:52.800366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:23.944687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303232+0000","last_change":"2024-09-11T12:56:50.289677+0000","last_active":"2024-09-11T12:56:52.303232+0000","last_peered":"2024-09-11T12:56:52.303232+0000","last_clean":"2024-09-11T12:56:52.303232+0000","last_became_active":"2024-09-11T12:56:50.289134+0000","last_became_peered":"2024-09-11T12:56:50.289134+0000","last_unstale":"2024-09-11T12:56:52.303232+0000","last_undegraded":"2024-09-11T12:56:52.303232+0000","last_fullsized":"2024-09-11T12:56:52.303232+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:32.513785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799631+0000","last_change":"2024-09-11T12:56:51.322508+0000","last_active":"2024-09-11T12:56:52.799631+0000","last_peered":"2024-09-11T12:56:52.799631+0000","last_clean":"2024-09-11T12:56:52.799631+0000","last_became_active":"2024-09-11T12:56:51.322012+0000","last_became_peered":"2024-09-11T12:56:51.322012+0000","last_unstale":"2024-09-11T12:56:52.799631+0000","last_undegraded":"2024-09-11T12:56:52.799631+0000","last_fullsized":"2024-09-11T12:56:52.799631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:42.264156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799817+0000","last_change":"2024-09-11T12:56:50.301293+0000","last_active":"2024-09-11T12:56:52.799817+0000","last_peered":"2024-09-11T12:56:52.799817+0000","last_clean":"2024-09-11T12:56:52.799817+0000","last_became_active":"2024-09-11T12:56:50.300991+0000","last_became_peered":"2024-09-11T12:56:50.300991+0000","last_unstale":"2024-09-11T12:56:52.799817+0000","last_undegraded":"2024-09-11T12:56:52.799817+0000","last_fullsized":"2024-09-11T12:56:52.799817+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:12.510525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797554+0000","last_change":"2024-09-11T12:56:51.317750+0000","last_active":"2024-09-11T12:56:52.797554+0000","last_peered":"2024-09-11T12:56:52.797554+0000","last_clean":"2024-09-11T12:56:52.797554+0000","last_became_active":"2024-09-11T12:56:51.317447+0000","last_became_peered":"2024-09-11T12:56:51.317447+0000","last_unstale":"2024-09-11T12:56:52.797554+0000","last_undegraded":"2024-09-11T12:56:52.797554+0000","last_fullsized":"2024-09-11T12:56:52.797554+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:42.979901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323429+0000","last_change":"2024-09-11T12:56:50.311418+0000","last_active":"2024-09-11T12:56:52.323429+0000","last_peered":"2024-09-11T12:56:52.323429+0000","last_clean":"2024-09-11T12:56:52.323429+0000","last_became_active":"2024-09-11T12:56:50.311129+0000","last_became_peered":"2024-09-11T12:56:50.311129+0000","last_unstale":"2024-09-11T12:56:52.323429+0000","last_undegraded":"2024-09-11T12:56:52.323429+0000","last_fullsized":"2024-09-11T12:56:52.323429+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:50.911770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320804+0000","last_change":"2024-09-11T12:56:51.320138+0000","last_active":"2024-09-11T12:56:52.320804+0000","last_peered":"2024-09-11T12:56:52.320804+0000","last_clean":"2024-09-11T12:56:52.320804+0000","last_became_active":"2024-09-11T12:56:51.319860+0000","last_became_peered":"2024-09-11T12:56:51.319860+0000","last_unstale":"2024-09-11T12:56:52.320804+0000","last_undegraded":"2024-09-11T12:56:52.320804+0000","last_fullsized":"2024-09-11T12:56:52.320804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:23.462856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799936+0000","last_change":"2024-09-11T12:56:50.303975+0000","last_active":"2024-09-11T12:56:52.799936+0000","last_peered":"2024-09-11T12:56:52.799936+0000","last_clean":"2024-09-11T12:56:52.799936+0000","last_became_active":"2024-09-11T12:56:50.303570+0000","last_became_peered":"2024-09-11T12:56:50.303570+0000","last_unstale":"2024-09-11T12:56:52.799936+0000","last_undegraded":"2024-09-11T12:56:52.799936+0000","last_fullsized":"2024-09-11T12:56:52.799936+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:44.652693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":372,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799951+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:52.799951+0000","last_peered":"2024-09-11T12:56:52.799951+0000","last_clean":"2024-09-11T12:56:52.799951+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:52.799951+0000","last_undegraded":"2024-09-11T12:56:52.799951+0000","last_fullsized":"2024-09-11T12:56:52.799951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37792,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334048,"statfs":{"total":96636764160,"available":96598065152,"internally_reserved":0,"allocated":2498560,"data_stored":1424041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509532,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1896448,"data_stored":833569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1896448,"data_stored":833569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1912832,"data_stored":835763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509532,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1908736,"data_stored":833763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1908736,"data_stored":834263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1908736,"data_stored":835181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542236,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2494464,"data_stored":1423959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:56:59.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:57:00.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 -- 172.21.15.78:0/3238750764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0314004d50 msgr2=0x7f03140051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 --2- 172.21.15.78:0/3238750764 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0314004d50 0x7f03140051d0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ec0 tx=0x7f032801d5d0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0335000640 1 -- 172.21.15.78:0/3238750764 reap_dead start 2024-09-11T12:57:00.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 -- 172.21.15.78:0/3238750764 shutdown_connections 2024-09-11T12:57:00.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 -- 172.21.15.78:0/3238750764 >> 172.21.15.78:0/3238750764 conn(0x7f033006ca80 msgr2=0x7f03300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:00.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 -- 172.21.15.78:0/3238750764 shutdown_connections 2024-09-11T12:57:00.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 -- 172.21.15.78:0/3238750764 wait complete. 2024-09-11T12:57:00.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.137+0000 7f0337000640 1 Processor -- start 2024-09-11T12:57:00.143 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- start start 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314004d50 0x7f03301e4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 0x7f033015e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b56b0 con 0x7f03301f3d90 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f03300b5530 con 0x7f03301560d0 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0330157ae0 con 0x7f0314004d50 2024-09-11T12:57:00.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46360/0 (socket says 172.21.15.78:46360) 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 -- 172.21.15.78:0/2601298245 learned_addr learned my addr 172.21.15.78:0/2601298245 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f032fe00640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314004d50 0x7f03301e4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f032f400640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 0x7f033015e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 -- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314004d50 msgr2=0x7f03301e4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314004d50 0x7f03301e4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 -- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 msgr2=0x7f033015e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 0x7f033015e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 -- 172.21.15.78:0/2601298245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03301f7450 con 0x7f03301f3d90 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f032f400640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 0x7f033015e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0335000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ec0 tx=0x7f0328009760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328009a90 con 0x7f03301f3d90 2024-09-11T12:57:00.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03280a94d0 con 0x7f03301f3d90 2024-09-11T12:57:00.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280a9cf0 con 0x7f03301f3d90 2024-09-11T12:57:00.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03301f7730 con 0x7f03301f3d90 2024-09-11T12:57:00.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03301f7cf0 con 0x7f03301f3d90 2024-09-11T12:57:00.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.141+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0330073b60 con 0x7f03301f3d90 2024-09-11T12:57:00.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.145+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f0328005040 con 0x7f03301f3d90 2024-09-11T12:57:00.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.145+0000 7f0327e00640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 0x7f0310074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.145+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f032812dc30 con 0x7f03301f3d90 2024-09-11T12:57:00.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.145+0000 7f032fe00640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 0x7f0310074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.145+0000 7f032fe00640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 0x7f0310074320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0320000c10 tx=0x7f0320006b20 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.149+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f03280c2030 con 0x7f03301f3d90 2024-09-11T12:57:00.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.309+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f03300695e0 con 0x7f0310071e60 2024-09-11T12:57:00.319 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.313+0000 7f0327e00640 1 -- 172.21.15.78:0/2601298245 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252480 (secure 0 0 0) 0x7f03300695e0 con 0x7f0310071e60 2024-09-11T12:57:00.321 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:00.323 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 msgr2=0x7f0310074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 0x7f0310074320 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f0320000c10 tx=0x7f0320006b20 comp rx=0 tx=0).stop 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 msgr2=0x7f03301591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f0328009ec0 tx=0x7f0328009760 comp rx=0 tx=0).stop 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 shutdown_connections 2024-09-11T12:57:00.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f03301f3d90 0x7f03301591f0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0310071e60 0x7f0310074320 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f03301560d0 0x7f033015e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 --2- 172.21.15.78:0/2601298245 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0314004d50 0x7f03301e4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 >> 172.21.15.78:0/2601298245 conn(0x7f033006ca80 msgr2=0x7f03300721a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 shutdown_connections 2024-09-11T12:57:00.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.317+0000 7f0337000640 1 -- 172.21.15.78:0/2601298245 wait complete. 2024-09-11T12:57:00.343 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-11T12:56:58.509903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":182,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":282232,"kb_used_data":16040,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754692488,"statfs":{"total":773094113280,"available":772805107712,"internally_reserved":0,"allocated":16424960,"data_stored":7854108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001874"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800594+0000","last_change":"2024-09-11T12:56:51.325800+0000","last_active":"2024-09-11T12:56:52.800594+0000","last_peered":"2024-09-11T12:56:52.800594+0000","last_clean":"2024-09-11T12:56:52.800594+0000","last_became_active":"2024-09-11T12:56:51.325534+0000","last_became_peered":"2024-09-11T12:56:51.325534+0000","last_unstale":"2024-09-11T12:56:52.800594+0000","last_undegraded":"2024-09-11T12:56:52.800594+0000","last_fullsized":"2024-09-11T12:56:52.800594+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:42:37.218816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799665+0000","last_change":"2024-09-11T12:56:51.323065+0000","last_active":"2024-09-11T12:56:52.799665+0000","last_peered":"2024-09-11T12:56:52.799665+0000","last_clean":"2024-09-11T12:56:52.799665+0000","last_became_active":"2024-09-11T12:56:51.322215+0000","last_became_peered":"2024-09-11T12:56:51.322215+0000","last_unstale":"2024-09-11T12:56:52.799665+0000","last_undegraded":"2024-09-11T12:56:52.799665+0000","last_fullsized":"2024-09-11T12:56:52.799665+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:12.506659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323538+0000","last_change":"2024-09-11T12:56:51.312717+0000","last_active":"2024-09-11T12:56:52.323538+0000","last_peered":"2024-09-11T12:56:52.323538+0000","last_clean":"2024-09-11T12:56:52.323538+0000","last_became_active":"2024-09-11T12:56:51.312438+0000","last_became_peered":"2024-09-11T12:56:51.312438+0000","last_unstale":"2024-09-11T12:56:52.323538+0000","last_undegraded":"2024-09-11T12:56:52.323538+0000","last_fullsized":"2024-09-11T12:56:52.323538+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:18.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800400+0000","last_change":"2024-09-11T12:56:51.328683+0000","last_active":"2024-09-11T12:56:52.800400+0000","last_peered":"2024-09-11T12:56:52.800400+0000","last_clean":"2024-09-11T12:56:52.800400+0000","last_became_active":"2024-09-11T12:56:51.328383+0000","last_became_peered":"2024-09-11T12:56:51.328383+0000","last_unstale":"2024-09-11T12:56:52.800400+0000","last_undegraded":"2024-09-11T12:56:52.800400+0000","last_fullsized":"2024-09-11T12:56:52.800400+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798092+0000","last_change":"2024-09-11T12:56:51.314989+0000","last_active":"2024-09-11T12:56:52.798092+0000","last_peered":"2024-09-11T12:56:52.798092+0000","last_clean":"2024-09-11T12:56:52.798092+0000","last_became_active":"2024-09-11T12:56:51.314693+0000","last_became_peered":"2024-09-11T12:56:51.314693+0000","last_unstale":"2024-09-11T12:56:52.798092+0000","last_undegraded":"2024-09-11T12:56:52.798092+0000","last_fullsized":"2024-09-11T12:56:52.798092+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:59:14.806808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323841+0000","last_change":"2024-09-11T12:56:51.322503+0000","last_active":"2024-09-11T12:56:52.323841+0000","last_peered":"2024-09-11T12:56:52.323841+0000","last_clean":"2024-09-11T12:56:52.323841+0000","last_became_active":"2024-09-11T12:56:51.322178+0000","last_became_peered":"2024-09-11T12:56:51.322178+0000","last_unstale":"2024-09-11T12:56:52.323841+0000","last_undegraded":"2024-09-11T12:56:52.323841+0000","last_fullsized":"2024-09-11T12:56:52.323841+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:36.089725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302972+0000","last_change":"2024-09-11T12:56:51.314700+0000","last_active":"2024-09-11T12:56:52.302972+0000","last_peered":"2024-09-11T12:56:52.302972+0000","last_clean":"2024-09-11T12:56:52.302972+0000","last_became_active":"2024-09-11T12:56:51.314451+0000","last_became_peered":"2024-09-11T12:56:51.314451+0000","last_unstale":"2024-09-11T12:56:52.302972+0000","last_undegraded":"2024-09-11T12:56:52.302972+0000","last_fullsized":"2024-09-11T12:56:52.302972+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:55.290264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300452+0000","last_change":"2024-09-11T12:56:51.306750+0000","last_active":"2024-09-11T12:56:52.300452+0000","last_peered":"2024-09-11T12:56:52.300452+0000","last_clean":"2024-09-11T12:56:52.300452+0000","last_became_active":"2024-09-11T12:56:51.306324+0000","last_became_peered":"2024-09-11T12:56:51.306324+0000","last_unstale":"2024-09-11T12:56:52.300452+0000","last_undegraded":"2024-09-11T12:56:52.300452+0000","last_fullsized":"2024-09-11T12:56:52.300452+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304279+0000","last_change":"2024-09-11T12:56:51.323990+0000","last_active":"2024-09-11T12:56:52.304279+0000","last_peered":"2024-09-11T12:56:52.304279+0000","last_clean":"2024-09-11T12:56:52.304279+0000","last_became_active":"2024-09-11T12:56:51.323693+0000","last_became_peered":"2024-09-11T12:56:51.323693+0000","last_unstale":"2024-09-11T12:56:52.304279+0000","last_undegraded":"2024-09-11T12:56:52.304279+0000","last_fullsized":"2024-09-11T12:56:52.304279+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:39:38.455546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321464+0000","last_change":"2024-09-11T12:56:51.323632+0000","last_active":"2024-09-11T12:56:52.321464+0000","last_peered":"2024-09-11T12:56:52.321464+0000","last_clean":"2024-09-11T12:56:52.321464+0000","last_became_active":"2024-09-11T12:56:51.322881+0000","last_became_peered":"2024-09-11T12:56:51.322881+0000","last_unstale":"2024-09-11T12:56:52.321464+0000","last_undegraded":"2024-09-11T12:56:52.321464+0000","last_fullsized":"2024-09-11T12:56:52.321464+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:55:00.102634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303354+0000","last_change":"2024-09-11T12:56:51.318511+0000","last_active":"2024-09-11T12:56:52.303354+0000","last_peered":"2024-09-11T12:56:52.303354+0000","last_clean":"2024-09-11T12:56:52.303354+0000","last_became_active":"2024-09-11T12:56:51.318236+0000","last_became_peered":"2024-09-11T12:56:51.318236+0000","last_unstale":"2024-09-11T12:56:52.303354+0000","last_undegraded":"2024-09-11T12:56:52.303354+0000","last_fullsized":"2024-09-11T12:56:52.303354+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:13.058353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800128+0000","last_change":"2024-09-11T12:56:51.320498+0000","last_active":"2024-09-11T12:56:52.800128+0000","last_peered":"2024-09-11T12:56:52.800128+0000","last_clean":"2024-09-11T12:56:52.800128+0000","last_became_active":"2024-09-11T12:56:51.320113+0000","last_became_peered":"2024-09-11T12:56:51.320113+0000","last_unstale":"2024-09-11T12:56:52.800128+0000","last_undegraded":"2024-09-11T12:56:52.800128+0000","last_fullsized":"2024-09-11T12:56:52.800128+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:29.453042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799646+0000","last_change":"2024-09-11T12:56:51.331608+0000","last_active":"2024-09-11T12:56:52.799646+0000","last_peered":"2024-09-11T12:56:52.799646+0000","last_clean":"2024-09-11T12:56:52.799646+0000","last_became_active":"2024-09-11T12:56:51.331170+0000","last_became_peered":"2024-09-11T12:56:51.331170+0000","last_unstale":"2024-09-11T12:56:52.799646+0000","last_undegraded":"2024-09-11T12:56:52.799646+0000","last_fullsized":"2024-09-11T12:56:52.799646+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:19.401069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302918+0000","last_change":"2024-09-11T12:56:51.314000+0000","last_active":"2024-09-11T12:56:52.302918+0000","last_peered":"2024-09-11T12:56:52.302918+0000","last_clean":"2024-09-11T12:56:52.302918+0000","last_became_active":"2024-09-11T12:56:51.313716+0000","last_became_peered":"2024-09-11T12:56:51.313716+0000","last_unstale":"2024-09-11T12:56:52.302918+0000","last_undegraded":"2024-09-11T12:56:52.302918+0000","last_fullsized":"2024-09-11T12:56:52.302918+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:00.097772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799766+0000","last_change":"2024-09-11T12:56:51.320626+0000","last_active":"2024-09-11T12:56:52.799766+0000","last_peered":"2024-09-11T12:56:52.799766+0000","last_clean":"2024-09-11T12:56:52.799766+0000","last_became_active":"2024-09-11T12:56:51.320268+0000","last_became_peered":"2024-09-11T12:56:51.320268+0000","last_unstale":"2024-09-11T12:56:52.799766+0000","last_undegraded":"2024-09-11T12:56:52.799766+0000","last_fullsized":"2024-09-11T12:56:52.799766+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:09.579181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300490+0000","last_change":"2024-09-11T12:56:51.307274+0000","last_active":"2024-09-11T12:56:52.300490+0000","last_peered":"2024-09-11T12:56:52.300490+0000","last_clean":"2024-09-11T12:56:52.300490+0000","last_became_active":"2024-09-11T12:56:51.307078+0000","last_became_peered":"2024-09-11T12:56:51.307078+0000","last_unstale":"2024-09-11T12:56:52.300490+0000","last_undegraded":"2024-09-11T12:56:52.300490+0000","last_fullsized":"2024-09-11T12:56:52.300490+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:36.089725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797989+0000","last_change":"2024-09-11T12:56:51.321448+0000","last_active":"2024-09-11T12:56:52.797989+0000","last_peered":"2024-09-11T12:56:52.797989+0000","last_clean":"2024-09-11T12:56:52.797989+0000","last_became_active":"2024-09-11T12:56:51.321101+0000","last_became_peered":"2024-09-11T12:56:51.321101+0000","last_unstale":"2024-09-11T12:56:52.797989+0000","last_undegraded":"2024-09-11T12:56:52.797989+0000","last_fullsized":"2024-09-11T12:56:52.797989+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:54:06.506829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800345+0000","last_change":"2024-09-11T12:56:51.322295+0000","last_active":"2024-09-11T12:56:52.800345+0000","last_peered":"2024-09-11T12:56:52.800345+0000","last_clean":"2024-09-11T12:56:52.800345+0000","last_became_active":"2024-09-11T12:56:51.322029+0000","last_became_peered":"2024-09-11T12:56:51.322029+0000","last_unstale":"2024-09-11T12:56:52.800345+0000","last_undegraded":"2024-09-11T12:56:52.800345+0000","last_fullsized":"2024-09-11T12:56:52.800345+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:16:47.295276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321278+0000","last_change":"2024-09-11T12:56:51.323737+0000","last_active":"2024-09-11T12:56:52.321278+0000","last_peered":"2024-09-11T12:56:52.321278+0000","last_clean":"2024-09-11T12:56:52.321278+0000","last_became_active":"2024-09-11T12:56:51.323259+0000","last_became_peered":"2024-09-11T12:56:51.323259+0000","last_unstale":"2024-09-11T12:56:52.321278+0000","last_undegraded":"2024-09-11T12:56:52.321278+0000","last_fullsized":"2024-09-11T12:56:52.321278+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:36.162519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797823+0000","last_change":"2024-09-11T12:56:51.322436+0000","last_active":"2024-09-11T12:56:52.797823+0000","last_peered":"2024-09-11T12:56:52.797823+0000","last_clean":"2024-09-11T12:56:52.797823+0000","last_became_active":"2024-09-11T12:56:51.322152+0000","last_became_peered":"2024-09-11T12:56:51.322152+0000","last_unstale":"2024-09-11T12:56:52.797823+0000","last_undegraded":"2024-09-11T12:56:52.797823+0000","last_fullsized":"2024-09-11T12:56:52.797823+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:04:27.249091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797752+0000","last_change":"2024-09-11T12:56:51.309729+0000","last_active":"2024-09-11T12:56:52.797752+0000","last_peered":"2024-09-11T12:56:52.797752+0000","last_clean":"2024-09-11T12:56:52.797752+0000","last_became_active":"2024-09-11T12:56:51.309458+0000","last_became_peered":"2024-09-11T12:56:51.309458+0000","last_unstale":"2024-09-11T12:56:52.797752+0000","last_undegraded":"2024-09-11T12:56:52.797752+0000","last_fullsized":"2024-09-11T12:56:52.797752+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:28:57.142651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320856+0000","last_change":"2024-09-11T12:56:51.327363+0000","last_active":"2024-09-11T12:56:52.320856+0000","last_peered":"2024-09-11T12:56:52.320856+0000","last_clean":"2024-09-11T12:56:52.320856+0000","last_became_active":"2024-09-11T12:56:51.327039+0000","last_became_peered":"2024-09-11T12:56:51.327039+0000","last_unstale":"2024-09-11T12:56:52.320856+0000","last_undegraded":"2024-09-11T12:56:52.320856+0000","last_fullsized":"2024-09-11T12:56:52.320856+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:00:39.941859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323443+0000","last_change":"2024-09-11T12:56:51.319865+0000","last_active":"2024-09-11T12:56:52.323443+0000","last_peered":"2024-09-11T12:56:52.323443+0000","last_clean":"2024-09-11T12:56:52.323443+0000","last_became_active":"2024-09-11T12:56:51.317358+0000","last_became_peered":"2024-09-11T12:56:51.317358+0000","last_unstale":"2024-09-11T12:56:52.323443+0000","last_undegraded":"2024-09-11T12:56:52.323443+0000","last_fullsized":"2024-09-11T12:56:52.323443+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:44:09.579181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300631+0000","last_change":"2024-09-11T12:56:51.315972+0000","last_active":"2024-09-11T12:56:52.300631+0000","last_peered":"2024-09-11T12:56:52.300631+0000","last_clean":"2024-09-11T12:56:52.300631+0000","last_became_active":"2024-09-11T12:56:51.315276+0000","last_became_peered":"2024-09-11T12:56:51.315276+0000","last_unstale":"2024-09-11T12:56:52.300631+0000","last_undegraded":"2024-09-11T12:56:52.300631+0000","last_fullsized":"2024-09-11T12:56:52.300631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:45.659768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799878+0000","last_change":"2024-09-11T12:56:51.325966+0000","last_active":"2024-09-11T12:56:52.799878+0000","last_peered":"2024-09-11T12:56:52.799878+0000","last_clean":"2024-09-11T12:56:52.799878+0000","last_became_active":"2024-09-11T12:56:51.325626+0000","last_became_peered":"2024-09-11T12:56:51.325626+0000","last_unstale":"2024-09-11T12:56:52.799878+0000","last_undegraded":"2024-09-11T12:56:52.799878+0000","last_fullsized":"2024-09-11T12:56:52.799878+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:39:18.866490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303623+0000","last_change":"2024-09-11T12:56:51.329476+0000","last_active":"2024-09-11T12:56:52.303623+0000","last_peered":"2024-09-11T12:56:52.303623+0000","last_clean":"2024-09-11T12:56:52.303623+0000","last_became_active":"2024-09-11T12:56:51.327136+0000","last_became_peered":"2024-09-11T12:56:51.327136+0000","last_unstale":"2024-09-11T12:56:52.303623+0000","last_undegraded":"2024-09-11T12:56:52.303623+0000","last_fullsized":"2024-09-11T12:56:52.303623+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:23.047576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323708+0000","last_change":"2024-09-11T12:56:51.322218+0000","last_active":"2024-09-11T12:56:52.323708+0000","last_peered":"2024-09-11T12:56:52.323708+0000","last_clean":"2024-09-11T12:56:52.323708+0000","last_became_active":"2024-09-11T12:56:51.321925+0000","last_became_peered":"2024-09-11T12:56:51.321925+0000","last_unstale":"2024-09-11T12:56:52.323708+0000","last_undegraded":"2024-09-11T12:56:52.323708+0000","last_fullsized":"2024-09-11T12:56:52.323708+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:05:29.453042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797910+0000","last_change":"2024-09-11T12:56:51.309714+0000","last_active":"2024-09-11T12:56:52.797910+0000","last_peered":"2024-09-11T12:56:52.797910+0000","last_clean":"2024-09-11T12:56:52.797910+0000","last_became_active":"2024-09-11T12:56:51.309402+0000","last_became_peered":"2024-09-11T12:56:51.309402+0000","last_unstale":"2024-09-11T12:56:52.797910+0000","last_undegraded":"2024-09-11T12:56:52.797910+0000","last_fullsized":"2024-09-11T12:56:52.797910+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:11:48.842364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799598+0000","last_change":"2024-09-11T12:56:51.323081+0000","last_active":"2024-09-11T12:56:52.799598+0000","last_peered":"2024-09-11T12:56:52.799598+0000","last_clean":"2024-09-11T12:56:52.799598+0000","last_became_active":"2024-09-11T12:56:51.322737+0000","last_became_peered":"2024-09-11T12:56:51.322737+0000","last_unstale":"2024-09-11T12:56:52.799598+0000","last_undegraded":"2024-09-11T12:56:52.799598+0000","last_fullsized":"2024-09-11T12:56:52.799598+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:19.076624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303123+0000","last_change":"2024-09-11T12:56:51.321697+0000","last_active":"2024-09-11T12:56:52.303123+0000","last_peered":"2024-09-11T12:56:52.303123+0000","last_clean":"2024-09-11T12:56:52.303123+0000","last_became_active":"2024-09-11T12:56:51.321385+0000","last_became_peered":"2024-09-11T12:56:51.321385+0000","last_unstale":"2024-09-11T12:56:52.303123+0000","last_undegraded":"2024-09-11T12:56:52.303123+0000","last_fullsized":"2024-09-11T12:56:52.303123+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:10:49.339713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323504+0000","last_change":"2024-09-11T12:56:51.332411+0000","last_active":"2024-09-11T12:56:52.323504+0000","last_peered":"2024-09-11T12:56:52.323504+0000","last_clean":"2024-09-11T12:56:52.323504+0000","last_became_active":"2024-09-11T12:56:51.331877+0000","last_became_peered":"2024-09-11T12:56:51.331877+0000","last_unstale":"2024-09-11T12:56:52.323504+0000","last_undegraded":"2024-09-11T12:56:52.323504+0000","last_fullsized":"2024-09-11T12:56:52.323504+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:50.219522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797484+0000","last_change":"2024-09-11T12:56:51.310078+0000","last_active":"2024-09-11T12:56:52.797484+0000","last_peered":"2024-09-11T12:56:52.797484+0000","last_clean":"2024-09-11T12:56:52.797484+0000","last_became_active":"2024-09-11T12:56:51.309774+0000","last_became_peered":"2024-09-11T12:56:51.309774+0000","last_unstale":"2024-09-11T12:56:52.797484+0000","last_undegraded":"2024-09-11T12:56:52.797484+0000","last_fullsized":"2024-09-11T12:56:52.797484+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:56:09.595235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300955+0000","last_change":"2024-09-11T12:56:51.307445+0000","last_active":"2024-09-11T12:56:52.300955+0000","last_peered":"2024-09-11T12:56:52.300955+0000","last_clean":"2024-09-11T12:56:52.300955+0000","last_became_active":"2024-09-11T12:56:51.307116+0000","last_became_peered":"2024-09-11T12:56:51.307116+0000","last_unstale":"2024-09-11T12:56:52.300955+0000","last_undegraded":"2024-09-11T12:56:52.300955+0000","last_fullsized":"2024-09-11T12:56:52.300955+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:18.820529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"150'4","reported_seq":19,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797963+0000","last_change":"2024-09-11T12:56:50.290099+0000","last_active":"2024-09-11T12:56:52.797963+0000","last_peered":"2024-09-11T12:56:52.797963+0000","last_clean":"2024-09-11T12:56:52.797963+0000","last_became_active":"2024-09-11T12:56:50.289571+0000","last_became_peered":"2024-09-11T12:56:50.289571+0000","last_unstale":"2024-09-11T12:56:52.797963+0000","last_undegraded":"2024-09-11T12:56:52.797963+0000","last_fullsized":"2024-09-11T12:56:52.797963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:40:53.672036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321489+0000","last_change":"2024-09-11T12:56:51.321133+0000","last_active":"2024-09-11T12:56:52.321489+0000","last_peered":"2024-09-11T12:56:52.321489+0000","last_clean":"2024-09-11T12:56:52.321489+0000","last_became_active":"2024-09-11T12:56:51.319696+0000","last_became_peered":"2024-09-11T12:56:51.319696+0000","last_unstale":"2024-09-11T12:56:52.321489+0000","last_undegraded":"2024-09-11T12:56:52.321489+0000","last_fullsized":"2024-09-11T12:56:52.321489+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:32:47.236732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304187+0000","last_change":"2024-09-11T12:56:50.305706+0000","last_active":"2024-09-11T12:56:52.304187+0000","last_peered":"2024-09-11T12:56:52.304187+0000","last_clean":"2024-09-11T12:56:52.304187+0000","last_became_active":"2024-09-11T12:56:50.303736+0000","last_became_peered":"2024-09-11T12:56:50.303736+0000","last_unstale":"2024-09-11T12:56:52.304187+0000","last_undegraded":"2024-09-11T12:56:52.304187+0000","last_fullsized":"2024-09-11T12:56:52.304187+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:16:26.722834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304075+0000","last_change":"2024-09-11T12:56:51.315464+0000","last_active":"2024-09-11T12:56:52.304075+0000","last_peered":"2024-09-11T12:56:52.304075+0000","last_clean":"2024-09-11T12:56:52.304075+0000","last_became_active":"2024-09-11T12:56:51.315149+0000","last_became_peered":"2024-09-11T12:56:51.315149+0000","last_unstale":"2024-09-11T12:56:52.304075+0000","last_undegraded":"2024-09-11T12:56:52.304075+0000","last_fullsized":"2024-09-11T12:56:52.304075+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:58:12.506659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323786+0000","last_change":"2024-09-11T12:56:50.306449+0000","last_active":"2024-09-11T12:56:52.323786+0000","last_peered":"2024-09-11T12:56:52.323786+0000","last_clean":"2024-09-11T12:56:52.323786+0000","last_became_active":"2024-09-11T12:56:50.306013+0000","last_became_peered":"2024-09-11T12:56:50.306013+0000","last_unstale":"2024-09-11T12:56:52.323786+0000","last_undegraded":"2024-09-11T12:56:52.323786+0000","last_fullsized":"2024-09-11T12:56:52.323786+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:54.283189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304002+0000","last_change":"2024-09-11T12:56:51.309155+0000","last_active":"2024-09-11T12:56:52.304002+0000","last_peered":"2024-09-11T12:56:52.304002+0000","last_clean":"2024-09-11T12:56:52.304002+0000","last_became_active":"2024-09-11T12:56:51.308914+0000","last_became_peered":"2024-09-11T12:56:51.308914+0000","last_unstale":"2024-09-11T12:56:52.304002+0000","last_undegraded":"2024-09-11T12:56:52.304002+0000","last_fullsized":"2024-09-11T12:56:52.304002+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:56:06.482630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800249+0000","last_change":"2024-09-11T12:56:50.302084+0000","last_active":"2024-09-11T12:56:52.800249+0000","last_peered":"2024-09-11T12:56:52.800249+0000","last_clean":"2024-09-11T12:56:52.800249+0000","last_became_active":"2024-09-11T12:56:50.301802+0000","last_became_peered":"2024-09-11T12:56:50.301802+0000","last_unstale":"2024-09-11T12:56:52.800249+0000","last_undegraded":"2024-09-11T12:56:52.800249+0000","last_fullsized":"2024-09-11T12:56:52.800249+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:48:23.661282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321008+0000","last_change":"2024-09-11T12:56:51.324291+0000","last_active":"2024-09-11T12:56:52.321008+0000","last_peered":"2024-09-11T12:56:52.321008+0000","last_clean":"2024-09-11T12:56:52.321008+0000","last_became_active":"2024-09-11T12:56:51.322861+0000","last_became_peered":"2024-09-11T12:56:51.322861+0000","last_unstale":"2024-09-11T12:56:52.321008+0000","last_undegraded":"2024-09-11T12:56:52.321008+0000","last_fullsized":"2024-09-11T12:56:52.321008+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:00.745036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797897+0000","last_change":"2024-09-11T12:56:50.289506+0000","last_active":"2024-09-11T12:56:52.797897+0000","last_peered":"2024-09-11T12:56:52.797897+0000","last_clean":"2024-09-11T12:56:52.797897+0000","last_became_active":"2024-09-11T12:56:50.289233+0000","last_became_peered":"2024-09-11T12:56:50.289233+0000","last_unstale":"2024-09-11T12:56:52.797897+0000","last_undegraded":"2024-09-11T12:56:52.797897+0000","last_fullsized":"2024-09-11T12:56:52.797897+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:47:10.824978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300697+0000","last_change":"2024-09-11T12:56:51.308409+0000","last_active":"2024-09-11T12:56:52.300697+0000","last_peered":"2024-09-11T12:56:52.300697+0000","last_clean":"2024-09-11T12:56:52.300697+0000","last_became_active":"2024-09-11T12:56:51.308123+0000","last_became_peered":"2024-09-11T12:56:51.308123+0000","last_unstale":"2024-09-11T12:56:52.300697+0000","last_undegraded":"2024-09-11T12:56:52.300697+0000","last_fullsized":"2024-09-11T12:56:52.300697+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:06.109961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300536+0000","last_change":"2024-09-11T12:56:50.305183+0000","last_active":"2024-09-11T12:56:52.300536+0000","last_peered":"2024-09-11T12:56:52.300536+0000","last_clean":"2024-09-11T12:56:52.300536+0000","last_became_active":"2024-09-11T12:56:50.304940+0000","last_became_peered":"2024-09-11T12:56:50.304940+0000","last_unstale":"2024-09-11T12:56:52.300536+0000","last_undegraded":"2024-09-11T12:56:52.300536+0000","last_fullsized":"2024-09-11T12:56:52.300536+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:16:22.040500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303185+0000","last_change":"2024-09-11T12:56:51.318619+0000","last_active":"2024-09-11T12:56:52.303185+0000","last_peered":"2024-09-11T12:56:52.303185+0000","last_clean":"2024-09-11T12:56:52.303185+0000","last_became_active":"2024-09-11T12:56:51.318318+0000","last_became_peered":"2024-09-11T12:56:51.318318+0000","last_unstale":"2024-09-11T12:56:52.303185+0000","last_undegraded":"2024-09-11T12:56:52.303185+0000","last_fullsized":"2024-09-11T12:56:52.303185+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:10.975605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799362+0000","last_change":"2024-09-11T12:56:50.308680+0000","last_active":"2024-09-11T12:56:52.799362+0000","last_peered":"2024-09-11T12:56:52.799362+0000","last_clean":"2024-09-11T12:56:52.799362+0000","last_became_active":"2024-09-11T12:56:50.308390+0000","last_became_peered":"2024-09-11T12:56:50.308390+0000","last_unstale":"2024-09-11T12:56:52.799362+0000","last_undegraded":"2024-09-11T12:56:52.799362+0000","last_fullsized":"2024-09-11T12:56:52.799362+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:16.923224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323799+0000","last_change":"2024-09-11T12:56:51.325404+0000","last_active":"2024-09-11T12:56:52.323799+0000","last_peered":"2024-09-11T12:56:52.323799+0000","last_clean":"2024-09-11T12:56:52.323799+0000","last_became_active":"2024-09-11T12:56:51.325127+0000","last_became_peered":"2024-09-11T12:56:51.325127+0000","last_unstale":"2024-09-11T12:56:52.323799+0000","last_undegraded":"2024-09-11T12:56:52.323799+0000","last_fullsized":"2024-09-11T12:56:52.323799+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:17.930299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303558+0000","last_change":"2024-09-11T12:56:50.292337+0000","last_active":"2024-09-11T12:56:52.303558+0000","last_peered":"2024-09-11T12:56:52.303558+0000","last_clean":"2024-09-11T12:56:52.303558+0000","last_became_active":"2024-09-11T12:56:50.292057+0000","last_became_peered":"2024-09-11T12:56:50.292057+0000","last_unstale":"2024-09-11T12:56:52.303558+0000","last_undegraded":"2024-09-11T12:56:52.303558+0000","last_fullsized":"2024-09-11T12:56:52.303558+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:10:19.743245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300862+0000","last_change":"2024-09-11T12:56:51.321692+0000","last_active":"2024-09-11T12:56:52.300862+0000","last_peered":"2024-09-11T12:56:52.300862+0000","last_clean":"2024-09-11T12:56:52.300862+0000","last_became_active":"2024-09-11T12:56:51.321415+0000","last_became_peered":"2024-09-11T12:56:51.321415+0000","last_unstale":"2024-09-11T12:56:52.300862+0000","last_undegraded":"2024-09-11T12:56:52.300862+0000","last_fullsized":"2024-09-11T12:56:52.300862+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:18.097421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798012+0000","last_change":"2024-09-11T12:56:50.300136+0000","last_active":"2024-09-11T12:56:52.798012+0000","last_peered":"2024-09-11T12:56:52.798012+0000","last_clean":"2024-09-11T12:56:52.798012+0000","last_became_active":"2024-09-11T12:56:50.299878+0000","last_became_peered":"2024-09-11T12:56:50.299878+0000","last_unstale":"2024-09-11T12:56:52.798012+0000","last_undegraded":"2024-09-11T12:56:52.798012+0000","last_fullsized":"2024-09-11T12:56:52.798012+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:22:03.290036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303822+0000","last_change":"2024-09-11T12:56:51.308310+0000","last_active":"2024-09-11T12:56:52.303822+0000","last_peered":"2024-09-11T12:56:52.303822+0000","last_clean":"2024-09-11T12:56:52.303822+0000","last_became_active":"2024-09-11T12:56:51.307974+0000","last_became_peered":"2024-09-11T12:56:51.307974+0000","last_unstale":"2024-09-11T12:56:52.303822+0000","last_undegraded":"2024-09-11T12:56:52.303822+0000","last_fullsized":"2024-09-11T12:56:52.303822+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:39:19.076624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"150'3","reported_seq":18,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797808+0000","last_change":"2024-09-11T12:56:50.302419+0000","last_active":"2024-09-11T12:56:52.797808+0000","last_peered":"2024-09-11T12:56:52.797808+0000","last_clean":"2024-09-11T12:56:52.797808+0000","last_became_active":"2024-09-11T12:56:50.302121+0000","last_became_peered":"2024-09-11T12:56:50.302121+0000","last_unstale":"2024-09-11T12:56:52.797808+0000","last_undegraded":"2024-09-11T12:56:52.797808+0000","last_fullsized":"2024-09-11T12:56:52.797808+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:42:04.424616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321351+0000","last_change":"2024-09-11T12:56:51.323861+0000","last_active":"2024-09-11T12:56:52.321351+0000","last_peered":"2024-09-11T12:56:52.321351+0000","last_clean":"2024-09-11T12:56:52.321351+0000","last_became_active":"2024-09-11T12:56:51.323610+0000","last_became_peered":"2024-09-11T12:56:51.323610+0000","last_unstale":"2024-09-11T12:56:52.321351+0000","last_undegraded":"2024-09-11T12:56:52.321351+0000","last_fullsized":"2024-09-11T12:56:52.321351+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:48:39.091060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"150'6","reported_seq":21,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323678+0000","last_change":"2024-09-11T12:56:50.306460+0000","last_active":"2024-09-11T12:56:52.323678+0000","last_peered":"2024-09-11T12:56:52.323678+0000","last_clean":"2024-09-11T12:56:52.323678+0000","last_became_active":"2024-09-11T12:56:50.306176+0000","last_became_peered":"2024-09-11T12:56:50.306176+0000","last_unstale":"2024-09-11T12:56:52.323678+0000","last_undegraded":"2024-09-11T12:56:52.323678+0000","last_fullsized":"2024-09-11T12:56:52.323678+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:57:34.597272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323539+0000","last_change":"2024-09-11T12:56:51.316949+0000","last_active":"2024-09-11T12:56:52.323539+0000","last_peered":"2024-09-11T12:56:52.323539+0000","last_clean":"2024-09-11T12:56:52.323539+0000","last_became_active":"2024-09-11T12:56:51.316608+0000","last_became_peered":"2024-09-11T12:56:51.316608+0000","last_unstale":"2024-09-11T12:56:52.323539+0000","last_undegraded":"2024-09-11T12:56:52.323539+0000","last_fullsized":"2024-09-11T12:56:52.323539+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:19.401069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800398+0000","last_change":"2024-09-11T12:56:50.309887+0000","last_active":"2024-09-11T12:56:52.800398+0000","last_peered":"2024-09-11T12:56:52.800398+0000","last_clean":"2024-09-11T12:56:52.800398+0000","last_became_active":"2024-09-11T12:56:50.309619+0000","last_became_peered":"2024-09-11T12:56:50.309619+0000","last_unstale":"2024-09-11T12:56:52.800398+0000","last_undegraded":"2024-09-11T12:56:52.800398+0000","last_fullsized":"2024-09-11T12:56:52.800398+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:05.102886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303717+0000","last_change":"2024-09-11T12:56:51.315637+0000","last_active":"2024-09-11T12:56:52.303717+0000","last_peered":"2024-09-11T12:56:52.303717+0000","last_clean":"2024-09-11T12:56:52.303717+0000","last_became_active":"2024-09-11T12:56:51.313997+0000","last_became_peered":"2024-09-11T12:56:51.313997+0000","last_unstale":"2024-09-11T12:56:52.303717+0000","last_undegraded":"2024-09-11T12:56:52.303717+0000","last_fullsized":"2024-09-11T12:56:52.303717+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:41.783318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321186+0000","last_change":"2024-09-11T12:56:50.298086+0000","last_active":"2024-09-11T12:56:52.321186+0000","last_peered":"2024-09-11T12:56:52.321186+0000","last_clean":"2024-09-11T12:56:52.321186+0000","last_became_active":"2024-09-11T12:56:50.297806+0000","last_became_peered":"2024-09-11T12:56:50.297806+0000","last_unstale":"2024-09-11T12:56:52.321186+0000","last_undegraded":"2024-09-11T12:56:52.321186+0000","last_fullsized":"2024-09-11T12:56:52.321186+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:19:10.786582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.302960+0000","last_change":"2024-09-11T12:56:51.314477+0000","last_active":"2024-09-11T12:56:52.302960+0000","last_peered":"2024-09-11T12:56:52.302960+0000","last_clean":"2024-09-11T12:56:52.302960+0000","last_became_active":"2024-09-11T12:56:51.314217+0000","last_became_peered":"2024-09-11T12:56:51.314217+0000","last_unstale":"2024-09-11T12:56:52.302960+0000","last_undegraded":"2024-09-11T12:56:52.302960+0000","last_fullsized":"2024-09-11T12:56:52.302960+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:48:29.509403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300348+0000","last_change":"2024-09-11T12:56:50.299386+0000","last_active":"2024-09-11T12:56:52.300348+0000","last_peered":"2024-09-11T12:56:52.300348+0000","last_clean":"2024-09-11T12:56:52.300348+0000","last_became_active":"2024-09-11T12:56:50.299158+0000","last_became_peered":"2024-09-11T12:56:50.299158+0000","last_unstale":"2024-09-11T12:56:52.300348+0000","last_undegraded":"2024-09-11T12:56:52.300348+0000","last_fullsized":"2024-09-11T12:56:52.300348+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:07:54.283189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320803+0000","last_change":"2024-09-11T12:56:51.326787+0000","last_active":"2024-09-11T12:56:52.320803+0000","last_peered":"2024-09-11T12:56:52.320803+0000","last_clean":"2024-09-11T12:56:52.320803+0000","last_became_active":"2024-09-11T12:56:51.325559+0000","last_became_peered":"2024-09-11T12:56:51.325559+0000","last_unstale":"2024-09-11T12:56:52.320803+0000","last_undegraded":"2024-09-11T12:56:52.320803+0000","last_fullsized":"2024-09-11T12:56:52.320803+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:05:05.908814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"150'8","reported_seq":23,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799371+0000","last_change":"2024-09-11T12:56:50.308722+0000","last_active":"2024-09-11T12:56:52.799371+0000","last_peered":"2024-09-11T12:56:52.799371+0000","last_clean":"2024-09-11T12:56:52.799371+0000","last_became_active":"2024-09-11T12:56:50.308431+0000","last_became_peered":"2024-09-11T12:56:50.308431+0000","last_unstale":"2024-09-11T12:56:52.799371+0000","last_undegraded":"2024-09-11T12:56:52.799371+0000","last_fullsized":"2024-09-11T12:56:52.799371+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:24:35.082649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321540+0000","last_change":"2024-09-11T12:56:51.312223+0000","last_active":"2024-09-11T12:56:52.321540+0000","last_peered":"2024-09-11T12:56:52.321540+0000","last_clean":"2024-09-11T12:56:52.321540+0000","last_became_active":"2024-09-11T12:56:51.311973+0000","last_became_peered":"2024-09-11T12:56:51.311973+0000","last_unstale":"2024-09-11T12:56:52.321540+0000","last_undegraded":"2024-09-11T12:56:52.321540+0000","last_fullsized":"2024-09-11T12:56:52.321540+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:31:39.565642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.798055+0000","last_change":"2024-09-11T12:56:50.289863+0000","last_active":"2024-09-11T12:56:52.798055+0000","last_peered":"2024-09-11T12:56:52.798055+0000","last_clean":"2024-09-11T12:56:52.798055+0000","last_became_active":"2024-09-11T12:56:50.289217+0000","last_became_peered":"2024-09-11T12:56:50.289217+0000","last_unstale":"2024-09-11T12:56:52.798055+0000","last_undegraded":"2024-09-11T12:56:52.798055+0000","last_fullsized":"2024-09-11T12:56:52.798055+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:03:52.682768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.304319+0000","last_change":"2024-09-11T12:56:51.327503+0000","last_active":"2024-09-11T12:56:52.304319+0000","last_peered":"2024-09-11T12:56:52.304319+0000","last_clean":"2024-09-11T12:56:52.304319+0000","last_became_active":"2024-09-11T12:56:51.327198+0000","last_became_peered":"2024-09-11T12:56:51.327198+0000","last_unstale":"2024-09-11T12:56:52.304319+0000","last_undegraded":"2024-09-11T12:56:52.304319+0000","last_fullsized":"2024-09-11T12:56:52.304319+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:48:05.531019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300963+0000","last_change":"2024-09-11T12:56:50.302013+0000","last_active":"2024-09-11T12:56:52.300963+0000","last_peered":"2024-09-11T12:56:52.300963+0000","last_clean":"2024-09-11T12:56:52.300963+0000","last_became_active":"2024-09-11T12:56:50.299221+0000","last_became_peered":"2024-09-11T12:56:50.299221+0000","last_unstale":"2024-09-11T12:56:52.300963+0000","last_undegraded":"2024-09-11T12:56:52.300963+0000","last_fullsized":"2024-09-11T12:56:52.300963+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:17:12.051278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303663+0000","last_change":"2024-09-11T12:56:51.319142+0000","last_active":"2024-09-11T12:56:52.303663+0000","last_peered":"2024-09-11T12:56:52.303663+0000","last_clean":"2024-09-11T12:56:52.303663+0000","last_became_active":"2024-09-11T12:56:51.318865+0000","last_became_peered":"2024-09-11T12:56:51.318865+0000","last_unstale":"2024-09-11T12:56:52.303663+0000","last_undegraded":"2024-09-11T12:56:52.303663+0000","last_fullsized":"2024-09-11T12:56:52.303663+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:33:06.964539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"150'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800166+0000","last_change":"2024-09-11T12:56:50.311439+0000","last_active":"2024-09-11T12:56:52.800166+0000","last_peered":"2024-09-11T12:56:52.800166+0000","last_clean":"2024-09-11T12:56:52.800166+0000","last_became_active":"2024-09-11T12:56:50.311186+0000","last_became_peered":"2024-09-11T12:56:50.311186+0000","last_unstale":"2024-09-11T12:56:52.800166+0000","last_undegraded":"2024-09-11T12:56:52.800166+0000","last_fullsized":"2024-09-11T12:56:52.800166+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:17.090345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799791+0000","last_change":"2024-09-11T12:56:51.329352+0000","last_active":"2024-09-11T12:56:52.799791+0000","last_peered":"2024-09-11T12:56:52.799791+0000","last_clean":"2024-09-11T12:56:52.799791+0000","last_became_active":"2024-09-11T12:56:51.327853+0000","last_became_peered":"2024-09-11T12:56:51.327853+0000","last_unstale":"2024-09-11T12:56:52.799791+0000","last_undegraded":"2024-09-11T12:56:52.799791+0000","last_fullsized":"2024-09-11T12:56:52.799791+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:13:50.219522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303923+0000","last_change":"2024-09-11T12:56:50.297243+0000","last_active":"2024-09-11T12:56:52.303923+0000","last_peered":"2024-09-11T12:56:52.303923+0000","last_clean":"2024-09-11T12:56:52.303923+0000","last_became_active":"2024-09-11T12:56:50.296921+0000","last_became_peered":"2024-09-11T12:56:50.296921+0000","last_unstale":"2024-09-11T12:56:52.303923+0000","last_undegraded":"2024-09-11T12:56:52.303923+0000","last_fullsized":"2024-09-11T12:56:52.303923+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:55.224946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321061+0000","last_change":"2024-09-11T12:56:51.310844+0000","last_active":"2024-09-11T12:56:52.321061+0000","last_peered":"2024-09-11T12:56:52.321061+0000","last_clean":"2024-09-11T12:56:52.321061+0000","last_became_active":"2024-09-11T12:56:51.310553+0000","last_became_peered":"2024-09-11T12:56:51.310553+0000","last_unstale":"2024-09-11T12:56:52.321061+0000","last_undegraded":"2024-09-11T12:56:52.321061+0000","last_fullsized":"2024-09-11T12:56:52.321061+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:09:40.902725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323402+0000","last_change":"2024-09-11T12:56:50.309614+0000","last_active":"2024-09-11T12:56:52.323402+0000","last_peered":"2024-09-11T12:56:52.323402+0000","last_clean":"2024-09-11T12:56:52.323402+0000","last_became_active":"2024-09-11T12:56:50.309309+0000","last_became_peered":"2024-09-11T12:56:50.309309+0000","last_unstale":"2024-09-11T12:56:52.323402+0000","last_undegraded":"2024-09-11T12:56:52.323402+0000","last_fullsized":"2024-09-11T12:56:52.323402+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:11:51.352295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303899+0000","last_change":"2024-09-11T12:56:51.308171+0000","last_active":"2024-09-11T12:56:52.303899+0000","last_peered":"2024-09-11T12:56:52.303899+0000","last_clean":"2024-09-11T12:56:52.303899+0000","last_became_active":"2024-09-11T12:56:51.307772+0000","last_became_peered":"2024-09-11T12:56:51.307772+0000","last_unstale":"2024-09-11T12:56:52.303899+0000","last_undegraded":"2024-09-11T12:56:52.303899+0000","last_fullsized":"2024-09-11T12:56:52.303899+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:42.264156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797642+0000","last_change":"2024-09-11T12:56:50.297571+0000","last_active":"2024-09-11T12:56:52.797642+0000","last_peered":"2024-09-11T12:56:52.797642+0000","last_clean":"2024-09-11T12:56:52.797642+0000","last_became_active":"2024-09-11T12:56:50.297307+0000","last_became_peered":"2024-09-11T12:56:50.297307+0000","last_unstale":"2024-09-11T12:56:52.797642+0000","last_undegraded":"2024-09-11T12:56:52.797642+0000","last_fullsized":"2024-09-11T12:56:52.797642+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:48:51.965420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799437+0000","last_change":"2024-09-11T12:56:51.323087+0000","last_active":"2024-09-11T12:56:52.799437+0000","last_peered":"2024-09-11T12:56:52.799437+0000","last_clean":"2024-09-11T12:56:52.799437+0000","last_became_active":"2024-09-11T12:56:51.322826+0000","last_became_peered":"2024-09-11T12:56:51.322826+0000","last_unstale":"2024-09-11T12:56:52.799437+0000","last_undegraded":"2024-09-11T12:56:52.799437+0000","last_fullsized":"2024-09-11T12:56:52.799437+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:29:41.783318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323586+0000","last_change":"2024-09-11T12:56:50.309534+0000","last_active":"2024-09-11T12:56:52.323586+0000","last_peered":"2024-09-11T12:56:52.323586+0000","last_clean":"2024-09-11T12:56:52.323586+0000","last_became_active":"2024-09-11T12:56:50.309270+0000","last_became_peered":"2024-09-11T12:56:50.309270+0000","last_unstale":"2024-09-11T12:56:52.323586+0000","last_undegraded":"2024-09-11T12:56:52.323586+0000","last_fullsized":"2024-09-11T12:56:52.323586+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:25:36.222739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797520+0000","last_change":"2024-09-11T12:56:51.330131+0000","last_active":"2024-09-11T12:56:52.797520+0000","last_peered":"2024-09-11T12:56:52.797520+0000","last_clean":"2024-09-11T12:56:52.797520+0000","last_became_active":"2024-09-11T12:56:51.329855+0000","last_became_peered":"2024-09-11T12:56:51.329855+0000","last_unstale":"2024-09-11T12:56:52.797520+0000","last_undegraded":"2024-09-11T12:56:52.797520+0000","last_fullsized":"2024-09-11T12:56:52.797520+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:45:21.554037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303092+0000","last_change":"2024-09-11T12:56:50.295252+0000","last_active":"2024-09-11T12:56:52.303092+0000","last_peered":"2024-09-11T12:56:52.303092+0000","last_clean":"2024-09-11T12:56:52.303092+0000","last_became_active":"2024-09-11T12:56:50.294990+0000","last_became_peered":"2024-09-11T12:56:50.294990+0000","last_unstale":"2024-09-11T12:56:52.303092+0000","last_undegraded":"2024-09-11T12:56:52.303092+0000","last_fullsized":"2024-09-11T12:56:52.303092+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:39:07.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797893+0000","last_change":"2024-09-11T12:56:51.319122+0000","last_active":"2024-09-11T12:56:52.797893+0000","last_peered":"2024-09-11T12:56:52.797893+0000","last_clean":"2024-09-11T12:56:52.797893+0000","last_became_active":"2024-09-11T12:56:51.318832+0000","last_became_peered":"2024-09-11T12:56:51.318832+0000","last_unstale":"2024-09-11T12:56:52.797893+0000","last_undegraded":"2024-09-11T12:56:52.797893+0000","last_fullsized":"2024-09-11T12:56:52.797893+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:25:39.391737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"150'2","reported_seq":17,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800669+0000","last_change":"2024-09-11T12:56:50.307695+0000","last_active":"2024-09-11T12:56:52.800669+0000","last_peered":"2024-09-11T12:56:52.800669+0000","last_clean":"2024-09-11T12:56:52.800669+0000","last_became_active":"2024-09-11T12:56:50.307465+0000","last_became_peered":"2024-09-11T12:56:50.307465+0000","last_unstale":"2024-09-11T12:56:52.800669+0000","last_undegraded":"2024-09-11T12:56:52.800669+0000","last_fullsized":"2024-09-11T12:56:52.800669+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:50.337765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321645+0000","last_change":"2024-09-11T12:56:51.323587+0000","last_active":"2024-09-11T12:56:52.321645+0000","last_peered":"2024-09-11T12:56:52.321645+0000","last_clean":"2024-09-11T12:56:52.321645+0000","last_became_active":"2024-09-11T12:56:51.323303+0000","last_became_peered":"2024-09-11T12:56:51.323303+0000","last_unstale":"2024-09-11T12:56:52.321645+0000","last_undegraded":"2024-09-11T12:56:52.321645+0000","last_fullsized":"2024-09-11T12:56:52.321645+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:27:34.683781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"151'1","reported_seq":16,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321541+0000","last_change":"2024-09-11T12:56:50.303866+0000","last_active":"2024-09-11T12:56:52.321541+0000","last_peered":"2024-09-11T12:56:52.321541+0000","last_clean":"2024-09-11T12:56:52.321541+0000","last_became_active":"2024-09-11T12:56:50.303577+0000","last_became_peered":"2024-09-11T12:56:50.303577+0000","last_unstale":"2024-09-11T12:56:52.321541+0000","last_undegraded":"2024-09-11T12:56:52.321541+0000","last_fullsized":"2024-09-11T12:56:52.321541+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:44:05.285265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300734+0000","last_change":"2024-09-11T12:56:51.316304+0000","last_active":"2024-09-11T12:56:52.300734+0000","last_peered":"2024-09-11T12:56:52.300734+0000","last_clean":"2024-09-11T12:56:52.300734+0000","last_became_active":"2024-09-11T12:56:51.316048+0000","last_became_peered":"2024-09-11T12:56:51.316048+0000","last_unstale":"2024-09-11T12:56:52.300734+0000","last_undegraded":"2024-09-11T12:56:52.300734+0000","last_fullsized":"2024-09-11T12:56:52.300734+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:15:51.344841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.321336+0000","last_change":"2024-09-11T12:56:50.306743+0000","last_active":"2024-09-11T12:56:52.321336+0000","last_peered":"2024-09-11T12:56:52.321336+0000","last_clean":"2024-09-11T12:56:52.321336+0000","last_became_active":"2024-09-11T12:56:50.306472+0000","last_became_peered":"2024-09-11T12:56:50.306472+0000","last_unstale":"2024-09-11T12:56:52.321336+0000","last_undegraded":"2024-09-11T12:56:52.321336+0000","last_fullsized":"2024-09-11T12:56:52.321336+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:58:32.955887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.301024+0000","last_change":"2024-09-11T12:56:51.307726+0000","last_active":"2024-09-11T12:56:52.301024+0000","last_peered":"2024-09-11T12:56:52.301024+0000","last_clean":"2024-09-11T12:56:52.301024+0000","last_became_active":"2024-09-11T12:56:51.306896+0000","last_became_peered":"2024-09-11T12:56:51.306896+0000","last_unstale":"2024-09-11T12:56:52.301024+0000","last_undegraded":"2024-09-11T12:56:52.301024+0000","last_fullsized":"2024-09-11T12:56:52.301024+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:29:21.819257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.300730+0000","last_change":"2024-09-11T12:56:50.297814+0000","last_active":"2024-09-11T12:56:52.300730+0000","last_peered":"2024-09-11T12:56:52.300730+0000","last_clean":"2024-09-11T12:56:52.300730+0000","last_became_active":"2024-09-11T12:56:50.296767+0000","last_became_peered":"2024-09-11T12:56:50.296767+0000","last_unstale":"2024-09-11T12:56:52.300730+0000","last_undegraded":"2024-09-11T12:56:52.300730+0000","last_fullsized":"2024-09-11T12:56:52.300730+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:09.968529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800083+0000","last_change":"2024-09-11T12:56:51.325130+0000","last_active":"2024-09-11T12:56:52.800083+0000","last_peered":"2024-09-11T12:56:52.800083+0000","last_clean":"2024-09-11T12:56:52.800083+0000","last_became_active":"2024-09-11T12:56:51.324863+0000","last_became_peered":"2024-09-11T12:56:51.324863+0000","last_unstale":"2024-09-11T12:56:52.800083+0000","last_undegraded":"2024-09-11T12:56:52.800083+0000","last_fullsized":"2024-09-11T12:56:52.800083+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:20:56.211711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303888+0000","last_change":"2024-09-11T12:56:50.296255+0000","last_active":"2024-09-11T12:56:52.303888+0000","last_peered":"2024-09-11T12:56:52.303888+0000","last_clean":"2024-09-11T12:56:52.303888+0000","last_became_active":"2024-09-11T12:56:50.296043+0000","last_became_peered":"2024-09-11T12:56:50.296043+0000","last_unstale":"2024-09-11T12:56:52.303888+0000","last_undegraded":"2024-09-11T12:56:52.303888+0000","last_fullsized":"2024-09-11T12:56:52.303888+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:08.325587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.800366+0000","last_change":"2024-09-11T12:56:51.328451+0000","last_active":"2024-09-11T12:56:52.800366+0000","last_peered":"2024-09-11T12:56:52.800366+0000","last_clean":"2024-09-11T12:56:52.800366+0000","last_became_active":"2024-09-11T12:56:51.326555+0000","last_became_peered":"2024-09-11T12:56:51.326555+0000","last_unstale":"2024-09-11T12:56:52.800366+0000","last_undegraded":"2024-09-11T12:56:52.800366+0000","last_fullsized":"2024-09-11T12:56:52.800366+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:18:23.944687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.303232+0000","last_change":"2024-09-11T12:56:50.289677+0000","last_active":"2024-09-11T12:56:52.303232+0000","last_peered":"2024-09-11T12:56:52.303232+0000","last_clean":"2024-09-11T12:56:52.303232+0000","last_became_active":"2024-09-11T12:56:50.289134+0000","last_became_peered":"2024-09-11T12:56:50.289134+0000","last_unstale":"2024-09-11T12:56:52.303232+0000","last_undegraded":"2024-09-11T12:56:52.303232+0000","last_fullsized":"2024-09-11T12:56:52.303232+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:42:32.513785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799631+0000","last_change":"2024-09-11T12:56:51.322508+0000","last_active":"2024-09-11T12:56:52.799631+0000","last_peered":"2024-09-11T12:56:52.799631+0000","last_clean":"2024-09-11T12:56:52.799631+0000","last_became_active":"2024-09-11T12:56:51.322012+0000","last_became_peered":"2024-09-11T12:56:51.322012+0000","last_unstale":"2024-09-11T12:56:52.799631+0000","last_undegraded":"2024-09-11T12:56:52.799631+0000","last_fullsized":"2024-09-11T12:56:52.799631+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:42.264156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799817+0000","last_change":"2024-09-11T12:56:50.301293+0000","last_active":"2024-09-11T12:56:52.799817+0000","last_peered":"2024-09-11T12:56:52.799817+0000","last_clean":"2024-09-11T12:56:52.799817+0000","last_became_active":"2024-09-11T12:56:50.300991+0000","last_became_peered":"2024-09-11T12:56:50.300991+0000","last_unstale":"2024-09-11T12:56:52.799817+0000","last_undegraded":"2024-09-11T12:56:52.799817+0000","last_fullsized":"2024-09-11T12:56:52.799817+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:12.510525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.797554+0000","last_change":"2024-09-11T12:56:51.317750+0000","last_active":"2024-09-11T12:56:52.797554+0000","last_peered":"2024-09-11T12:56:52.797554+0000","last_clean":"2024-09-11T12:56:52.797554+0000","last_became_active":"2024-09-11T12:56:51.317447+0000","last_became_peered":"2024-09-11T12:56:51.317447+0000","last_unstale":"2024-09-11T12:56:52.797554+0000","last_undegraded":"2024-09-11T12:56:52.797554+0000","last_fullsized":"2024-09-11T12:56:52.797554+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:29:42.979901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.323429+0000","last_change":"2024-09-11T12:56:50.311418+0000","last_active":"2024-09-11T12:56:52.323429+0000","last_peered":"2024-09-11T12:56:52.323429+0000","last_clean":"2024-09-11T12:56:52.323429+0000","last_became_active":"2024-09-11T12:56:50.311129+0000","last_became_peered":"2024-09-11T12:56:50.311129+0000","last_unstale":"2024-09-11T12:56:52.323429+0000","last_undegraded":"2024-09-11T12:56:52.323429+0000","last_fullsized":"2024-09-11T12:56:52.323429+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:13:50.911770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.320804+0000","last_change":"2024-09-11T12:56:51.320138+0000","last_active":"2024-09-11T12:56:52.320804+0000","last_peered":"2024-09-11T12:56:52.320804+0000","last_clean":"2024-09-11T12:56:52.320804+0000","last_became_active":"2024-09-11T12:56:51.319860+0000","last_became_peered":"2024-09-11T12:56:51.319860+0000","last_unstale":"2024-09-11T12:56:52.320804+0000","last_undegraded":"2024-09-11T12:56:52.320804+0000","last_fullsized":"2024-09-11T12:56:52.320804+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:50.275883+0000","last_clean_scrub_stamp":"2024-09-11T12:56:50.275883+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:10:23.462856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799936+0000","last_change":"2024-09-11T12:56:50.303975+0000","last_active":"2024-09-11T12:56:52.799936+0000","last_peered":"2024-09-11T12:56:52.799936+0000","last_clean":"2024-09-11T12:56:52.799936+0000","last_became_active":"2024-09-11T12:56:50.303570+0000","last_became_peered":"2024-09-11T12:56:50.303570+0000","last_unstale":"2024-09-11T12:56:52.799936+0000","last_undegraded":"2024-09-11T12:56:52.799936+0000","last_fullsized":"2024-09-11T12:56:52.799936+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":148,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:56:49.268808+0000","last_clean_scrub_stamp":"2024-09-11T12:56:49.268808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:04:44.652693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":372,"reported_epoch":151,"state":"active+clean","last_fresh":"2024-09-11T12:56:52.799951+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:56:52.799951+0000","last_peered":"2024-09-11T12:56:52.799951+0000","last_clean":"2024-09-11T12:56:52.799951+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:56:52.799951+0000","last_undegraded":"2024-09-11T12:56:52.799951+0000","last_fullsized":"2024-09-11T12:56:52.799951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509532,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37792,"kb_used_data":2440,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334048,"statfs":{"total":96636764160,"available":96598065152,"internally_reserved":0,"allocated":2498560,"data_stored":1424041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509532,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1896448,"data_stored":833569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37208,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334632,"statfs":{"total":96636764160,"available":96598663168,"internally_reserved":0,"allocated":1896448,"data_stored":833569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37224,"kb_used_data":1868,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334616,"statfs":{"total":96636764160,"available":96598646784,"internally_reserved":0,"allocated":1912832,"data_stored":835763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509532,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33056,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338784,"statfs":{"total":96636764160,"available":96602914816,"internally_reserved":0,"allocated":1908736,"data_stored":833763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509532,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1908736,"data_stored":834263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509532,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33060,"kb_used_data":1864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338780,"statfs":{"total":96636764160,"available":96602910720,"internally_reserved":0,"allocated":1908736,"data_stored":835181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542236,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33624,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338216,"statfs":{"total":96636764160,"available":96602333184,"internally_reserved":0,"allocated":2494464,"data_stored":1423959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:57:00.348 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:57:00.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:57:00.540 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- 172.21.15.78:0/1072371762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9380cf790 msgr2=0x7fa9380cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- 172.21.15.78:0/1072371762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9380cf790 0x7fa9380cd290 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fa938070ec0 tx=0x7fa92401c8c0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- 172.21.15.78:0/1072371762 shutdown_connections 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- 172.21.15.78:0/1072371762 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa9380cf790 0x7fa9380cd290 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- 172.21.15.78:0/1072371762 >> v2:172.21.15.157:3300/0 conn(0x7fa938140400 0x7fa93815e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- 172.21.15.78:0/1072371762 >> 172.21.15.78:0/1072371762 conn(0x7fa9380b2ec0 msgr2=0x7fa9380b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- 172.21.15.78:0/1072371762 shutdown_connections 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- 172.21.15.78:0/1072371762 wait complete. 2024-09-11T12:57:00.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 Processor -- start 2024-09-11T12:57:00.542 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- start start 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9380cf790 0x7fa9380ae350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9381407e0 0x7fa9381fadd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa938073500 con 0x7fa938140400 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa938073380 con 0x7fa9380cf790 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa93da00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa938073680 con 0x7fa9381407e0 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46382/0 (socket says 172.21.15.78:46382) 2024-09-11T12:57:00.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936000640 1 -- 172.21.15.78:0/965569286 learned_addr learned my addr 172.21.15.78:0/965569286 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936a00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9380cf790 0x7fa9380ae350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa937400640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9381407e0 0x7fa9381fadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936000640 1 -- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9381407e0 msgr2=0x7fa9381fadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.537+0000 7fa936000640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9381407e0 0x7fa9381fadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa936000640 1 -- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9380cf790 msgr2=0x7fa9380ae350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa936000640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9380cf790 0x7fa9380ae350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa936000640 1 -- 172.21.15.78:0/965569286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9380af0f0 con 0x7fa938140400 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa936000640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b0040 tx=0x7fa9240a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240bb070 con 0x7fa938140400 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa924005030 con 0x7fa938140400 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9240b2040 con 0x7fa938140400 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9381fb940 con 0x7fa938140400 2024-09-11T12:57:00.545 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380ad870 con 0x7fa938140400 2024-09-11T12:57:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa938073b00 con 0x7fa938140400 2024-09-11T12:57:00.546 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.541+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fa924005480 con 0x7fa938140400 2024-09-11T12:57:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.545+0000 7fa92ea00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 0x7fa918074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.545+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fa924129ee0 con 0x7fa938140400 2024-09-11T12:57:00.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.545+0000 7fa936a00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 0x7fa918074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.548 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.545+0000 7fa936a00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 0x7fa918074320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa92800af60 tx=0x7fa92800a710 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.549+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa9240164b0 con 0x7fa938140400 2024-09-11T12:57:00.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.773+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa9380af6f0 con 0x7fa938140400 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.773+0000 7fa92ea00640 1 -- 172.21.15.78:0/965569286 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) ==== 190+0+147 (secure 0 0 0) 0x7fa9240f6ff0 con 0x7fa938140400 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:57:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:57:00.780 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 msgr2=0x7fa918074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 0x7fa918074320 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa92800af60 tx=0x7fa92800a710 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 msgr2=0x7fa9380cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fa9240b0040 tx=0x7fa9240a57b0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 shutdown_connections 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa9381407e0 0x7fa9381fadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa938140400 0x7fa9380cb950 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa918071e60 0x7fa918074320 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 --2- 172.21.15.78:0/965569286 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa9380cf790 0x7fa9380ae350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 >> 172.21.15.78:0/965569286 conn(0x7fa9380b2ec0 msgr2=0x7fa938070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:00.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 shutdown_connections 2024-09-11T12:57:00.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.777+0000 7fa93da00640 1 -- 172.21.15.78:0/965569286 wait complete. 2024-09-11T12:57:00.801 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 -- 172.21.15.78:0/4015674742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 msgr2=0x7f9bc40a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 --2- 172.21.15.78:0/4015674742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc40a9d00 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f9bb0009900 tx=0x7f9bb001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 -- 172.21.15.78:0/4015674742 shutdown_connections 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 --2- 172.21.15.78:0/4015674742 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc40a9d00 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 --2- 172.21.15.78:0/4015674742 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40b14b0 0x7f9bc415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 -- 172.21.15.78:0/4015674742 >> 172.21.15.78:0/4015674742 conn(0x7f9bc4074b90 msgr2=0x7f9bc40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 -- 172.21.15.78:0/4015674742 shutdown_connections 2024-09-11T12:57:00.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.937+0000 7f9bcc400640 1 -- 172.21.15.78:0/4015674742 wait complete. 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 Processor -- start 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- start start 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc40b14b0 0x7f9bc41fad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40d0bf0 0x7f9bc41f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc41693a0 con 0x7f9bc406fdf0 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9bc4169220 con 0x7f9bc40b14b0 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9bc4169520 con 0x7f9bc40d0bf0 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46404/0 (socket says 172.21.15.78:46404) 2024-09-11T12:57:00.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 -- 172.21.15.78:0/3758704722 learned_addr learned my addr 172.21.15.78:0/3758704722 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9000640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc40b14b0 0x7f9bc41fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bca400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40d0bf0 0x7f9bc41f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 -- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40d0bf0 msgr2=0x7f9bc41f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40d0bf0 0x7f9bc41f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 -- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc40b14b0 msgr2=0x7f9bc41fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc40b14b0 0x7f9bc41fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 -- 172.21.15.78:0/3758704722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bc406f4a0 con 0x7f9bc406fdf0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc9a00640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800ead0 tx=0x7f9bb800ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb800ce40 con 0x7f9bc406fdf0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9bb8004c20 con 0x7f9bc406fdf0 2024-09-11T12:57:00.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bc406ccb0 con 0x7f9bc406fdf0 2024-09-11T12:57:00.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bb8005280 con 0x7f9bc406fdf0 2024-09-11T12:57:00.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bc406c9d0 con 0x7f9bc406fdf0 2024-09-11T12:57:00.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.941+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9bc40a9d00 con 0x7f9bc406fdf0 2024-09-11T12:57:00.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.945+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f9bb8005420 con 0x7f9bc406fdf0 2024-09-11T12:57:00.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.945+0000 7f9bc1600640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 0x7f9ba4074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:00.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.945+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f9bb8093a90 con 0x7f9bc406fdf0 2024-09-11T12:57:00.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.945+0000 7f9bc9000640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 0x7f9ba4074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:00.952 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.945+0000 7f9bc9000640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 0x7f9ba4074350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00af000 tx=0x7f9bb001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:00.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:00.949+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9bb8016030 con 0x7f9bc406fdf0 2024-09-11T12:57:01.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9bc4144e40 con 0x7f9bc406fdf0 2024-09-11T12:57:01.153 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bc1600640 1 -- 172.21.15.78:0/3758704722 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+5236 (secure 0 0 0) 0x7f9bb805e210 con 0x7f9bc406fdf0 2024-09-11T12:57:01.153 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:01.154 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":111,"btime":"2024-09-11T12:56:54:322400+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:57:01.154 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 111 2024-09-11T12:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 msgr2=0x7f9ba4074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 0x7f9ba4074350 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9bb00af000 tx=0x7f9bb001b120 comp rx=0 tx=0).stop 2024-09-11T12:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 msgr2=0x7f9bc4144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.155 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f9bb800ead0 tx=0x7f9bb800ef90 comp rx=0 tx=0).stop 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 shutdown_connections 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9bc40d0bf0 0x7f9bc41f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9ba4071e90 0x7f9ba4074350 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9bc40b14b0 0x7f9bc41fad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 --2- 172.21.15.78:0/3758704722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9bc406fdf0 0x7f9bc4144900 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 >> 172.21.15.78:0/3758704722 conn(0x7f9bc4074b90 msgr2=0x7f9bc4148670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 shutdown_connections 2024-09-11T12:57:01.156 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.149+0000 7f9bcc400640 1 -- 172.21.15.78:0/3758704722 wait complete. 2024-09-11T12:57:01.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 111, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:56:50.675090+0000', 'modified': '2024-09-11T12:56:53.710536+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8631': {'gid': 8631, 'name': 'c', 'rank': 0, 'incarnation': 107, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.78:6835/963798948', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6833', 'nonce': 963798948}, {'type': 'v1', 'addr': '172.21.15.78:6835', 'nonce': 963798948}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8631, 'qdb_cluster': [8631]} 2024-09-11T12:57:01.577 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:57:01.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-11T12:57:01.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 -- 172.21.15.78:0/1145336495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 msgr2=0x7fb71c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 --2- 172.21.15.78:0/1145336495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb71c004c90 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7fb720006fc0 tx=0x7fb72001cf00 comp rx=0 tx=0).stop 2024-09-11T12:57:01.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 -- 172.21.15.78:0/1145336495 shutdown_connections 2024-09-11T12:57:01.722 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 --2- 172.21.15.78:0/1145336495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb71c004c90 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 -- 172.21.15.78:0/1145336495 >> 172.21.15.78:0/1145336495 conn(0x7fb7380746c0 msgr2=0x7fb7380ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 -- 172.21.15.78:0/1145336495 shutdown_connections 2024-09-11T12:57:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.717+0000 7fb73ca00640 1 -- 172.21.15.78:0/1145336495 wait complete. 2024-09-11T12:57:01.723 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 Processor -- start 2024-09-11T12:57:01.724 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- start start 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7381f8000 0x7fb7381f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb738154fe0 0x7fb738155460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb73813f310 con 0x7fb71c004810 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb73813f190 con 0x7fb738154fe0 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb73813f490 con 0x7fb7381f8000 2024-09-11T12:57:01.729 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46430/0 (socket says 172.21.15.78:46430) 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 -- 172.21.15.78:0/3801372138 learned_addr learned my addr 172.21.15.78:0/3801372138 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7381f8000 0x7fb7381f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737e00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb738154fe0 0x7fb738155460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 -- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7381f8000 msgr2=0x7fb7381f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7381f8000 0x7fb7381f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 -- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb738154fe0 msgr2=0x7fb738155460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb738154fe0 0x7fb738155460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 -- 172.21.15.78:0/3801372138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb73806ef40 con 0x7fb71c004810 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb737400640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e390 tx=0x7fb72800e850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb72f400640 1 -- 172.21.15.78:0/3801372138 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7280062e0 con 0x7fb71c004810 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb72f400640 1 -- 172.21.15.78:0/3801372138 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb728017020 con 0x7fb71c004810 2024-09-11T12:57:01.730 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb72f400640 1 -- 172.21.15.78:0/3801372138 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728003860 con 0x7fb71c004810 2024-09-11T12:57:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb73806e6a0 con 0x7fb71c004810 2024-09-11T12:57:01.731 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.721+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb73806ec60 con 0x7fb71c004810 2024-09-11T12:57:01.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72f400640 1 -- 172.21.15.78:0/3801372138 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fb72800d070 con 0x7fb71c004810 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72f400640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 0x7fb714074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72f400640 1 -- 172.21.15.78:0/3801372138 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fb728094200 con 0x7fb71c004810 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 0x7fb714074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 Processor -- start 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 -- start start 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 0x7fb714074350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb7200ac000 tx=0x7fb720019d40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53080fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a5307c2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a530620270 con 0x55a5307c3340 2024-09-11T12:57:01.734 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a53060f770 con 0x55a5308103c0 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a5306137b0 con 0x55a53080faa0 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb737e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb737e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46432/0 (socket says 172.21.15.78:46432) 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb737e00640 1 -- 172.21.15.78:0/4040984391 learned_addr learned my addr 172.21.15.78:0/4040984391 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb737400640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53080fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.725+0000 7fb72fe00640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a5307c2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 -- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 msgr2=0x55a53080fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53080fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 -- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 msgr2=0x55a5307c2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a5307c2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 -- 172.21.15.78:0/4040984391 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5305aef90 con 0x55a5307c3340 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53080fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:01.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb738140fe0 tx=0x7fb730003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146d11d5b7a80077 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb727e00640 1 -- 172.21.15.78:0/4040984391 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730004770 con 0x55a5307c3340 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb727e00640 1 -- 172.21.15.78:0/4040984391 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb730009e10 con 0x55a5307c3340 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb727e00640 1 -- 172.21.15.78:0/4040984391 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb730004d90 con 0x55a5307c3340 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- 172.21.15.78:0/4040984391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 msgr2=0x55a5307c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7fb738140fe0 tx=0x7fb730003d50 comp rx=0 tx=0).stop 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- 172.21.15.78:0/4040984391 shutdown_connections 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a5307c3720 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a5307c2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4040984391 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53080fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- 172.21.15.78:0/4040984391 >> 172.21.15.78:0/4040984391 conn(0x55a5307f0b60 msgr2=0x55a5307f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- 172.21.15.78:0/4040984391 shutdown_connections 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- 172.21.15.78:0/4040984391 wait complete. 2024-09-11T12:57:01.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- start start 2024-09-11T12:57:01.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 Processor -- start 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a53080faa0 0x55a5307c5920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a5308103c0 0x55a5307c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a5306df290 con 0x55a5307c3340 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a5305bbab0 con 0x55a53080faa0 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a5305b9170 con 0x55a5308103c0 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb7414c8000 10 client.0 _ll_register_callbacks cb 0x55a5307b0830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:57:01.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46444/0 (socket says 172.21.15.78:46444) 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 -- 172.21.15.78:0/3177930416 learned_addr learned my addr 172.21.15.78:0/3177930416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737e00640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a5308103c0 0x55a5307c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a53080faa0 0x55a5307c5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 -- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a5308103c0 msgr2=0x55a5307c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a5308103c0 0x55a5307c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 -- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a53080faa0 msgr2=0x55a5307c5920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a53080faa0 0x55a5307c5920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.729+0000 7fb737400640 1 -- 172.21.15.78:0/3177930416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5306946f0 con 0x55a5307c3340 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb737400640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb7280611f0 tx=0x7fb728060c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fecfa33e68920313 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728016070 con 0x55a5307c3340 2024-09-11T12:57:01.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fb72805cbd0 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a5305a8f10 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7fb728061960 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 10 client.8802 fetch_fsmap learned FSMap version 111 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55a53066bc40 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a5305a8f10 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 111) ==== 3502+0+0 (secure 0 0 0) 0x7fb72805cdd0 con 0x55a5307c3340 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 10 client.8802 fetch_fsmap finished waiting for FSMap version 111 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 10 client.8802 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 20 client.8802 populate_metadata read hostname 'smithi078' 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a5307c6ba0 0x55a530821d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 4 client.8802 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:57:01.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 --> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5306221c0 con 0x55a5307c6ba0 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a5307c6ba0 0x55a530821d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a5307c6ba0 0x55a530821d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75b186ffdd67c6f8 server_cookie=cb27a03800745910 in_seq=0 out_seq=0 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 10 client.8802 ms_handle_connect on v2:172.21.15.78:6833/963798948 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 1 -- 172.21.15.78:0/3177930416 <== mds.0 v2:172.21.15.78:6833/963798948 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a5306221c0 con 0x55a5307c6ba0 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 10 client.8802 handle_command_reply: tid=0 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 client.8802 shutdown 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 msgr2=0x55a53080f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7fb7280611f0 tx=0x7fb728060c20 comp rx=0 tx=0).stop 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 shutdown_connections 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a5308103c0 0x55a5307c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a5307c6ba0 0x55a530821d60 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a53080faa0 0x55a5307c5920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3177930416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53080f3a0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 >> 172.21.15.78:0/3177930416 conn(0x55a5307f0b60 msgr2=0x55a5307c2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb724c00640 0 client.8802 ms_handle_reset on v2:172.21.15.78:6833/963798948 2024-09-11T12:57:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 shutdown_connections 2024-09-11T12:57:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 1 -- 172.21.15.78:0/3177930416 wait complete. 2024-09-11T12:57:01.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.733+0000 7fb7414c8000 20 client.8802 trim_cache size 0 max 16384 2024-09-11T12:57:01.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 Processor -- start 2024-09-11T12:57:01.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- start start 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53087f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a53087fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fb7382001e0 con 0x55a5307c3340 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fb73806a830 con 0x55a5308103c0 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fb73806e010 con 0x55a53080faa0 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46450/0 (socket says 172.21.15.78:46450) 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 -- 172.21.15.78:0/3352213699 learned_addr learned my addr 172.21.15.78:0/3352213699 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb72fe00640 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53087f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737e00640 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a53087fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 -- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 msgr2=0x55a53087f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53087f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 -- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 msgr2=0x55a53087fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a53087fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 -- 172.21.15.78:0/3352213699 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5305c4e70 con 0x55a5307c3340 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb737400640 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb728096d00 tx=0x7fb7280609d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea28613859a72a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb727400640 1 -- 172.21.15.78:0/3352213699 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728016070 con 0x55a5307c3340 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb727400640 1 -- 172.21.15.78:0/3352213699 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb72807a030 con 0x55a5307c3340 2024-09-11T12:57:01.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb727400640 1 -- 172.21.15.78:0/3352213699 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72805cdd0 con 0x55a5307c3340 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- 172.21.15.78:0/3352213699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 msgr2=0x55a53087f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fb728096d00 tx=0x7fb7280609d0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- 172.21.15.78:0/3352213699 shutdown_connections 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5308103c0 0x55a53087fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a53087f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 --2- 172.21.15.78:0/3352213699 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5307c3340 0x55a53087f440 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- 172.21.15.78:0/3352213699 >> 172.21.15.78:0/3352213699 conn(0x55a5307f0b60 msgr2=0x55a53081d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- 172.21.15.78:0/3352213699 shutdown_connections 2024-09-11T12:57:01.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.753+0000 7fb7414c8000 1 -- 172.21.15.78:0/3352213699 wait complete. 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- start start 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 Processor -- start 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5307c3340 0x55a530883700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a530883c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55a530617de0 con 0x55a5308103c0 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55a530612ac0 con 0x55a5307c3340 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55a530622ea0 con 0x55a53080faa0 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 10 client.0 _ll_register_callbacks cb 0x7fb7280610e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46458/0 (socket says 172.21.15.78:46458) 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 -- 172.21.15.78:0/4084703546 learned_addr learned my addr 172.21.15.78:0/4084703546 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737400640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5307c3340 0x55a530883700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb72fe00640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a530883c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 -- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 msgr2=0x55a530883c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a530883c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 -- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5307c3340 msgr2=0x55a530883700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5307c3340 0x55a530883700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 -- 172.21.15.78:0/4084703546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a530599bf0 con 0x55a5308103c0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737e00640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb730009a50 tx=0x7fb7300092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea4da332f4f80837 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73002c060 con 0x55a5308103c0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fb730027440 con 0x55a5308103c0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb72807a030 con 0x55a5308103c0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=111) ==== 24+0+0 (secure 0 0 0) 0x7fb730027680 con 0x55a5308103c0 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 10 client.8808 fetch_fsmap learned FSMap version 111 2024-09-11T12:57:01.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=111,monmap=2+}) -- 0x55a5305b5c60 con 0x55a5308103c0 2024-09-11T12:57:01.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb72807a030 con 0x55a5308103c0 2024-09-11T12:57:01.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 111) ==== 3502+0+0 (secure 0 0 0) 0x7fb730035020 con 0x55a5308103c0 2024-09-11T12:57:01.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 10 client.8808 fetch_fsmap finished waiting for FSMap version 111 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 10 client.8808 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 20 client.8808 populate_metadata read hostname 'smithi078' 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a530825790 0x55a530824730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 4 client.8808 mds_command: new command op to 8631 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 --> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] -- command(tid 0: {"prefix": "status"}) -- 0x55a53061dc60 con 0x55a530825790 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.757+0000 7fb737400640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a530825790 0x55a530824730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb737400640 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a530825790 0x55a530824730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267f8703e2f47c4c server_cookie=595c0cddef32b46e in_seq=0 out_seq=0 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb724c00640 10 client.8808 ms_handle_connect on v2:172.21.15.78:6833/963798948 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb724c00640 1 -- 172.21.15.78:0/4084703546 <== mds.0 v2:172.21.15.78:6833/963798948 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55a53061dc60 con 0x55a530825790 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb724c00640 10 client.8808 handle_command_reply: tid=0 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 client.8808 shutdown 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 msgr2=0x55a530884180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fb730009a50 tx=0x7fb7300092b0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 shutdown_connections 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55a5308103c0 0x55a530884180 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55a53080faa0 0x55a530883c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.78:6833/963798948,v1:172.21.15.78:6835/963798948] conn(0x55a530825790 0x55a530824730 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 --2- 172.21.15.78:0/4084703546 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55a5307c3340 0x55a530883700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 >> 172.21.15.78:0/4084703546 conn(0x55a5307f0b60 msgr2=0x55a5307c5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb724c00640 0 client.8808 ms_handle_reset on v2:172.21.15.78:6833/963798948 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 shutdown_connections 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 1 -- 172.21.15.78:0/4084703546 wait complete. 2024-09-11T12:57:01.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb7414c8000 20 client.8808 trim_cache size 0 max 16384 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 8631, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 10.452920188, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 111, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 151, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 151, 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 14.986936132 2024-09-11T12:57:01.769 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 msgr2=0x7fb714074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 0x7fb714074350 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb7200ac000 tx=0x7fb720019d40 comp rx=0 tx=0).stop 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 msgr2=0x7fb7380701f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7fb72800e390 tx=0x7fb72800e850 comp rx=0 tx=0).stop 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 shutdown_connections 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fb714071e90 0x7fb714074350 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fb738154fe0 0x7fb738155460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fb7381f8000 0x7fb7381f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 --2- 172.21.15.78:0/3801372138 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fb71c004810 0x7fb7380701f0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 >> 172.21.15.78:0/3801372138 conn(0x7fb7380746c0 msgr2=0x7fb7380ce4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 shutdown_connections 2024-09-11T12:57:01.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.761+0000 7fb73ca00640 1 -- 172.21.15.78:0/3801372138 wait complete. 2024-09-11T12:57:01.784 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:57:01.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- 172.21.15.78:0/631594197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 msgr2=0x7f888c15af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 --2- 172.21.15.78:0/631594197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c15af90 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f888c074d50 tx=0x7f887801daa0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- 172.21.15.78:0/631594197 shutdown_connections 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 --2- 172.21.15.78:0/631594197 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c15af90 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 --2- 172.21.15.78:0/631594197 >> v2:172.21.15.157:3300/0 conn(0x7f888c157ae0 0x7f888c070a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- 172.21.15.78:0/631594197 >> 172.21.15.78:0/631594197 conn(0x7f888c06ca80 msgr2=0x7f888c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- 172.21.15.78:0/631594197 shutdown_connections 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- 172.21.15.78:0/631594197 wait complete. 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 Processor -- start 2024-09-11T12:57:01.926 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.921+0000 7f8893800640 1 -- start start 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0b4db0 0x7f888c0afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c157ae0 0x7f888c138630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f888c166f10 con 0x7f888c0736d0 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f888c0b5730 con 0x7f888c0b4db0 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f888c167090 con 0x7f888c157ae0 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46478/0 (socket says 172.21.15.78:46478) 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 -- 172.21.15.78:0/1928073313 learned_addr learned my addr 172.21.15.78:0/1928073313 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f888be00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0b4db0 0x7f888c0afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.928 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8891800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c157ae0 0x7f888c138630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 -- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c157ae0 msgr2=0x7f888c138630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c157ae0 0x7f888c138630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 -- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0b4db0 msgr2=0x7f888c0afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:01.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0b4db0 0x7f888c0afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 -- 172.21.15.78:0/1928073313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f888c13b640 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8890e00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f887c00eb00 tx=0x7f887c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c004b30 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f887c004cf0 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c005280 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f888c13b920 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f888c13bee0 con 0x7f888c0736d0 2024-09-11T12:57:01.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f888c15af90 con 0x7f888c0736d0 2024-09-11T12:57:01.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.925+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f887c005420 con 0x7f888c0736d0 2024-09-11T12:57:01.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.929+0000 7f8888c00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 0x7f886c074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:01.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.929+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f887c093ae0 con 0x7f888c0736d0 2024-09-11T12:57:01.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.929+0000 7f888be00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 0x7f886c074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:01.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.929+0000 7f888be00640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 0x7f886c074320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f88780b0000 tx=0x7f887801c2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:01.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:01.933+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f887c05e420 con 0x7f888c0736d0 2024-09-11T12:57:02.135 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f888c156ca0 con 0x7f888c0736d0 2024-09-11T12:57:02.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8888c00640 1 -- 172.21.15.78:0/1928073313 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f887c062970 con 0x7f888c0736d0 2024-09-11T12:57:02.136 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:57:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 msgr2=0x7f886c074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 0x7f886c074320 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f88780b0000 tx=0x7f887801c2e0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 msgr2=0x7f888c13afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f887c00eb00 tx=0x7f887c00c2d0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.138 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 shutdown_connections 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f888c157ae0 0x7f888c138630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f886c071e60 0x7f886c074320 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f888c0b4db0 0x7f888c0afc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 --2- 172.21.15.78:0/1928073313 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f888c0736d0 0x7f888c13afd0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 >> 172.21.15.78:0/1928073313 conn(0x7f888c06ca80 msgr2=0x7f888c15e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 shutdown_connections 2024-09-11T12:57:02.139 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02.133+0000 7f8893800640 1 -- 172.21.15.78:0/1928073313 wait complete. 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:02.158 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:57:02.159 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:57:02.159 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:57:02.159 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:02.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.159 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.168 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 42012sec preferred_lft 42012sec 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.169 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:57:02.170 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:57:02.171 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.171 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:02.171 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:57:02.171 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:57:02.171 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:02.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.172 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:57:02.232 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:57:02.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.234 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:02.287 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:02.288 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:57:02.288 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:57:02.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:57:02.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:02.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.425 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:57:02.434 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:57:02.434 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:57:02.434 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:02.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.434 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:57:02.498 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:02.498 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:02.547 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:02.547 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:02.548 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:02.598 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:57:02.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.599 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:02.687 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:02.687 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:57:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:57:02.700 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:57:02.704 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:57:02.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:02.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:02.724 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:57:02.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:02.823 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:02.829 INFO:teuthology.orchestra.run.smithi078.stdout:client7259 2024-09-11T12:57:02.830 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:02.830 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:02.838 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:57:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- 172.21.15.78:0/1737450760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 msgr2=0x7f20500736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 --2- 172.21.15.78:0/1737450760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500736d0 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f205015dad0 tx=0x7f204001c770 comp rx=0 tx=0).stop 2024-09-11T12:57:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- 172.21.15.78:0/1737450760 shutdown_connections 2024-09-11T12:57:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 --2- 172.21.15.78:0/1737450760 >> v2:172.21.15.157:3300/0 conn(0x7f20500b4db0 0x7f20500b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 --2- 172.21.15.78:0/1737450760 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500736d0 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- 172.21.15.78:0/1737450760 >> 172.21.15.78:0/1737450760 conn(0x7f205006ca80 msgr2=0x7f20500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- 172.21.15.78:0/1737450760 shutdown_connections 2024-09-11T12:57:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- 172.21.15.78:0/1737450760 wait complete. 2024-09-11T12:57:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 Processor -- start 2024-09-11T12:57:03.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.021+0000 7f2055200640 1 -- start start 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20500b4db0 0x7f205014a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2050157ae0 0x7f20500ce700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f205006f280 con 0x7f2050070a00 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f205006f100 con 0x7f2050157ae0 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f205006f400 con 0x7f20500b4db0 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46496/0 (socket says 172.21.15.78:46496) 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 -- 172.21.15.78:0/1089541838 learned_addr learned my addr 172.21.15.78:0/1089541838 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204d400640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20500b4db0 0x7f205014a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204e800640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2050157ae0 0x7f20500ce700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:03.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 -- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20500b4db0 msgr2=0x7f205014a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20500b4db0 0x7f205014a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 -- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2050157ae0 msgr2=0x7f20500ce700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2050157ae0 0x7f20500ce700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 -- 172.21.15.78:0/1089541838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20500cec40 con 0x7f2050070a00 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f204de00640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f204800c780 tx=0x7f204800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048017070 con 0x7f2050070a00 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2048005070 con 0x7f2050070a00 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048005480 con 0x7f2050070a00 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205015c9c0 con 0x7f2050070a00 2024-09-11T12:57:03.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f205015d100 con 0x7f2050070a00 2024-09-11T12:57:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20500736d0 con 0x7f2050070a00 2024-09-11T12:57:03.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.025+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f204801a070 con 0x7f2050070a00 2024-09-11T12:57:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.029+0000 7f2046000640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 0x7f2030074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.029+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f20480926a0 con 0x7f2050070a00 2024-09-11T12:57:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.029+0000 7f204d400640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 0x7f2030074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:03.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.029+0000 7f204d400640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 0x7f2030074320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20400af000 tx=0x7f204001afb0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:03.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.033+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f204805d5b0 con 0x7f2050070a00 2024-09-11T12:57:03.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.233+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f20500afe70 con 0x7f2050070a00 2024-09-11T12:57:03.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.233+0000 7f2046000640 1 -- 172.21.15.78:0/1089541838 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v111) ==== 53+0+73 (secure 0 0 0) 0x7f204805ceb0 con 0x7f2050070a00 2024-09-11T12:57:03.239 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 msgr2=0x7f2030074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 0x7f2030074320 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f20400af000 tx=0x7f204001afb0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 msgr2=0x7f20500cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f204800c780 tx=0x7f204800cc40 comp rx=0 tx=0).stop 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 shutdown_connections 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2030071e60 0x7f2030074320 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2050157ae0 0x7f20500ce700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f20500b4db0 0x7f205014a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 --2- 172.21.15.78:0/1089541838 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2050070a00 0x7f20500cf5f0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 >> 172.21.15.78:0/1089541838 conn(0x7f205006ca80 msgr2=0x7f205014ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 shutdown_connections 2024-09-11T12:57:03.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:03.237+0000 7f2055200640 1 -- 172.21.15.78:0/1089541838 wait complete. 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:57:03.260 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:03.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.261 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:57:03.271 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 42020sec preferred_lft 42020sec 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:57:03.272 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.273 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:03.274 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:57:03.274 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:57:03.274 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:03.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.275 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:57:03.338 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:57:03.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.340 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:57:03.394 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:57:03.394 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:57:03.394 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:57:03.394 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:03.395 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:57:03.396 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:57:03.493 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:57:03.537 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:03.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.544 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:57:03.553 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:57:03.553 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:57:03.554 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:03.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.555 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:57:03.622 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:03.623 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:03.678 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:03.678 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:03.678 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:03.729 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:57:03.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.729 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:03.822 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:03.822 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:03.837 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:03.837 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:03.837 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:57:03.837 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:57:03.837 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:57:03.841 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:57:03.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:03.841 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:03.860 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:57:03.948 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:03.952 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:03.958 INFO:teuthology.orchestra.run.smithi157.stdout:client7265 2024-09-11T12:57:03.959 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:03.959 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:03.967 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 -- 172.21.15.78:0/689311641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f195c0052e0 msgr2=0x7f195c005920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 --2- 172.21.15.78:0/689311641 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f195c0052e0 0x7f195c005920 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f1960001860 tx=0x7f196001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f1979e00640 1 -- 172.21.15.78:0/689311641 reap_dead start 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 -- 172.21.15.78:0/689311641 shutdown_connections 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 -- 172.21.15.78:0/689311641 >> 172.21.15.78:0/689311641 conn(0x7f19740b3050 msgr2=0x7f19740b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 -- 172.21.15.78:0/689311641 shutdown_connections 2024-09-11T12:57:04.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.113+0000 7f197be00640 1 -- 172.21.15.78:0/689311641 wait complete. 2024-09-11T12:57:04.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 Processor -- start 2024-09-11T12:57:04.120 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- start start 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0052e0 0x7f197414cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 0x7f1974147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1974165960 con 0x7f1974162090 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f19741657e0 con 0x7f19740c5b10 2024-09-11T12:57:04.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1974165ae0 con 0x7f195c0052e0 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46304/0 (socket says 172.21.15.78:46304) 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 -- 172.21.15.78:0/2173505926 learned_addr learned my addr 172.21.15.78:0/2173505926 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979400640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0052e0 0x7f197414cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1978a00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 0x7f1974147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 -- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0052e0 msgr2=0x7f197414cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0052e0 0x7f197414cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 -- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 msgr2=0x7f1974147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 0x7f1974147640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 -- 172.21.15.78:0/2173505926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19740c5f90 con 0x7f1974162090 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1978a00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 0x7f1974147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:04.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1979e00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f197406e9e0 tx=0x7f196c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00f8c0 con 0x7f1974162090 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f196c00fa60 con 0x7f1974162090 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00fe70 con 0x7f1974162090 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741fac80 con 0x7f1974162090 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.117+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741fb240 con 0x7f1974162090 2024-09-11T12:57:04.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1974071e10 con 0x7f1974162090 2024-09-11T12:57:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f196c003600 con 0x7f1974162090 2024-09-11T12:57:04.125 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f1971600640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 0x7f1954074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f196c091190 con 0x7f1974162090 2024-09-11T12:57:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f1979400640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 0x7f1954074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.121+0000 7f1979400640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 0x7f1954074350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968006f60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:04.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.125+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f196c021020 con 0x7f1974162090 2024-09-11T12:57:04.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.325+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f1974070ce0 con 0x7f1974162090 2024-09-11T12:57:04.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.573+0000 7f1971600640 1 -- 172.21.15.78:0/2173505926 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v112) ==== 136+0+0 (secure 0 0 0) 0x7f19740c5f90 con 0x7f1974162090 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 msgr2=0x7f1954074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 0x7f1954074350 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f1968006fd0 tx=0x7f1968006f60 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 msgr2=0x7f1974162510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f197406e9e0 tx=0x7f196c011c40 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 shutdown_connections 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1974162090 0x7f1974162510 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1954071e90 0x7f1954074350 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f19740c5b10 0x7f1974147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 --2- 172.21.15.78:0/2173505926 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f195c0052e0 0x7f197414cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 >> 172.21.15.78:0/2173505926 conn(0x7f19740b3050 msgr2=0x7f197415ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:04.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 shutdown_connections 2024-09-11T12:57:04.583 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.577+0000 7f197be00640 1 -- 172.21.15.78:0/2173505926 wait complete. 2024-09-11T12:57:04.602 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- 172.21.15.78:0/2735323021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 msgr2=0x7fed6c073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 --2- 172.21.15.78:0/2735323021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c073f70 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7fed64009cb0 tx=0x7fed6401cda0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- 172.21.15.78:0/2735323021 shutdown_connections 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 --2- 172.21.15.78:0/2735323021 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c073f70 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- 172.21.15.78:0/2735323021 >> 172.21.15.78:0/2735323021 conn(0x7fed6c06ca80 msgr2=0x7fed6c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:04.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- 172.21.15.78:0/2735323021 shutdown_connections 2024-09-11T12:57:04.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- 172.21.15.78:0/2735323021 wait complete. 2024-09-11T12:57:04.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 Processor -- start 2024-09-11T12:57:04.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.741+0000 7fed72a00640 1 -- start start 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0b4780 0x7fed6c15ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed6c15bc60 0x7fed6c0cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c169ed0 con 0x7fed6c0ccb20 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fed6c169d50 con 0x7fed6c0b4780 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fed6c16a050 con 0x7fed6c15bc60 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46336/0 (socket says 172.21.15.78:46336) 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 -- 172.21.15.78:0/2697490300 learned_addr learned my addr 172.21.15.78:0/2697490300 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed70a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed6c15bc60 0x7fed6c0cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6be00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0b4780 0x7fed6c15ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 -- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed6c15bc60 msgr2=0x7fed6c0cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed6c15bc60 0x7fed6c0cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 -- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0b4780 msgr2=0x7fed6c15ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0b4780 0x7fed6c15ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 -- 172.21.15.78:0/2697490300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6c0b1d50 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed6b400640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c3c0 tx=0x7fed5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed580052d0 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed58007020 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6c1fb570 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed58007470 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6c1fbb30 con 0x7fed6c0ccb20 2024-09-11T12:57:04.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed6c073f70 con 0x7fed6c0ccb20 2024-09-11T12:57:04.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.745+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fed580067e0 con 0x7fed6c0ccb20 2024-09-11T12:57:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.749+0000 7fed63e00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 0x7fed4c074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.749+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fed580923e0 con 0x7fed6c0ccb20 2024-09-11T12:57:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.749+0000 7fed6be00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 0x7fed4c074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:04.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.749+0000 7fed6be00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 0x7fed4c074320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007fd0 tx=0x7fed5c007e40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:04.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.753+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fed58001090 con 0x7fed6c0ccb20 2024-09-11T12:57:04.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.937+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fed6c14a4d0 con 0x7fed4c071e60 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed63e00640 1 -- 172.21.15.78:0/2697490300 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fed6c14a4d0 con 0x7fed4c071e60 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:04.944 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:57:04.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 msgr2=0x7fed4c074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.946 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 0x7fed4c074320 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed5c007fd0 tx=0x7fed5c007e40 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 msgr2=0x7fed6c149420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fed5800c3c0 tx=0x7fed5800c880 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 shutdown_connections 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fed6c15bc60 0x7fed6c0cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fed6c0ccb20 0x7fed6c149420 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fed4c071e60 0x7fed4c074320 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 --2- 172.21.15.78:0/2697490300 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fed6c0b4780 0x7fed6c15ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 >> 172.21.15.78:0/2697490300 conn(0x7fed6c06ca80 msgr2=0x7fed6c14e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 shutdown_connections 2024-09-11T12:57:04.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:04.941+0000 7fed72a00640 1 -- 172.21.15.78:0/2697490300 wait complete. 2024-09-11T12:57:04.966 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:57:05.110 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 -- 172.21.15.78:0/1766129977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80cd770 msgr2=0x7f09b80cdc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 --2- 172.21.15.78:0/1766129977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80cd770 0x7f09b80cdc20 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009900 tx=0x7f09b001c8b0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 -- 172.21.15.78:0/1766129977 shutdown_connections 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 --2- 172.21.15.78:0/1766129977 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80cd770 0x7f09b80cdc20 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 --2- 172.21.15.78:0/1766129977 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80b3420 0x7f09b80cf6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 -- 172.21.15.78:0/1766129977 >> 172.21.15.78:0/1766129977 conn(0x7f09b8071dd0 msgr2=0x7f09b80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 -- 172.21.15.78:0/1766129977 shutdown_connections 2024-09-11T12:57:05.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.105+0000 7f09bca00640 1 -- 172.21.15.78:0/1766129977 wait complete. 2024-09-11T12:57:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 Processor -- start 2024-09-11T12:57:05.112 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- start start 2024-09-11T12:57:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b80b3420 0x7f09b8153860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80cd770 0x7f09b8142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80af0c0 con 0x7f09b80b3040 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f09b80aef40 con 0x7f09b80b3420 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f09b80af240 con 0x7f09b80cd770 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46346/0 (socket says 172.21.15.78:46346) 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 -- 172.21.15.78:0/2094466134 learned_addr learned my addr 172.21.15.78:0/2094466134 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09ad400640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b80b3420 0x7f09b8153860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b6800640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80cd770 0x7f09b8142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 -- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80cd770 msgr2=0x7f09b8142190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.115 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80cd770 0x7f09b8142190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 -- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b80b3420 msgr2=0x7f09b8153860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b80b3420 0x7f09b8153860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 -- 172.21.15.78:0/2094466134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b80afc90 con 0x7f09b80b3040 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09b5e00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8a0 tx=0x7f09a800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8003b80 con 0x7f09b80b3040 2024-09-11T12:57:05.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09a8003d40 con 0x7f09b80b3040 2024-09-11T12:57:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800aab0 con 0x7f09b80b3040 2024-09-11T12:57:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b81f5460 con 0x7f09b80b3040 2024-09-11T12:57:05.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b81f5a20 con 0x7f09b80b3040 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.109+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09b80cdd90 con 0x7f09b80b3040 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.113+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f09a800ac50 con 0x7f09b80b3040 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.113+0000 7f09aea00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 0x7f0994074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.113+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f09a8090550 con 0x7f09b80b3040 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.113+0000 7f09ad400640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 0x7f0994074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.119 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.113+0000 7f09ad400640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 0x7f0994074350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09b8072740 tx=0x7f09a00091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:05.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.117+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f09b80ce160 con 0x7f09b80b3040 2024-09-11T12:57:05.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.281+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f09b8072490 con 0x7f09b80b3040 2024-09-11T12:57:05.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.281+0000 7f09aea00640 1 -- 172.21.15.78:0/2094466134 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f09a805d660 con 0x7f09b80b3040 2024-09-11T12:57:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 msgr2=0x7f0994074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 0x7f0994074350 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f09b8072740 tx=0x7f09a00091e0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 msgr2=0x7f09b80b02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f09a800c8a0 tx=0x7f09a800cd60 comp rx=0 tx=0).stop 2024-09-11T12:57:05.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 shutdown_connections 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f09b80cd770 0x7f09b8142190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0994071e90 0x7f0994074350 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f09b80b3420 0x7f09b8153860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 --2- 172.21.15.78:0/2094466134 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f09b80b3040 0x7f09b80b02c0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 >> 172.21.15.78:0/2094466134 conn(0x7f09b8071dd0 msgr2=0x7f09b8156f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 shutdown_connections 2024-09-11T12:57:05.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.285+0000 7f09bca00640 1 -- 172.21.15.78:0/2094466134 wait complete. 2024-09-11T12:57:05.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.309 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol41230/16b05905-b236-4562-b853-703113e7d08d) 2024-09-11T12:57:05.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.336 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:57:05.388 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:57:05.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.389 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol41230/16b05905-b236-4562-b853-703113e7d08d) 2024-09-11T12:57:05.456 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:05.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol41230/.meta 2024-09-11T12:57:05.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.523 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp25450/subvol78091/47a17d3c-6124-4369-a42a-52d20d17bc38) 2024-09-11T12:57:05.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.586 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:57:05.636 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:57:05.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:05.636 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp25450/subvol78091/47a17d3c-6124-4369-a42a-52d20d17bc38) 2024-09-11T12:57:05.704 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:05.704 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp25450/subvol78091/.meta 2024-09-11T12:57:05.766 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol41230 2024-09-11T12:57:05.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 -- 172.21.15.78:0/1457959683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561406f1a0 msgr2=0x7f56140d13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 --2- 172.21.15.78:0/1457959683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561406f1a0 0x7f56140d13c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f5600009900 tx=0x7f560001c850 comp rx=0 tx=0).stop 2024-09-11T12:57:05.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 -- 172.21.15.78:0/1457959683 shutdown_connections 2024-09-11T12:57:05.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 --2- 172.21.15.78:0/1457959683 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f561406f1a0 0x7f56140d13c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 -- 172.21.15.78:0/1457959683 >> 172.21.15.78:0/1457959683 conn(0x7f56140745a0 msgr2=0x7f56140749b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:05.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 -- 172.21.15.78:0/1457959683 shutdown_connections 2024-09-11T12:57:05.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.949+0000 7f561ae00640 1 -- 172.21.15.78:0/1457959683 wait complete. 2024-09-11T12:57:05.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 Processor -- start 2024-09-11T12:57:05.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- start start 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561406f1a0 0x7f5614146330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5614154bc0 0x7f561414cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f561416bdc0 con 0x7f56141547e0 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f561416bc40 con 0x7f561406f1a0 2024-09-11T12:57:05.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f561416bf40 con 0x7f5614154bc0 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46362/0 (socket says 172.21.15.78:46362) 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 -- 172.21.15.78:0/2668370620 learned_addr learned my addr 172.21.15.78:0/2668370620 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613e00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561406f1a0 0x7f5614146330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5618e00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5614154bc0 0x7f561414cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 -- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5614154bc0 msgr2=0x7f561414cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5614154bc0 0x7f561414cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 -- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561406f1a0 msgr2=0x7f5614146330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561406f1a0 0x7f5614146330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 -- 172.21.15.78:0/2668370620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56141695b0 con 0x7f56141547e0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f5613400640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5600009850 tx=0x7f5600004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56000b2040 con 0x7f56141547e0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5600005030 con 0x7f56141547e0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600005440 con 0x7f56141547e0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56140b3200 con 0x7f56141547e0 2024-09-11T12:57:05.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5614146d10 con 0x7f56141547e0 2024-09-11T12:57:05.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.953+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56140d13c0 con 0x7f56141547e0 2024-09-11T12:57:05.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.957+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f56000046d0 con 0x7f56141547e0 2024-09-11T12:57:05.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.957+0000 7f560be00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 0x7f55f4074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:05.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.957+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f560012c810 con 0x7f56141547e0 2024-09-11T12:57:05.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.957+0000 7f5613e00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 0x7f55f4074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:05.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.957+0000 7f5613e00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 0x7f55f4074320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5604001d90 tx=0x7f5604009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:05.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:05.961+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f56000164d0 con 0x7f56141547e0 2024-09-11T12:57:06.157 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.153+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol41230", "target": ["mon-mgr", ""]}) -- 0x7f5614069570 con 0x7f55f4071e60 2024-09-11T12:57:06.163 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:06.161+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:06.164 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:06.161+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:06.164 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:06.161+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:06.164 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:06.161+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:06.164 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:06.161+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:06.213 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f560be00640 1 -- 172.21.15.78:0/2668370620 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f5614069570 con 0x7f55f4071e60 2024-09-11T12:57:06.213 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/_nogroup/subvol41230/16b05905-b236-4562-b853-703113e7d08d 2024-09-11T12:57:06.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 msgr2=0x7f55f4074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 0x7f55f4074320 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5604001d90 tx=0x7f5604009040 comp rx=0 tx=0).stop 2024-09-11T12:57:06.215 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 msgr2=0x7f5614141520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f5600009850 tx=0x7f5600004040 comp rx=0 tx=0).stop 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 shutdown_connections 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5614154bc0 0x7f561414cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f56141547e0 0x7f5614141520 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f55f4071e60 0x7f55f4074320 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 --2- 172.21.15.78:0/2668370620 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f561406f1a0 0x7f5614146330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.209+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 >> 172.21.15.78:0/2668370620 conn(0x7f56140745a0 msgr2=0x7f561406ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.213+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 shutdown_connections 2024-09-11T12:57:06.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.213+0000 7f561ae00640 1 -- 172.21.15.78:0/2668370620 wait complete. 2024-09-11T12:57:06.234 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol78091 subvol_grp25450 2024-09-11T12:57:06.384 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- 172.21.15.78:0/151972558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8073e90 msgr2=0x7fd7d8071e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- 172.21.15.78:0/151972558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8073e90 0x7fd7d8071e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd7c400aa10 tx=0x7fd7c401dc00 comp rx=0 tx=0).stop 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- 172.21.15.78:0/151972558 shutdown_connections 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- 172.21.15.78:0/151972558 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8073e90 0x7fd7d8071e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- 172.21.15.78:0/151972558 >> v2:172.21.15.157:3300/0 conn(0x7fd7d8151050 0x7fd7d8073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- 172.21.15.78:0/151972558 >> 172.21.15.78:0/151972558 conn(0x7fd7d80b3050 msgr2=0x7fd7d80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- 172.21.15.78:0/151972558 shutdown_connections 2024-09-11T12:57:06.385 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- 172.21.15.78:0/151972558 wait complete. 2024-09-11T12:57:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 Processor -- start 2024-09-11T12:57:06.386 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- start start 2024-09-11T12:57:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.387 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d8073e90 0x7fd7d8148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8151050 0x7fd7d814fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d815e910 con 0x7fd7d8072350 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd7d815e790 con 0x7fd7d8073e90 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7de600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd7d815ea90 con 0x7fd7d8151050 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7d7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7d7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46384/0 (socket says 172.21.15.78:46384) 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7d7400640 1 -- 172.21.15.78:0/303187614 learned_addr learned my addr 172.21.15.78:0/303187614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7d6a00640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d8073e90 0x7fd7d8148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.381+0000 7fd7d7e00640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8151050 0x7fd7d814fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 -- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8151050 msgr2=0x7fd7d814fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8151050 0x7fd7d814fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.388 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 -- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d8073e90 msgr2=0x7fd7d8148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d8073e90 0x7fd7d8148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 -- 172.21.15.78:0/303187614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7d814f5b0 con 0x7fd7d8072350 2024-09-11T12:57:06.389 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7d7400640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e970 tx=0x7fd7c800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:06.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c800cd80 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7c8004020 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7c8004470 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7d8150420 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7d814ead0 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7d8071e10 con 0x7fd7d8072350 2024-09-11T12:57:06.391 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7fd7c8003600 con 0x7fd7d8072350 2024-09-11T12:57:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 0x7fd7b8074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.385+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fd7c8093310 con 0x7fd7d8072350 2024-09-11T12:57:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.389+0000 7fd7d6a00640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 0x7fd7b8074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.389+0000 7fd7d6a00640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 0x7fd7b8074320 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b0000 tx=0x7fd7c40a8040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:06.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.389+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd7d8150420 con 0x7fd7d8072350 2024-09-11T12:57:06.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.581+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol78091", "group_name": "subvol_grp25450", "target": ["mon-mgr", ""]}) -- 0x7fd7d8070ce0 con 0x7fd7b8071e60 2024-09-11T12:57:06.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7cf400640 1 -- 172.21.15.78:0/303187614 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fd7d8070ce0 con 0x7fd7b8071e60 2024-09-11T12:57:06.637 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp25450/subvol78091/47a17d3c-6124-4369-a42a-52d20d17bc38 2024-09-11T12:57:06.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 msgr2=0x7fd7b8074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 0x7fd7b8074320 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40b0000 tx=0x7fd7c40a8040 comp rx=0 tx=0).stop 2024-09-11T12:57:06.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 msgr2=0x7fd7d80aed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fd7c800e970 tx=0x7fd7c800ee30 comp rx=0 tx=0).stop 2024-09-11T12:57:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 shutdown_connections 2024-09-11T12:57:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd7d8151050 0x7fd7d814fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd7b8071e60 0x7fd7b8074320 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd7d8073e90 0x7fd7d8148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 --2- 172.21.15.78:0/303187614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd7d8072350 0x7fd7d80aed80 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 >> 172.21.15.78:0/303187614 conn(0x7fd7d80b3050 msgr2=0x7fd7d8158e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:06.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 shutdown_connections 2024-09-11T12:57:06.641 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.633+0000 7fd7de600640 1 -- 172.21.15.78:0/303187614 wait complete. 2024-09-11T12:57:06.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:06.658 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol41230/.meta) 2024-09-11T12:57:06.678 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:57:06.678 INFO:teuthology.orchestra.run.smithi078.stdout:version = 2 2024-09-11T12:57:06.678 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:57:06.678 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/_nogroup/subvol41230/16b05905-b236-4562-b853-703113e7d08d 2024-09-11T12:57:06.678 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:57:06.679 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:06.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:06.680 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp25450/subvol78091/.meta) 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout:version = 2 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/subvol_grp25450/subvol78091/47a17d3c-6124-4369-a42a-52d20d17bc38 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:57:06.743 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:06.745 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol41230 2024-09-11T12:57:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 -- 172.21.15.78:0/1824717286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e00cf6e0 msgr2=0x7f50e00734d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 --2- 172.21.15.78:0/1824717286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e00cf6e0 0x7f50e00734d0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f50d8009840 tx=0x7f50d801c840 comp rx=0 tx=0).stop 2024-09-11T12:57:06.930 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 -- 172.21.15.78:0/1824717286 shutdown_connections 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 --2- 172.21.15.78:0/1824717286 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e00cf6e0 0x7f50e00734d0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 --2- 172.21.15.78:0/1824717286 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0072f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 -- 172.21.15.78:0/1824717286 >> 172.21.15.78:0/1824717286 conn(0x7f50e00d0410 msgr2=0x7f50e00d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 -- 172.21.15.78:0/1824717286 shutdown_connections 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.925+0000 7f50e5000640 1 -- 172.21.15.78:0/1824717286 wait complete. 2024-09-11T12:57:06.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 Processor -- start 2024-09-11T12:57:06.932 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- start start 2024-09-11T12:57:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e00cf6e0 0x7f50e015b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0139210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0168f80 con 0x7f50e0071d90 2024-09-11T12:57:06.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50e0168e00 con 0x7f50e00cf6e0 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50e0169100 con 0x7f50e0144640 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46398/0 (socket says 172.21.15.78:46398) 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 -- 172.21.15.78:0/1151181444 learned_addr learned my addr 172.21.15.78:0/1151181444 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50de800640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0139210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dd400640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e00cf6e0 0x7f50e015b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 -- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 msgr2=0x7f50e0139210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0139210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 -- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e00cf6e0 msgr2=0x7f50e015b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e00cf6e0 0x7f50e015b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 -- 172.21.15.78:0/1151181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e0070970 con 0x7f50e0071d90 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50de800640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:06.935 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50dde00640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e9f0 tx=0x7f50d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d000cd80 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50d0004c20 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50e0070bf0 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50d0005020 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50e0164ae0 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.929+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50e015f590 con 0x7f50e0071d90 2024-09-11T12:57:06.936 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.933+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f50d00051c0 con 0x7f50e0071d90 2024-09-11T12:57:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.933+0000 7f50d6000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 0x7f50c0074350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.933+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f50d00939f0 con 0x7f50e0071d90 2024-09-11T12:57:06.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.933+0000 7f50dd400640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 0x7f50c0074350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:06.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.933+0000 7f50dd400640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 0x7f50c0074350 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f50e0137870 tx=0x7f50cc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:06.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:06.937+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f50e0070970 con 0x7f50e0071d90 2024-09-11T12:57:07.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.125+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol41230", "target": ["mon-mgr", ""]}) -- 0x7f50e00732c0 con 0x7f50c0071e90 2024-09-11T12:57:07.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.141+0000 7f50d6000640 1 -- 172.21.15.78:0/1151181444 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f50e00732c0 con 0x7f50c0071e90 2024-09-11T12:57:07.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 msgr2=0x7f50c0074350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 0x7f50c0074350 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f50e0137870 tx=0x7f50cc009040 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 msgr2=0x7f50e01fb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f50d000e9f0 tx=0x7f50d000eeb0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 shutdown_connections 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50e0144640 0x7f50e0139210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50c0071e90 0x7f50c0074350 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50e00cf6e0 0x7f50e015b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 --2- 172.21.15.78:0/1151181444 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50e0071d90 0x7f50e01fb710 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 >> 172.21.15.78:0/1151181444 conn(0x7f50e00d0410 msgr2=0x7f50e0143750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 shutdown_connections 2024-09-11T12:57:07.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.145+0000 7f50e5000640 1 -- 172.21.15.78:0/1151181444 wait complete. 2024-09-11T12:57:07.151 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.149+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.152 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.149+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.152 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.149+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.152 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.149+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.152 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.149+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.168 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.165+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.168 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.165+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.169 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.165+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.169 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.165+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.169 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:07.165+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:07.174 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol78091 subvol_grp25450 2024-09-11T12:57:07.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- 172.21.15.78:0/57079154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 msgr2=0x7f74a0073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:07.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- 172.21.15.78:0/57079154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a0073d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f748c009840 tx=0x7f748c01c820 comp rx=0 tx=0).stop 2024-09-11T12:57:07.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- 172.21.15.78:0/57079154 shutdown_connections 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- 172.21.15.78:0/57079154 >> v2:172.21.15.78:3300/0 conn(0x7f74a0074280 0x7f74a00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- 172.21.15.78:0/57079154 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a0073d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- 172.21.15.78:0/57079154 >> 172.21.15.78:0/57079154 conn(0x7f74a00b2730 msgr2=0x7f74a006c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- 172.21.15.78:0/57079154 shutdown_connections 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- 172.21.15.78:0/57079154 wait complete. 2024-09-11T12:57:07.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 Processor -- start 2024-09-11T12:57:07.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- start start 2024-09-11T12:57:07.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a0074280 0x7f74a0141430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:07.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:07.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a006ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:07.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0167500 con 0x7f74a015e8a0 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f74a0167380 con 0x7f74a0074280 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f74a4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f74a0167680 con 0x7f74a015ec80 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f749de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f749de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:46456/0 (socket says 172.21.15.78:46456) 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f749de00640 1 -- 172.21.15.78:0/1558902910 learned_addr learned my addr 172.21.15.78:0/1558902910 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f749e800640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a0074280 0x7f74a0141430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.321+0000 7f749f200640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a006ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 -- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 msgr2=0x7f74a006ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a006ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 -- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a0074280 msgr2=0x7f74a0141430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a0074280 0x7f74a0141430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:07.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 -- 172.21.15.78:0/1558902910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a01f72e0 con 0x7f74a015e8a0 2024-09-11T12:57:07.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f749de00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f748c009790 tx=0x7f748c005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:07.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0b2040 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f748c004320 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a01f7580 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c0052d0 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a01f7b40 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a00ce070 con 0x7f74a015e8a0 2024-09-11T12:57:07.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.325+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 23) ==== 95646+0+0 (secure 0 0 0) 0x7f748c0a5750 con 0x7f74a015e8a0 2024-09-11T12:57:07.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.329+0000 7f7496a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 0x7f7480074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:07.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.329+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f748c12c840 con 0x7f74a015e8a0 2024-09-11T12:57:07.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.329+0000 7f749e800640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 0x7f7480074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:07.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.329+0000 7f749e800640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 0x7f7480074320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7490007ed0 tx=0x7f7490007e60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:07.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.333+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f748c0bd060 con 0x7f74a015e8a0 2024-09-11T12:57:07.526 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:07.521+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol78091", "group_name": "subvol_grp25450", "target": ["mon-mgr", ""]}) -- 0x7f74a0071cf0 con 0x7f7480071e60 2024-09-11T12:57:08.599 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:08.597+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f748c0f3630 con 0x7f74a015e8a0 2024-09-11T12:57:12.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:57:12.790 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:57:16.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.793+0000 7f7496a00640 1 -- 172.21.15.78:0/1558902910 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f74a0071cf0 con 0x7f7480071e60 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 msgr2=0x7f7480074320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 0x7f7480074320 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7490007ed0 tx=0x7f7490007e60 comp rx=0 tx=0).stop 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 msgr2=0x7f74a00c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f748c009790 tx=0x7f748c005db0 comp rx=0 tx=0).stop 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 shutdown_connections 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f74a015ec80 0x7f74a006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:16.796 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f74a015e8a0 0x7f74a00c56c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7480071e60 0x7f7480074320 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 --2- 172.21.15.78:0/1558902910 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f74a0074280 0x7f74a0141430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 >> 172.21.15.78:0/1558902910 conn(0x7f74a00b2730 msgr2=0x7f74a0143c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 shutdown_connections 2024-09-11T12:57:16.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16.797+0000 7f74a4a00640 1 -- 172.21.15.78:0/1558902910 wait complete. 2024-09-11T12:57:16.824 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:57:16.874 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:16.881 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:57:16.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:16.889 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:57:16.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:17.014 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:17.113 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:57:17.113 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:17.113 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:17.113 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:17.122 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:17.122 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:17.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 -- 172.21.15.78:0/3675202515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 msgr2=0x7f9f24071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 --2- 172.21.15.78:0/3675202515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f24071d60 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c009c80 tx=0x7f9f1c01cd70 comp rx=0 tx=0).stop 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 -- 172.21.15.78:0/3675202515 shutdown_connections 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 --2- 172.21.15.78:0/3675202515 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f24071d60 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 --2- 172.21.15.78:0/3675202515 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f2415ba80 0x7f9f2406fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 -- 172.21.15.78:0/3675202515 >> 172.21.15.78:0/3675202515 conn(0x7f9f2406ca80 msgr2=0x7f9f240b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 -- 172.21.15.78:0/3675202515 shutdown_connections 2024-09-11T12:57:17.358 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.357+0000 7f9f29e00640 1 -- 172.21.15.78:0/3675202515 wait complete. 2024-09-11T12:57:17.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 Processor -- start 2024-09-11T12:57:17.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- start start 2024-09-11T12:57:17.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.361 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f24074240 0x7f9f241faf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2415ba80 0x7f9f24144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f9f241678c0 con 0x7f9f24070190 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f9f24167740 con 0x7f9f2415ba80 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f9f24167a40 con 0x7f9f24074240 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36498/0 (socket says 172.21.15.78:36498) 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 -- 172.21.15.78:0/406592064 learned_addr learned my addr 172.21.15.78:0/406592064 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22000640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f24074240 0x7f9f241faf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f23400640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2415ba80 0x7f9f24144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 -- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f24074240 msgr2=0x7f9f241faf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f24074240 0x7f9f241faf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 -- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2415ba80 msgr2=0x7f9f24144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.362 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2415ba80 0x7f9f24144790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 -- 172.21.15.78:0/406592064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f240b1b10 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f22a00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c530 tx=0x7f9f1400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14017070 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f140053c0 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f14012890 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f24162a00 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f24162ca0 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.361+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f24071d60 con 0x7f9f24070190 2024-09-11T12:57:17.363 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.365+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f9f1401a070 con 0x7f9f24070190 2024-09-11T12:57:17.364 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.365+0000 7f9f1b400640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 0x7f9f040744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.365+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f9f1408f8f0 con 0x7f9f24070190 2024-09-11T12:57:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.365+0000 7f9f22000640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 0x7f9f040744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.365 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.365+0000 7f9f22000640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 0x7f9f040744d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9f100058d0 tx=0x7f9f10005860 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:17.369 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.369+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f9f1405d360 con 0x7f9f24070190 2024-09-11T12:57:17.550 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.549+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9f240b1ff0 con 0x7f9f24070190 2024-09-11T12:57:17.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.549+0000 7f9f1b400640 1 -- 172.21.15.78:0/406592064 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) ==== 80+0+0 (secure 0 0 0) 0x7f9f1405cc60 con 0x7f9f24070190 2024-09-11T12:57:17.551 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 msgr2=0x7f9f040744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 0x7f9f040744d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f9f100058d0 tx=0x7f9f10005860 comp rx=0 tx=0).stop 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 msgr2=0x7f9f2406efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f9f1400c530 tx=0x7f9f1400c9f0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 shutdown_connections 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f9f04072010 0x7f9f040744d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f9f2415ba80 0x7f9f24144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f9f24074240 0x7f9f241faf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 --2- 172.21.15.78:0/406592064 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f9f24070190 0x7f9f2406efd0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 >> 172.21.15.78:0/406592064 conn(0x7f9f2406ca80 msgr2=0x7f9f2415c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:17.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 shutdown_connections 2024-09-11T12:57:17.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.553+0000 7f9f29e00640 1 -- 172.21.15.78:0/406592064 wait complete. 2024-09-11T12:57:17.573 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:57:17.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:17.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:17.620 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:57:17.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:17.620 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:57:17.672 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:17.672 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:17.672 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:17.681 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:17.682 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:17.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 -- 172.21.15.78:0/1165523135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c9c004b50 msgr2=0x7f1c9c004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 --2- 172.21.15.78:0/1165523135 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1c9c004b50 0x7f1c9c004f70 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f1ca800c2e0 tx=0x7f1ca801cf60 comp rx=0 tx=0).stop 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cb6800640 1 -- 172.21.15.78:0/1165523135 reap_dead start 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 -- 172.21.15.78:0/1165523135 shutdown_connections 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 -- 172.21.15.78:0/1165523135 >> 172.21.15.78:0/1165523135 conn(0x7f1cb806ca80 msgr2=0x7f1cb80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 -- 172.21.15.78:0/1165523135 shutdown_connections 2024-09-11T12:57:17.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.877+0000 7f1cbca00640 1 -- 172.21.15.78:0/1165523135 wait complete. 2024-09-11T12:57:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 Processor -- start 2024-09-11T12:57:17.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- start start 2024-09-11T12:57:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 0x7f1cb81f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb80b1a90 0x7f1cb80aed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb806f2e0 con 0x7f1cb80b0a20 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb806f160 con 0x7f1cb80b1a90 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb806f460 con 0x7f1c9c004b50 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36516/0 (socket says 172.21.15.78:36516) 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 -- 172.21.15.78:0/267514241 learned_addr learned my addr 172.21.15.78:0/267514241 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb5400640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb80b1a90 0x7f1cb80aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb5e00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 0x7f1cb81f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 -- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 msgr2=0x7f1cb81f8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 0x7f1cb81f8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 -- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb80b1a90 msgr2=0x7f1cb80aed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:17.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb80b1a90 0x7f1cb80aed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 -- 172.21.15.78:0/267514241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb81fc0c0 con 0x7f1cb80b0a20 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb5e00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 0x7f1cb81f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cb6800640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80b2c30 tx=0x7f1cb0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0017070 con 0x7f1cb80b0a20 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1cb0015070 con 0x7f1cb80b0a20 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0004d90 con 0x7f1cb80b0a20 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb8158610 con 0x7f1cb80b0a20 2024-09-11T12:57:17.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.881+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb8158bd0 con 0x7f1cb80b0a20 2024-09-11T12:57:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cb8158430 con 0x7f1cb80b0a20 2024-09-11T12:57:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f1cb0003980 con 0x7f1cb80b0a20 2024-09-11T12:57:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cae000640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 0x7f1c980744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f1cb0090980 con 0x7f1cb80b0a20 2024-09-11T12:57:17.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cb5e00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 0x7f1c980744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:17.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.885+0000 7f1cb5e00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 0x7f1c980744d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ab000 tx=0x7f1ca8019da0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:17.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:17.889+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1cb8158430 con 0x7f1cb80b0a20 2024-09-11T12:57:18.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1cb80b4db0 con 0x7f1cb80b0a20 2024-09-11T12:57:18.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cae000640 1 -- 172.21.15.78:0/267514241 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v151) ==== 80+0+0 (secure 0 0 0) 0x7f1cb005dda0 con 0x7f1cb80b0a20 2024-09-11T12:57:18.068 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 msgr2=0x7f1c980744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 0x7f1c980744d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1ca80ab000 tx=0x7f1ca8019da0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 msgr2=0x7f1cb80aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f1cb80b2c30 tx=0x7f1cb0010e90 comp rx=0 tx=0).stop 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 shutdown_connections 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f1cb80b0a20 0x7f1cb80aa510 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f1c98072010 0x7f1c980744d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f1cb80b1a90 0x7f1cb80aed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 --2- 172.21.15.78:0/267514241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f1c9c004b50 0x7f1cb81f8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 >> 172.21.15.78:0/267514241 conn(0x7f1cb806ca80 msgr2=0x7f1cb815cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:18.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 shutdown_connections 2024-09-11T12:57:18.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.069+0000 7f1cbca00640 1 -- 172.21.15.78:0/267514241 wait complete. 2024-09-11T12:57:18.089 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:57:18.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:18.089 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:18.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:57:18.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:18.121 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:57:18.174 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:18.175 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:18.175 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:18.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:18.184 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:18.184 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:18.184 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:18.230 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:18.230 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- 172.21.15.78:0/1724020885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4074240 msgr2=0x7faea415ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 --2- 172.21.15.78:0/1724020885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4074240 0x7faea415ba80 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7faea40b5790 tx=0x7fae9401c870 comp rx=0 tx=0).stop 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- 172.21.15.78:0/1724020885 shutdown_connections 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 --2- 172.21.15.78:0/1724020885 >> v2:172.21.15.157:3300/0 conn(0x7faea4070360 0x7faea415ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 --2- 172.21.15.78:0/1724020885 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4074240 0x7faea415ba80 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.373 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- 172.21.15.78:0/1724020885 >> 172.21.15.78:0/1724020885 conn(0x7faea406ca80 msgr2=0x7faea40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- 172.21.15.78:0/1724020885 shutdown_connections 2024-09-11T12:57:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- 172.21.15.78:0/1724020885 wait complete. 2024-09-11T12:57:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 Processor -- start 2024-09-11T12:57:18.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 -- start start 2024-09-11T12:57:18.376 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.373+0000 7faea9000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea4074240 0x7faea41513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea415e5b0 0x7faea4159f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faea4167740 con 0x7faea4070360 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faea4072920 con 0x7faea4074240 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faea41678c0 con 0x7faea415e5b0 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36536/0 (socket says 172.21.15.78:36536) 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 -- 172.21.15.78:0/1653950436 learned_addr learned my addr 172.21.15.78:0/1653950436 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 -- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea415e5b0 msgr2=0x7faea4159f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea415e5b0 0x7faea4159f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 -- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea4074240 msgr2=0x7faea41513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.377 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea4074240 0x7faea41513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 -- 172.21.15.78:0/1653950436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea4144ef0 con 0x7faea4070360 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea1e00640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fae940b0040 tx=0x7fae94004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae940b2040 con 0x7faea4070360 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae94005140 con 0x7faea4070360 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea41fa8a0 con 0x7faea4070360 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae94005590 con 0x7faea4070360 2024-09-11T12:57:18.378 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea41fae60 con 0x7faea4070360 2024-09-11T12:57:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea415ea20 con 0x7faea4070360 2024-09-11T12:57:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.377+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fae940046d0 con 0x7faea4070360 2024-09-11T12:57:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.381+0000 7fae9a000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 0x7fae800744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.379 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.381+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fae94129f00 con 0x7faea4070360 2024-09-11T12:57:18.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.381+0000 7fae9be00640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 0x7fae800744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.380 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.381+0000 7fae9be00640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 0x7fae800744d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faea4152060 tx=0x7fae8c006bb0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:18.383 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.385+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fae940bc060 con 0x7faea4070360 2024-09-11T12:57:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.585+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faea40b50d0 con 0x7faea4070360 2024-09-11T12:57:18.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.585+0000 7fae9a000640 1 -- 172.21.15.78:0/1653950436 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5236 (secure 0 0 0) 0x7fae94016490 con 0x7faea4070360 2024-09-11T12:57:18.586 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:18.586 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"btime":"2024-09-11T12:57:04:575682+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:57:18.586 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2024-09-11T12:57:18.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 msgr2=0x7fae800744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 0x7fae800744d0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7faea4152060 tx=0x7fae8c006bb0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 msgr2=0x7faea41449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fae940b0040 tx=0x7fae94004220 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 shutdown_connections 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faea415e5b0 0x7faea4159f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fae80072010 0x7fae800744d0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faea4074240 0x7faea41513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 --2- 172.21.15.78:0/1653950436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faea4070360 0x7faea41449b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 >> 172.21.15.78:0/1653950436 conn(0x7faea406ca80 msgr2=0x7faea40b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 shutdown_connections 2024-09-11T12:57:18.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.589+0000 7faea9000640 1 -- 172.21.15.78:0/1653950436 wait complete. 2024-09-11T12:57:18.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- 172.21.15.78:0/3825713946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c40b4db0 msgr2=0x7f86c40b5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- 172.21.15.78:0/3825713946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c40b4db0 0x7f86c40b5220 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f86bc009c80 tx=0x7f86bc01cd40 comp rx=0 tx=0).stop 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- 172.21.15.78:0/3825713946 shutdown_connections 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- 172.21.15.78:0/3825713946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c40b4db0 0x7f86c40b5220 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- 172.21.15.78:0/3825713946 >> v2:172.21.15.157:3300/0 conn(0x7f86c40707f0 0x7f86c415db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- 172.21.15.78:0/3825713946 >> 172.21.15.78:0/3825713946 conn(0x7f86c406ca80 msgr2=0x7f86c40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- 172.21.15.78:0/3825713946 shutdown_connections 2024-09-11T12:57:18.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- 172.21.15.78:0/3825713946 wait complete. 2024-09-11T12:57:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 Processor -- start 2024-09-11T12:57:18.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- start start 2024-09-11T12:57:18.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 0x7f86c4144a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c4073ed0 0x7f86c41fb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f86c4072530 con 0x7f86c40b4db0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f86c40723b0 con 0x7f86c40707f0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.749+0000 7f86ca400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f86c40726b0 con 0x7f86c4073ed0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36542/0 (socket says 172.21.15.78:36542) 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 -- 172.21.15.78:0/557765968 learned_addr learned my addr 172.21.15.78:0/557765968 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c2a00640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c4073ed0 0x7f86c41fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 0x7f86c4144a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 -- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c4073ed0 msgr2=0x7f86c41fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c4073ed0 0x7f86c41fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 -- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 msgr2=0x7f86c4144a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 0x7f86c4144a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 -- 172.21.15.78:0/557765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86c4148720 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 0x7f86c4144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86c3e00640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f86bc01c7b0 tx=0x7f86bc005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0b2040 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86bc0bb070 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86c4148a00 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86bc004240 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86c4163e60 con 0x7f86c40b4db0 2024-09-11T12:57:18.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86c415db60 con 0x7f86c40b4db0 2024-09-11T12:57:18.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.753+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f86bc0043e0 con 0x7f86c40b4db0 2024-09-11T12:57:18.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.757+0000 7f86bb400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 0x7f86a40744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:18.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.757+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f86bc12ad60 con 0x7f86c40b4db0 2024-09-11T12:57:18.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.757+0000 7f86c3400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 0x7f86a40744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:18.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.757+0000 7f86c3400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 0x7f86a40744d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007fd0 tx=0x7f86b4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:18.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.761+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f86bc016990 con 0x7f86c40b4db0 2024-09-11T12:57:18.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.961+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86c40b3970 con 0x7f86c40b4db0 2024-09-11T12:57:18.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.961+0000 7f86bb400640 1 -- 172.21.15.78:0/557765968 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5236 (secure 0 0 0) 0x7f86bc0f87d0 con 0x7f86c40b4db0 2024-09-11T12:57:18.962 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:18.962 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"btime":"2024-09-11T12:57:04:575682+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:57:18.962 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 msgr2=0x7f86a40744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 0x7f86a40744d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f86b4007fd0 tx=0x7f86b4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 msgr2=0x7f86c4150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f86bc01c7b0 tx=0x7f86bc005c20 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 shutdown_connections 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f86c40b4db0 0x7f86c4150500 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f86c4073ed0 0x7f86c41fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f86a4072010 0x7f86a40744d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 --2- 172.21.15.78:0/557765968 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f86c40707f0 0x7f86c4144a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 >> 172.21.15.78:0/557765968 conn(0x7f86c406ca80 msgr2=0x7f86c406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 shutdown_connections 2024-09-11T12:57:18.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:18.965+0000 7f86ca400640 1 -- 172.21.15.78:0/557765968 wait complete. 2024-09-11T12:57:18.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 -- 172.21.15.78:0/3023736624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714415ee90 msgr2=0x7f71440b4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 --2- 172.21.15.78:0/3023736624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714415ee90 0x7f71440b4520 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7130009900 tx=0x7f713001c700 comp rx=0 tx=0).stop 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 -- 172.21.15.78:0/3023736624 shutdown_connections 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 --2- 172.21.15.78:0/3023736624 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f714415ee90 0x7f71440b4520 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 --2- 172.21.15.78:0/3023736624 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144070020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.126 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 -- 172.21.15.78:0/3023736624 >> 172.21.15.78:0/3023736624 conn(0x7f714406ca80 msgr2=0x7f71440b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:19.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 -- 172.21.15.78:0/3023736624 shutdown_connections 2024-09-11T12:57:19.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.125+0000 7f714b800640 1 -- 172.21.15.78:0/3023736624 wait complete. 2024-09-11T12:57:19.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 Processor -- start 2024-09-11T12:57:19.127 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- start start 2024-09-11T12:57:19.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.129 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714415ee90 0x7f7144152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f714416b2d0 con 0x7f7144070560 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f714416b150 con 0x7f714415ee90 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f714416b450 con 0x7f7144161f70 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36562/0 (socket says 172.21.15.78:36562) 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 -- 172.21.15.78:0/3204636514 learned_addr learned my addr 172.21.15.78:0/3204636514 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7149800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7143e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714415ee90 0x7f7144152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 -- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 msgr2=0x7f7144159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 -- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714415ee90 msgr2=0x7f7144152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714415ee90 0x7f7144152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.130 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 -- 172.21.15.78:0/3204636514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71441582d0 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7149800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7148e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f713400b410 tx=0x7f713400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134015020 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7134011920 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7144158e40 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7134011d70 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7144159400 con 0x7f7144070560 2024-09-11T12:57:19.131 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.129+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71440b4520 con 0x7f7144070560 2024-09-11T12:57:19.132 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.133+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f7134018020 con 0x7f7144070560 2024-09-11T12:57:19.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.133+0000 7f7140c00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 0x7f7124074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.133+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f7134090380 con 0x7f7144070560 2024-09-11T12:57:19.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.133+0000 7f7143e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 0x7f7124074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.133 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.133+0000 7f7143e00640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 0x7f7124074500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f71300ae000 tx=0x7f713001af40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.136 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.137+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f713401f020 con 0x7f7144070560 2024-09-11T12:57:19.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.305+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7144073a30 con 0x7f7144070560 2024-09-11T12:57:19.307 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.305+0000 7f7140c00640 1 -- 172.21.15.78:0/3204636514 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16739 (secure 0 0 0) 0x7f713405ddf0 con 0x7f7144070560 2024-09-11T12:57:19.307 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:19.307 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":151,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:52.296488+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:48.378165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:49.654186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:19.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 msgr2=0x7f7124074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.309 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 0x7f7124074500 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f71300ae000 tx=0x7f713001af40 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 msgr2=0x7f71441587f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f713400b410 tx=0x7f713400b8d0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 shutdown_connections 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7144161f70 0x7f7144159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7124072040 0x7f7124074500 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f714415ee90 0x7f7144152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 --2- 172.21.15.78:0/3204636514 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7144070560 0x7f71441587f0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 >> 172.21.15.78:0/3204636514 conn(0x7f714406ca80 msgr2=0x7f714415d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 shutdown_connections 2024-09-11T12:57:19.310 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.309+0000 7f714b800640 1 -- 172.21.15.78:0/3204636514 wait complete. 2024-09-11T12:57:19.329 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:57:19.329 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 -- 172.21.15.78:0/1111104051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 msgr2=0x7ff6200cf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 --2- 172.21.15.78:0/1111104051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff6200cf7a0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7ff618009bc0 tx=0x7ff61801ca40 comp rx=0 tx=0).stop 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 -- 172.21.15.78:0/1111104051 shutdown_connections 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 --2- 172.21.15.78:0/1111104051 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff6200cf7a0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 --2- 172.21.15.78:0/1111104051 >> v2:172.21.15.157:3300/0 conn(0x7ff62015bcc0 0x7ff620160060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 -- 172.21.15.78:0/1111104051 >> 172.21.15.78:0/1111104051 conn(0x7ff62006f3a0 msgr2=0x7ff6200cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 -- 172.21.15.78:0/1111104051 shutdown_connections 2024-09-11T12:57:19.471 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.469+0000 7ff625600640 1 -- 172.21.15.78:0/1111104051 wait complete. 2024-09-11T12:57:19.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 Processor -- start 2024-09-11T12:57:19.472 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- start start 2024-09-11T12:57:19.474 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 0x7ff6201526c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff62015bcc0 0x7ff620155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff620168f40 con 0x7ff62015c780 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff620168dc0 con 0x7ff62015b8e0 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6201690c0 con 0x7ff62015bcc0 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36576/0 (socket says 172.21.15.78:36576) 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 -- 172.21.15.78:0/4159355835 learned_addr learned my addr 172.21.15.78:0/4159355835 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61ea00640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 0x7ff6201526c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61e000640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff62015bcc0 0x7ff620155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 -- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff62015bcc0 msgr2=0x7ff620155210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff62015bcc0 0x7ff620155210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.475 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 -- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 msgr2=0x7ff6201526c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 0x7ff6201526c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 -- 172.21.15.78:0/4159355835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff620155750 con 0x7ff62015c780 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61ea00640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 0x7ff6201526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff61f400640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff6180b0040 tx=0x7ff618003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6180b2040 con 0x7ff62015c780 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff6180bb070 con 0x7ff62015c780 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff618004370 con 0x7ff62015c780 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff620154750 con 0x7ff62015c780 2024-09-11T12:57:19.476 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.473+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff620156b50 con 0x7ff62015c780 2024-09-11T12:57:19.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff620160060 con 0x7ff62015c780 2024-09-11T12:57:19.477 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7ff618004510 con 0x7ff62015c780 2024-09-11T12:57:19.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff616a00640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 0x7ff6000744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7ff61812ae40 con 0x7ff62015c780 2024-09-11T12:57:19.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff61ea00640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 0x7ff6000744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.478 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.477+0000 7ff61ea00640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 0x7ff6000744d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff610001230 tx=0x7ff610009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.481 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.481+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff618016750 con 0x7ff62015c780 2024-09-11T12:57:19.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.681+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff6200695e0 con 0x7ff62015c780 2024-09-11T12:57:19.683 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff616a00640 1 -- 172.21.15.78:0/4159355835 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5236 (secure 0 0 0) 0x7ff6180f88b0 con 0x7ff62015c780 2024-09-11T12:57:19.684 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:19.684 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"btime":"2024-09-11T12:57:04:575682+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:57:19.684 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2024-09-11T12:57:19.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 msgr2=0x7ff6000744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 0x7ff6000744d0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff610001230 tx=0x7ff610009040 comp rx=0 tx=0).stop 2024-09-11T12:57:19.686 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 msgr2=0x7ff620152d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7ff6180b0040 tx=0x7ff618003f40 comp rx=0 tx=0).stop 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 shutdown_connections 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff62015c780 0x7ff620152d00 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff62015bcc0 0x7ff620155210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff600072010 0x7ff6000744d0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 --2- 172.21.15.78:0/4159355835 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff62015b8e0 0x7ff6201526c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 >> 172.21.15.78:0/4159355835 conn(0x7ff62006f3a0 msgr2=0x7ff620153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 shutdown_connections 2024-09-11T12:57:19.687 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.685+0000 7ff625600640 1 -- 172.21.15.78:0/4159355835 wait complete. 2024-09-11T12:57:19.706 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:19.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- 172.21.15.78:0/2349854123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 msgr2=0x7f0b100b54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 --2- 172.21.15.78:0/2349854123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b100b54b0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f0b100d1760 tx=0x7f0afc01c800 comp rx=0 tx=0).stop 2024-09-11T12:57:19.849 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- 172.21.15.78:0/2349854123 shutdown_connections 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 --2- 172.21.15.78:0/2349854123 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b100b54b0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 --2- 172.21.15.78:0/2349854123 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b10074060 0x7f0b1006f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 --2- 172.21.15.78:0/2349854123 >> v2:172.21.15.157:3300/0 conn(0x7f0b1015db00 0x7f0b10073b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- 172.21.15.78:0/2349854123 >> 172.21.15.78:0/2349854123 conn(0x7f0b1006c650 msgr2=0x7f0b100d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- 172.21.15.78:0/2349854123 shutdown_connections 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- 172.21.15.78:0/2349854123 wait complete. 2024-09-11T12:57:19.850 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 Processor -- start 2024-09-11T12:57:19.851 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.849+0000 7f0b16a00640 1 -- start start 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10074060 0x7f0b10157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b1015db00 0x7f0b10161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10161f00 con 0x7f0b1015dee0 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10161d80 con 0x7f0b10074060 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0b10162080 con 0x7f0b1015db00 2024-09-11T12:57:19.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10074060 0x7f0b10157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36590/0 (socket says 172.21.15.78:36590) 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b1015db00 0x7f0b10161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 -- 172.21.15.78:0/449965678 learned_addr learned my addr 172.21.15.78:0/449965678 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 -- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b1015db00 msgr2=0x7f0b10161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b1015db00 0x7f0b10161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 -- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10074060 msgr2=0x7f0b10157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10074060 0x7f0b10157d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:19.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 -- 172.21.15.78:0/449965678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b10162b90 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b14a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c940 tx=0x7f0b0800ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004460 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b08007890 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08007ce0 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b10157240 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b10157800 con 0x7f0b1015dee0 2024-09-11T12:57:19.854 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.853+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b10162530 con 0x7f0b1015dee0 2024-09-11T12:57:19.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.857+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f0b08003600 con 0x7f0b1015dee0 2024-09-11T12:57:19.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.857+0000 7f0b07e00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 0x7f0af00744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:19.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.857+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f0b08090b00 con 0x7f0b1015dee0 2024-09-11T12:57:19.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.857+0000 7f0b0fe00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 0x7f0af00744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:19.857 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.857+0000 7f0b0fe00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 0x7f0af00744d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b10167dd0 tx=0x7f0b00007600 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:19.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:19.861+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0b08013030 con 0x7f0b1015dee0 2024-09-11T12:57:20.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b100b4e20 con 0x7f0b1015dee0 2024-09-11T12:57:20.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b07e00640 1 -- 172.21.15.78:0/449965678 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5236 (secure 0 0 0) 0x7f0b0805df20 con 0x7f0b1015dee0 2024-09-11T12:57:20.064 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:20.064 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":112,"btime":"2024-09-11T12:57:04:575682+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106}],"filesystems":[{"mdsmap":{"epoch":111,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:56:50.675090+0000","modified":"2024-09-11T12:56:53.710536+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":107,"state":"up:active","state_seq":3,"addr":"172.21.15.78:6835/963798948","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":963798948},{"type":"v1","addr":"172.21.15.78:6835","nonce":963798948}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8631,"qdb_cluster":[8631]},"id":12}]} 2024-09-11T12:57:20.065 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 112 2024-09-11T12:57:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 msgr2=0x7f0af00744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 0x7f0af00744d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b10167dd0 tx=0x7f0b00007600 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 msgr2=0x7f0b101401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800c940 tx=0x7f0b0800ce00 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 shutdown_connections 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b1015dee0 0x7f0b101401a0 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b1015db00 0x7f0b10161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0af0072010 0x7f0af00744d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 --2- 172.21.15.78:0/449965678 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b10074060 0x7f0b10157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 >> 172.21.15.78:0/449965678 conn(0x7f0b1006c650 msgr2=0x7f0b100d0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 shutdown_connections 2024-09-11T12:57:20.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.065+0000 7f0b16a00640 1 -- 172.21.15.78:0/449965678 wait complete. 2024-09-11T12:57:20.086 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.225+0000 7fe163e00640 1 -- 172.21.15.78:0/1810152081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1641601d0 msgr2=0x7fe164141460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.225+0000 7fe163e00640 1 --2- 172.21.15.78:0/1810152081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1641601d0 0x7fe164141460 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009840 tx=0x7fe15c01c820 comp rx=0 tx=0).stop 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- 172.21.15.78:0/1810152081 shutdown_connections 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 --2- 172.21.15.78:0/1810152081 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe1641601d0 0x7fe164141460 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- 172.21.15.78:0/1810152081 >> 172.21.15.78:0/1810152081 conn(0x7fe1640746c0 msgr2=0x7fe1640ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- 172.21.15.78:0/1810152081 shutdown_connections 2024-09-11T12:57:20.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- 172.21.15.78:0/1810152081 wait complete. 2024-09-11T12:57:20.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 Processor -- start 2024-09-11T12:57:20.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- start start 2024-09-11T12:57:20.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe16415f9b0 0x7fe164155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164153a50 0x7fe16413d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe164141dc0 con 0x7fe16415fd90 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe164141c40 con 0x7fe164153a50 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe164141f40 con 0x7fe16415f9b0 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36600/0 (socket says 172.21.15.78:36600) 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 -- 172.21.15.78:0/3929812755 learned_addr learned my addr 172.21.15.78:0/3929812755 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe163400640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164153a50 0x7fe16413d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162a00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe16415f9b0 0x7fe164155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 -- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe16415f9b0 msgr2=0x7fe164155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe16415f9b0 0x7fe164155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 -- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164153a50 msgr2=0x7fe16413d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164153a50 0x7fe16413d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 -- 172.21.15.78:0/3929812755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe164155c20 con 0x7fe16415fd90 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.229+0000 7fe162000640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c420 tx=0x7fe15000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150017070 con 0x7fe16415fd90 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe150003600 con 0x7fe16415fd90 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe164134d30 con 0x7fe16415fd90 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150003a90 con 0x7fe16415fd90 2024-09-11T12:57:20.233 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe164151f90 con 0x7fe16415fd90 2024-09-11T12:57:20.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe164141460 con 0x7fe16415fd90 2024-09-11T12:57:20.234 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fe15001a070 con 0x7fe16415fd90 2024-09-11T12:57:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 0x7fe144074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7fe15008f780 con 0x7fe16415fd90 2024-09-11T12:57:20.235 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.233+0000 7fe162a00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 0x7fe144074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.237+0000 7fe162a00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 0x7fe144074500 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1540012f0 tx=0x7fe154009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.237+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe15005cba0 con 0x7fe16415fd90 2024-09-11T12:57:20.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.405+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe164134b20 con 0x7fe16415fd90 2024-09-11T12:57:20.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.405+0000 7fe15b400640 1 -- 172.21.15.78:0/3929812755 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) ==== 74+0+16739 (secure 0 0 0) 0x7fe150065430 con 0x7fe16415fd90 2024-09-11T12:57:20.406 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:20.407 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":151,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:56:52.296488+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:56:48.378165+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-11T12:56:49.654186+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 msgr2=0x7fe144074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 0x7fe144074500 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe1540012f0 tx=0x7fe154009040 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 msgr2=0x7fe164151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fe15000c420 tx=0x7fe15000c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 shutdown_connections 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe144072040 0x7fe144074500 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe164153a50 0x7fe16413d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe16415fd90 0x7fe164151a50 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 --2- 172.21.15.78:0/3929812755 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe16415f9b0 0x7fe164155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 >> 172.21.15.78:0/3929812755 conn(0x7fe1640746c0 msgr2=0x7fe16406f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 shutdown_connections 2024-09-11T12:57:20.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.409+0000 7fe163e00640 1 -- 172.21.15.78:0/3929812755 wait complete. 2024-09-11T12:57:20.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:57:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 -- 172.21.15.78:0/1650896424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c074240 msgr2=0x7f271c15ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 --2- 172.21.15.78:0/1650896424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c074240 0x7f271c15ba80 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f271c0b2f70 tx=0x7f271001c890 comp rx=0 tx=0).stop 2024-09-11T12:57:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 -- 172.21.15.78:0/1650896424 shutdown_connections 2024-09-11T12:57:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 --2- 172.21.15.78:0/1650896424 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 0x7f271c0700a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 --2- 172.21.15.78:0/1650896424 >> v2:172.21.15.157:3300/0 conn(0x7f271c15e5b0 0x7f271c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 --2- 172.21.15.78:0/1650896424 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c074240 0x7f271c15ba80 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 -- 172.21.15.78:0/1650896424 >> 172.21.15.78:0/1650896424 conn(0x7f271c06ca80 msgr2=0x7f271c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 -- 172.21.15.78:0/1650896424 shutdown_connections 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.569+0000 7f2723e00640 1 -- 172.21.15.78:0/1650896424 wait complete. 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 Processor -- start 2024-09-11T12:57:20.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- start start 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 0x7f271c153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 0x7f271c1538b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f271c072900 con 0x7f271c15e5b0 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f271c072780 con 0x7f271c074240 2024-09-11T12:57:20.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f271c0b4520 con 0x7f271c06fc50 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36618/0 (socket says 172.21.15.78:36618) 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 -- 172.21.15.78:0/3460761952 learned_addr learned my addr 172.21.15.78:0/3460761952 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721400640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 0x7f271c153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2720a00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 0x7f271c1538b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 -- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 msgr2=0x7f271c153f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 0x7f271c153f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 -- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 msgr2=0x7f271c1538b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 0x7f271c1538b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 -- 172.21.15.78:0/3460761952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f271c1545d0 con 0x7f271c15e5b0 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2720a00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 0x7f271c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2721e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f271400e990 tx=0x7f271400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714004d30 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f271400cb00 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f271c152af0 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2714005020 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.573+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f271c1530b0 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f271c15ba60 con 0x7f271c15e5b0 2024-09-11T12:57:20.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f27140051c0 con 0x7f271c15e5b0 2024-09-11T12:57:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2719600640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 0x7f26fc0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(151..151 src has 1..151) ==== 5709+0+0 (secure 0 0 0) 0x7f2714090d00 con 0x7f271c15e5b0 2024-09-11T12:57:20.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2721400640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 0x7f26fc0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.579 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.577+0000 7f2721400640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 0x7f26fc0744d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27100af000 tx=0x7f271001b0d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.582 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.581+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2714016030 con 0x7f271c15e5b0 2024-09-11T12:57:20.784 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.785+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f271c14ed20 con 0x7f271c15e5b0 2024-09-11T12:57:20.784 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:57:20.785+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:57:20.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.785+0000 7f2719600640 1 -- 172.21.15.78:0/3460761952 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v113) ==== 161+0+0 (secure 0 0 0) 0x7f271405e120 con 0x7f271c15e5b0 2024-09-11T12:57:20.786 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 msgr2=0x7f26fc0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 0x7f26fc0744d0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27100af000 tx=0x7f271001b0d0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 msgr2=0x7f271c1f8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f271400e990 tx=0x7f271400ee50 comp rx=0 tx=0).stop 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 shutdown_connections 2024-09-11T12:57:20.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f271c15e5b0 0x7f271c1f8580 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26fc072010 0x7f26fc0744d0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f271c074240 0x7f271c1538b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 --2- 172.21.15.78:0/3460761952 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f271c06fc50 0x7f271c153f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 >> 172.21.15.78:0/3460761952 conn(0x7f271c06ca80 msgr2=0x7f271c15cb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 shutdown_connections 2024-09-11T12:57:20.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.789+0000 7f2723e00640 1 -- 172.21.15.78:0/3460761952 wait complete. 2024-09-11T12:57:20.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:57:20.853 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:57:20.957 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- 172.21.15.78:0/792574389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec0ce380 msgr2=0x7f46ec0ce830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 --2- 172.21.15.78:0/792574389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec0ce380 0x7f46ec0ce830 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f46e4009bc0 tx=0x7f46e401cae0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- 172.21.15.78:0/792574389 shutdown_connections 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 --2- 172.21.15.78:0/792574389 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec0ce380 0x7f46ec0ce830 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 --2- 172.21.15.78:0/792574389 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec15b980 0x7f46ec06f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- 172.21.15.78:0/792574389 >> 172.21.15.78:0/792574389 conn(0x7f46ec0b2260 msgr2=0x7f46ec0b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- 172.21.15.78:0/792574389 shutdown_connections 2024-09-11T12:57:20.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- 172.21.15.78:0/792574389 wait complete. 2024-09-11T12:57:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 Processor -- start 2024-09-11T12:57:20.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.957+0000 7f46f1c00640 1 -- start start 2024-09-11T12:57:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ec06f210 0x7f46ec1f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec0ce380 0x7f46ec13de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0ceef0 con 0x7f46ec15b980 2024-09-11T12:57:20.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec0ced70 con 0x7f46ec06f210 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46ec0afe30 con 0x7f46ec0ce380 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36664/0 (socket says 172.21.15.78:36664) 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 -- 172.21.15.78:0/3747775315 learned_addr learned my addr 172.21.15.78:0/3747775315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46e3e00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec0ce380 0x7f46ec13de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eaa00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ec06f210 0x7f46ec1f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 -- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec0ce380 msgr2=0x7f46ec13de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec0ce380 0x7f46ec13de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 -- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ec06f210 msgr2=0x7f46ec1f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ec06f210 0x7f46ec1f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 -- 172.21.15.78:0/3747775315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec1f56f0 con 0x7f46ec15b980 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46eb400640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f46e401a710 tx=0x7f46e4003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e40b2040 con 0x7f46ec15b980 2024-09-11T12:57:20.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46e40a5e50 con 0x7f46ec15b980 2024-09-11T12:57:20.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e4003fd0 con 0x7f46ec15b980 2024-09-11T12:57:20.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec1f59b0 con 0x7f46ec15b980 2024-09-11T12:57:20.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1f5f70 con 0x7f46ec15b980 2024-09-11T12:57:20.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.961+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec0ce830 con 0x7f46ec15b980 2024-09-11T12:57:20.963 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.965+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f46e40be060 con 0x7f46ec15b980 2024-09-11T12:57:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.965+0000 7f46e2a00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 0x7f46c80744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.965+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5795+0+0 (secure 0 0 0) 0x7f46e412a180 con 0x7f46ec15b980 2024-09-11T12:57:20.964 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.965+0000 7f46eaa00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 0x7f46c80744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:20.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.965+0000 7f46eaa00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 0x7f46c80744d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0016e0 tx=0x7f46dc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:20.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:20.969+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f46e4016790 con 0x7f46ec15b980 2024-09-11T12:57:21.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.173+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f46ec06fab0 con 0x7f46ec15b980 2024-09-11T12:57:21.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46e2a00640 1 -- 172.21.15.78:0/3747775315 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v114) ==== 104+0+0 (secure 0 0 0) 0x7f46e40f7bf0 con 0x7f46ec15b980 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 msgr2=0x7f46c80744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 0x7f46c80744d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f46dc0016e0 tx=0x7f46dc009040 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 msgr2=0x7f46ec13e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f46e401a710 tx=0x7f46e4003f60 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 shutdown_connections 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46ec15b980 0x7f46ec13e390 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46ec0ce380 0x7f46ec13de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46c8072010 0x7f46c80744d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 --2- 172.21.15.78:0/3747775315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46ec06f210 0x7f46ec1f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:21.791 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 >> 172.21.15.78:0/3747775315 conn(0x7f46ec0b2260 msgr2=0x7f46ec15bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:21.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 shutdown_connections 2024-09-11T12:57:21.792 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:21.789+0000 7f46f1c00640 1 -- 172.21.15.78:0/3747775315 wait complete. 2024-09-11T12:57:21.814 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:57:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 -- 172.21.15.78:0/763493363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0703e0 msgr2=0x7fd21c15da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 --2- 172.21.15.78:0/763493363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0703e0 0x7fd21c15da00 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fd21c0cc840 tx=0x7fd20c01c800 comp rx=0 tx=0).stop 2024-09-11T12:57:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 -- 172.21.15.78:0/763493363 shutdown_connections 2024-09-11T12:57:22.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 --2- 172.21.15.78:0/763493363 >> v2:172.21.15.157:3300/0 conn(0x7fd21c0cf3e0 0x7fd21c15acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 --2- 172.21.15.78:0/763493363 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0703e0 0x7fd21c15da00 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 -- 172.21.15.78:0/763493363 >> 172.21.15.78:0/763493363 conn(0x7fd21c0b2c70 msgr2=0x7fd21c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 -- 172.21.15.78:0/763493363 shutdown_connections 2024-09-11T12:57:22.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.001+0000 7fd220a00640 1 -- 172.21.15.78:0/763493363 wait complete. 2024-09-11T12:57:22.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 Processor -- start 2024-09-11T12:57:22.005 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- start start 2024-09-11T12:57:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 0x7fd21c164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21c0cf3e0 0x7fd21c1533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c073640 con 0x7fd21c0707c0 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd21c0734c0 con 0x7fd21c0703e0 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd21c0737c0 con 0x7fd21c0cf3e0 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36688/0 (socket says 172.21.15.78:36688) 2024-09-11T12:57:22.008 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 -- 172.21.15.78:0/20943402 learned_addr learned my addr 172.21.15.78:0/20943402 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd21b200640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21c0cf3e0 0x7fd21c1533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd21a800640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 0x7fd21c164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 -- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21c0cf3e0 msgr2=0x7fd21c1533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21c0cf3e0 0x7fd21c1533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 -- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 msgr2=0x7fd21c164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.009 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 0x7fd21c164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 -- 172.21.15.78:0/20943402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd21c1f8950 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd21a800640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 0x7fd21c164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd213e00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c9f0 tx=0x7fd20400ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd204007bb0 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd204007e40 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd21c1f8c30 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20400f690 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.005+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd21c1f91f0 con 0x7fd21c0707c0 2024-09-11T12:57:22.010 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd21c15da00 con 0x7fd21c0707c0 2024-09-11T12:57:22.011 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fd20400f830 con 0x7fd21c0707c0 2024-09-11T12:57:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd212a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 0x7fd1f80744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 5795+0+0 (secure 0 0 0) 0x7fd204090c00 con 0x7fd21c0707c0 2024-09-11T12:57:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd21a800640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 0x7fd1f80744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.012 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.009+0000 7fd21a800640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 0x7fd1f80744d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0af000 tx=0x7fd20c01b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:22.013 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.013+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd204014030 con 0x7fd21c0707c0 2024-09-11T12:57:22.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.181+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd21c13f470 con 0x7fd21c0707c0 2024-09-11T12:57:22.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.793+0000 7fd212a00640 1 -- 172.21.15.78:0/20943402 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v153) ==== 181+0+0 (secure 0 0 0) 0x7fd20405e670 con 0x7fd21c0707c0 2024-09-11T12:57:22.794 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:57:22.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 msgr2=0x7fd1f80744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.797 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 0x7fd1f80744d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0af000 tx=0x7fd20c01b040 comp rx=0 tx=0).stop 2024-09-11T12:57:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 msgr2=0x7fd21c1f5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fd20400c9f0 tx=0x7fd20400ceb0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 shutdown_connections 2024-09-11T12:57:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd21c0cf3e0 0x7fd21c1533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.798 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd21c0707c0 0x7fd21c1f5c40 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd1f8072010 0x7fd1f80744d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 --2- 172.21.15.78:0/20943402 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd21c0703e0 0x7fd21c164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 >> 172.21.15.78:0/20943402 conn(0x7fd21c0b2c70 msgr2=0x7fd21c15b660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 shutdown_connections 2024-09-11T12:57:22.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.797+0000 7fd220a00640 1 -- 172.21.15.78:0/20943402 wait complete. 2024-09-11T12:57:22.821 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:57:22.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- 172.21.15.78:0/196955263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40cf520 msgr2=0x7fbbd40cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- 172.21.15.78:0/196955263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40cf520 0x7fbbd40cf960 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0009900 tx=0x7fbbc001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- 172.21.15.78:0/196955263 shutdown_connections 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- 172.21.15.78:0/196955263 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40cf520 0x7fbbd40cf960 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- 172.21.15.78:0/196955263 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd4152990 0x7fbbd415d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- 172.21.15.78:0/196955263 >> 172.21.15.78:0/196955263 conn(0x7fbbd40b3050 msgr2=0x7fbbd40b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- 172.21.15.78:0/196955263 shutdown_connections 2024-09-11T12:57:22.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- 172.21.15.78:0/196955263 wait complete. 2024-09-11T12:57:22.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 Processor -- start 2024-09-11T12:57:22.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- start start 2024-09-11T12:57:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd40cf520 0x7fbbd41488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd4152990 0x7fbbd41f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4169290 con 0x7fbbd40b5530 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd4169110 con 0x7fbbd4152990 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbdba00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd4169410 con 0x7fbbd40cf520 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbd9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbd9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:36710/0 (socket says 172.21.15.78:36710) 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.973+0000 7fbbd9000640 1 -- 172.21.15.78:0/384605508 learned_addr learned my addr 172.21.15.78:0/384605508 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9a00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd4152990 0x7fbbd41f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd3e00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd40cf520 0x7fbbd41488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 -- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd40cf520 msgr2=0x7fbbd41488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd40cf520 0x7fbbd41488f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 -- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd4152990 msgr2=0x7fbbd41f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd4152990 0x7fbbd41f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:22.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 -- 172.21.15.78:0/384605508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4166230 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd9000640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400ba50 tx=0x7fbbc400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4019070 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc400ccf0 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd41fb860 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc40146a0 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd41fc340 con 0x7fbbd40b5530 2024-09-11T12:57:22.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd41fb650 con 0x7fbbd40b5530 2024-09-11T12:57:22.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.977+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fbbc401c070 con 0x7fbbd40b5530 2024-09-11T12:57:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.981+0000 7fbbd0c00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 0x7fbbb40744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.981+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5357+0+0 (secure 0 0 0) 0x7fbbc4090370 con 0x7fbbd40b5530 2024-09-11T12:57:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.981+0000 7fbbd3e00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 0x7fbbb40744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:22.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.981+0000 7fbbd3e00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 0x7fbbb40744d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00af000 tx=0x7fbbc001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:22.983 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:22.985+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbbc405dde0 con 0x7fbbd40b5530 2024-09-11T12:57:23.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.153+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbbd40cf960 con 0x7fbbd40b5530 2024-09-11T12:57:23.795 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.797+0000 7fbbd0c00640 1 -- 172.21.15.78:0/384605508 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v154) ==== 169+0+0 (secure 0 0 0) 0x7fbbc405d6e0 con 0x7fbbd40b5530 2024-09-11T12:57:23.795 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:57:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.797+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 msgr2=0x7fbbb40744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.797+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 0x7fbbb40744d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00af000 tx=0x7fbbc001b120 comp rx=0 tx=0).stop 2024-09-11T12:57:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.797+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 msgr2=0x7fbbd4165bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.797+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400ba50 tx=0x7fbbc400bf10 comp rx=0 tx=0).stop 2024-09-11T12:57:23.799 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 shutdown_connections 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbb4072010 0x7fbbb40744d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbd4152990 0x7fbbd41f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbd40cf520 0x7fbbd41488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 --2- 172.21.15.78:0/384605508 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbd40b5530 0x7fbbd4165bc0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 >> 172.21.15.78:0/384605508 conn(0x7fbbd40b3050 msgr2=0x7fbbd414be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 shutdown_connections 2024-09-11T12:57:23.800 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.801+0000 7fbbdba00640 1 -- 172.21.15.78:0/384605508 wait complete. 2024-09-11T12:57:23.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:57:23.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 -- 172.21.15.78:0/1303481894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540cf7c0 msgr2=0x7ff15415cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:23.967 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 --2- 172.21.15.78:0/1303481894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540cf7c0 0x7ff15415cad0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7ff1540af2d0 tx=0x7ff14001c720 comp rx=0 tx=0).stop 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 -- 172.21.15.78:0/1303481894 shutdown_connections 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 --2- 172.21.15.78:0/1303481894 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540cf7c0 0x7ff15415cad0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 --2- 172.21.15.78:0/1303481894 >> v2:172.21.15.157:3300/0 conn(0x7ff1540aed20 0x7ff1540cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 -- 172.21.15.78:0/1303481894 >> 172.21.15.78:0/1303481894 conn(0x7ff15406ca80 msgr2=0x7ff1540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 -- 172.21.15.78:0/1303481894 shutdown_connections 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.965+0000 7ff15a200640 1 -- 172.21.15.78:0/1303481894 wait complete. 2024-09-11T12:57:23.968 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 Processor -- start 2024-09-11T12:57:23.969 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- start start 2024-09-11T12:57:23.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:23.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1540cf7c0 0x7ff1541fa190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:23.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 0x7ff154073090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:23.970 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff1540cd250 con 0x7ff1540aed20 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff1540cd0d0 con 0x7ff15415d010 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff1540cd3d0 con 0x7ff1540cf7c0 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59568/0 (socket says 172.21.15.78:59568) 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 -- 172.21.15.78:0/3192360221 learned_addr learned my addr 172.21.15.78:0/3192360221 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff152a00640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1540cf7c0 0x7ff1541fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153e00640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 0x7ff154073090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 -- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1540cf7c0 msgr2=0x7ff1541fa190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1540cf7c0 0x7ff1541fa190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 -- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 msgr2=0x7ff154073090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 0x7ff154073090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 -- 172.21.15.78:0/3192360221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1541fabe0 con 0x7ff1540aed20 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153e00640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 0x7ff154073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:23.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff153400640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff14400ead0 tx=0x7ff14400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14400ce40 con 0x7ff1540aed20 2024-09-11T12:57:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff144004c20 con 0x7ff1540aed20 2024-09-11T12:57:23.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1541faec0 con 0x7ff1540aed20 2024-09-11T12:57:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff144005280 con 0x7ff1540aed20 2024-09-11T12:57:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.969+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15414e510 con 0x7ff1540aed20 2024-09-11T12:57:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1540cf280 con 0x7ff1540aed20 2024-09-11T12:57:23.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7ff144005420 con 0x7ff1540aed20 2024-09-11T12:57:23.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff14b400640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 0x7ff1340744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:23.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7ff144090e50 con 0x7ff1540aed20 2024-09-11T12:57:23.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff152a00640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 0x7ff1340744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:23.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.973+0000 7ff152a00640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 0x7ff1340744d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff14001af60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:23.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:23.977+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff144016030 con 0x7ff1540aed20 2024-09-11T12:57:24.154 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.153+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7ff1541fb090 con 0x7ff1540aed20 2024-09-11T12:57:24.158 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.157+0000 7ff14b400640 1 -- 172.21.15.78:0/3192360221 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7ff14405e270 con 0x7ff1540aed20 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 msgr2=0x7ff1340744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 0x7ff1340744d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff1400af000 tx=0x7ff14001af60 comp rx=0 tx=0).stop 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 msgr2=0x7ff154149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7ff14400ead0 tx=0x7ff14400ef90 comp rx=0 tx=0).stop 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 shutdown_connections 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff134072010 0x7ff1340744d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff15415d010 0x7ff154073090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff1540cf7c0 0x7ff1541fa190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 --2- 172.21.15.78:0/3192360221 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff1540aed20 0x7ff154149540 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 >> 172.21.15.78:0/3192360221 conn(0x7ff15406ca80 msgr2=0x7ff15414d840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 shutdown_connections 2024-09-11T12:57:24.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.161+0000 7ff15a200640 1 -- 172.21.15.78:0/3192360221 wait complete. 2024-09-11T12:57:24.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 -- 172.21.15.78:0/3004472610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9180d0bf0 msgr2=0x7fe9180d1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 --2- 172.21.15.78:0/3004472610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9180d0bf0 0x7fe9180d1080 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fe9180d1780 tx=0x7fe91001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 -- 172.21.15.78:0/3004472610 shutdown_connections 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 --2- 172.21.15.78:0/3004472610 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe9180d0bf0 0x7fe9180d1080 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 --2- 172.21.15.78:0/3004472610 >> v2:172.21.15.157:3300/0 conn(0x7fe9180b14b0 0x7fe91815c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 -- 172.21.15.78:0/3004472610 >> 172.21.15.78:0/3004472610 conn(0x7fe918074b90 msgr2=0x7fe9180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 -- 172.21.15.78:0/3004472610 shutdown_connections 2024-09-11T12:57:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.321+0000 7fe91cc00640 1 -- 172.21.15.78:0/3004472610 wait complete. 2024-09-11T12:57:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 Processor -- start 2024-09-11T12:57:24.323 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- start start 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9180b14b0 0x7fe91813b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9180d0bf0 0x7fe918149c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe918168e50 con 0x7fe91806fdf0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe918168cd0 con 0x7fe9180d0bf0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe918168fd0 con 0x7fe9180b14b0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59582/0 (socket says 172.21.15.78:59582) 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 -- 172.21.15.78:0/940465868 learned_addr learned my addr 172.21.15.78:0/940465868 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe916800640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9180d0bf0 0x7fe918149c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915400640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9180b14b0 0x7fe91813b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.326 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 -- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9180b14b0 msgr2=0x7fe91813b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9180b14b0 0x7fe91813b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 -- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9180d0bf0 msgr2=0x7fe918149c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9180d0bf0 0x7fe918149c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 -- 172.21.15.78:0/940465868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91814a4a0 con 0x7fe91806fdf0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe915e00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe90800cc60 tx=0x7fe90800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800f650 con 0x7fe91806fdf0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90800f8e0 con 0x7fe91806fdf0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91806c880 con 0x7fe91806fdf0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90800fd30 con 0x7fe91806fdf0 2024-09-11T12:57:24.327 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91814a190 con 0x7fe91806fdf0 2024-09-11T12:57:24.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.325+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91815c100 con 0x7fe91806fdf0 2024-09-11T12:57:24.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.329+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fe908018400 con 0x7fe91806fdf0 2024-09-11T12:57:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.329+0000 7fe90e000640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 0x7fe8f80744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.329+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7fe908090fd0 con 0x7fe91806fdf0 2024-09-11T12:57:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.329+0000 7fe915400640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 0x7fe8f80744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.329 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.329+0000 7fe915400640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 0x7fe8f80744d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe9180ab160 tx=0x7fe904009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:24.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.333+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe908014030 con 0x7fe91806fdf0 2024-09-11T12:57:24.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.557+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7fe91815b880 con 0x7fe91806fdf0 2024-09-11T12:57:24.805 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe90e000640 1 -- 172.21.15.78:0/940465868 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v428) ==== 145+0+0 (secure 0 0 0) 0x7fe90805ea40 con 0x7fe91806fdf0 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 msgr2=0x7fe8f80744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 0x7fe8f80744d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fe9180ab160 tx=0x7fe904009040 comp rx=0 tx=0).stop 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 msgr2=0x7fe91806d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fe90800cc60 tx=0x7fe90800a620 comp rx=0 tx=0).stop 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 shutdown_connections 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8f8072010 0x7fe8f80744d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe9180d0bf0 0x7fe918149c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe9180b14b0 0x7fe91813b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 --2- 172.21.15.78:0/940465868 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe91806fdf0 0x7fe91806d360 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 >> 172.21.15.78:0/940465868 conn(0x7fe918074b90 msgr2=0x7fe91814d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 shutdown_connections 2024-09-11T12:57:24.808 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.805+0000 7fe91cc00640 1 -- 172.21.15.78:0/940465868 wait complete. 2024-09-11T12:57:24.827 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-11T12:57:24.828 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-11T12:57:24.828 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-11T12:57:24.828 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 -- 172.21.15.78:0/3048470578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 msgr2=0x7f88dc0b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 --2- 172.21.15.78:0/3048470578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc0b4db0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f88d4009840 tx=0x7f88d401c760 comp rx=0 tx=0).stop 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 -- 172.21.15.78:0/3048470578 shutdown_connections 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 --2- 172.21.15.78:0/3048470578 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc0b4db0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 --2- 172.21.15.78:0/3048470578 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 0x7f88dc070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 -- 172.21.15.78:0/3048470578 >> 172.21.15.78:0/3048470578 conn(0x7f88dc06ca80 msgr2=0x7f88dc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 -- 172.21.15.78:0/3048470578 shutdown_connections 2024-09-11T12:57:24.971 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.969+0000 7f88e4600640 1 -- 172.21.15.78:0/3048470578 wait complete. 2024-09-11T12:57:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 Processor -- start 2024-09-11T12:57:24.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- start start 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 0x7f88dc1588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc14f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc06f270 con 0x7f88dc14fa60 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88dc06f0f0 con 0x7f88dc157e30 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f88dc06f3f0 con 0x7f88dc157a50 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59592/0 (socket says 172.21.15.78:59592) 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 -- 172.21.15.78:0/2629605120 learned_addr learned my addr 172.21.15.78:0/2629605120 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:24.974 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e1c00640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 0x7f88dc1588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e1200640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc14f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 -- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 msgr2=0x7f88dc1588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 0x7f88dc1588c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 -- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 msgr2=0x7f88dc14f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc14f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:24.975 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 -- 172.21.15.78:0/2629605120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88dc150040 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e2600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f88d40b0040 tx=0x7f88d40a6540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d40bd020 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88d4004330 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88dc0ad9a0 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88d4003820 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.973+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88dc0adf60 con 0x7f88dc14fa60 2024-09-11T12:57:24.976 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88dc070780 con 0x7f88dc14fa60 2024-09-11T12:57:24.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f88d40c0020 con 0x7f88dc14fa60 2024-09-11T12:57:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88d9600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 0x7f88bc0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7f88d412b4a0 con 0x7f88dc14fa60 2024-09-11T12:57:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88e1c00640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 0x7f88bc0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:24.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.977+0000 7f88e1c00640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 0x7f88bc0744d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88dc158850 tx=0x7f88d0010040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:24.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:24.981+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f88d40c7030 con 0x7f88dc14fa60 2024-09-11T12:57:25.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88dc0cc580 con 0x7f88dc14fa60 2024-09-11T12:57:25.149 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88d9600640 1 -- 172.21.15.78:0/2629605120 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v154) ==== 74+0+13040 (secure 0 0 0) 0x7f88d410de20 con 0x7f88dc14fa60 2024-09-11T12:57:25.149 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:25.149 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":154,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:23.795766+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":149,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.78:6835/963798948":"2024-09-12T12:57:20.786507+0000","172.21.15.78:6833/963798948":"2024-09-12T12:57:20.786507+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 msgr2=0x7f88bc0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 0x7f88bc0744d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f88dc158850 tx=0x7f88d0010040 comp rx=0 tx=0).stop 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 msgr2=0x7f88dc0ae480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f88d40b0040 tx=0x7f88d40a6540 comp rx=0 tx=0).stop 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 shutdown_connections 2024-09-11T12:57:25.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f88dc14fa60 0x7f88dc0ae480 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f88bc072010 0x7f88bc0744d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f88dc157e30 0x7f88dc14f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 --2- 172.21.15.78:0/2629605120 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f88dc157a50 0x7f88dc1588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 >> 172.21.15.78:0/2629605120 conn(0x7f88dc06ca80 msgr2=0x7f88dc14a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.149+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 shutdown_connections 2024-09-11T12:57:25.152 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.153+0000 7f88e4600640 1 -- 172.21.15.78:0/2629605120 wait complete. 2024-09-11T12:57:25.171 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-11T12:49:33.130780+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T12:57:25.171 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- 172.21.15.78:0/89518917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 msgr2=0x7f4ca80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 --2- 172.21.15.78:0/89518917 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4ca80051d0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f4cbc009ec0 tx=0x7f4cbc01d610 comp rx=0 tx=0).stop 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cc3e00640 1 -- 172.21.15.78:0/89518917 reap_dead start 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- 172.21.15.78:0/89518917 shutdown_connections 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- 172.21.15.78:0/89518917 >> 172.21.15.78:0/89518917 conn(0x7f4cc4071dd0 msgr2=0x7f4cc40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- 172.21.15.78:0/89518917 shutdown_connections 2024-09-11T12:57:25.317 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- 172.21.15.78:0/89518917 wait complete. 2024-09-11T12:57:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 Processor -- start 2024-09-11T12:57:25.318 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.317+0000 7f4cca400640 1 -- start start 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cc41f3e60 0x7f4cc41577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cc4152000 0x7f4cc415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc406fd90 con 0x7f4ca8004d50 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc4151e80 con 0x7f4cc4152000 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f4cc406ff10 con 0x7f4cc41f3e60 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59618/0 (socket says 172.21.15.78:59618) 2024-09-11T12:57:25.320 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 -- 172.21.15.78:0/2935030574 learned_addr learned my addr 172.21.15.78:0/2935030574 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 -- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cc41f3e60 msgr2=0x7f4cc41577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cc41f3e60 0x7f4cc41577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 -- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cc4152000 msgr2=0x7f4cc415d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cc4152000 0x7f4cc415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 -- 172.21.15.78:0/2935030574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc4152900 con 0x7f4ca8004d50 2024-09-11T12:57:25.321 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cc3400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e970 tx=0x7f4cb400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb400ccc0 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cb400ce60 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4005260 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc4152be0 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc41531a0 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cc40b1a10 con 0x7f4ca8004d50 2024-09-11T12:57:25.322 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.321+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f4cb4005400 con 0x7f4ca8004d50 2024-09-11T12:57:25.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.325+0000 7f4cbb400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 0x7f4ca4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.325+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7f4cb4090e60 con 0x7f4ca8004d50 2024-09-11T12:57:25.324 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.325+0000 7f4cc2a00640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 0x7f4ca4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.325 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.325+0000 7f4cc2a00640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 0x7f4ca4074500 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4151400 tx=0x7f4cb00015d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:25.328 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.329+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f4cb405e8d0 con 0x7f4ca8004d50 2024-09-11T12:57:25.493 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.493+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4cc4157d20 con 0x7f4ca8004d50 2024-09-11T12:57:25.494 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.493+0000 7f4cbb400640 1 -- 172.21.15.78:0/2935030574 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v154) ==== 93+0+10 (secure 0 0 0) 0x7f4cb405e1d0 con 0x7f4ca8004d50 2024-09-11T12:57:25.495 INFO:teuthology.orchestra.run.smithi078.stdout:pg_num: 1 2024-09-11T12:57:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 msgr2=0x7f4ca4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.496 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 0x7f4ca4074500 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4151400 tx=0x7f4cb00015d0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 msgr2=0x7f4cc4153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f4cb400e970 tx=0x7f4cb400ee30 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 shutdown_connections 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f4ca4072040 0x7f4ca4074500 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f4cc4152000 0x7f4cc415d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f4cc41f3e60 0x7f4cc41577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 --2- 172.21.15.78:0/2935030574 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4ca8004d50 0x7f4cc4153980 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 >> 172.21.15.78:0/2935030574 conn(0x7f4cc4071dd0 msgr2=0x7f4cc40b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 shutdown_connections 2024-09-11T12:57:25.497 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.497+0000 7f4cca400640 1 -- 172.21.15.78:0/2935030574 wait complete. 2024-09-11T12:57:25.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 -- 172.21.15.78:0/903208311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157e30 msgr2=0x7efc840b4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 --2- 172.21.15.78:0/903208311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157e30 0x7efc840b4db0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7efc7c009840 tx=0x7efc7c01c760 comp rx=0 tx=0).stop 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 -- 172.21.15.78:0/903208311 shutdown_connections 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 --2- 172.21.15.78:0/903208311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157e30 0x7efc840b4db0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 --2- 172.21.15.78:0/903208311 >> v2:172.21.15.157:3300/0 conn(0x7efc84157a50 0x7efc84070780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 -- 172.21.15.78:0/903208311 >> 172.21.15.78:0/903208311 conn(0x7efc8406ca80 msgr2=0x7efc840b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 -- 172.21.15.78:0/903208311 shutdown_connections 2024-09-11T12:57:25.659 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.657+0000 7efc8a800640 1 -- 172.21.15.78:0/903208311 wait complete. 2024-09-11T12:57:25.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 Processor -- start 2024-09-11T12:57:25.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- start start 2024-09-11T12:57:25.661 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157a50 0x7efc84152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc84157e30 0x7efc840ce160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efc8406f230 con 0x7efc84153b60 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efc8406f0b0 con 0x7efc84157e30 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efc8406f3b0 con 0x7efc84157a50 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59634/0 (socket says 172.21.15.78:59634) 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 -- 172.21.15.78:0/2858678984 learned_addr learned my addr 172.21.15.78:0/2858678984 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83400640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157a50 0x7efc84152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc82a00640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc84157e30 0x7efc840ce160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 -- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157a50 msgr2=0x7efc84152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157a50 0x7efc84152ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.662 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 -- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc84157e30 msgr2=0x7efc840ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:25.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc84157e30 0x7efc840ce160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:25.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 -- 172.21.15.78:0/2858678984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8415bd00 con 0x7efc84153b60 2024-09-11T12:57:25.663 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc83e00640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7efc7c01a390 tx=0x7efc7c0a6660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c0040c0 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8415bfe0 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc7c004350 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc7c005290 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.661+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc840ce6a0 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc840cda30 con 0x7efc84153b60 2024-09-11T12:57:25.664 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7efc7c0c0020 con 0x7efc84153b60 2024-09-11T12:57:25.665 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc7b400640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 0x7efc640744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7efc7c12b580 con 0x7efc84153b60 2024-09-11T12:57:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc83400640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 0x7efc640744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:25.666 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.665+0000 7efc83400640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 0x7efc640744d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efc74009980 tx=0x7efc74009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:25.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.669+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efc7c0c7020 con 0x7efc84153b60 2024-09-11T12:57:25.894 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:25.893+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7efc84070780 con 0x7efc84153b60 2024-09-11T12:57:26.813 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.813+0000 7efc7b400640 1 -- 172.21.15.78:0/2858678984 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v430) ==== 166+0+0 (secure 0 0 0) 0x7efc7c016410 con 0x7efc84153b60 2024-09-11T12:57:26.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 msgr2=0x7efc640744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 0x7efc640744d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7efc74009980 tx=0x7efc74009500 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 msgr2=0x7efc840ae8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7efc7c01a390 tx=0x7efc7c0a6660 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 shutdown_connections 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efc84153b60 0x7efc840ae8c0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efc64072010 0x7efc640744d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efc84157e30 0x7efc840ce160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 --2- 172.21.15.78:0/2858678984 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efc84157a50 0x7efc84152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 >> 172.21.15.78:0/2858678984 conn(0x7efc8406ca80 msgr2=0x7efc84156b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 shutdown_connections 2024-09-11T12:57:26.816 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.817+0000 7efc8a800640 1 -- 172.21.15.78:0/2858678984 wait complete. 2024-09-11T12:57:26.839 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-11T12:57:26.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- 172.21.15.78:0/2275129734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe90415ba80 msgr2=0x7fe90406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:26.985 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 --2- 172.21.15.78:0/2275129734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe90415ba80 0x7fe90406fc50 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009840 tx=0x7fe8f001c7f0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- 172.21.15.78:0/2275129734 shutdown_connections 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 --2- 172.21.15.78:0/2275129734 >> v2:172.21.15.157:3300/0 conn(0x7fe904070190 0x7fe904071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 --2- 172.21.15.78:0/2275129734 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe90415ba80 0x7fe90406fc50 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- 172.21.15.78:0/2275129734 >> 172.21.15.78:0/2275129734 conn(0x7fe90406ca80 msgr2=0x7fe9040b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- 172.21.15.78:0/2275129734 shutdown_connections 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- 172.21.15.78:0/2275129734 wait complete. 2024-09-11T12:57:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 Processor -- start 2024-09-11T12:57:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.985+0000 7fe908e00640 1 -- start start 2024-09-11T12:57:26.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe904070190 0x7fe904154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:26.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90415ba80 0x7fe90414dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe9041658b0 con 0x7fe904074240 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe904165730 con 0x7fe90415ba80 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe904165a30 con 0x7fe904070190 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59642/0 (socket says 172.21.15.78:59642) 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 -- 172.21.15.78:0/3305324580 learned_addr learned my addr 172.21.15.78:0/3305324580 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe902800640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90415ba80 0x7fe90414dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe904070190 0x7fe904154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 -- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe904070190 msgr2=0x7fe904154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe904070190 0x7fe904154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 -- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90415ba80 msgr2=0x7fe90414dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90415ba80 0x7fe90414dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 -- 172.21.15.78:0/3305324580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9040735e0 con 0x7fe904074240 2024-09-11T12:57:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe901400640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009790 tx=0x7fe8f00a57a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00b2040 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8f0005020 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00044f0 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe904153980 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe904153f40 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.989+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe904071d60 con 0x7fe904074240 2024-09-11T12:57:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.993+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fe8f0005430 con 0x7fe904074240 2024-09-11T12:57:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.993+0000 7fe8fa000640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 0x7fe8e40744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.993+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7fe8f0129e40 con 0x7fe904074240 2024-09-11T12:57:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.993+0000 7fe901e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 0x7fe8e40744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:26.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.993+0000 7fe901e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 0x7fe8e40744d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000f70 tx=0x7fe8f4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:26.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:26.997+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe8f00bc060 con 0x7fe904074240 2024-09-11T12:57:27.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.161+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe904075720 con 0x7fe904074240 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.161+0000 7fe8fa000640 1 -- 172.21.15.78:0/3305324580 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+1998 (secure 0 0 0) 0x7fe8f0016410 con 0x7fe904074240 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:mon advanced mon_op_complaint_time 120 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 6675.830787 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 4602.351245 2024-09-11T12:57:27.162 INFO:teuthology.orchestra.run.smithi078.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 4910.344210 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 4772.185724 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 3797.308906 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 3701.863896 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 3801.222813 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 3681.100574 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds 20/20 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_mds_balancer 20/20 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced debug_ms 2/2 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_frag true 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_debug_scatterstat true 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds dev mds_verify_scatter true 2024-09-11T12:57:27.163 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_mon_op_timeout 900 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:mds advanced rados_osd_op_timeout 900 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced client_mount_timeout 600 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_client 20/20 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced debug_ms 1/1 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_mon_op_timeout 900 2024-09-11T12:57:27.164 INFO:teuthology.orchestra.run.smithi078.stdout:client advanced rados_osd_op_timeout 900 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 msgr2=0x7fe8e40744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 0x7fe8e40744d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4000f70 tx=0x7fe8f4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 msgr2=0x7fe9041f8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009790 tx=0x7fe8f00a57a0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 shutdown_connections 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe8e4072010 0x7fe8e40744d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe90415ba80 0x7fe90414dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe904074240 0x7fe9041f8c10 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 --2- 172.21.15.78:0/3305324580 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe904070190 0x7fe904154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 >> 172.21.15.78:0/3305324580 conn(0x7fe90406ca80 msgr2=0x7fe9040cf130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 shutdown_connections 2024-09-11T12:57:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.165+0000 7fe908e00640 1 -- 172.21.15.78:0/3305324580 wait complete. 2024-09-11T12:57:27.185 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 -- 172.21.15.78:0/1097331343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 msgr2=0x7f6db0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/1097331343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6db0004c90 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f6db8001740 tx=0x7f6db801c800 comp rx=0 tx=0).stop 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 -- 172.21.15.78:0/1097331343 shutdown_connections 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/1097331343 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6db0004c90 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 -- 172.21.15.78:0/1097331343 >> 172.21.15.78:0/1097331343 conn(0x7f6dcc071dd0 msgr2=0x7f6dcc0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 -- 172.21.15.78:0/1097331343 shutdown_connections 2024-09-11T12:57:27.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.329+0000 7f6dd1e00640 1 -- 172.21.15.78:0/1097331343 wait complete. 2024-09-11T12:57:27.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 Processor -- start 2024-09-11T12:57:27.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- start start 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 0x7f6dcc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc13bfd0 0x7f6dcc13c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc141f20 con 0x7f6db0004810 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6dcc141da0 con 0x7f6dcc13bfd0 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6dcc0b1a10 con 0x7f6dcc0b11d0 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59652/0 (socket says 172.21.15.78:59652) 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 -- 172.21.15.78:0/390360185 learned_addr learned my addr 172.21.15.78:0/390360185 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dca000640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 0x7f6dcc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcb400640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc13bfd0 0x7f6dcc13c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 -- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 msgr2=0x7f6dcc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 0x7f6dcc15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 -- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc13bfd0 msgr2=0x7f6dcc13c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc13bfd0 0x7f6dcc13c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 -- 172.21.15.78:0/390360185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dcc13c960 con 0x7f6db0004810 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dca000640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 0x7f6dcc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dcaa00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006680 tx=0x7f6dbc0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc015070 con 0x7f6db0004810 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6dbc011900 con 0x7f6db0004810 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dcc0b1660 con 0x7f6db0004810 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc011cf0 con 0x7f6db0004810 2024-09-11T12:57:27.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.333+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dcc06ec70 con 0x7f6db0004810 2024-09-11T12:57:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dcc0b1f70 con 0x7f6db0004810 2024-09-11T12:57:27.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f6dbc018070 con 0x7f6db0004810 2024-09-11T12:57:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dc3400640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 0x7f6dac0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7f6dbc091110 con 0x7f6db0004810 2024-09-11T12:57:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dca000640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 0x7f6dac0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.337+0000 7f6dca000640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 0x7f6dac0744d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6db80a9000 tx=0x7f6db80b1040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.341+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6dbc05eb80 con 0x7f6db0004810 2024-09-11T12:57:27.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.509+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6dcc076290 con 0x7f6db0004810 2024-09-11T12:57:27.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dc3400640 1 -- 172.21.15.78:0/390360185 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f6dbc05e480 con 0x7f6db0004810 2024-09-11T12:57:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 msgr2=0x7f6dac0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 0x7f6dac0744d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6db80a9000 tx=0x7f6db80b1040 comp rx=0 tx=0).stop 2024-09-11T12:57:27.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 msgr2=0x7f6dcc1fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc006680 tx=0x7f6dbc0039d0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 shutdown_connections 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6dac072010 0x7f6dac0744d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6dcc13bfd0 0x7f6dcc13c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6dcc0b11d0 0x7f6dcc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 --2- 172.21.15.78:0/390360185 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6db0004810 0x7f6dcc1fb190 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 >> 172.21.15.78:0/390360185 conn(0x7f6dcc071dd0 msgr2=0x7f6dcc0ccc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 shutdown_connections 2024-09-11T12:57:27.515 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.513+0000 7f6dd1e00640 1 -- 172.21.15.78:0/390360185 wait complete. 2024-09-11T12:57:27.533 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:57:27.534 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:27.642 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:27.728 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:57:27.729 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:27.729 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:27.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:27.738 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:27.738 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:27.748 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:27.748 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:27.748 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:27.787 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:27.787 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:27.787 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:27.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:27.798 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-11T12:57:27.798 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- 172.21.15.78:0/2522925241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 msgr2=0x7fc4e40700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/2522925241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e40700e0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4074f40 tx=0x7fc4dc01cd80 comp rx=0 tx=0).stop 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- 172.21.15.78:0/2522925241 shutdown_connections 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/2522925241 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e40700e0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/2522925241 >> v2:172.21.15.157:3300/0 conn(0x7fc4e415aec0 0x7fc4e4070830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/2522925241 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 0x7fc4e4074260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- 172.21.15.78:0/2522925241 >> 172.21.15.78:0/2522925241 conn(0x7fc4e406ca80 msgr2=0x7fc4e40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- 172.21.15.78:0/2522925241 shutdown_connections 2024-09-11T12:57:27.973 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- 172.21.15.78:0/2522925241 wait complete. 2024-09-11T12:57:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 Processor -- start 2024-09-11T12:57:27.977 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- start start 2024-09-11T12:57:27.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 0x7fc4e4151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e415aec0 0x7fc4e4152300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e415f2f0 con 0x7fc4e406fc50 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4e415f170 con 0x7fc4e415aec0 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4e4151010 con 0x7fc4e40cf760 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59680/0 (socket says 172.21.15.78:59680) 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 -- 172.21.15.78:0/3258438950 learned_addr learned my addr 172.21.15.78:0/3258438950 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e2800640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e415aec0 0x7fc4e4152300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.979 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4dbe00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 0x7fc4e4151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 -- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 msgr2=0x7fc4e4151dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 0x7fc4e4151dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 -- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e415aec0 msgr2=0x7fc4e4152300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e415aec0 0x7fc4e4152300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.973+0000 7fc4e1e00640 1 -- 172.21.15.78:0/3258438950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4e4154d20 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4e1e00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d400ccc0 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4d400ce60 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4005280 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4e4155000 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4e414a9d0 con 0x7fc4e406fc50 2024-09-11T12:57:27.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4e4070400 con 0x7fc4e406fc50 2024-09-11T12:57:27.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fc4d4005420 con 0x7fc4e406fc50 2024-09-11T12:57:27.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 0x7fc4c00744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:27.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7fc4d4091040 con 0x7fc4e406fc50 2024-09-11T12:57:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.977+0000 7fc4dbe00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 0x7fc4c00744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.981+0000 7fc4dbe00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 0x7fc4c00744d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005e40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:27.982 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:27.981+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc4d405eab0 con 0x7fc4e406fc50 2024-09-11T12:57:28.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.181+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc4e4074240 con 0x7fc4e406fc50 2024-09-11T12:57:28.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.181+0000 7fc4da000640 1 -- 172.21.15.78:0/3258438950 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+3922 (secure 0 0 0) 0x7fc4d405e3b0 con 0x7fc4e406fc50 2024-09-11T12:57:28.182 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:28.182 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":114,"btime":"2024-09-11T12:57:21:788617+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/1027148600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":1027148600},{"type":"v1","addr":"172.21.15.78:6834","nonce":1027148600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":105},{"gid":7151,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/1452874547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":1452874547},{"type":"v1","addr":"172.21.15.157:6801","nonce":1452874547}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":7154,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/2590056176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":2590056176},{"type":"v1","addr":"172.21.15.157:6835","nonce":2590056176}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":106},{"gid":8958,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3650176442","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3650176442},{"type":"v1","addr":"172.21.15.78:6835","nonce":3650176442}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":114}],"filesystems":[]} 2024-09-11T12:57:28.182 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 114 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 msgr2=0x7fc4c00744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 0x7fc4c00744d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc005fd0 tx=0x7fc4cc005e40 comp rx=0 tx=0).stop 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 msgr2=0x7fc4e413a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400e9f0 tx=0x7fc4d400eeb0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 shutdown_connections 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4c0072010 0x7fc4c00744d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4e415aec0 0x7fc4e4152300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4e40cf760 0x7fc4e4151dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 --2- 172.21.15.78:0/3258438950 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4e406fc50 0x7fc4e413a6f0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 >> 172.21.15.78:0/3258438950 conn(0x7fc4e406ca80 msgr2=0x7fc4e40ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:28.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 shutdown_connections 2024-09-11T12:57:28.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.185+0000 7fc4e8c00640 1 -- 172.21.15.78:0/3258438950 wait complete. 2024-09-11T12:57:28.203 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-11T12:57:28.203 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-11T12:57:28.203 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:57:28.203 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:57:28.203 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-11T12:57:28.203 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-11T12:57:28.204 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:57:28.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:57:28.204 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-11T12:57:28.204 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-11T12:57:28.204 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:57:28.204 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:57:28.204 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-11T12:57:28.205 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-11T12:57:28.205 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:57:28.205 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:57:28.216 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:57:28.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-11T12:57:28.217 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:57:28.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-11T12:57:28.219 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:57:28.219 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-11T12:57:28.220 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:57:28.221 DEBUG:teuthology.orchestra.run.smithi157:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-11T12:57:28.265 INFO:tasks.ceph.mds.a:Started 2024-09-11T12:57:28.265 INFO:tasks.ceph.mds.d:Started 2024-09-11T12:57:28.266 DEBUG:teuthology.parallel:result is None 2024-09-11T12:57:28.266 DEBUG:teuthology.parallel:result is None 2024-09-11T12:57:28.266 INFO:tasks.ceph.mds.b:Started 2024-09-11T12:57:28.266 DEBUG:teuthology.parallel:result is None 2024-09-11T12:57:28.267 INFO:tasks.ceph.mds.c:Started 2024-09-11T12:57:28.267 DEBUG:teuthology.parallel:result is None 2024-09-11T12:57:28.267 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-11T12:57:28.351 INFO:tasks.ceph.mds.c.smithi078.stdout:starting mds.c at 2024-09-11T12:57:28.352 INFO:tasks.ceph.mds.a.smithi078.stdout:starting mds.a at 2024-09-11T12:57:28.356 INFO:tasks.ceph.mds.b.smithi157.stdout:starting mds.b at 2024-09-11T12:57:28.356 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:57:28.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 -- 172.21.15.78:0/3174419121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0cf2f0 msgr2=0x7f5b1c1413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 --2- 172.21.15.78:0/3174419121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1413d0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009c80 tx=0x7f5b1801cd10 comp rx=0 tx=0).stop 2024-09-11T12:57:28.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 -- 172.21.15.78:0/3174419121 shutdown_connections 2024-09-11T12:57:28.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 --2- 172.21.15.78:0/3174419121 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1413d0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 --2- 172.21.15.78:0/3174419121 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c15f930 0x7f5b1c13efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 -- 172.21.15.78:0/3174419121 >> 172.21.15.78:0/3174419121 conn(0x7f5b1c0746c0 msgr2=0x7f5b1c0ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:28.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 -- 172.21.15.78:0/3174419121 shutdown_connections 2024-09-11T12:57:28.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.417+0000 7f5b23200640 1 -- 172.21.15.78:0/3174419121 wait complete. 2024-09-11T12:57:28.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 Processor -- start 2024-09-11T12:57:28.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- start start 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1fb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 0x7f5b1c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c168c70 con 0x7f5b1c15f930 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5b1c168af0 con 0x7f5b1c15fd10 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5b1c168df0 con 0x7f5b1c0cf2f0 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b22800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 0x7f5b1c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59716/0 (socket says 172.21.15.78:59716) 2024-09-11T12:57:28.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1fb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:28.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 -- 172.21.15.78:0/2944628160 learned_addr learned my addr 172.21.15.78:0/2944628160 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 -- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c0cf2f0 msgr2=0x7f5b1c1fb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1fb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 -- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 msgr2=0x7f5b1c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 0x7f5b1c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 -- 172.21.15.78:0/2944628160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b1c1380b0 con 0x7f5b1c15f930 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b22800640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 0x7f5b1c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b21400640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c001ec0 tx=0x7f5b0c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c017070 con 0x7f5b1c15f930 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b0c0053c0 con 0x7f5b1c15f930 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c003860 con 0x7f5b1c15f930 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b1c06e030 con 0x7f5b1c15f930 2024-09-11T12:57:28.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b1c1fb1b0 con 0x7f5b1c15f930 2024-09-11T12:57:28.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.421+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b1c1413d0 con 0x7f5b1c15f930 2024-09-11T12:57:28.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.425+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f5b0c01a070 con 0x7f5b1c15f930 2024-09-11T12:57:28.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.425+0000 7f5b16000640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 0x7f5b000744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:28.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.425+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 4943+0+0 (secure 0 0 0) 0x7f5b0c08f820 con 0x7f5b1c15f930 2024-09-11T12:57:28.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.425+0000 7f5b21e00640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 0x7f5b000744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:28.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.425+0000 7f5b21e00640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 0x7f5b000744d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1f5580 tx=0x7f5b1000b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:28.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.429+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5b0c05cc40 con 0x7f5b1c15f930 2024-09-11T12:57:28.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.605+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5b1c06e4e0 con 0x7f5b1c15f930 2024-09-11T12:57:28.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.825+0000 7f5b16000640 1 -- 172.21.15.78:0/2944628160 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v156) ==== 97+0+0 (secure 0 0 0) 0x7f5b0c061190 con 0x7f5b1c15f930 2024-09-11T12:57:28.825 INFO:teuthology.orchestra.run.smithi078.stderr: removed all blocklist entries 2024-09-11T12:57:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 msgr2=0x7f5b000744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 0x7f5b000744d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c1f5580 tx=0x7f5b1000b390 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 msgr2=0x7f5b1c137b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c001ec0 tx=0x7f5b0c00c890 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 shutdown_connections 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5b00072010 0x7f5b000744d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5b1c15fd10 0x7f5b1c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5b1c15f930 0x7f5b1c137b70 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 --2- 172.21.15.78:0/2944628160 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5b1c0cf2f0 0x7f5b1c1fb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 >> 172.21.15.78:0/2944628160 conn(0x7f5b1c0746c0 msgr2=0x7f5b1c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 shutdown_connections 2024-09-11T12:57:28.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.829+0000 7f5b23200640 1 -- 172.21.15.78:0/2944628160 wait complete. 2024-09-11T12:57:28.852 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-11T12:57:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- 172.21.15.78:0/4233843977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 msgr2=0x7fa35c06ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 --2- 172.21.15.78:0/4233843977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c06ff80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fa354009840 tx=0x7fa35401c820 comp rx=0 tx=0).stop 2024-09-11T12:57:28.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- 172.21.15.78:0/4233843977 shutdown_connections 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 --2- 172.21.15.78:0/4233843977 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c06ff80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- 172.21.15.78:0/4233843977 >> 172.21.15.78:0/4233843977 conn(0x7fa35c0cfeb0 msgr2=0x7fa35c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- 172.21.15.78:0/4233843977 shutdown_connections 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- 172.21.15.78:0/4233843977 wait complete. 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 Processor -- start 2024-09-11T12:57:28.998 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:28.997+0000 7fa362c00640 1 -- start start 2024-09-11T12:57:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35c0ce3c0 0x7fa35c13bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c158920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c168e20 con 0x7fa35c0cee70 2024-09-11T12:57:29.001 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa35c0b3940 con 0x7fa35c142bf0 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa35c168fa0 con 0x7fa35c0ce3c0 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59724/0 (socket says 172.21.15.78:59724) 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 -- 172.21.15.78:0/2737828397 learned_addr learned my addr 172.21.15.78:0/2737828397 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa361800640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35c0ce3c0 0x7fa35c13bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362200640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c158920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 -- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35c0ce3c0 msgr2=0x7fa35c13bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35c0ce3c0 0x7fa35c13bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 -- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 msgr2=0x7fa35c158920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c158920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 -- 172.21.15.78:0/2737828397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35c133c40 con 0x7fa35c0cee70 2024-09-11T12:57:29.002 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa360e00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c490 tx=0x7fa34800c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3480052d0 con 0x7fa35c0cee70 2024-09-11T12:57:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa348005490 con 0x7fa35c0cee70 2024-09-11T12:57:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348012870 con 0x7fa35c0cee70 2024-09-11T12:57:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35c159170 con 0x7fa35c0cee70 2024-09-11T12:57:29.003 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa35c159730 con 0x7fa35c0cee70 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa35c06ff80 con 0x7fa35c0cee70 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.001+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fa348012a10 con 0x7fa35c0cee70 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.005+0000 7fa359600640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 0x7fa33c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.005+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4857+0+0 (secure 0 0 0) 0x7fa34808ffa0 con 0x7fa35c0cee70 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.005+0000 7fa361800640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 0x7fa33c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.004 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.005+0000 7fa361800640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 0x7fa33c0744d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa350001850 tx=0x7fa3500017e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.007 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.009+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa35c133c40 con 0x7fa35c0cee70 2024-09-11T12:57:29.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fa35c13cba0 con 0x7fa35c0cee70 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa359600640 1 -- 172.21.15.78:0/2737828397 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+5304 (secure 0 0 0) 0x7fa34805d3c0 con 0x7fa35c0cee70 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stdout: "auth_dump": [ 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.232 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.a", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFm5RflKBAA6PWvL3VJkPgoyjoaOk+w6A==", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.b", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmZT3xNBAAFdRCsOmYtLrKnh1B3fh1pg==", 2024-09-11T12:57:29.233 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.c", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmbdIIMBAAd+BO72fUBPguM7Fxj1J57g==", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:57:29.234 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mds.d", 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGuJjABAANU9By48jQ7uZugABM0LdKA==", 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.0", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCvkeFmDcvhBBAAxiQRieocv95IGUp1xzYYEQ==", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.1", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCykeFmwJ1wGxAAfI4dPiRrz+92szBgFNbDxA==", 2024-09-11T12:57:29.236 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.2", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC1keFmzfy6LhAAh9kmpVfSILmCM/Wt+I+c8w==", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.237 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.3", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC5keFmxPBpChAAPvLTlqUQqCMPfnBbC2PHSA==", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.4", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQC9keFmY68WJRAAStxxMtdKTVreI8uuxcCuMA==", 2024-09-11T12:57:29.238 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.5", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDAkeFmjr0lOxAAvWZwxRU51CIWSTJ8vb3o5Q==", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.6", 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDEkeFmuFeQFBAA2JD9OOph5tBW5hGdEIUcGw==", 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.241 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "osd.7", 2024-09-11T12:57:29.241 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQDHkeFm6iXQKRAArQmpcuKFYtluSNlLEXVpFw==", 2024-09-11T12:57:29.241 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.241 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow profile osd", 2024-09-11T12:57:29.242 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile osd", 2024-09-11T12:57:29.242 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.0", 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w==", 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow r", 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.243 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.1", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFm9zy9DRAA+8Mfc2o4DQRHDp8f40lLfA==", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.2", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmAmYnChAAxHBa+Q0Ev9HpUiSa7LPMxw==", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.244 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.3", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCskeFmGWOiExAAaCTiIX0v0A6LkDQcABXpDA==", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow r", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow rw", 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow rwx" 2024-09-11T12:57:29.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "client.admin", 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmcOdnDRAA99TRxoiO/MisP3RHrsuF1A==", 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.246 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mgr": "allow *", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow *", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.x", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmrN+DJBAAF5eTOT9MAu+xGGwwlVb5+g==", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:57:29.247 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "entity": "mgr.y", 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "key": "AQCrkeFmhx2wIBAAVuhWyy7wFcndTIOgfUujDg==", 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "caps": { 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mds": "allow *", 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "mon": "allow profile mgr", 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: "osd": "allow *" 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.248 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-09-11T12:57:29.249 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:57:29.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 msgr2=0x7fa33c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 0x7fa33c0744d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa350001850 tx=0x7fa3500017e0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 msgr2=0x7fa35c158270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fa34800c490 tx=0x7fa34800c950 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 shutdown_connections 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa33c072010 0x7fa33c0744d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa35c142bf0 0x7fa35c158920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa35c0cee70 0x7fa35c158270 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 --2- 172.21.15.78:0/2737828397 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa35c0ce3c0 0x7fa35c13bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 >> 172.21.15.78:0/2737828397 conn(0x7fa35c0cfeb0 msgr2=0x7fa35c13e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 shutdown_connections 2024-09-11T12:57:29.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.233+0000 7fa362c00640 1 -- 172.21.15.78:0/2737828397 wait complete. 2024-09-11T12:57:29.253 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-11T12:57:29.394 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 -- 172.21.15.78:0/1595985003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727415e5b0 msgr2=0x7f727415e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 --2- 172.21.15.78:0/1595985003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727415e5b0 0x7f727415e9f0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f7274073760 tx=0x7f726401c890 comp rx=0 tx=0).stop 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 -- 172.21.15.78:0/1595985003 shutdown_connections 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 --2- 172.21.15.78:0/1595985003 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727415e5b0 0x7f727415e9f0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 --2- 172.21.15.78:0/1595985003 >> v2:172.21.15.157:3300/0 conn(0x7f727406fc50 0x7f72740700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 --2- 172.21.15.78:0/1595985003 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7274074240 0x7f727415ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 -- 172.21.15.78:0/1595985003 >> 172.21.15.78:0/1595985003 conn(0x7f727406ca80 msgr2=0x7f72740b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 -- 172.21.15.78:0/1595985003 shutdown_connections 2024-09-11T12:57:29.395 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.393+0000 7f727c800640 1 -- 172.21.15.78:0/1595985003 wait complete. 2024-09-11T12:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 Processor -- start 2024-09-11T12:57:29.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- start start 2024-09-11T12:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274074240 0x7f727415c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727415e5b0 0x7f727415c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.398 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7274072900 con 0x7f727406fc50 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7274072780 con 0x7f7274074240 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72740b4520 con 0x7f727415e5b0 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59746/0 (socket says 172.21.15.78:59746) 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 -- 172.21.15.78:0/3009184858 learned_addr learned my addr 172.21.15.78:0/3009184858 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279400640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274074240 0x7f727415c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727a800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727415e5b0 0x7f727415c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 -- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727415e5b0 msgr2=0x7f727415c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727415e5b0 0x7f727415c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 -- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274074240 msgr2=0x7f727415c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274074240 0x7f727415c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 -- 172.21.15.78:0/3009184858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7274161710 con 0x7f727406fc50 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f7279e00640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f726800e940 tx=0x7f726800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726800cd80 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7268005020 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741619f0 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268005450 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.397+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7274161fb0 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f727415ba80 con 0x7f727406fc50 2024-09-11T12:57:29.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f72680047b0 con 0x7f727406fc50 2024-09-11T12:57:29.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f726e000640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 0x7f72580744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4857+0+0 (secure 0 0 0) 0x7f7268090f00 con 0x7f727406fc50 2024-09-11T12:57:29.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f7279400640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 0x7f72580744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.402 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.401+0000 7f7279400640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 0x7f72580744d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72640af000 tx=0x7f72640a7040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.405 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.405+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f726805e2a0 con 0x7f727406fc50 2024-09-11T12:57:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f72741621f0 con 0x7f727406fc50 2024-09-11T12:57:29.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f726e000640 1 -- 172.21.15.78:0/3009184858 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v158) ==== 84+0+65 (secure 0 0 0) 0x7f72680627f0 con 0x7f727406fc50 2024-09-11T12:57:29.573 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:29.573 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:57:29.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:29.574 INFO:teuthology.orchestra.run.smithi078.stdout: "poolnum": 1, 2024-09-11T12:57:29.574 INFO:teuthology.orchestra.run.smithi078.stdout: "poolname": ".mgr" 2024-09-11T12:57:29.574 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:29.574 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:57:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 msgr2=0x7f72580744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 0x7f72580744d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f72640af000 tx=0x7f72640a7040 comp rx=0 tx=0).stop 2024-09-11T12:57:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 msgr2=0x7f7274154df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.573+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f726800e940 tx=0x7f726800ee00 comp rx=0 tx=0).stop 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 shutdown_connections 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f727415e5b0 0x7f727415c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7258072010 0x7f72580744d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7274074240 0x7f727415c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 --2- 172.21.15.78:0/3009184858 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f727406fc50 0x7f7274154df0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 >> 172.21.15.78:0/3009184858 conn(0x7f727406ca80 msgr2=0x7f7274164d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 shutdown_connections 2024-09-11T12:57:29.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.577+0000 7f727c800640 1 -- 172.21.15.78:0/3009184858 wait complete. 2024-09-11T12:57:29.594 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-11T12:57:29.595 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-11T12:57:29.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 -- 172.21.15.78:0/3118288955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec0c4080 msgr2=0x7f60ec0c4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 --2- 172.21.15.78:0/3118288955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec0c4080 0x7f60ec0c4890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009840 tx=0x7f60dc01c7f0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 -- 172.21.15.78:0/3118288955 shutdown_connections 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 --2- 172.21.15.78:0/3118288955 >> v2:172.21.15.157:3300/0 conn(0x7f60ec072660 0x7f60ec072ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 --2- 172.21.15.78:0/3118288955 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec0c4080 0x7f60ec0c4890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 -- 172.21.15.78:0/3118288955 >> 172.21.15.78:0/3118288955 conn(0x7f60ec073940 msgr2=0x7f60ec073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 -- 172.21.15.78:0/3118288955 shutdown_connections 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.741+0000 7f60f1a00640 1 -- 172.21.15.78:0/3118288955 wait complete. 2024-09-11T12:57:29.744 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 Processor -- start 2024-09-11T12:57:29.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- start start 2024-09-11T12:57:29.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec072660 0x7f60ec144c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec1615a0 0x7f60ec0cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec16ab10 con 0x7f60ec0c4080 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec16a990 con 0x7f60ec1615a0 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f60ec16ac90 con 0x7f60ec072660 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59760/0 (socket says 172.21.15.78:59760) 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 -- 172.21.15.78:0/2685967790 learned_addr learned my addr 172.21.15.78:0/2685967790 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60ebe00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec072660 0x7f60ec144c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1000640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec1615a0 0x7f60ec0cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 -- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec072660 msgr2=0x7f60ec144c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec072660 0x7f60ec144c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 -- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec1615a0 msgr2=0x7f60ec0cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:29.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec1615a0 0x7f60ec0cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 -- 172.21.15.78:0/2685967790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec0cbf30 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60eb400640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c390 tx=0x7f60d800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011790 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d8017070 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d8011ce0 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec0caed0 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.745+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec0cb450 con 0x7f60ec0c4080 2024-09-11T12:57:29.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec072ad0 con 0x7f60ec0c4080 2024-09-11T12:57:29.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f60d801c460 con 0x7f60ec0c4080 2024-09-11T12:57:29.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60e3e00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 0x7f60cc0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:29.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 4857+0+0 (secure 0 0 0) 0x7f60d80901c0 con 0x7f60ec0c4080 2024-09-11T12:57:29.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60ebe00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 0x7f60cc0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:29.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.749+0000 7f60ebe00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 0x7f60cc0744d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0af000 tx=0x7f60dc01b030 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:29.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.753+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f60d805d5e0 con 0x7f60ec0c4080 2024-09-11T12:57:29.925 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:29.925+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f60ec15fa80 con 0x7f60ec0c4080 2024-09-11T12:57:30.836 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.837+0000 7f60e3e00640 1 -- 172.21.15.78:0/2685967790 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v159) ==== 138+0+0 (secure 0 0 0) 0x7f60d8064e20 con 0x7f60ec0c4080 2024-09-11T12:57:30.837 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' created 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 msgr2=0x7f60cc0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 0x7f60cc0744d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f60dc0af000 tx=0x7f60dc01b030 comp rx=0 tx=0).stop 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 msgr2=0x7f60ec0cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f60d800c390 tx=0x7f60d800c850 comp rx=0 tx=0).stop 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 shutdown_connections 2024-09-11T12:57:30.840 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f60cc072010 0x7f60cc0744d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f60ec1615a0 0x7f60ec0cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f60ec0c4080 0x7f60ec0cc170 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 --2- 172.21.15.78:0/2685967790 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f60ec072660 0x7f60ec144c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 >> 172.21.15.78:0/2685967790 conn(0x7f60ec073940 msgr2=0x7f60ec073d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 shutdown_connections 2024-09-11T12:57:30.841 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:30.841+0000 7f60f1a00640 1 -- 172.21.15.78:0/2685967790 wait complete. 2024-09-11T12:57:30.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- 172.21.15.78:0/926554804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b40cc080 msgr2=0x7f72b40702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- 172.21.15.78:0/926554804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b40cc080 0x7f72b40702b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f72b40cccb0 tx=0x7f72a401cb10 comp rx=0 tx=0).stop 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- 172.21.15.78:0/926554804 shutdown_connections 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- 172.21.15.78:0/926554804 >> v2:172.21.15.157:3300/0 conn(0x7f72b40d0ec0 0x7f72b40d1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- 172.21.15.78:0/926554804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b40cc080 0x7f72b40702b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- 172.21.15.78:0/926554804 >> 172.21.15.78:0/926554804 conn(0x7f72b40ceab0 msgr2=0x7f72b40ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- 172.21.15.78:0/926554804 shutdown_connections 2024-09-11T12:57:31.024 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- 172.21.15.78:0/926554804 wait complete. 2024-09-11T12:57:31.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 Processor -- start 2024-09-11T12:57:31.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- start start 2024-09-11T12:57:31.027 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cc080 0x7f72b41f6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 0x7f72b40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4168e30 con 0x7f72b40d0ec0 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f72b4168cb0 con 0x7f72b40cc080 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b8a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f72b4168fb0 con 0x7f72b415cee0 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59764/0 (socket says 172.21.15.78:59764) 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 -- 172.21.15.78:0/89014269 learned_addr learned my addr 172.21.15.78:0/89014269 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1e00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cc080 0x7f72b41f6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b2800640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 0x7f72b40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 -- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 msgr2=0x7f72b40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:31.028 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 0x7f72b40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 -- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cc080 msgr2=0x7f72b41f6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cc080 0x7f72b41f6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b1400640 1 -- 172.21.15.78:0/89014269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72b414c1b0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.025+0000 7f72b2800640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 0x7f72b40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72b1400640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c430 tx=0x7f72a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0011790 con 0x7f72b40d0ec0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a0017070 con 0x7f72b40d0ec0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72b41f68b0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a0011ce0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72b414acb0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72b40702b0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f72a001c460 con 0x7f72b40d0ec0 2024-09-11T12:57:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 0x7f72940744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 5227+0+0 (secure 0 0 0) 0x7f72a0090450 con 0x7f72b40d0ec0 2024-09-11T12:57:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.029+0000 7f72b1e00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 0x7f72940744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:31.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.033+0000 7f72b1e00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 0x7f72940744d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f72a40af000 tx=0x7f72a401b350 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:31.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.037+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f72a005df00 con 0x7f72b40d0ec0 2024-09-11T12:57:31.202 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.201+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f72b40b39c0 con 0x7f72b40d0ec0 2024-09-11T12:57:31.839 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72aa000640 1 -- 172.21.15.78:0/89014269 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v160) ==== 170+0+0 (secure 0 0 0) 0x7f72a005d800 con 0x7f72b40d0ec0 2024-09-11T12:57:31.840 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' created 2024-09-11T12:57:31.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 msgr2=0x7f72940744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:31.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 0x7f72940744d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f72a40af000 tx=0x7f72a401b350 comp rx=0 tx=0).stop 2024-09-11T12:57:31.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 msgr2=0x7f72b4148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:31.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f72a000c430 tx=0x7f72a000c8f0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 shutdown_connections 2024-09-11T12:57:31.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f72b415cee0 0x7f72b40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f72b40d0ec0 0x7f72b4148010 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7294072010 0x7f72940744d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.843 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 --2- 172.21.15.78:0/89014269 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f72b40cc080 0x7f72b41f6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:31.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 >> 172.21.15.78:0/89014269 conn(0x7f72b40ceab0 msgr2=0x7f72b41527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:31.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 shutdown_connections 2024-09-11T12:57:31.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:31.841+0000 7f72b8a00640 1 -- 172.21.15.78:0/89014269 wait complete. 2024-09-11T12:57:31.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-11T12:57:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- 172.21.15.78:0/1758482112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 msgr2=0x7faff41655c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 --2- 172.21.15.78:0/1758482112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff41655c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fafec009930 tx=0x7fafec01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- 172.21.15.78:0/1758482112 shutdown_connections 2024-09-11T12:57:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 --2- 172.21.15.78:0/1758482112 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff41655c0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.025 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- 172.21.15.78:0/1758482112 >> 172.21.15.78:0/1758482112 conn(0x7faff4071dd0 msgr2=0x7faff40721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:32.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- 172.21.15.78:0/1758482112 shutdown_connections 2024-09-11T12:57:32.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- 172.21.15.78:0/1758482112 wait complete. 2024-09-11T12:57:32.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 Processor -- start 2024-09-11T12:57:32.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.025+0000 7faff8c00640 1 -- start start 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff40b2120 0x7faff414d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff41518b0 0x7faff40b3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7faff4168f40 con 0x7faff40ce320 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7faff4168dc0 con 0x7faff41518b0 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7faff41690c0 con 0x7faff40b2120 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59780/0 (socket says 172.21.15.78:59780) 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 -- 172.21.15.78:0/538277599 learned_addr learned my addr 172.21.15.78:0/538277599 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:32.029 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1e00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff40b2120 0x7faff414d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff2800640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff41518b0 0x7faff40b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 -- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff40b2120 msgr2=0x7faff414d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff40b2120 0x7faff414d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 -- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff41518b0 msgr2=0x7faff40b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff41518b0 0x7faff40b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 -- 172.21.15.78:0/538277599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faff41fb2b0 con 0x7faff40ce320 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff1400640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c420 tx=0x7fafe000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0017070 con 0x7faff40ce320 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fafe0003600 con 0x7faff40ce320 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faff41fb550 con 0x7faff40ce320 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafe0003a90 con 0x7faff40ce320 2024-09-11T12:57:32.030 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faff41fbb10 con 0x7faff40ce320 2024-09-11T12:57:32.031 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.029+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faff40cdc10 con 0x7faff40ce320 2024-09-11T12:57:32.032 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.033+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fafe001a070 con 0x7faff40ce320 2024-09-11T12:57:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.033+0000 7fafea000640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 0x7fafd4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.033+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(160..160 src has 1..160) ==== 5609+0+0 (secure 0 0 0) 0x7fafe008fc30 con 0x7faff40ce320 2024-09-11T12:57:32.033 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.033+0000 7faff1e00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 0x7fafd4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:32.034 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.033+0000 7faff1e00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 0x7fafd4074500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafe40012f0 tx=0x7fafe4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:32.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.037+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fafe0021020 con 0x7faff40ce320 2024-09-11T12:57:32.236 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.237+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7faff415ba90 con 0x7faff40ce320 2024-09-11T12:57:32.876 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.873+0000 7fafea000640 1 -- 172.21.15.78:0/538277599 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v117) ==== 174+0+0 (secure 0 0 0) 0x7fafe005d050 con 0x7faff40ce320 2024-09-11T12:57:32.880 INFO:teuthology.orchestra.run.smithi078.stderr:new fs with metadata pool 26 and data pool 27 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 msgr2=0x7fafd4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 0x7fafd4074500 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fafe40012f0 tx=0x7fafe4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 msgr2=0x7faff414c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c420 tx=0x7fafe000c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 shutdown_connections 2024-09-11T12:57:32.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fafd4072040 0x7fafd4074500 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7faff41518b0 0x7faff40b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7faff40ce320 0x7faff414c900 secure :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fafe000c420 tx=0x7fafe000c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 --2- 172.21.15.78:0/538277599 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7faff40b2120 0x7faff414d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:32.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.881+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 >> 172.21.15.78:0/538277599 conn(0x7faff4071dd0 msgr2=0x7faff415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:32.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.885+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 shutdown_connections 2024-09-11T12:57:32.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:32.885+0000 7faff8c00640 1 -- 172.21.15.78:0/538277599 wait complete. 2024-09-11T12:57:32.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- 172.21.15.78:0/843133853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 msgr2=0x7f0b0006f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- 172.21.15.78:0/843133853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b0006f690 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f0af8009900 tx=0x7f0af801c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- 172.21.15.78:0/843133853 shutdown_connections 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- 172.21.15.78:0/843133853 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b0006f690 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- 172.21.15.78:0/843133853 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 0x7f0b000cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- 172.21.15.78:0/843133853 >> 172.21.15.78:0/843133853 conn(0x7f0b0006ca80 msgr2=0x7f0b000b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- 172.21.15.78:0/843133853 shutdown_connections 2024-09-11T12:57:33.060 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- 172.21.15.78:0/843133853 wait complete. 2024-09-11T12:57:33.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 Processor -- start 2024-09-11T12:57:33.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- start start 2024-09-11T12:57:33.063 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 0x7f0b000b1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 0x7f0b00157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f0b000b5250 con 0x7f0b0006f200 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f0b000b50d0 con 0x7f0b00159fc0 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0b05600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f0b00167e30 con 0x7f0b000cd910 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59808/0 (socket says 172.21.15.78:59808) 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 -- 172.21.15.78:0/1215375908 learned_addr learned my addr 172.21.15.78:0/1215375908 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afe000640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 0x7f0b000b1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0aff400640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 0x7f0b00157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 -- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 msgr2=0x7f0b000b1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 0x7f0b000b1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 -- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 msgr2=0x7f0b00157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 0x7f0b00157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 -- 172.21.15.78:0/1215375908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b00149d20 con 0x7f0b0006f200 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0aff400640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 0x7f0b00157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.061+0000 7f0afea00640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0af000ba80 tx=0x7f0af000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af000cba0 con 0x7f0b0006f200 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0af000ce30 con 0x7f0b0006f200 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af0003be0 con 0x7f0b0006f200 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0015bd80 con 0x7f0b0006f200 2024-09-11T12:57:33.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0015c340 con 0x7f0b0006f200 2024-09-11T12:57:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b0006f690 con 0x7f0b0006f200 2024-09-11T12:57:33.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f0af0006d10 con 0x7f0b0006f200 2024-09-11T12:57:33.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.065+0000 7f0af6a00640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 0x7f0ae00744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.069+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5709+0+0 (secure 0 0 0) 0x7f0af00900b0 con 0x7f0b0006f200 2024-09-11T12:57:33.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.069+0000 7f0afe000640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 0x7f0ae00744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.069+0000 7f0afe000640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 0x7f0ae00744d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0b000b0cc0 tx=0x7f0aec00b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.073+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0af005d4d0 con 0x7f0b0006f200 2024-09-11T12:57:33.250 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.249+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b00156dd0 con 0x7f0b0006f200 2024-09-11T12:57:33.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.249+0000 7f0af6a00640 1 -- 172.21.15.78:0/1215375908 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16756 (secure 0 0 0) 0x7f0af0061a20 con 0x7f0b0006f200 2024-09-11T12:57:33.251 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:33.251 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":161,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:32.239590+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:57:29.938274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-11T12:57:31.215025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:33.253 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 msgr2=0x7f0ae00744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 0x7f0ae00744d0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0b000b0cc0 tx=0x7f0aec00b390 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 msgr2=0x7f0b001497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f0af000ba80 tx=0x7f0af000bf40 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 shutdown_connections 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f0ae0072010 0x7f0ae00744d0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f0b00159fc0 0x7f0b00157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f0b000cd910 0x7f0b000b1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 --2- 172.21.15.78:0/1215375908 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f0b0006f200 0x7f0b001497e0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 >> 172.21.15.78:0/1215375908 conn(0x7f0b0006ca80 msgr2=0x7f0b00159130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 shutdown_connections 2024-09-11T12:57:33.254 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.253+0000 7f0b05600640 1 -- 172.21.15.78:0/1215375908 wait complete. 2024-09-11T12:57:33.272 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 -- 172.21.15.78:0/3960703875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc618073710 msgr2=0x7fc61815ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 --2- 172.21.15.78:0/3960703875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc618073710 0x7fc61815ae20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fc6180afa30 tx=0x7fc60401c800 comp rx=0 tx=0).stop 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 -- 172.21.15.78:0/3960703875 shutdown_connections 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 --2- 172.21.15.78:0/3960703875 >> v2:172.21.15.157:3300/0 conn(0x7fc61815b360 0x7fc6180b4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 --2- 172.21.15.78:0/3960703875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc618073710 0x7fc61815ae20 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 -- 172.21.15.78:0/3960703875 >> 172.21.15.78:0/3960703875 conn(0x7fc61806ca80 msgr2=0x7fc6180b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 -- 172.21.15.78:0/3960703875 shutdown_connections 2024-09-11T12:57:33.414 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.413+0000 7fc620000640 1 -- 172.21.15.78:0/3960703875 wait complete. 2024-09-11T12:57:33.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 Processor -- start 2024-09-11T12:57:33.415 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- start start 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc618157a50 0x7fc6181457c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61815e040 0x7fc6180b18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc61806f410 con 0x7fc61815dbc0 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc61806f290 con 0x7fc618157a50 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc618166dc0 con 0x7fc61815e040 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.417 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59814/0 (socket says 172.21.15.78:59814) 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 -- 172.21.15.78:0/3218662351 learned_addr learned my addr 172.21.15.78:0/3218662351 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61e000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61815e040 0x7fc6180b18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61d600640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc618157a50 0x7fc6181457c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 -- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61815e040 msgr2=0x7fc6180b18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61815e040 0x7fc6180b18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 -- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc618157a50 msgr2=0x7fc6181457c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc618157a50 0x7fc6181457c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 -- 172.21.15.78:0/3218662351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc61815e610 con 0x7fc61815dbc0 2024-09-11T12:57:33.418 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc61cc00640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc604000c10 tx=0x7fc604005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6040b2040 con 0x7fc61815dbc0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc604004130 con 0x7fc61815dbc0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc604005020 con 0x7fc61815dbc0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc61815e8f0 con 0x7fc61815dbc0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6180af850 con 0x7fc61815dbc0 2024-09-11T12:57:33.419 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.417+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc61815ae20 con 0x7fc61815dbc0 2024-09-11T12:57:33.420 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.421+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fc6040bf020 con 0x7fc61815dbc0 2024-09-11T12:57:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.421+0000 7fc615600640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 0x7fc5f8074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.421+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5709+0+0 (secure 0 0 0) 0x7fc60412a910 con 0x7fc61815dbc0 2024-09-11T12:57:33.421 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.421+0000 7fc61d600640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 0x7fc5f8074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.421+0000 7fc61d600640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 0x7fc5f8074500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0016e0 tx=0x7fc60c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.424 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.425+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc6040164b0 con 0x7fc61815dbc0 2024-09-11T12:57:33.590 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.589+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc618072450 con 0x7fc61815dbc0 2024-09-11T12:57:33.591 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc615600640 1 -- 172.21.15.78:0/3218662351 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) ==== 74+0+16756 (secure 0 0 0) 0x7fc6040f8380 con 0x7fc61815dbc0 2024-09-11T12:57:33.592 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:33.592 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":161,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:32.239590+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:57:29.938274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-11T12:57:31.215025+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 msgr2=0x7fc5f8074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 0x7fc5f8074500 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc60c0016e0 tx=0x7fc60c009040 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 msgr2=0x7fc6180b1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fc604000c10 tx=0x7fc604005e70 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 shutdown_connections 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc61815e040 0x7fc6180b18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc61815dbc0 0x7fc6180b1380 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5f8072040 0x7fc5f8074500 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 --2- 172.21.15.78:0/3218662351 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc618157a50 0x7fc6181457c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 >> 172.21.15.78:0/3218662351 conn(0x7fc61806ca80 msgr2=0x7fc6181548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:33.594 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 shutdown_connections 2024-09-11T12:57:33.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.593+0000 7fc620000640 1 -- 172.21.15.78:0/3218662351 wait complete. 2024-09-11T12:57:33.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-11T12:57:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 -- 172.21.15.78:0/379567869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a4c004810 msgr2=0x7f6a4c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/379567869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a4c004810 0x7f6a4c004c90 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f6a54001860 tx=0x7f6a5401f030 comp rx=0 tx=0).stop 2024-09-11T12:57:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 -- 172.21.15.78:0/379567869 shutdown_connections 2024-09-11T12:57:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/379567869 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a4c004810 0x7f6a4c004c90 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 -- 172.21.15.78:0/379567869 >> 172.21.15.78:0/379567869 conn(0x7f6a68071dd0 msgr2=0x7f6a680721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 -- 172.21.15.78:0/379567869 shutdown_connections 2024-09-11T12:57:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.757+0000 7f6a6cc00640 1 -- 172.21.15.78:0/379567869 wait complete. 2024-09-11T12:57:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 Processor -- start 2024-09-11T12:57:33.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- start start 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a4c004810 0x7f6a681f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a681f75b0 0x7f6a680b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680cdfa0 con 0x7f6a6813de30 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6a680cde20 con 0x7f6a681f75b0 2024-09-11T12:57:33.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6a680ce120 con 0x7f6a4c004810 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:59824/0 (socket says 172.21.15.78:59824) 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 -- 172.21.15.78:0/3615957028 learned_addr learned my addr 172.21.15.78:0/3615957028 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65e00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a4c004810 0x7f6a681f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a66800640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a681f75b0 0x7f6a680b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 -- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a4c004810 msgr2=0x7f6a681f44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a4c004810 0x7f6a681f44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 -- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a681f75b0 msgr2=0x7f6a680b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a681f75b0 0x7f6a680b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:33.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 -- 172.21.15.78:0/3615957028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a681f4ac0 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a65400640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6a54001860 tx=0x7f6a54001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54003600 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a540b7030 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54003af0 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a681f4da0 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a681f5360 con 0x7f6a6813de30 2024-09-11T12:57:33.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.761+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a680b1d90 con 0x7f6a6813de30 2024-09-11T12:57:33.764 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.765+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f6a54003c90 con 0x7f6a6813de30 2024-09-11T12:57:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.765+0000 7f6a5e000640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 0x7f6a48074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.765+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(161..161 src has 1..161) ==== 5709+0+0 (secure 0 0 0) 0x7f6a5412dc40 con 0x7f6a6813de30 2024-09-11T12:57:33.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.765+0000 7f6a65e00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 0x7f6a48074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:33.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.765+0000 7f6a65e00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 0x7f6a48074500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001bf0 tx=0x7f6a58006040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:33.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.769+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6a54018de0 con 0x7f6a6813de30 2024-09-11T12:57:33.972 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:33.973+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6a681408e0 con 0x7f6a6813de30 2024-09-11T12:57:34.834 INFO:tasks.ceph.mds.d.smithi157.stderr:2024-09-11T12:57:34.835+0000 7fc7adc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-11T12:57:34.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a5e000640 1 -- 172.21.15.78:0/3615957028 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v120) ==== 118+0+0 (secure 0 0 0) 0x7f6a540fafe0 con 0x7f6a6813de30 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 msgr2=0x7f6a48074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 0x7f6a48074500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001bf0 tx=0x7f6a58006040 comp rx=0 tx=0).stop 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 msgr2=0x7f6a680b0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f6a54001860 tx=0x7f6a54001520 comp rx=0 tx=0).stop 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 shutdown_connections 2024-09-11T12:57:34.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6a48072040 0x7f6a48074500 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6a681f75b0 0x7f6a680b1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6a6813de30 0x7f6a680b0df0 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 --2- 172.21.15.78:0/3615957028 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6a4c004810 0x7f6a681f44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 >> 172.21.15.78:0/3615957028 conn(0x7f6a68071dd0 msgr2=0x7f6a6815a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.885+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 shutdown_connections 2024-09-11T12:57:34.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:34.889+0000 7f6a6cc00640 1 -- 172.21.15.78:0/3615957028 wait complete. 2024-09-11T12:57:34.911 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-11T12:57:34.911 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- 172.21.15.78:0/138389006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd990004810 msgr2=0x7fd990004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 --2- 172.21.15.78:0/138389006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd990004810 0x7fd990004c90 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd994005fd0 tx=0x7fd99401cd60 comp rx=0 tx=0).stop 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- 172.21.15.78:0/138389006 shutdown_connections 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 --2- 172.21.15.78:0/138389006 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd990004810 0x7fd990004c90 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- 172.21.15.78:0/138389006 >> 172.21.15.78:0/138389006 conn(0x7fd9a8071dd0 msgr2=0x7fd9a80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- 172.21.15.78:0/138389006 shutdown_connections 2024-09-11T12:57:35.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- 172.21.15.78:0/138389006 wait complete. 2024-09-11T12:57:35.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 Processor -- start 2024-09-11T12:57:35.062 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- start start 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd990004810 0x7fd9a8138ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 0x7fd9a81503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80cdfa0 con 0x7fd9a8150900 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd9a80cde20 con 0x7fd9a8139220 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.061+0000 7fd9b0000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd9a80b1a10 con 0x7fd990004810 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:35.064 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47180/0 (socket says 172.21.15.78:47180) 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 -- 172.21.15.78:0/1955986452 learned_addr learned my addr 172.21.15.78:0/1955986452 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9acc00640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 0x7fd9a81503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ad600640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd990004810 0x7fd9a8138ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 -- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd990004810 msgr2=0x7fd9a8138ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd990004810 0x7fd9a8138ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 -- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 msgr2=0x7fd9a81503c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 0x7fd9a81503c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 -- 172.21.15.78:0/1955986452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9a815b160 con 0x7fd9a8150900 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9acc00640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 0x7fd9a81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9ae000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000c7c0 con 0x7fd9a8150900 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9a000c960 con 0x7fd9a8150900 2024-09-11T12:57:35.065 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000cbb0 con 0x7fd9a8150900 2024-09-11T12:57:35.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9a815b440 con 0x7fd9a8150900 2024-09-11T12:57:35.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9a815ba00 con 0x7fd9a8150900 2024-09-11T12:57:35.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9a815e4d0 con 0x7fd9a8150900 2024-09-11T12:57:35.066 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.065+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fd9a001c020 con 0x7fd9a8150900 2024-09-11T12:57:35.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.069+0000 7fd9a5600640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 0x7fd9880744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.069+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fd9a008ff50 con 0x7fd9a8150900 2024-09-11T12:57:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.069+0000 7fd9ad600640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 0x7fd9880744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:35.068 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.069+0000 7fd9ad600640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 0x7fd9880744d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd99c001060 tx=0x7fd99c001b40 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:35.071 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.073+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd9a0023020 con 0x7fd9a8150900 2024-09-11T12:57:35.272 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.273+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd9a806fbe0 con 0x7fd9a8150900 2024-09-11T12:57:35.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.885+0000 7fd9a5600640 1 -- 172.21.15.78:0/1955986452 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v121) ==== 122+0+0 (secure 0 0 0) 0x7fd9a005d9c0 con 0x7fd9a8150900 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 msgr2=0x7fd9880744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 0x7fd9880744d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fd99c001060 tx=0x7fd99c001b40 comp rx=0 tx=0).stop 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 msgr2=0x7fd9a815bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a000ed10 comp rx=0 tx=0).stop 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 shutdown_connections 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd9a8150900 0x7fd9a815bf20 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd988072010 0x7fd9880744d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd9a8139220 0x7fd9a81503c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.888 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 --2- 172.21.15.78:0/1955986452 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd990004810 0x7fd9a8138ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 >> 172.21.15.78:0/1955986452 conn(0x7fd9a8071dd0 msgr2=0x7fd9a80b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 shutdown_connections 2024-09-11T12:57:35.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:35.889+0000 7fd9b0000640 1 -- 172.21.15.78:0/1955986452 wait complete. 2024-09-11T12:57:35.908 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-11T12:57:35.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 -- 172.21.15.78:0/2140081781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f883006eb40 msgr2=0x7f883006efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 --2- 172.21.15.78:0/2140081781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f883006eb40 0x7f883006efb0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f8830074d50 tx=0x7f882801c770 comp rx=0 tx=0).stop 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 -- 172.21.15.78:0/2140081781 shutdown_connections 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 --2- 172.21.15.78:0/2140081781 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f883006eb40 0x7f883006efb0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 --2- 172.21.15.78:0/2140081781 >> v2:172.21.15.157:3300/0 conn(0x7f8830070a00 0x7f88300736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 -- 172.21.15.78:0/2140081781 >> 172.21.15.78:0/2140081781 conn(0x7f883006ca80 msgr2=0x7f88300b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 -- 172.21.15.78:0/2140081781 shutdown_connections 2024-09-11T12:57:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.049+0000 7f8838000640 1 -- 172.21.15.78:0/2140081781 wait complete. 2024-09-11T12:57:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 Processor -- start 2024-09-11T12:57:36.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- start start 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883006eb40 0x7f883013d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8830070a00 0x7f8830159660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f88300b5220 con 0x7f8830157ae0 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f88300b50a0 con 0x7f883006eb40 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f883014c590 con 0x7f8830070a00 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47196/0 (socket says 172.21.15.78:47196) 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 -- 172.21.15.78:0/3653263157 learned_addr learned my addr 172.21.15.78:0/3653263157 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8834c00640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8830070a00 0x7f8830159660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8835600640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883006eb40 0x7f883013d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 -- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8830070a00 msgr2=0x7f8830159660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8830070a00 0x7f8830159660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 -- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883006eb40 msgr2=0x7f883013d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883006eb40 0x7f883013d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 -- 172.21.15.78:0/3653263157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8830068b40 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8836000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8828009790 tx=0x7f88280a5b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280b2040 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8828004320 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88300aff30 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88280052d0 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.053+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f883014c800 con 0x7f8830157ae0 2024-09-11T12:57:36.056 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883006efb0 con 0x7f8830157ae0 2024-09-11T12:57:36.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f8828005470 con 0x7f8830157ae0 2024-09-11T12:57:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f882d600640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 0x7f88100744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f882812a190 con 0x7f8830157ae0 2024-09-11T12:57:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f8835600640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 0x7f88100744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.057+0000 7f8835600640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 0x7f88100744d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f00 tx=0x7f8824009650 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.061 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.061+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f88280bd060 con 0x7f8830157ae0 2024-09-11T12:57:36.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88300b1c80 con 0x7f8830157ae0 2024-09-11T12:57:36.265 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f882d600640 1 -- 172.21.15.78:0/3653263157 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5230 (secure 0 0 0) 0x7f88280163f0 con 0x7f8830157ae0 2024-09-11T12:57:36.265 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:36.265 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":121,"btime":"2024-09-11T12:57:35:886343+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:36.266 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 121 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 msgr2=0x7f88100744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 0x7f88100744d0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8824009f00 tx=0x7f8824009650 comp rx=0 tx=0).stop 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 msgr2=0x7f8830159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f8828009790 tx=0x7f88280a5b60 comp rx=0 tx=0).stop 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 shutdown_connections 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8830157ae0 0x7f8830159ba0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8830070a00 0x7f8830159660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8810072010 0x7f88100744d0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 --2- 172.21.15.78:0/3653263157 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f883006eb40 0x7f883013d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 >> 172.21.15.78:0/3653263157 conn(0x7f883006ca80 msgr2=0x7f883015af90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 shutdown_connections 2024-09-11T12:57:36.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.265+0000 7f8838000640 1 -- 172.21.15.78:0/3653263157 wait complete. 2024-09-11T12:57:36.287 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- 172.21.15.78:0/188699653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 msgr2=0x7fcff4073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 --2- 172.21.15.78:0/188699653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4073f70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7fcfec009870 tx=0x7fcfec01c820 comp rx=0 tx=0).stop 2024-09-11T12:57:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- 172.21.15.78:0/188699653 shutdown_connections 2024-09-11T12:57:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 --2- 172.21.15.78:0/188699653 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4073f70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- 172.21.15.78:0/188699653 >> 172.21.15.78:0/188699653 conn(0x7fcff406ca80 msgr2=0x7fcff40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- 172.21.15.78:0/188699653 shutdown_connections 2024-09-11T12:57:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- 172.21.15.78:0/188699653 wait complete. 2024-09-11T12:57:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 Processor -- start 2024-09-11T12:57:36.430 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.429+0000 7fcffb600640 1 -- start start 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff40b4780 0x7fcff4154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff415bc60 0x7fcff413e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4169e30 con 0x7fcff40ccb20 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcff4169cb0 con 0x7fcff415bc60 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcff4169fb0 con 0x7fcff40b4780 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47222/0 (socket says 172.21.15.78:47222) 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 -- 172.21.15.78:0/2893842987 learned_addr learned my addr 172.21.15.78:0/2893842987 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff8c00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff40b4780 0x7fcff4154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff9600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff415bc60 0x7fcff413e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 -- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff40b4780 msgr2=0x7fcff4154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff40b4780 0x7fcff4154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 -- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff415bc60 msgr2=0x7fcff413e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff415bc60 0x7fcff413e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.433 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 -- 172.21.15.78:0/2893842987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff4153890 con 0x7fcff40ccb20 2024-09-11T12:57:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff3e00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c3c0 tx=0x7fcfe000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe00068a0 con 0x7fcff40ccb20 2024-09-11T12:57:36.434 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe0006b50 con 0x7fcff40ccb20 2024-09-11T12:57:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0007020 con 0x7fcff40ccb20 2024-09-11T12:57:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff4153b70 con 0x7fcff40ccb20 2024-09-11T12:57:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff4154130 con 0x7fcff40ccb20 2024-09-11T12:57:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff4073f70 con 0x7fcff40ccb20 2024-09-11T12:57:36.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.433+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fcfe0019020 con 0x7fcff40ccb20 2024-09-11T12:57:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.437+0000 7fcff0c00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 0x7fcfd4074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.437+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fcfe008fb40 con 0x7fcff40ccb20 2024-09-11T12:57:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.437+0000 7fcff8c00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 0x7fcfd4074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.436 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.437+0000 7fcff8c00640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 0x7fcfd4074500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001f20 tx=0x7fcfe4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.439 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.441+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcfe005cee0 con 0x7fcff40ccb20 2024-09-11T12:57:36.606 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.605+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcff414d3b0 con 0x7fcff40ccb20 2024-09-11T12:57:36.607 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcff0c00640 1 -- 172.21.15.78:0/2893842987 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16743 (secure 0 0 0) 0x7fcfe0061430 con 0x7fcff40ccb20 2024-09-11T12:57:36.608 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:36.608 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":162,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:33.860596+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:57:29.938274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-11T12:57:31.215025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 msgr2=0x7fcfd4074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 0x7fcfd4074500 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4001f20 tx=0x7fcfe4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 msgr2=0x7fcff4156240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7fcfe000c3c0 tx=0x7fcfe000c880 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 shutdown_connections 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcfd4072040 0x7fcfd4074500 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcff415bc60 0x7fcff413e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcff40ccb20 0x7fcff4156240 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 --2- 172.21.15.78:0/2893842987 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcff40b4780 0x7fcff4154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 >> 172.21.15.78:0/2893842987 conn(0x7fcff406ca80 msgr2=0x7fcff414c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 shutdown_connections 2024-09-11T12:57:36.611 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.609+0000 7fcffb600640 1 -- 172.21.15.78:0/2893842987 wait complete. 2024-09-11T12:57:36.630 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:57:36.630 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:57:36.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- 172.21.15.78:0/2204815041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 msgr2=0x7f46b4004f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 --2- 172.21.15.78:0/2204815041 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46b4004f70 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f46c0009b60 tx=0x7f46c001d270 comp rx=0 tx=0).stop 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d1800640 1 -- 172.21.15.78:0/2204815041 reap_dead start 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- 172.21.15.78:0/2204815041 shutdown_connections 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- 172.21.15.78:0/2204815041 >> 172.21.15.78:0/2204815041 conn(0x7f46cc06ca80 msgr2=0x7f46cc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- 172.21.15.78:0/2204815041 shutdown_connections 2024-09-11T12:57:36.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- 172.21.15.78:0/2204815041 wait complete. 2024-09-11T12:57:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 Processor -- start 2024-09-11T12:57:36.774 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- start start 2024-09-11T12:57:36.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 0x7f46cc0732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 0x7f46cc14dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0ceb20 con 0x7f46b4004b50 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc0ce9a0 con 0x7f46cc1f8650 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.773+0000 7f46d3800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f46cc0ceca0 con 0x7f46cc14b5f0 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47238/0 (socket says 172.21.15.78:47238) 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 -- 172.21.15.78:0/265432959 learned_addr learned my addr 172.21.15.78:0/265432959 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46cbe00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 0x7f46cc0732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d1800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 0x7f46cc14dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 -- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 msgr2=0x7f46cc14dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 0x7f46cc14dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 -- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 msgr2=0x7f46cc0732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 0x7f46cc0732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 -- 172.21.15.78:0/265432959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc073810 con 0x7f46b4004b50 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d1800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 0x7f46cc14dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46cbe00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 0x7f46cc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d0e00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f46c00af040 tx=0x7f46c00055b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00be070 con 0x7f46b4004b50 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46c00b2040 con 0x7f46b4004b50 2024-09-11T12:57:36.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc074210 con 0x7f46b4004b50 2024-09-11T12:57:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00b1050 con 0x7f46b4004b50 2024-09-11T12:57:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc0727f0 con 0x7f46b4004b50 2024-09-11T12:57:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46cc157f50 con 0x7f46b4004b50 2024-09-11T12:57:36.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.777+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f46c00c1070 con 0x7f46b4004b50 2024-09-11T12:57:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.781+0000 7f46c8c00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 0x7f46ac0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.781+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f46c012a840 con 0x7f46b4004b50 2024-09-11T12:57:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.781+0000 7f46cbe00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 0x7f46ac0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:36.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.781+0000 7f46cbe00640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 0x7f46ac0744d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f46b8001250 tx=0x7f46b8001120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:36.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.785+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f46c0016f80 con 0x7f46b4004b50 2024-09-11T12:57:36.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.949+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f46cc157990 con 0x7f46ac072010 2024-09-11T12:57:36.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.953+0000 7f46c8c00640 1 -- 172.21.15.78:0/265432959 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f46cc157990 con 0x7f46ac072010 2024-09-11T12:57:36.957 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:36.959 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 msgr2=0x7f46ac0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 0x7f46ac0744d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f46b8001250 tx=0x7f46b8001120 comp rx=0 tx=0).stop 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 msgr2=0x7f46cc073c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f46c00af040 tx=0x7f46c00055b0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 shutdown_connections 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f46cc14b5f0 0x7f46cc14dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f46ac072010 0x7f46ac0744d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f46cc1f8650 0x7f46cc0732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 --2- 172.21.15.78:0/265432959 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f46b4004b50 0x7f46cc073c30 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 >> 172.21.15.78:0/265432959 conn(0x7f46cc06ca80 msgr2=0x7f46cc14cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 shutdown_connections 2024-09-11T12:57:36.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:36.957+0000 7f46d3800640 1 -- 172.21.15.78:0/265432959 wait complete. 2024-09-11T12:57:36.978 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T12:57:36.518587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287496,"kb_used_data":17160,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687224,"statfs":{"total":773094113280,"available":772799717376,"internally_reserved":0,"allocated":17571840,"data_stored":8359708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.678984"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019008+0000","last_change":"2024-09-11T12:57:32.889914+0000","last_active":"2024-09-11T12:57:34.019008+0000","last_peered":"2024-09-11T12:57:34.019008+0000","last_clean":"2024-09-11T12:57:34.019008+0000","last_became_active":"2024-09-11T12:57:32.889275+0000","last_became_peered":"2024-09-11T12:57:32.889275+0000","last_unstale":"2024-09-11T12:57:34.019008+0000","last_undegraded":"2024-09-11T12:57:34.019008+0000","last_fullsized":"2024-09-11T12:57:34.019008+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:18.112007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868810+0000","last_change":"2024-09-11T12:57:32.875358+0000","last_active":"2024-09-11T12:57:33.868810+0000","last_peered":"2024-09-11T12:57:33.868810+0000","last_clean":"2024-09-11T12:57:33.868810+0000","last_became_active":"2024-09-11T12:57:32.875087+0000","last_became_peered":"2024-09-11T12:57:32.875087+0000","last_unstale":"2024-09-11T12:57:33.868810+0000","last_undegraded":"2024-09-11T12:57:33.868810+0000","last_fullsized":"2024-09-11T12:57:33.868810+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:05:27.222559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018203+0000","last_change":"2024-09-11T12:57:32.891769+0000","last_active":"2024-09-11T12:57:34.018203+0000","last_peered":"2024-09-11T12:57:34.018203+0000","last_clean":"2024-09-11T12:57:34.018203+0000","last_became_active":"2024-09-11T12:57:32.891473+0000","last_became_peered":"2024-09-11T12:57:32.891473+0000","last_unstale":"2024-09-11T12:57:34.018203+0000","last_undegraded":"2024-09-11T12:57:34.018203+0000","last_fullsized":"2024-09-11T12:57:34.018203+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:06.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892480+0000","last_change":"2024-09-11T12:57:32.885624+0000","last_active":"2024-09-11T12:57:33.892480+0000","last_peered":"2024-09-11T12:57:33.892480+0000","last_clean":"2024-09-11T12:57:33.892480+0000","last_became_active":"2024-09-11T12:57:32.885310+0000","last_became_peered":"2024-09-11T12:57:32.885310+0000","last_unstale":"2024-09-11T12:57:33.892480+0000","last_undegraded":"2024-09-11T12:57:33.892480+0000","last_fullsized":"2024-09-11T12:57:33.892480+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:47:38.893184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017760+0000","last_change":"2024-09-11T12:57:32.879696+0000","last_active":"2024-09-11T12:57:34.017760+0000","last_peered":"2024-09-11T12:57:34.017760+0000","last_clean":"2024-09-11T12:57:34.017760+0000","last_became_active":"2024-09-11T12:57:32.879423+0000","last_became_peered":"2024-09-11T12:57:32.879423+0000","last_unstale":"2024-09-11T12:57:34.017760+0000","last_undegraded":"2024-09-11T12:57:34.017760+0000","last_fullsized":"2024-09-11T12:57:34.017760+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:58.339628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017538+0000","last_change":"2024-09-11T12:57:32.888460+0000","last_active":"2024-09-11T12:57:34.017538+0000","last_peered":"2024-09-11T12:57:34.017538+0000","last_clean":"2024-09-11T12:57:34.017538+0000","last_became_active":"2024-09-11T12:57:32.887843+0000","last_became_peered":"2024-09-11T12:57:32.887843+0000","last_unstale":"2024-09-11T12:57:34.017538+0000","last_undegraded":"2024-09-11T12:57:34.017538+0000","last_fullsized":"2024-09-11T12:57:34.017538+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:20:35.486224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017594+0000","last_change":"2024-09-11T12:57:32.870308+0000","last_active":"2024-09-11T12:57:34.017594+0000","last_peered":"2024-09-11T12:57:34.017594+0000","last_clean":"2024-09-11T12:57:34.017594+0000","last_became_active":"2024-09-11T12:57:32.869987+0000","last_became_peered":"2024-09-11T12:57:32.869987+0000","last_unstale":"2024-09-11T12:57:34.017594+0000","last_undegraded":"2024-09-11T12:57:34.017594+0000","last_fullsized":"2024-09-11T12:57:34.017594+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:56.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017230+0000","last_change":"2024-09-11T12:57:32.879231+0000","last_active":"2024-09-11T12:57:34.017230+0000","last_peered":"2024-09-11T12:57:34.017230+0000","last_clean":"2024-09-11T12:57:34.017230+0000","last_became_active":"2024-09-11T12:57:32.878661+0000","last_became_peered":"2024-09-11T12:57:32.878661+0000","last_unstale":"2024-09-11T12:57:34.017230+0000","last_undegraded":"2024-09-11T12:57:34.017230+0000","last_fullsized":"2024-09-11T12:57:34.017230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:43.164510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869043+0000","last_change":"2024-09-11T12:57:32.892255+0000","last_active":"2024-09-11T12:57:33.869043+0000","last_peered":"2024-09-11T12:57:33.869043+0000","last_clean":"2024-09-11T12:57:33.869043+0000","last_became_active":"2024-09-11T12:57:32.891982+0000","last_became_peered":"2024-09-11T12:57:32.891982+0000","last_unstale":"2024-09-11T12:57:33.869043+0000","last_undegraded":"2024-09-11T12:57:33.869043+0000","last_fullsized":"2024-09-11T12:57:33.869043+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:03.382048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865309+0000","last_change":"2024-09-11T12:57:32.889197+0000","last_active":"2024-09-11T12:57:33.865309+0000","last_peered":"2024-09-11T12:57:33.865309+0000","last_clean":"2024-09-11T12:57:33.865309+0000","last_became_active":"2024-09-11T12:57:32.888905+0000","last_became_peered":"2024-09-11T12:57:32.888905+0000","last_unstale":"2024-09-11T12:57:33.865309+0000","last_undegraded":"2024-09-11T12:57:33.865309+0000","last_fullsized":"2024-09-11T12:57:33.865309+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:56.172538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018214+0000","last_change":"2024-09-11T12:57:32.881863+0000","last_active":"2024-09-11T12:57:34.018214+0000","last_peered":"2024-09-11T12:57:34.018214+0000","last_clean":"2024-09-11T12:57:34.018214+0000","last_became_active":"2024-09-11T12:57:32.881567+0000","last_became_peered":"2024-09-11T12:57:32.881567+0000","last_unstale":"2024-09-11T12:57:34.018214+0000","last_undegraded":"2024-09-11T12:57:34.018214+0000","last_fullsized":"2024-09-11T12:57:34.018214+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:55.080391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895227+0000","last_change":"2024-09-11T12:57:32.881782+0000","last_active":"2024-09-11T12:57:33.895227+0000","last_peered":"2024-09-11T12:57:33.895227+0000","last_clean":"2024-09-11T12:57:33.895227+0000","last_became_active":"2024-09-11T12:57:32.881485+0000","last_became_peered":"2024-09-11T12:57:32.881485+0000","last_unstale":"2024-09-11T12:57:33.895227+0000","last_undegraded":"2024-09-11T12:57:33.895227+0000","last_fullsized":"2024-09-11T12:57:33.895227+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:22.179372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895272+0000","last_change":"2024-09-11T12:57:32.880158+0000","last_active":"2024-09-11T12:57:33.895272+0000","last_peered":"2024-09-11T12:57:33.895272+0000","last_clean":"2024-09-11T12:57:33.895272+0000","last_became_active":"2024-09-11T12:57:32.879440+0000","last_became_peered":"2024-09-11T12:57:32.879440+0000","last_unstale":"2024-09-11T12:57:33.895272+0000","last_undegraded":"2024-09-11T12:57:33.895272+0000","last_fullsized":"2024-09-11T12:57:33.895272+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:23.453105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018055+0000","last_change":"2024-09-11T12:57:32.885869+0000","last_active":"2024-09-11T12:57:34.018055+0000","last_peered":"2024-09-11T12:57:34.018055+0000","last_clean":"2024-09-11T12:57:34.018055+0000","last_became_active":"2024-09-11T12:57:32.885578+0000","last_became_peered":"2024-09-11T12:57:32.885578+0000","last_unstale":"2024-09-11T12:57:34.018055+0000","last_undegraded":"2024-09-11T12:57:34.018055+0000","last_fullsized":"2024-09-11T12:57:34.018055+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:50.295398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017897+0000","last_change":"2024-09-11T12:57:32.882128+0000","last_active":"2024-09-11T12:57:34.017897+0000","last_peered":"2024-09-11T12:57:34.017897+0000","last_clean":"2024-09-11T12:57:34.017897+0000","last_became_active":"2024-09-11T12:57:32.881828+0000","last_became_peered":"2024-09-11T12:57:32.881828+0000","last_unstale":"2024-09-11T12:57:34.017897+0000","last_undegraded":"2024-09-11T12:57:34.017897+0000","last_fullsized":"2024-09-11T12:57:34.017897+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:35.838657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864973+0000","last_change":"2024-09-11T12:57:32.891797+0000","last_active":"2024-09-11T12:57:33.864973+0000","last_peered":"2024-09-11T12:57:33.864973+0000","last_clean":"2024-09-11T12:57:33.864973+0000","last_became_active":"2024-09-11T12:57:32.886475+0000","last_became_peered":"2024-09-11T12:57:32.886475+0000","last_unstale":"2024-09-11T12:57:33.864973+0000","last_undegraded":"2024-09-11T12:57:33.864973+0000","last_fullsized":"2024-09-11T12:57:33.864973+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:20.643732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019337+0000","last_change":"2024-09-11T12:57:32.890144+0000","last_active":"2024-09-11T12:57:34.019337+0000","last_peered":"2024-09-11T12:57:34.019337+0000","last_clean":"2024-09-11T12:57:34.019337+0000","last_became_active":"2024-09-11T12:57:32.889648+0000","last_became_peered":"2024-09-11T12:57:32.889648+0000","last_unstale":"2024-09-11T12:57:34.019337+0000","last_undegraded":"2024-09-11T12:57:34.019337+0000","last_fullsized":"2024-09-11T12:57:34.019337+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:46.344170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865469+0000","last_change":"2024-09-11T12:57:32.891604+0000","last_active":"2024-09-11T12:57:33.865469+0000","last_peered":"2024-09-11T12:57:33.865469+0000","last_clean":"2024-09-11T12:57:33.865469+0000","last_became_active":"2024-09-11T12:57:32.891355+0000","last_became_peered":"2024-09-11T12:57:32.891355+0000","last_unstale":"2024-09-11T12:57:33.865469+0000","last_undegraded":"2024-09-11T12:57:33.865469+0000","last_fullsized":"2024-09-11T12:57:33.865469+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:47:38.893184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865488+0000","last_change":"2024-09-11T12:57:32.889683+0000","last_active":"2024-09-11T12:57:33.865488+0000","last_peered":"2024-09-11T12:57:33.865488+0000","last_clean":"2024-09-11T12:57:33.865488+0000","last_became_active":"2024-09-11T12:57:32.889031+0000","last_became_peered":"2024-09-11T12:57:32.889031+0000","last_unstale":"2024-09-11T12:57:33.865488+0000","last_undegraded":"2024-09-11T12:57:33.865488+0000","last_fullsized":"2024-09-11T12:57:33.865488+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:56.776240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017827+0000","last_change":"2024-09-11T12:57:32.880402+0000","last_active":"2024-09-11T12:57:34.017827+0000","last_peered":"2024-09-11T12:57:34.017827+0000","last_clean":"2024-09-11T12:57:34.017827+0000","last_became_active":"2024-09-11T12:57:32.880076+0000","last_became_peered":"2024-09-11T12:57:32.880076+0000","last_unstale":"2024-09-11T12:57:34.017827+0000","last_undegraded":"2024-09-11T12:57:34.017827+0000","last_fullsized":"2024-09-11T12:57:34.017827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:36.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892383+0000","last_change":"2024-09-11T12:57:32.891776+0000","last_active":"2024-09-11T12:57:33.892383+0000","last_peered":"2024-09-11T12:57:33.892383+0000","last_clean":"2024-09-11T12:57:33.892383+0000","last_became_active":"2024-09-11T12:57:32.891505+0000","last_became_peered":"2024-09-11T12:57:32.891505+0000","last_unstale":"2024-09-11T12:57:33.892383+0000","last_undegraded":"2024-09-11T12:57:33.892383+0000","last_fullsized":"2024-09-11T12:57:33.892383+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:00:31.139145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867741+0000","last_change":"2024-09-11T12:57:32.881143+0000","last_active":"2024-09-11T12:57:33.867741+0000","last_peered":"2024-09-11T12:57:33.867741+0000","last_clean":"2024-09-11T12:57:33.867741+0000","last_became_active":"2024-09-11T12:57:32.879722+0000","last_became_peered":"2024-09-11T12:57:32.879722+0000","last_unstale":"2024-09-11T12:57:33.867741+0000","last_undegraded":"2024-09-11T12:57:33.867741+0000","last_fullsized":"2024-09-11T12:57:33.867741+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:07.853228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867957+0000","last_change":"2024-09-11T12:57:32.871767+0000","last_active":"2024-09-11T12:57:33.867957+0000","last_peered":"2024-09-11T12:57:33.867957+0000","last_clean":"2024-09-11T12:57:33.867957+0000","last_became_active":"2024-09-11T12:57:32.871346+0000","last_became_peered":"2024-09-11T12:57:32.871346+0000","last_unstale":"2024-09-11T12:57:33.867957+0000","last_undegraded":"2024-09-11T12:57:33.867957+0000","last_fullsized":"2024-09-11T12:57:33.867957+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:54.862519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892401+0000","last_change":"2024-09-11T12:57:32.895534+0000","last_active":"2024-09-11T12:57:33.892401+0000","last_peered":"2024-09-11T12:57:33.892401+0000","last_clean":"2024-09-11T12:57:33.892401+0000","last_became_active":"2024-09-11T12:57:32.895226+0000","last_became_peered":"2024-09-11T12:57:32.895226+0000","last_unstale":"2024-09-11T12:57:33.892401+0000","last_undegraded":"2024-09-11T12:57:33.892401+0000","last_fullsized":"2024-09-11T12:57:33.892401+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:56.172538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868883+0000","last_change":"2024-09-11T12:57:32.881232+0000","last_active":"2024-09-11T12:57:33.868883+0000","last_peered":"2024-09-11T12:57:33.868883+0000","last_clean":"2024-09-11T12:57:33.868883+0000","last_became_active":"2024-09-11T12:57:32.880585+0000","last_became_peered":"2024-09-11T12:57:32.880585+0000","last_unstale":"2024-09-11T12:57:33.868883+0000","last_undegraded":"2024-09-11T12:57:33.868883+0000","last_fullsized":"2024-09-11T12:57:33.868883+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:32.907631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018983+0000","last_change":"2024-09-11T12:57:32.881047+0000","last_active":"2024-09-11T12:57:34.018983+0000","last_peered":"2024-09-11T12:57:34.018983+0000","last_clean":"2024-09-11T12:57:34.018983+0000","last_became_active":"2024-09-11T12:57:32.877623+0000","last_became_peered":"2024-09-11T12:57:32.877623+0000","last_unstale":"2024-09-11T12:57:34.018983+0000","last_undegraded":"2024-09-11T12:57:34.018983+0000","last_fullsized":"2024-09-11T12:57:34.018983+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:46:21.507100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895278+0000","last_change":"2024-09-11T12:57:32.881829+0000","last_active":"2024-09-11T12:57:33.895278+0000","last_peered":"2024-09-11T12:57:33.895278+0000","last_clean":"2024-09-11T12:57:33.895278+0000","last_became_active":"2024-09-11T12:57:32.881523+0000","last_became_peered":"2024-09-11T12:57:32.881523+0000","last_unstale":"2024-09-11T12:57:33.895278+0000","last_undegraded":"2024-09-11T12:57:33.895278+0000","last_fullsized":"2024-09-11T12:57:33.895278+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:56.281712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017907+0000","last_change":"2024-09-11T12:57:32.883199+0000","last_active":"2024-09-11T12:57:34.017907+0000","last_peered":"2024-09-11T12:57:34.017907+0000","last_clean":"2024-09-11T12:57:34.017907+0000","last_became_active":"2024-09-11T12:57:32.882937+0000","last_became_peered":"2024-09-11T12:57:32.882937+0000","last_unstale":"2024-09-11T12:57:34.017907+0000","last_undegraded":"2024-09-11T12:57:34.017907+0000","last_fullsized":"2024-09-11T12:57:34.017907+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:59:37.794812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895059+0000","last_change":"2024-09-11T12:57:32.889094+0000","last_active":"2024-09-11T12:57:33.895059+0000","last_peered":"2024-09-11T12:57:33.895059+0000","last_clean":"2024-09-11T12:57:33.895059+0000","last_became_active":"2024-09-11T12:57:32.888755+0000","last_became_peered":"2024-09-11T12:57:32.888755+0000","last_unstale":"2024-09-11T12:57:33.895059+0000","last_undegraded":"2024-09-11T12:57:33.895059+0000","last_fullsized":"2024-09-11T12:57:33.895059+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:27.156526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867826+0000","last_change":"2024-09-11T12:57:32.880207+0000","last_active":"2024-09-11T12:57:33.867826+0000","last_peered":"2024-09-11T12:57:33.867826+0000","last_clean":"2024-09-11T12:57:33.867826+0000","last_became_active":"2024-09-11T12:57:32.879912+0000","last_became_peered":"2024-09-11T12:57:32.879912+0000","last_unstale":"2024-09-11T12:57:33.867826+0000","last_undegraded":"2024-09-11T12:57:33.867826+0000","last_fullsized":"2024-09-11T12:57:33.867826+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:06:11.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864975+0000","last_change":"2024-09-11T12:57:32.890567+0000","last_active":"2024-09-11T12:57:33.864975+0000","last_peered":"2024-09-11T12:57:33.864975+0000","last_clean":"2024-09-11T12:57:33.864975+0000","last_became_active":"2024-09-11T12:57:32.890295+0000","last_became_peered":"2024-09-11T12:57:32.890295+0000","last_unstale":"2024-09-11T12:57:33.864975+0000","last_undegraded":"2024-09-11T12:57:33.864975+0000","last_fullsized":"2024-09-11T12:57:33.864975+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:55.359021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017629+0000","last_change":"2024-09-11T12:57:32.880259+0000","last_active":"2024-09-11T12:57:34.017629+0000","last_peered":"2024-09-11T12:57:34.017629+0000","last_clean":"2024-09-11T12:57:34.017629+0000","last_became_active":"2024-09-11T12:57:32.879843+0000","last_became_peered":"2024-09-11T12:57:32.879843+0000","last_unstale":"2024-09-11T12:57:34.017629+0000","last_undegraded":"2024-09-11T12:57:34.017629+0000","last_fullsized":"2024-09-11T12:57:34.017629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:12.272662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868882+0000","last_change":"2024-09-11T12:57:32.887467+0000","last_active":"2024-09-11T12:57:33.868882+0000","last_peered":"2024-09-11T12:57:33.868882+0000","last_clean":"2024-09-11T12:57:33.868882+0000","last_became_active":"2024-09-11T12:57:32.887165+0000","last_became_peered":"2024-09-11T12:57:32.887165+0000","last_unstale":"2024-09-11T12:57:33.868882+0000","last_undegraded":"2024-09-11T12:57:33.868882+0000","last_fullsized":"2024-09-11T12:57:33.868882+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:47.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"161'4","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895583+0000","last_change":"2024-09-11T12:57:31.864514+0000","last_active":"2024-09-11T12:57:33.895583+0000","last_peered":"2024-09-11T12:57:33.895583+0000","last_clean":"2024-09-11T12:57:33.895583+0000","last_became_active":"2024-09-11T12:57:31.861904+0000","last_became_peered":"2024-09-11T12:57:31.861904+0000","last_unstale":"2024-09-11T12:57:33.895583+0000","last_undegraded":"2024-09-11T12:57:33.895583+0000","last_fullsized":"2024-09-11T12:57:33.895583+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:15.260922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868699+0000","last_change":"2024-09-11T12:57:32.884362+0000","last_active":"2024-09-11T12:57:33.868699+0000","last_peered":"2024-09-11T12:57:33.868699+0000","last_clean":"2024-09-11T12:57:33.868699+0000","last_became_active":"2024-09-11T12:57:32.884021+0000","last_became_peered":"2024-09-11T12:57:32.884021+0000","last_unstale":"2024-09-11T12:57:33.868699+0000","last_undegraded":"2024-09-11T12:57:33.868699+0000","last_fullsized":"2024-09-11T12:57:33.868699+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:22.179372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868997+0000","last_change":"2024-09-11T12:57:31.861806+0000","last_active":"2024-09-11T12:57:33.868997+0000","last_peered":"2024-09-11T12:57:33.868997+0000","last_clean":"2024-09-11T12:57:33.868997+0000","last_became_active":"2024-09-11T12:57:31.861491+0000","last_became_peered":"2024-09-11T12:57:31.861491+0000","last_unstale":"2024-09-11T12:57:33.868997+0000","last_undegraded":"2024-09-11T12:57:33.868997+0000","last_fullsized":"2024-09-11T12:57:33.868997+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:16.158063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018506+0000","last_change":"2024-09-11T12:57:32.879520+0000","last_active":"2024-09-11T12:57:34.018506+0000","last_peered":"2024-09-11T12:57:34.018506+0000","last_clean":"2024-09-11T12:57:34.018506+0000","last_became_active":"2024-09-11T12:57:32.878396+0000","last_became_peered":"2024-09-11T12:57:32.878396+0000","last_unstale":"2024-09-11T12:57:34.018506+0000","last_undegraded":"2024-09-11T12:57:34.018506+0000","last_fullsized":"2024-09-11T12:57:34.018506+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:12.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018287+0000","last_change":"2024-09-11T12:57:31.865369+0000","last_active":"2024-09-11T12:57:34.018287+0000","last_peered":"2024-09-11T12:57:34.018287+0000","last_clean":"2024-09-11T12:57:34.018287+0000","last_became_active":"2024-09-11T12:57:31.865083+0000","last_became_peered":"2024-09-11T12:57:31.865083+0000","last_unstale":"2024-09-11T12:57:34.018287+0000","last_undegraded":"2024-09-11T12:57:34.018287+0000","last_fullsized":"2024-09-11T12:57:34.018287+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:10.063118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865284+0000","last_change":"2024-09-11T12:57:32.892581+0000","last_active":"2024-09-11T12:57:33.865284+0000","last_peered":"2024-09-11T12:57:33.865284+0000","last_clean":"2024-09-11T12:57:33.865284+0000","last_became_active":"2024-09-11T12:57:32.892322+0000","last_became_peered":"2024-09-11T12:57:32.892322+0000","last_unstale":"2024-09-11T12:57:33.865284+0000","last_undegraded":"2024-09-11T12:57:33.865284+0000","last_fullsized":"2024-09-11T12:57:33.865284+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:00:31.139145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892593+0000","last_change":"2024-09-11T12:57:31.871779+0000","last_active":"2024-09-11T12:57:33.892593+0000","last_peered":"2024-09-11T12:57:33.892593+0000","last_clean":"2024-09-11T12:57:33.892593+0000","last_became_active":"2024-09-11T12:57:31.871344+0000","last_became_peered":"2024-09-11T12:57:31.871344+0000","last_unstale":"2024-09-11T12:57:33.892593+0000","last_undegraded":"2024-09-11T12:57:33.892593+0000","last_fullsized":"2024-09-11T12:57:33.892593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:42.155435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892379+0000","last_change":"2024-09-11T12:57:32.882843+0000","last_active":"2024-09-11T12:57:33.892379+0000","last_peered":"2024-09-11T12:57:33.892379+0000","last_clean":"2024-09-11T12:57:33.892379+0000","last_became_active":"2024-09-11T12:57:32.882566+0000","last_became_peered":"2024-09-11T12:57:32.882566+0000","last_unstale":"2024-09-11T12:57:33.892379+0000","last_undegraded":"2024-09-11T12:57:33.892379+0000","last_fullsized":"2024-09-11T12:57:33.892379+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:44:07.586644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017870+0000","last_change":"2024-09-11T12:57:31.869332+0000","last_active":"2024-09-11T12:57:34.017870+0000","last_peered":"2024-09-11T12:57:34.017870+0000","last_clean":"2024-09-11T12:57:34.017870+0000","last_became_active":"2024-09-11T12:57:31.869052+0000","last_became_peered":"2024-09-11T12:57:31.869052+0000","last_unstale":"2024-09-11T12:57:34.017870+0000","last_undegraded":"2024-09-11T12:57:34.017870+0000","last_fullsized":"2024-09-11T12:57:34.017870+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:21.226658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017278+0000","last_change":"2024-09-11T12:57:32.889980+0000","last_active":"2024-09-11T12:57:34.017278+0000","last_peered":"2024-09-11T12:57:34.017278+0000","last_clean":"2024-09-11T12:57:34.017278+0000","last_became_active":"2024-09-11T12:57:32.889676+0000","last_became_peered":"2024-09-11T12:57:32.889676+0000","last_unstale":"2024-09-11T12:57:34.017278+0000","last_undegraded":"2024-09-11T12:57:34.017278+0000","last_fullsized":"2024-09-11T12:57:34.017278+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:24.561952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017603+0000","last_change":"2024-09-11T12:57:31.868458+0000","last_active":"2024-09-11T12:57:34.017603+0000","last_peered":"2024-09-11T12:57:34.017603+0000","last_clean":"2024-09-11T12:57:34.017603+0000","last_became_active":"2024-09-11T12:57:31.868173+0000","last_became_peered":"2024-09-11T12:57:31.868173+0000","last_unstale":"2024-09-11T12:57:34.017603+0000","last_undegraded":"2024-09-11T12:57:34.017603+0000","last_fullsized":"2024-09-11T12:57:34.017603+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:06.844153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892380+0000","last_change":"2024-09-11T12:57:32.888265+0000","last_active":"2024-09-11T12:57:33.892380+0000","last_peered":"2024-09-11T12:57:33.892380+0000","last_clean":"2024-09-11T12:57:33.892380+0000","last_became_active":"2024-09-11T12:57:32.887920+0000","last_became_peered":"2024-09-11T12:57:32.887920+0000","last_unstale":"2024-09-11T12:57:33.892380+0000","last_undegraded":"2024-09-11T12:57:33.892380+0000","last_fullsized":"2024-09-11T12:57:33.892380+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.461881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867280+0000","last_change":"2024-09-11T12:57:31.852792+0000","last_active":"2024-09-11T12:57:33.867280+0000","last_peered":"2024-09-11T12:57:33.867280+0000","last_clean":"2024-09-11T12:57:33.867280+0000","last_became_active":"2024-09-11T12:57:31.852499+0000","last_became_peered":"2024-09-11T12:57:31.852499+0000","last_unstale":"2024-09-11T12:57:33.867280+0000","last_undegraded":"2024-09-11T12:57:33.867280+0000","last_fullsized":"2024-09-11T12:57:33.867280+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:19.415493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892641+0000","last_change":"2024-09-11T12:57:32.891524+0000","last_active":"2024-09-11T12:57:33.892641+0000","last_peered":"2024-09-11T12:57:33.892641+0000","last_clean":"2024-09-11T12:57:33.892641+0000","last_became_active":"2024-09-11T12:57:32.891188+0000","last_became_peered":"2024-09-11T12:57:32.891188+0000","last_unstale":"2024-09-11T12:57:33.892641+0000","last_undegraded":"2024-09-11T12:57:33.892641+0000","last_fullsized":"2024-09-11T12:57:33.892641+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:53:05.458153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018520+0000","last_change":"2024-09-11T12:57:31.863925+0000","last_active":"2024-09-11T12:57:34.018520+0000","last_peered":"2024-09-11T12:57:34.018520+0000","last_clean":"2024-09-11T12:57:34.018520+0000","last_became_active":"2024-09-11T12:57:31.863660+0000","last_became_peered":"2024-09-11T12:57:31.863660+0000","last_unstale":"2024-09-11T12:57:34.018520+0000","last_undegraded":"2024-09-11T12:57:34.018520+0000","last_fullsized":"2024-09-11T12:57:34.018520+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:26.147451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018920+0000","last_change":"2024-09-11T12:57:32.884255+0000","last_active":"2024-09-11T12:57:34.018920+0000","last_peered":"2024-09-11T12:57:34.018920+0000","last_clean":"2024-09-11T12:57:34.018920+0000","last_became_active":"2024-09-11T12:57:32.883980+0000","last_became_peered":"2024-09-11T12:57:32.883980+0000","last_unstale":"2024-09-11T12:57:34.018920+0000","last_undegraded":"2024-09-11T12:57:34.018920+0000","last_fullsized":"2024-09-11T12:57:34.018920+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:26:18.792605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868435+0000","last_change":"2024-09-11T12:57:31.857213+0000","last_active":"2024-09-11T12:57:33.868435+0000","last_peered":"2024-09-11T12:57:33.868435+0000","last_clean":"2024-09-11T12:57:33.868435+0000","last_became_active":"2024-09-11T12:57:31.856897+0000","last_became_peered":"2024-09-11T12:57:31.856897+0000","last_unstale":"2024-09-11T12:57:33.868435+0000","last_undegraded":"2024-09-11T12:57:33.868435+0000","last_fullsized":"2024-09-11T12:57:33.868435+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:17.772531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892610+0000","last_change":"2024-09-11T12:57:32.891115+0000","last_active":"2024-09-11T12:57:33.892610+0000","last_peered":"2024-09-11T12:57:33.892610+0000","last_clean":"2024-09-11T12:57:33.892610+0000","last_became_active":"2024-09-11T12:57:32.890723+0000","last_became_peered":"2024-09-11T12:57:32.890723+0000","last_unstale":"2024-09-11T12:57:33.892610+0000","last_undegraded":"2024-09-11T12:57:33.892610+0000","last_fullsized":"2024-09-11T12:57:33.892610+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:56.776240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"161'3","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865425+0000","last_change":"2024-09-11T12:57:31.870529+0000","last_active":"2024-09-11T12:57:33.865425+0000","last_peered":"2024-09-11T12:57:33.865425+0000","last_clean":"2024-09-11T12:57:33.865425+0000","last_became_active":"2024-09-11T12:57:31.864182+0000","last_became_peered":"2024-09-11T12:57:31.864182+0000","last_unstale":"2024-09-11T12:57:33.865425+0000","last_undegraded":"2024-09-11T12:57:33.865425+0000","last_fullsized":"2024-09-11T12:57:33.865425+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:50.098425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019099+0000","last_change":"2024-09-11T12:57:32.887802+0000","last_active":"2024-09-11T12:57:34.019099+0000","last_peered":"2024-09-11T12:57:34.019099+0000","last_clean":"2024-09-11T12:57:34.019099+0000","last_became_active":"2024-09-11T12:57:32.887521+0000","last_became_peered":"2024-09-11T12:57:32.887521+0000","last_unstale":"2024-09-11T12:57:34.019099+0000","last_undegraded":"2024-09-11T12:57:34.019099+0000","last_fullsized":"2024-09-11T12:57:34.019099+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:24:14.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"161'6","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892688+0000","last_change":"2024-09-11T12:57:31.865574+0000","last_active":"2024-09-11T12:57:33.892688+0000","last_peered":"2024-09-11T12:57:33.892688+0000","last_clean":"2024-09-11T12:57:33.892688+0000","last_became_active":"2024-09-11T12:57:31.865300+0000","last_became_peered":"2024-09-11T12:57:31.865300+0000","last_unstale":"2024-09-11T12:57:33.892688+0000","last_undegraded":"2024-09-11T12:57:33.892688+0000","last_fullsized":"2024-09-11T12:57:33.892688+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:23.552877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017716+0000","last_change":"2024-09-11T12:57:32.879251+0000","last_active":"2024-09-11T12:57:34.017716+0000","last_peered":"2024-09-11T12:57:34.017716+0000","last_clean":"2024-09-11T12:57:34.017716+0000","last_became_active":"2024-09-11T12:57:32.878954+0000","last_became_peered":"2024-09-11T12:57:32.878954+0000","last_unstale":"2024-09-11T12:57:34.017716+0000","last_undegraded":"2024-09-11T12:57:34.017716+0000","last_fullsized":"2024-09-11T12:57:34.017716+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:34:14.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867773+0000","last_change":"2024-09-11T12:57:31.863811+0000","last_active":"2024-09-11T12:57:33.867773+0000","last_peered":"2024-09-11T12:57:33.867773+0000","last_clean":"2024-09-11T12:57:33.867773+0000","last_became_active":"2024-09-11T12:57:31.863559+0000","last_became_peered":"2024-09-11T12:57:31.863559+0000","last_unstale":"2024-09-11T12:57:33.867773+0000","last_undegraded":"2024-09-11T12:57:33.867773+0000","last_fullsized":"2024-09-11T12:57:33.867773+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:29.932458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865257+0000","last_change":"2024-09-11T12:57:32.878633+0000","last_active":"2024-09-11T12:57:33.865257+0000","last_peered":"2024-09-11T12:57:33.865257+0000","last_clean":"2024-09-11T12:57:33.865257+0000","last_became_active":"2024-09-11T12:57:32.878369+0000","last_became_peered":"2024-09-11T12:57:32.878369+0000","last_unstale":"2024-09-11T12:57:33.865257+0000","last_undegraded":"2024-09-11T12:57:33.865257+0000","last_fullsized":"2024-09-11T12:57:33.865257+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:44:07.586644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868035+0000","last_change":"2024-09-11T12:57:31.850870+0000","last_active":"2024-09-11T12:57:33.868035+0000","last_peered":"2024-09-11T12:57:33.868035+0000","last_clean":"2024-09-11T12:57:33.868035+0000","last_became_active":"2024-09-11T12:57:31.850421+0000","last_became_peered":"2024-09-11T12:57:31.850421+0000","last_unstale":"2024-09-11T12:57:33.868035+0000","last_undegraded":"2024-09-11T12:57:33.868035+0000","last_fullsized":"2024-09-11T12:57:33.868035+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:15.260922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018229+0000","last_change":"2024-09-11T12:57:32.884877+0000","last_active":"2024-09-11T12:57:34.018229+0000","last_peered":"2024-09-11T12:57:34.018229+0000","last_clean":"2024-09-11T12:57:34.018229+0000","last_became_active":"2024-09-11T12:57:32.884617+0000","last_became_peered":"2024-09-11T12:57:32.884617+0000","last_unstale":"2024-09-11T12:57:34.018229+0000","last_undegraded":"2024-09-11T12:57:34.018229+0000","last_fullsized":"2024-09-11T12:57:34.018229+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:51:04.146114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864884+0000","last_change":"2024-09-11T12:57:31.858024+0000","last_active":"2024-09-11T12:57:33.864884+0000","last_peered":"2024-09-11T12:57:33.864884+0000","last_clean":"2024-09-11T12:57:33.864884+0000","last_became_active":"2024-09-11T12:57:31.857769+0000","last_became_peered":"2024-09-11T12:57:31.857769+0000","last_unstale":"2024-09-11T12:57:33.864884+0000","last_undegraded":"2024-09-11T12:57:33.864884+0000","last_fullsized":"2024-09-11T12:57:33.864884+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:22.444030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018042+0000","last_change":"2024-09-11T12:57:32.876251+0000","last_active":"2024-09-11T12:57:34.018042+0000","last_peered":"2024-09-11T12:57:34.018042+0000","last_clean":"2024-09-11T12:57:34.018042+0000","last_became_active":"2024-09-11T12:57:32.875799+0000","last_became_peered":"2024-09-11T12:57:32.875799+0000","last_unstale":"2024-09-11T12:57:34.018042+0000","last_undegraded":"2024-09-11T12:57:34.018042+0000","last_fullsized":"2024-09-11T12:57:34.018042+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:48.165860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"161'8","reported_seq":23,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018457+0000","last_change":"2024-09-11T12:57:31.866327+0000","last_active":"2024-09-11T12:57:34.018457+0000","last_peered":"2024-09-11T12:57:34.018457+0000","last_clean":"2024-09-11T12:57:34.018457+0000","last_became_active":"2024-09-11T12:57:31.865182+0000","last_became_peered":"2024-09-11T12:57:31.865182+0000","last_unstale":"2024-09-11T12:57:34.018457+0000","last_undegraded":"2024-09-11T12:57:34.018457+0000","last_fullsized":"2024-09-11T12:57:34.018457+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:21.332155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869021+0000","last_change":"2024-09-11T12:57:32.884972+0000","last_active":"2024-09-11T12:57:33.869021+0000","last_peered":"2024-09-11T12:57:33.869021+0000","last_clean":"2024-09-11T12:57:33.869021+0000","last_became_active":"2024-09-11T12:57:32.884694+0000","last_became_peered":"2024-09-11T12:57:32.884694+0000","last_unstale":"2024-09-11T12:57:33.869021+0000","last_undegraded":"2024-09-11T12:57:33.869021+0000","last_fullsized":"2024-09-11T12:57:33.869021+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:22:00.383319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018620+0000","last_change":"2024-09-11T12:57:31.869598+0000","last_active":"2024-09-11T12:57:34.018620+0000","last_peered":"2024-09-11T12:57:34.018620+0000","last_clean":"2024-09-11T12:57:34.018620+0000","last_became_active":"2024-09-11T12:57:31.869347+0000","last_became_peered":"2024-09-11T12:57:31.869347+0000","last_unstale":"2024-09-11T12:57:34.018620+0000","last_undegraded":"2024-09-11T12:57:34.018620+0000","last_fullsized":"2024-09-11T12:57:34.018620+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:21.170297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868898+0000","last_change":"2024-09-11T12:57:32.881495+0000","last_active":"2024-09-11T12:57:33.868898+0000","last_peered":"2024-09-11T12:57:33.868898+0000","last_clean":"2024-09-11T12:57:33.868898+0000","last_became_active":"2024-09-11T12:57:32.880925+0000","last_became_peered":"2024-09-11T12:57:32.880925+0000","last_unstale":"2024-09-11T12:57:33.868898+0000","last_undegraded":"2024-09-11T12:57:33.868898+0000","last_fullsized":"2024-09-11T12:57:33.868898+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:59.660211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869067+0000","last_change":"2024-09-11T12:57:31.854091+0000","last_active":"2024-09-11T12:57:33.869067+0000","last_peered":"2024-09-11T12:57:33.869067+0000","last_clean":"2024-09-11T12:57:33.869067+0000","last_became_active":"2024-09-11T12:57:31.853803+0000","last_became_peered":"2024-09-11T12:57:31.853803+0000","last_unstale":"2024-09-11T12:57:33.869067+0000","last_undegraded":"2024-09-11T12:57:33.869067+0000","last_fullsized":"2024-09-11T12:57:33.869067+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:08:35.843980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018076+0000","last_change":"2024-09-11T12:57:32.888089+0000","last_active":"2024-09-11T12:57:34.018076+0000","last_peered":"2024-09-11T12:57:34.018076+0000","last_clean":"2024-09-11T12:57:34.018076+0000","last_became_active":"2024-09-11T12:57:32.887587+0000","last_became_peered":"2024-09-11T12:57:32.887587+0000","last_unstale":"2024-09-11T12:57:34.018076+0000","last_undegraded":"2024-09-11T12:57:34.018076+0000","last_fullsized":"2024-09-11T12:57:34.018076+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:04.655412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865532+0000","last_change":"2024-09-11T12:57:31.870185+0000","last_active":"2024-09-11T12:57:33.865532+0000","last_peered":"2024-09-11T12:57:33.865532+0000","last_clean":"2024-09-11T12:57:33.865532+0000","last_became_active":"2024-09-11T12:57:31.864175+0000","last_became_peered":"2024-09-11T12:57:31.864175+0000","last_unstale":"2024-09-11T12:57:33.865532+0000","last_undegraded":"2024-09-11T12:57:33.865532+0000","last_fullsized":"2024-09-11T12:57:33.865532+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:00.347455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865262+0000","last_change":"2024-09-11T12:57:32.891001+0000","last_active":"2024-09-11T12:57:33.865262+0000","last_peered":"2024-09-11T12:57:33.865262+0000","last_clean":"2024-09-11T12:57:33.865262+0000","last_became_active":"2024-09-11T12:57:32.890382+0000","last_became_peered":"2024-09-11T12:57:32.890382+0000","last_unstale":"2024-09-11T12:57:33.865262+0000","last_undegraded":"2024-09-11T12:57:33.865262+0000","last_fullsized":"2024-09-11T12:57:33.865262+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.461881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018168+0000","last_change":"2024-09-11T12:57:31.872206+0000","last_active":"2024-09-11T12:57:34.018168+0000","last_peered":"2024-09-11T12:57:34.018168+0000","last_clean":"2024-09-11T12:57:34.018168+0000","last_became_active":"2024-09-11T12:57:31.871859+0000","last_became_peered":"2024-09-11T12:57:31.871859+0000","last_unstale":"2024-09-11T12:57:34.018168+0000","last_undegraded":"2024-09-11T12:57:34.018168+0000","last_fullsized":"2024-09-11T12:57:34.018168+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:38.784985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865090+0000","last_change":"2024-09-11T12:57:32.882146+0000","last_active":"2024-09-11T12:57:33.865090+0000","last_peered":"2024-09-11T12:57:33.865090+0000","last_clean":"2024-09-11T12:57:33.865090+0000","last_became_active":"2024-09-11T12:57:32.881675+0000","last_became_peered":"2024-09-11T12:57:32.881675+0000","last_unstale":"2024-09-11T12:57:33.865090+0000","last_undegraded":"2024-09-11T12:57:33.865090+0000","last_fullsized":"2024-09-11T12:57:33.865090+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:10.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865288+0000","last_change":"2024-09-11T12:57:31.866404+0000","last_active":"2024-09-11T12:57:33.865288+0000","last_peered":"2024-09-11T12:57:33.865288+0000","last_clean":"2024-09-11T12:57:33.865288+0000","last_became_active":"2024-09-11T12:57:31.865858+0000","last_became_peered":"2024-09-11T12:57:31.865858+0000","last_unstale":"2024-09-11T12:57:33.865288+0000","last_undegraded":"2024-09-11T12:57:33.865288+0000","last_fullsized":"2024-09-11T12:57:33.865288+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:55.272637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018686+0000","last_change":"2024-09-11T12:57:32.886161+0000","last_active":"2024-09-11T12:57:34.018686+0000","last_peered":"2024-09-11T12:57:34.018686+0000","last_clean":"2024-09-11T12:57:34.018686+0000","last_became_active":"2024-09-11T12:57:32.885899+0000","last_became_peered":"2024-09-11T12:57:32.885899+0000","last_unstale":"2024-09-11T12:57:34.018686+0000","last_undegraded":"2024-09-11T12:57:34.018686+0000","last_fullsized":"2024-09-11T12:57:34.018686+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:12:33.922161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018426+0000","last_change":"2024-09-11T12:57:31.870069+0000","last_active":"2024-09-11T12:57:34.018426+0000","last_peered":"2024-09-11T12:57:34.018426+0000","last_clean":"2024-09-11T12:57:34.018426+0000","last_became_active":"2024-09-11T12:57:31.862696+0000","last_became_peered":"2024-09-11T12:57:31.862696+0000","last_unstale":"2024-09-11T12:57:34.018426+0000","last_undegraded":"2024-09-11T12:57:34.018426+0000","last_fullsized":"2024-09-11T12:57:34.018426+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:11.423174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867363+0000","last_change":"2024-09-11T12:57:32.877902+0000","last_active":"2024-09-11T12:57:33.867363+0000","last_peered":"2024-09-11T12:57:33.867363+0000","last_clean":"2024-09-11T12:57:33.867363+0000","last_became_active":"2024-09-11T12:57:32.876976+0000","last_became_peered":"2024-09-11T12:57:32.876976+0000","last_unstale":"2024-09-11T12:57:33.867363+0000","last_undegraded":"2024-09-11T12:57:33.867363+0000","last_fullsized":"2024-09-11T12:57:33.867363+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:00.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018135+0000","last_change":"2024-09-11T12:57:31.854629+0000","last_active":"2024-09-11T12:57:34.018135+0000","last_peered":"2024-09-11T12:57:34.018135+0000","last_clean":"2024-09-11T12:57:34.018135+0000","last_became_active":"2024-09-11T12:57:31.854373+0000","last_became_peered":"2024-09-11T12:57:31.854373+0000","last_unstale":"2024-09-11T12:57:34.018135+0000","last_undegraded":"2024-09-11T12:57:34.018135+0000","last_fullsized":"2024-09-11T12:57:34.018135+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:09.010285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867758+0000","last_change":"2024-09-11T12:57:32.884164+0000","last_active":"2024-09-11T12:57:33.867758+0000","last_peered":"2024-09-11T12:57:33.867758+0000","last_clean":"2024-09-11T12:57:33.867758+0000","last_became_active":"2024-09-11T12:57:32.883873+0000","last_became_peered":"2024-09-11T12:57:32.883873+0000","last_unstale":"2024-09-11T12:57:33.867758+0000","last_undegraded":"2024-09-11T12:57:33.867758+0000","last_fullsized":"2024-09-11T12:57:33.867758+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:22.235734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017362+0000","last_change":"2024-09-11T12:57:31.870348+0000","last_active":"2024-09-11T12:57:34.017362+0000","last_peered":"2024-09-11T12:57:34.017362+0000","last_clean":"2024-09-11T12:57:34.017362+0000","last_became_active":"2024-09-11T12:57:31.870073+0000","last_became_peered":"2024-09-11T12:57:31.870073+0000","last_unstale":"2024-09-11T12:57:34.017362+0000","last_undegraded":"2024-09-11T12:57:34.017362+0000","last_fullsized":"2024-09-11T12:57:34.017362+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:31.622668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895340+0000","last_change":"2024-09-11T12:57:32.886241+0000","last_active":"2024-09-11T12:57:33.895340+0000","last_peered":"2024-09-11T12:57:33.895340+0000","last_clean":"2024-09-11T12:57:33.895340+0000","last_became_active":"2024-09-11T12:57:32.885788+0000","last_became_peered":"2024-09-11T12:57:32.885788+0000","last_unstale":"2024-09-11T12:57:33.895340+0000","last_undegraded":"2024-09-11T12:57:33.895340+0000","last_fullsized":"2024-09-11T12:57:33.895340+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:01.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018971+0000","last_change":"2024-09-11T12:57:31.867186+0000","last_active":"2024-09-11T12:57:34.018971+0000","last_peered":"2024-09-11T12:57:34.018971+0000","last_clean":"2024-09-11T12:57:34.018971+0000","last_became_active":"2024-09-11T12:57:31.866919+0000","last_became_peered":"2024-09-11T12:57:31.866919+0000","last_unstale":"2024-09-11T12:57:34.018971+0000","last_undegraded":"2024-09-11T12:57:34.018971+0000","last_fullsized":"2024-09-11T12:57:34.018971+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:40.651487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019161+0000","last_change":"2024-09-11T12:57:32.885645+0000","last_active":"2024-09-11T12:57:34.019161+0000","last_peered":"2024-09-11T12:57:34.019161+0000","last_clean":"2024-09-11T12:57:34.019161+0000","last_became_active":"2024-09-11T12:57:32.885329+0000","last_became_peered":"2024-09-11T12:57:32.885329+0000","last_unstale":"2024-09-11T12:57:34.019161+0000","last_undegraded":"2024-09-11T12:57:34.019161+0000","last_fullsized":"2024-09-11T12:57:34.019161+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:59.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"162'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.894303+0000","last_change":"2024-09-11T12:57:31.866622+0000","last_active":"2024-09-11T12:57:33.894303+0000","last_peered":"2024-09-11T12:57:33.894303+0000","last_clean":"2024-09-11T12:57:33.894303+0000","last_became_active":"2024-09-11T12:57:31.866334+0000","last_became_peered":"2024-09-11T12:57:31.866334+0000","last_unstale":"2024-09-11T12:57:33.894303+0000","last_undegraded":"2024-09-11T12:57:33.894303+0000","last_fullsized":"2024-09-11T12:57:33.894303+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:05.932483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868590+0000","last_change":"2024-09-11T12:57:32.871089+0000","last_active":"2024-09-11T12:57:33.868590+0000","last_peered":"2024-09-11T12:57:33.868590+0000","last_clean":"2024-09-11T12:57:33.868590+0000","last_became_active":"2024-09-11T12:57:32.870622+0000","last_became_peered":"2024-09-11T12:57:32.870622+0000","last_unstale":"2024-09-11T12:57:33.868590+0000","last_undegraded":"2024-09-11T12:57:33.868590+0000","last_fullsized":"2024-09-11T12:57:33.868590+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:25:17.652515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018020+0000","last_change":"2024-09-11T12:57:31.868269+0000","last_active":"2024-09-11T12:57:34.018020+0000","last_peered":"2024-09-11T12:57:34.018020+0000","last_clean":"2024-09-11T12:57:34.018020+0000","last_became_active":"2024-09-11T12:57:31.868006+0000","last_became_peered":"2024-09-11T12:57:31.868006+0000","last_unstale":"2024-09-11T12:57:34.018020+0000","last_undegraded":"2024-09-11T12:57:34.018020+0000","last_fullsized":"2024-09-11T12:57:34.018020+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:55.272637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868233+0000","last_change":"2024-09-11T12:57:32.881512+0000","last_active":"2024-09-11T12:57:33.868233+0000","last_peered":"2024-09-11T12:57:33.868233+0000","last_clean":"2024-09-11T12:57:33.868233+0000","last_became_active":"2024-09-11T12:57:32.879930+0000","last_became_peered":"2024-09-11T12:57:32.879930+0000","last_unstale":"2024-09-11T12:57:33.868233+0000","last_undegraded":"2024-09-11T12:57:33.868233+0000","last_fullsized":"2024-09-11T12:57:33.868233+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:27.156526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018224+0000","last_change":"2024-09-11T12:57:31.867661+0000","last_active":"2024-09-11T12:57:34.018224+0000","last_peered":"2024-09-11T12:57:34.018224+0000","last_clean":"2024-09-11T12:57:34.018224+0000","last_became_active":"2024-09-11T12:57:31.867397+0000","last_became_peered":"2024-09-11T12:57:31.867397+0000","last_unstale":"2024-09-11T12:57:34.018224+0000","last_undegraded":"2024-09-11T12:57:34.018224+0000","last_fullsized":"2024-09-11T12:57:34.018224+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.189410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018313+0000","last_change":"2024-09-11T12:57:32.878537+0000","last_active":"2024-09-11T12:57:34.018313+0000","last_peered":"2024-09-11T12:57:34.018313+0000","last_clean":"2024-09-11T12:57:34.018313+0000","last_became_active":"2024-09-11T12:57:32.878256+0000","last_became_peered":"2024-09-11T12:57:32.878256+0000","last_unstale":"2024-09-11T12:57:34.018313+0000","last_undegraded":"2024-09-11T12:57:34.018313+0000","last_fullsized":"2024-09-11T12:57:34.018313+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:35:56.025964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892527+0000","last_change":"2024-09-11T12:57:31.874733+0000","last_active":"2024-09-11T12:57:33.892527+0000","last_peered":"2024-09-11T12:57:33.892527+0000","last_clean":"2024-09-11T12:57:33.892527+0000","last_became_active":"2024-09-11T12:57:31.874504+0000","last_became_peered":"2024-09-11T12:57:31.874504+0000","last_unstale":"2024-09-11T12:57:33.892527+0000","last_undegraded":"2024-09-11T12:57:33.892527+0000","last_fullsized":"2024-09-11T12:57:33.892527+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:55.731340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895352+0000","last_change":"2024-09-11T12:57:32.885287+0000","last_active":"2024-09-11T12:57:33.895352+0000","last_peered":"2024-09-11T12:57:33.895352+0000","last_clean":"2024-09-11T12:57:33.895352+0000","last_became_active":"2024-09-11T12:57:32.882854+0000","last_became_peered":"2024-09-11T12:57:32.882854+0000","last_unstale":"2024-09-11T12:57:33.895352+0000","last_undegraded":"2024-09-11T12:57:33.895352+0000","last_fullsized":"2024-09-11T12:57:33.895352+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:51.107501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017934+0000","last_change":"2024-09-11T12:57:31.871691+0000","last_active":"2024-09-11T12:57:34.017934+0000","last_peered":"2024-09-11T12:57:34.017934+0000","last_clean":"2024-09-11T12:57:34.017934+0000","last_became_active":"2024-09-11T12:57:31.869375+0000","last_became_peered":"2024-09-11T12:57:31.869375+0000","last_unstale":"2024-09-11T12:57:34.017934+0000","last_undegraded":"2024-09-11T12:57:34.017934+0000","last_fullsized":"2024-09-11T12:57:34.017934+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:53.853443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018399+0000","last_change":"2024-09-11T12:57:32.881504+0000","last_active":"2024-09-11T12:57:34.018399+0000","last_peered":"2024-09-11T12:57:34.018399+0000","last_clean":"2024-09-11T12:57:34.018399+0000","last_became_active":"2024-09-11T12:57:32.881210+0000","last_became_peered":"2024-09-11T12:57:32.881210+0000","last_unstale":"2024-09-11T12:57:34.018399+0000","last_undegraded":"2024-09-11T12:57:34.018399+0000","last_fullsized":"2024-09-11T12:57:34.018399+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:12.845778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867878+0000","last_change":"2024-09-11T12:57:31.855799+0000","last_active":"2024-09-11T12:57:33.867878+0000","last_peered":"2024-09-11T12:57:33.867878+0000","last_clean":"2024-09-11T12:57:33.867878+0000","last_became_active":"2024-09-11T12:57:31.855465+0000","last_became_peered":"2024-09-11T12:57:31.855465+0000","last_unstale":"2024-09-11T12:57:33.867878+0000","last_undegraded":"2024-09-11T12:57:33.867878+0000","last_fullsized":"2024-09-11T12:57:33.867878+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:51.452333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867847+0000","last_change":"2024-09-11T12:57:32.891156+0000","last_active":"2024-09-11T12:57:33.867847+0000","last_peered":"2024-09-11T12:57:33.867847+0000","last_clean":"2024-09-11T12:57:33.867847+0000","last_became_active":"2024-09-11T12:57:32.890728+0000","last_became_peered":"2024-09-11T12:57:32.890728+0000","last_unstale":"2024-09-11T12:57:33.867847+0000","last_undegraded":"2024-09-11T12:57:33.867847+0000","last_fullsized":"2024-09-11T12:57:33.867847+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:59.493090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.894982+0000","last_change":"2024-09-11T12:57:31.864289+0000","last_active":"2024-09-11T12:57:33.894982+0000","last_peered":"2024-09-11T12:57:33.894982+0000","last_clean":"2024-09-11T12:57:33.894982+0000","last_became_active":"2024-09-11T12:57:31.862678+0000","last_became_peered":"2024-09-11T12:57:31.862678+0000","last_unstale":"2024-09-11T12:57:33.894982+0000","last_undegraded":"2024-09-11T12:57:33.894982+0000","last_fullsized":"2024-09-11T12:57:33.894982+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:58:56.466714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018184+0000","last_change":"2024-09-11T12:57:32.890686+0000","last_active":"2024-09-11T12:57:34.018184+0000","last_peered":"2024-09-11T12:57:34.018184+0000","last_clean":"2024-09-11T12:57:34.018184+0000","last_became_active":"2024-09-11T12:57:32.890376+0000","last_became_peered":"2024-09-11T12:57:32.890376+0000","last_unstale":"2024-09-11T12:57:34.018184+0000","last_undegraded":"2024-09-11T12:57:34.018184+0000","last_fullsized":"2024-09-11T12:57:34.018184+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:47.914002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895395+0000","last_change":"2024-09-11T12:57:31.857498+0000","last_active":"2024-09-11T12:57:33.895395+0000","last_peered":"2024-09-11T12:57:33.895395+0000","last_clean":"2024-09-11T12:57:33.895395+0000","last_became_active":"2024-09-11T12:57:31.857220+0000","last_became_peered":"2024-09-11T12:57:31.857220+0000","last_unstale":"2024-09-11T12:57:33.895395+0000","last_undegraded":"2024-09-11T12:57:33.895395+0000","last_fullsized":"2024-09-11T12:57:33.895395+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:40.115214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":392,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892265+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:33.892265+0000","last_peered":"2024-09-11T12:57:33.892265+0000","last_clean":"2024-09-11T12:57:33.892265+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:33.892265+0000","last_undegraded":"2024-09-11T12:57:33.892265+0000","last_fullsized":"2024-09-11T12:57:33.892265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2646016,"data_stored":1488749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2043904,"data_stored":897351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2043904,"data_stored":897351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2060288,"data_stored":898485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2052096,"data_stored":896951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2039808,"data_stored":896769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2048000,"data_stored":896825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542244,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37924,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333916,"statfs":{"total":96636764160,"available":96597929984,"internally_reserved":0,"allocated":2637824,"data_stored":1487227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:57:36.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:57:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 -- 172.21.15.78:0/3376712148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3600070360 msgr2=0x7f3600071d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 --2- 172.21.15.78:0/3376712148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3600070360 0x7f3600071d60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f35f8009bf0 tx=0x7f35f801cad0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 -- 172.21.15.78:0/3376712148 shutdown_connections 2024-09-11T12:57:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 --2- 172.21.15.78:0/3376712148 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3600070360 0x7f3600071d60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 -- 172.21.15.78:0/3376712148 >> 172.21.15.78:0/3376712148 conn(0x7f360006ca80 msgr2=0x7f36000b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.157+0000 7f3605c00640 1 -- 172.21.15.78:0/3376712148 shutdown_connections 2024-09-11T12:57:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 -- 172.21.15.78:0/3376712148 wait complete. 2024-09-11T12:57:37.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 Processor -- start 2024-09-11T12:57:37.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 -- start start 2024-09-11T12:57:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600074240 0x7f3600150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f360015ba80 0x7f36001540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3600168ae0 con 0x7f3600070360 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3600168960 con 0x7f3600074240 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f3605c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3600168c60 con 0x7f360015ba80 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47242/0 (socket says 172.21.15.78:47242) 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 -- 172.21.15.78:0/1083577614 learned_addr learned my addr 172.21.15.78:0/1083577614 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35ff400640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f360015ba80 0x7f36001540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fe000640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600074240 0x7f3600150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 -- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f360015ba80 msgr2=0x7f36001540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f360015ba80 0x7f36001540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 -- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600074240 msgr2=0x7f3600150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600074240 0x7f3600150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 -- 172.21.15.78:0/1083577614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3600151650 con 0x7f3600070360 2024-09-11T12:57:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35fea00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c700 tx=0x7f35f000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.161+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0005270 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35f0005410 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3600151930 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0003840 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3600150230 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3600071d60 con 0x7f3600070360 2024-09-11T12:57:37.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f35f00039e0 con 0x7f3600070360 2024-09-11T12:57:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35f6a00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 0x7f35e0074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f35f0090160 con 0x7f3600070360 2024-09-11T12:57:37.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35fe000640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 0x7f35e0074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.165+0000 7f35fe000640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 0x7f35e0074500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3600166550 tx=0x7f35ec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.169+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f35f005d500 con 0x7f3600070360 2024-09-11T12:57:37.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.333+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3600151e50 con 0x7f35e0072040 2024-09-11T12:57:37.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.337+0000 7f35f6a00640 1 -- 172.21.15.78:0/1083577614 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f3600151e50 con 0x7f35e0072040 2024-09-11T12:57:37.339 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:37.341 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 msgr2=0x7f35e0074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 0x7f35e0074500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3600166550 tx=0x7f35ec009040 comp rx=0 tx=0).stop 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 msgr2=0x7f360014bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c700 tx=0x7f35f000cbc0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 shutdown_connections 2024-09-11T12:57:37.342 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f360015ba80 0x7f36001540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f35e0072040 0x7f35e0074500 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3600074240 0x7f3600150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 --2- 172.21.15.78:0/1083577614 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3600070360 0x7f360014bbb0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 >> 172.21.15.78:0/1083577614 conn(0x7f360006ca80 msgr2=0x7f360014f280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 shutdown_connections 2024-09-11T12:57:37.343 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.341+0000 7f3605c00640 1 -- 172.21.15.78:0/1083577614 wait complete. 2024-09-11T12:57:37.360 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T12:57:36.518587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287496,"kb_used_data":17160,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687224,"statfs":{"total":773094113280,"available":772799717376,"internally_reserved":0,"allocated":17571840,"data_stored":8359708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.678984"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019008+0000","last_change":"2024-09-11T12:57:32.889914+0000","last_active":"2024-09-11T12:57:34.019008+0000","last_peered":"2024-09-11T12:57:34.019008+0000","last_clean":"2024-09-11T12:57:34.019008+0000","last_became_active":"2024-09-11T12:57:32.889275+0000","last_became_peered":"2024-09-11T12:57:32.889275+0000","last_unstale":"2024-09-11T12:57:34.019008+0000","last_undegraded":"2024-09-11T12:57:34.019008+0000","last_fullsized":"2024-09-11T12:57:34.019008+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:18.112007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868810+0000","last_change":"2024-09-11T12:57:32.875358+0000","last_active":"2024-09-11T12:57:33.868810+0000","last_peered":"2024-09-11T12:57:33.868810+0000","last_clean":"2024-09-11T12:57:33.868810+0000","last_became_active":"2024-09-11T12:57:32.875087+0000","last_became_peered":"2024-09-11T12:57:32.875087+0000","last_unstale":"2024-09-11T12:57:33.868810+0000","last_undegraded":"2024-09-11T12:57:33.868810+0000","last_fullsized":"2024-09-11T12:57:33.868810+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:05:27.222559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018203+0000","last_change":"2024-09-11T12:57:32.891769+0000","last_active":"2024-09-11T12:57:34.018203+0000","last_peered":"2024-09-11T12:57:34.018203+0000","last_clean":"2024-09-11T12:57:34.018203+0000","last_became_active":"2024-09-11T12:57:32.891473+0000","last_became_peered":"2024-09-11T12:57:32.891473+0000","last_unstale":"2024-09-11T12:57:34.018203+0000","last_undegraded":"2024-09-11T12:57:34.018203+0000","last_fullsized":"2024-09-11T12:57:34.018203+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:49:06.231148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892480+0000","last_change":"2024-09-11T12:57:32.885624+0000","last_active":"2024-09-11T12:57:33.892480+0000","last_peered":"2024-09-11T12:57:33.892480+0000","last_clean":"2024-09-11T12:57:33.892480+0000","last_became_active":"2024-09-11T12:57:32.885310+0000","last_became_peered":"2024-09-11T12:57:32.885310+0000","last_unstale":"2024-09-11T12:57:33.892480+0000","last_undegraded":"2024-09-11T12:57:33.892480+0000","last_fullsized":"2024-09-11T12:57:33.892480+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:47:38.893184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017760+0000","last_change":"2024-09-11T12:57:32.879696+0000","last_active":"2024-09-11T12:57:34.017760+0000","last_peered":"2024-09-11T12:57:34.017760+0000","last_clean":"2024-09-11T12:57:34.017760+0000","last_became_active":"2024-09-11T12:57:32.879423+0000","last_became_peered":"2024-09-11T12:57:32.879423+0000","last_unstale":"2024-09-11T12:57:34.017760+0000","last_undegraded":"2024-09-11T12:57:34.017760+0000","last_fullsized":"2024-09-11T12:57:34.017760+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:58.339628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017538+0000","last_change":"2024-09-11T12:57:32.888460+0000","last_active":"2024-09-11T12:57:34.017538+0000","last_peered":"2024-09-11T12:57:34.017538+0000","last_clean":"2024-09-11T12:57:34.017538+0000","last_became_active":"2024-09-11T12:57:32.887843+0000","last_became_peered":"2024-09-11T12:57:32.887843+0000","last_unstale":"2024-09-11T12:57:34.017538+0000","last_undegraded":"2024-09-11T12:57:34.017538+0000","last_fullsized":"2024-09-11T12:57:34.017538+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:20:35.486224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017594+0000","last_change":"2024-09-11T12:57:32.870308+0000","last_active":"2024-09-11T12:57:34.017594+0000","last_peered":"2024-09-11T12:57:34.017594+0000","last_clean":"2024-09-11T12:57:34.017594+0000","last_became_active":"2024-09-11T12:57:32.869987+0000","last_became_peered":"2024-09-11T12:57:32.869987+0000","last_unstale":"2024-09-11T12:57:34.017594+0000","last_undegraded":"2024-09-11T12:57:34.017594+0000","last_fullsized":"2024-09-11T12:57:34.017594+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:34:56.322193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017230+0000","last_change":"2024-09-11T12:57:32.879231+0000","last_active":"2024-09-11T12:57:34.017230+0000","last_peered":"2024-09-11T12:57:34.017230+0000","last_clean":"2024-09-11T12:57:34.017230+0000","last_became_active":"2024-09-11T12:57:32.878661+0000","last_became_peered":"2024-09-11T12:57:32.878661+0000","last_unstale":"2024-09-11T12:57:34.017230+0000","last_undegraded":"2024-09-11T12:57:34.017230+0000","last_fullsized":"2024-09-11T12:57:34.017230+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:43.164510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869043+0000","last_change":"2024-09-11T12:57:32.892255+0000","last_active":"2024-09-11T12:57:33.869043+0000","last_peered":"2024-09-11T12:57:33.869043+0000","last_clean":"2024-09-11T12:57:33.869043+0000","last_became_active":"2024-09-11T12:57:32.891982+0000","last_became_peered":"2024-09-11T12:57:32.891982+0000","last_unstale":"2024-09-11T12:57:33.869043+0000","last_undegraded":"2024-09-11T12:57:33.869043+0000","last_fullsized":"2024-09-11T12:57:33.869043+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:30:03.382048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865309+0000","last_change":"2024-09-11T12:57:32.889197+0000","last_active":"2024-09-11T12:57:33.865309+0000","last_peered":"2024-09-11T12:57:33.865309+0000","last_clean":"2024-09-11T12:57:33.865309+0000","last_became_active":"2024-09-11T12:57:32.888905+0000","last_became_peered":"2024-09-11T12:57:32.888905+0000","last_unstale":"2024-09-11T12:57:33.865309+0000","last_undegraded":"2024-09-11T12:57:33.865309+0000","last_fullsized":"2024-09-11T12:57:33.865309+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:56.172538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018214+0000","last_change":"2024-09-11T12:57:32.881863+0000","last_active":"2024-09-11T12:57:34.018214+0000","last_peered":"2024-09-11T12:57:34.018214+0000","last_clean":"2024-09-11T12:57:34.018214+0000","last_became_active":"2024-09-11T12:57:32.881567+0000","last_became_peered":"2024-09-11T12:57:32.881567+0000","last_unstale":"2024-09-11T12:57:34.018214+0000","last_undegraded":"2024-09-11T12:57:34.018214+0000","last_fullsized":"2024-09-11T12:57:34.018214+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:10:55.080391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895227+0000","last_change":"2024-09-11T12:57:32.881782+0000","last_active":"2024-09-11T12:57:33.895227+0000","last_peered":"2024-09-11T12:57:33.895227+0000","last_clean":"2024-09-11T12:57:33.895227+0000","last_became_active":"2024-09-11T12:57:32.881485+0000","last_became_peered":"2024-09-11T12:57:32.881485+0000","last_unstale":"2024-09-11T12:57:33.895227+0000","last_undegraded":"2024-09-11T12:57:33.895227+0000","last_fullsized":"2024-09-11T12:57:33.895227+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:22.179372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895272+0000","last_change":"2024-09-11T12:57:32.880158+0000","last_active":"2024-09-11T12:57:33.895272+0000","last_peered":"2024-09-11T12:57:33.895272+0000","last_clean":"2024-09-11T12:57:33.895272+0000","last_became_active":"2024-09-11T12:57:32.879440+0000","last_became_peered":"2024-09-11T12:57:32.879440+0000","last_unstale":"2024-09-11T12:57:33.895272+0000","last_undegraded":"2024-09-11T12:57:33.895272+0000","last_fullsized":"2024-09-11T12:57:33.895272+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:23.453105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018055+0000","last_change":"2024-09-11T12:57:32.885869+0000","last_active":"2024-09-11T12:57:34.018055+0000","last_peered":"2024-09-11T12:57:34.018055+0000","last_clean":"2024-09-11T12:57:34.018055+0000","last_became_active":"2024-09-11T12:57:32.885578+0000","last_became_peered":"2024-09-11T12:57:32.885578+0000","last_unstale":"2024-09-11T12:57:34.018055+0000","last_undegraded":"2024-09-11T12:57:34.018055+0000","last_fullsized":"2024-09-11T12:57:34.018055+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:43:50.295398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017897+0000","last_change":"2024-09-11T12:57:32.882128+0000","last_active":"2024-09-11T12:57:34.017897+0000","last_peered":"2024-09-11T12:57:34.017897+0000","last_clean":"2024-09-11T12:57:34.017897+0000","last_became_active":"2024-09-11T12:57:32.881828+0000","last_became_peered":"2024-09-11T12:57:32.881828+0000","last_unstale":"2024-09-11T12:57:34.017897+0000","last_undegraded":"2024-09-11T12:57:34.017897+0000","last_fullsized":"2024-09-11T12:57:34.017897+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:31:35.838657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864973+0000","last_change":"2024-09-11T12:57:32.891797+0000","last_active":"2024-09-11T12:57:33.864973+0000","last_peered":"2024-09-11T12:57:33.864973+0000","last_clean":"2024-09-11T12:57:33.864973+0000","last_became_active":"2024-09-11T12:57:32.886475+0000","last_became_peered":"2024-09-11T12:57:32.886475+0000","last_unstale":"2024-09-11T12:57:33.864973+0000","last_undegraded":"2024-09-11T12:57:33.864973+0000","last_fullsized":"2024-09-11T12:57:33.864973+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:20.643732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019337+0000","last_change":"2024-09-11T12:57:32.890144+0000","last_active":"2024-09-11T12:57:34.019337+0000","last_peered":"2024-09-11T12:57:34.019337+0000","last_clean":"2024-09-11T12:57:34.019337+0000","last_became_active":"2024-09-11T12:57:32.889648+0000","last_became_peered":"2024-09-11T12:57:32.889648+0000","last_unstale":"2024-09-11T12:57:34.019337+0000","last_undegraded":"2024-09-11T12:57:34.019337+0000","last_fullsized":"2024-09-11T12:57:34.019337+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:09:46.344170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865469+0000","last_change":"2024-09-11T12:57:32.891604+0000","last_active":"2024-09-11T12:57:33.865469+0000","last_peered":"2024-09-11T12:57:33.865469+0000","last_clean":"2024-09-11T12:57:33.865469+0000","last_became_active":"2024-09-11T12:57:32.891355+0000","last_became_peered":"2024-09-11T12:57:32.891355+0000","last_unstale":"2024-09-11T12:57:33.865469+0000","last_undegraded":"2024-09-11T12:57:33.865469+0000","last_fullsized":"2024-09-11T12:57:33.865469+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:47:38.893184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865488+0000","last_change":"2024-09-11T12:57:32.889683+0000","last_active":"2024-09-11T12:57:33.865488+0000","last_peered":"2024-09-11T12:57:33.865488+0000","last_clean":"2024-09-11T12:57:33.865488+0000","last_became_active":"2024-09-11T12:57:32.889031+0000","last_became_peered":"2024-09-11T12:57:32.889031+0000","last_unstale":"2024-09-11T12:57:33.865488+0000","last_undegraded":"2024-09-11T12:57:33.865488+0000","last_fullsized":"2024-09-11T12:57:33.865488+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:56.776240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017827+0000","last_change":"2024-09-11T12:57:32.880402+0000","last_active":"2024-09-11T12:57:34.017827+0000","last_peered":"2024-09-11T12:57:34.017827+0000","last_clean":"2024-09-11T12:57:34.017827+0000","last_became_active":"2024-09-11T12:57:32.880076+0000","last_became_peered":"2024-09-11T12:57:32.880076+0000","last_unstale":"2024-09-11T12:57:34.017827+0000","last_undegraded":"2024-09-11T12:57:34.017827+0000","last_fullsized":"2024-09-11T12:57:34.017827+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:48:36.847587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892383+0000","last_change":"2024-09-11T12:57:32.891776+0000","last_active":"2024-09-11T12:57:33.892383+0000","last_peered":"2024-09-11T12:57:33.892383+0000","last_clean":"2024-09-11T12:57:33.892383+0000","last_became_active":"2024-09-11T12:57:32.891505+0000","last_became_peered":"2024-09-11T12:57:32.891505+0000","last_unstale":"2024-09-11T12:57:33.892383+0000","last_undegraded":"2024-09-11T12:57:33.892383+0000","last_fullsized":"2024-09-11T12:57:33.892383+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:00:31.139145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867741+0000","last_change":"2024-09-11T12:57:32.881143+0000","last_active":"2024-09-11T12:57:33.867741+0000","last_peered":"2024-09-11T12:57:33.867741+0000","last_clean":"2024-09-11T12:57:33.867741+0000","last_became_active":"2024-09-11T12:57:32.879722+0000","last_became_peered":"2024-09-11T12:57:32.879722+0000","last_unstale":"2024-09-11T12:57:33.867741+0000","last_undegraded":"2024-09-11T12:57:33.867741+0000","last_fullsized":"2024-09-11T12:57:33.867741+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:07.853228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867957+0000","last_change":"2024-09-11T12:57:32.871767+0000","last_active":"2024-09-11T12:57:33.867957+0000","last_peered":"2024-09-11T12:57:33.867957+0000","last_clean":"2024-09-11T12:57:33.867957+0000","last_became_active":"2024-09-11T12:57:32.871346+0000","last_became_peered":"2024-09-11T12:57:32.871346+0000","last_unstale":"2024-09-11T12:57:33.867957+0000","last_undegraded":"2024-09-11T12:57:33.867957+0000","last_fullsized":"2024-09-11T12:57:33.867957+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:54.862519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892401+0000","last_change":"2024-09-11T12:57:32.895534+0000","last_active":"2024-09-11T12:57:33.892401+0000","last_peered":"2024-09-11T12:57:33.892401+0000","last_clean":"2024-09-11T12:57:33.892401+0000","last_became_active":"2024-09-11T12:57:32.895226+0000","last_became_peered":"2024-09-11T12:57:32.895226+0000","last_unstale":"2024-09-11T12:57:33.892401+0000","last_undegraded":"2024-09-11T12:57:33.892401+0000","last_fullsized":"2024-09-11T12:57:33.892401+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:46:56.172538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868883+0000","last_change":"2024-09-11T12:57:32.881232+0000","last_active":"2024-09-11T12:57:33.868883+0000","last_peered":"2024-09-11T12:57:33.868883+0000","last_clean":"2024-09-11T12:57:33.868883+0000","last_became_active":"2024-09-11T12:57:32.880585+0000","last_became_peered":"2024-09-11T12:57:32.880585+0000","last_unstale":"2024-09-11T12:57:33.868883+0000","last_undegraded":"2024-09-11T12:57:33.868883+0000","last_fullsized":"2024-09-11T12:57:33.868883+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:16:32.907631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018983+0000","last_change":"2024-09-11T12:57:32.881047+0000","last_active":"2024-09-11T12:57:34.018983+0000","last_peered":"2024-09-11T12:57:34.018983+0000","last_clean":"2024-09-11T12:57:34.018983+0000","last_became_active":"2024-09-11T12:57:32.877623+0000","last_became_peered":"2024-09-11T12:57:32.877623+0000","last_unstale":"2024-09-11T12:57:34.018983+0000","last_undegraded":"2024-09-11T12:57:34.018983+0000","last_fullsized":"2024-09-11T12:57:34.018983+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:46:21.507100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895278+0000","last_change":"2024-09-11T12:57:32.881829+0000","last_active":"2024-09-11T12:57:33.895278+0000","last_peered":"2024-09-11T12:57:33.895278+0000","last_clean":"2024-09-11T12:57:33.895278+0000","last_became_active":"2024-09-11T12:57:32.881523+0000","last_became_peered":"2024-09-11T12:57:32.881523+0000","last_unstale":"2024-09-11T12:57:33.895278+0000","last_undegraded":"2024-09-11T12:57:33.895278+0000","last_fullsized":"2024-09-11T12:57:33.895278+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:56.281712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017907+0000","last_change":"2024-09-11T12:57:32.883199+0000","last_active":"2024-09-11T12:57:34.017907+0000","last_peered":"2024-09-11T12:57:34.017907+0000","last_clean":"2024-09-11T12:57:34.017907+0000","last_became_active":"2024-09-11T12:57:32.882937+0000","last_became_peered":"2024-09-11T12:57:32.882937+0000","last_unstale":"2024-09-11T12:57:34.017907+0000","last_undegraded":"2024-09-11T12:57:34.017907+0000","last_fullsized":"2024-09-11T12:57:34.017907+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:59:37.794812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895059+0000","last_change":"2024-09-11T12:57:32.889094+0000","last_active":"2024-09-11T12:57:33.895059+0000","last_peered":"2024-09-11T12:57:33.895059+0000","last_clean":"2024-09-11T12:57:33.895059+0000","last_became_active":"2024-09-11T12:57:32.888755+0000","last_became_peered":"2024-09-11T12:57:32.888755+0000","last_unstale":"2024-09-11T12:57:33.895059+0000","last_undegraded":"2024-09-11T12:57:33.895059+0000","last_fullsized":"2024-09-11T12:57:33.895059+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:27.156526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867826+0000","last_change":"2024-09-11T12:57:32.880207+0000","last_active":"2024-09-11T12:57:33.867826+0000","last_peered":"2024-09-11T12:57:33.867826+0000","last_clean":"2024-09-11T12:57:33.867826+0000","last_became_active":"2024-09-11T12:57:32.879912+0000","last_became_peered":"2024-09-11T12:57:32.879912+0000","last_unstale":"2024-09-11T12:57:33.867826+0000","last_undegraded":"2024-09-11T12:57:33.867826+0000","last_fullsized":"2024-09-11T12:57:33.867826+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:06:11.015832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864975+0000","last_change":"2024-09-11T12:57:32.890567+0000","last_active":"2024-09-11T12:57:33.864975+0000","last_peered":"2024-09-11T12:57:33.864975+0000","last_clean":"2024-09-11T12:57:33.864975+0000","last_became_active":"2024-09-11T12:57:32.890295+0000","last_became_peered":"2024-09-11T12:57:32.890295+0000","last_unstale":"2024-09-11T12:57:33.864975+0000","last_undegraded":"2024-09-11T12:57:33.864975+0000","last_fullsized":"2024-09-11T12:57:33.864975+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:36:55.359021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017629+0000","last_change":"2024-09-11T12:57:32.880259+0000","last_active":"2024-09-11T12:57:34.017629+0000","last_peered":"2024-09-11T12:57:34.017629+0000","last_clean":"2024-09-11T12:57:34.017629+0000","last_became_active":"2024-09-11T12:57:32.879843+0000","last_became_peered":"2024-09-11T12:57:32.879843+0000","last_unstale":"2024-09-11T12:57:34.017629+0000","last_undegraded":"2024-09-11T12:57:34.017629+0000","last_fullsized":"2024-09-11T12:57:34.017629+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:48:12.272662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868882+0000","last_change":"2024-09-11T12:57:32.887467+0000","last_active":"2024-09-11T12:57:33.868882+0000","last_peered":"2024-09-11T12:57:33.868882+0000","last_clean":"2024-09-11T12:57:33.868882+0000","last_became_active":"2024-09-11T12:57:32.887165+0000","last_became_peered":"2024-09-11T12:57:32.887165+0000","last_unstale":"2024-09-11T12:57:33.868882+0000","last_undegraded":"2024-09-11T12:57:33.868882+0000","last_fullsized":"2024-09-11T12:57:33.868882+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:36:47.672752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"161'4","reported_seq":19,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895583+0000","last_change":"2024-09-11T12:57:31.864514+0000","last_active":"2024-09-11T12:57:33.895583+0000","last_peered":"2024-09-11T12:57:33.895583+0000","last_clean":"2024-09-11T12:57:33.895583+0000","last_became_active":"2024-09-11T12:57:31.861904+0000","last_became_peered":"2024-09-11T12:57:31.861904+0000","last_unstale":"2024-09-11T12:57:33.895583+0000","last_undegraded":"2024-09-11T12:57:33.895583+0000","last_fullsized":"2024-09-11T12:57:33.895583+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:15.260922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868699+0000","last_change":"2024-09-11T12:57:32.884362+0000","last_active":"2024-09-11T12:57:33.868699+0000","last_peered":"2024-09-11T12:57:33.868699+0000","last_clean":"2024-09-11T12:57:33.868699+0000","last_became_active":"2024-09-11T12:57:32.884021+0000","last_became_peered":"2024-09-11T12:57:32.884021+0000","last_unstale":"2024-09-11T12:57:33.868699+0000","last_undegraded":"2024-09-11T12:57:33.868699+0000","last_fullsized":"2024-09-11T12:57:33.868699+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:22.179372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868997+0000","last_change":"2024-09-11T12:57:31.861806+0000","last_active":"2024-09-11T12:57:33.868997+0000","last_peered":"2024-09-11T12:57:33.868997+0000","last_clean":"2024-09-11T12:57:33.868997+0000","last_became_active":"2024-09-11T12:57:31.861491+0000","last_became_peered":"2024-09-11T12:57:31.861491+0000","last_unstale":"2024-09-11T12:57:33.868997+0000","last_undegraded":"2024-09-11T12:57:33.868997+0000","last_fullsized":"2024-09-11T12:57:33.868997+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:58:16.158063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018506+0000","last_change":"2024-09-11T12:57:32.879520+0000","last_active":"2024-09-11T12:57:34.018506+0000","last_peered":"2024-09-11T12:57:34.018506+0000","last_clean":"2024-09-11T12:57:34.018506+0000","last_became_active":"2024-09-11T12:57:32.878396+0000","last_became_peered":"2024-09-11T12:57:32.878396+0000","last_unstale":"2024-09-11T12:57:34.018506+0000","last_undegraded":"2024-09-11T12:57:34.018506+0000","last_fullsized":"2024-09-11T12:57:34.018506+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:41:12.005213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018287+0000","last_change":"2024-09-11T12:57:31.865369+0000","last_active":"2024-09-11T12:57:34.018287+0000","last_peered":"2024-09-11T12:57:34.018287+0000","last_clean":"2024-09-11T12:57:34.018287+0000","last_became_active":"2024-09-11T12:57:31.865083+0000","last_became_peered":"2024-09-11T12:57:31.865083+0000","last_unstale":"2024-09-11T12:57:34.018287+0000","last_undegraded":"2024-09-11T12:57:34.018287+0000","last_fullsized":"2024-09-11T12:57:34.018287+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:10.063118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865284+0000","last_change":"2024-09-11T12:57:32.892581+0000","last_active":"2024-09-11T12:57:33.865284+0000","last_peered":"2024-09-11T12:57:33.865284+0000","last_clean":"2024-09-11T12:57:33.865284+0000","last_became_active":"2024-09-11T12:57:32.892322+0000","last_became_peered":"2024-09-11T12:57:32.892322+0000","last_unstale":"2024-09-11T12:57:33.865284+0000","last_undegraded":"2024-09-11T12:57:33.865284+0000","last_fullsized":"2024-09-11T12:57:33.865284+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:00:31.139145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892593+0000","last_change":"2024-09-11T12:57:31.871779+0000","last_active":"2024-09-11T12:57:33.892593+0000","last_peered":"2024-09-11T12:57:33.892593+0000","last_clean":"2024-09-11T12:57:33.892593+0000","last_became_active":"2024-09-11T12:57:31.871344+0000","last_became_peered":"2024-09-11T12:57:31.871344+0000","last_unstale":"2024-09-11T12:57:33.892593+0000","last_undegraded":"2024-09-11T12:57:33.892593+0000","last_fullsized":"2024-09-11T12:57:33.892593+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:01:42.155435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892379+0000","last_change":"2024-09-11T12:57:32.882843+0000","last_active":"2024-09-11T12:57:33.892379+0000","last_peered":"2024-09-11T12:57:33.892379+0000","last_clean":"2024-09-11T12:57:33.892379+0000","last_became_active":"2024-09-11T12:57:32.882566+0000","last_became_peered":"2024-09-11T12:57:32.882566+0000","last_unstale":"2024-09-11T12:57:33.892379+0000","last_undegraded":"2024-09-11T12:57:33.892379+0000","last_fullsized":"2024-09-11T12:57:33.892379+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:44:07.586644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1a","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017870+0000","last_change":"2024-09-11T12:57:31.869332+0000","last_active":"2024-09-11T12:57:34.017870+0000","last_peered":"2024-09-11T12:57:34.017870+0000","last_clean":"2024-09-11T12:57:34.017870+0000","last_became_active":"2024-09-11T12:57:31.869052+0000","last_became_peered":"2024-09-11T12:57:31.869052+0000","last_unstale":"2024-09-11T12:57:34.017870+0000","last_undegraded":"2024-09-11T12:57:34.017870+0000","last_fullsized":"2024-09-11T12:57:34.017870+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:21.226658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017278+0000","last_change":"2024-09-11T12:57:32.889980+0000","last_active":"2024-09-11T12:57:34.017278+0000","last_peered":"2024-09-11T12:57:34.017278+0000","last_clean":"2024-09-11T12:57:34.017278+0000","last_became_active":"2024-09-11T12:57:32.889676+0000","last_became_peered":"2024-09-11T12:57:32.889676+0000","last_unstale":"2024-09-11T12:57:34.017278+0000","last_undegraded":"2024-09-11T12:57:34.017278+0000","last_fullsized":"2024-09-11T12:57:34.017278+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:24.561952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017603+0000","last_change":"2024-09-11T12:57:31.868458+0000","last_active":"2024-09-11T12:57:34.017603+0000","last_peered":"2024-09-11T12:57:34.017603+0000","last_clean":"2024-09-11T12:57:34.017603+0000","last_became_active":"2024-09-11T12:57:31.868173+0000","last_became_peered":"2024-09-11T12:57:31.868173+0000","last_unstale":"2024-09-11T12:57:34.017603+0000","last_undegraded":"2024-09-11T12:57:34.017603+0000","last_fullsized":"2024-09-11T12:57:34.017603+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:36:06.844153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892380+0000","last_change":"2024-09-11T12:57:32.888265+0000","last_active":"2024-09-11T12:57:33.892380+0000","last_peered":"2024-09-11T12:57:33.892380+0000","last_clean":"2024-09-11T12:57:33.892380+0000","last_became_active":"2024-09-11T12:57:32.887920+0000","last_became_peered":"2024-09-11T12:57:32.887920+0000","last_unstale":"2024-09-11T12:57:33.892380+0000","last_undegraded":"2024-09-11T12:57:33.892380+0000","last_fullsized":"2024-09-11T12:57:33.892380+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.461881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867280+0000","last_change":"2024-09-11T12:57:31.852792+0000","last_active":"2024-09-11T12:57:33.867280+0000","last_peered":"2024-09-11T12:57:33.867280+0000","last_clean":"2024-09-11T12:57:33.867280+0000","last_became_active":"2024-09-11T12:57:31.852499+0000","last_became_peered":"2024-09-11T12:57:31.852499+0000","last_unstale":"2024-09-11T12:57:33.867280+0000","last_undegraded":"2024-09-11T12:57:33.867280+0000","last_fullsized":"2024-09-11T12:57:33.867280+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:00:19.415493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892641+0000","last_change":"2024-09-11T12:57:32.891524+0000","last_active":"2024-09-11T12:57:33.892641+0000","last_peered":"2024-09-11T12:57:33.892641+0000","last_clean":"2024-09-11T12:57:33.892641+0000","last_became_active":"2024-09-11T12:57:32.891188+0000","last_became_peered":"2024-09-11T12:57:32.891188+0000","last_unstale":"2024-09-11T12:57:33.892641+0000","last_undegraded":"2024-09-11T12:57:33.892641+0000","last_fullsized":"2024-09-11T12:57:33.892641+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:53:05.458153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018520+0000","last_change":"2024-09-11T12:57:31.863925+0000","last_active":"2024-09-11T12:57:34.018520+0000","last_peered":"2024-09-11T12:57:34.018520+0000","last_clean":"2024-09-11T12:57:34.018520+0000","last_became_active":"2024-09-11T12:57:31.863660+0000","last_became_peered":"2024-09-11T12:57:31.863660+0000","last_unstale":"2024-09-11T12:57:34.018520+0000","last_undegraded":"2024-09-11T12:57:34.018520+0000","last_fullsized":"2024-09-11T12:57:34.018520+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:26.147451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018920+0000","last_change":"2024-09-11T12:57:32.884255+0000","last_active":"2024-09-11T12:57:34.018920+0000","last_peered":"2024-09-11T12:57:34.018920+0000","last_clean":"2024-09-11T12:57:34.018920+0000","last_became_active":"2024-09-11T12:57:32.883980+0000","last_became_peered":"2024-09-11T12:57:32.883980+0000","last_unstale":"2024-09-11T12:57:34.018920+0000","last_undegraded":"2024-09-11T12:57:34.018920+0000","last_fullsized":"2024-09-11T12:57:34.018920+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:26:18.792605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868435+0000","last_change":"2024-09-11T12:57:31.857213+0000","last_active":"2024-09-11T12:57:33.868435+0000","last_peered":"2024-09-11T12:57:33.868435+0000","last_clean":"2024-09-11T12:57:33.868435+0000","last_became_active":"2024-09-11T12:57:31.856897+0000","last_became_peered":"2024-09-11T12:57:31.856897+0000","last_unstale":"2024-09-11T12:57:33.868435+0000","last_undegraded":"2024-09-11T12:57:33.868435+0000","last_fullsized":"2024-09-11T12:57:33.868435+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:43:17.772531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892610+0000","last_change":"2024-09-11T12:57:32.891115+0000","last_active":"2024-09-11T12:57:33.892610+0000","last_peered":"2024-09-11T12:57:33.892610+0000","last_clean":"2024-09-11T12:57:33.892610+0000","last_became_active":"2024-09-11T12:57:32.890723+0000","last_became_peered":"2024-09-11T12:57:32.890723+0000","last_unstale":"2024-09-11T12:57:33.892610+0000","last_undegraded":"2024-09-11T12:57:33.892610+0000","last_fullsized":"2024-09-11T12:57:33.892610+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:56.776240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.15","version":"161'3","reported_seq":18,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865425+0000","last_change":"2024-09-11T12:57:31.870529+0000","last_active":"2024-09-11T12:57:33.865425+0000","last_peered":"2024-09-11T12:57:33.865425+0000","last_clean":"2024-09-11T12:57:33.865425+0000","last_became_active":"2024-09-11T12:57:31.864182+0000","last_became_peered":"2024-09-11T12:57:31.864182+0000","last_unstale":"2024-09-11T12:57:33.865425+0000","last_undegraded":"2024-09-11T12:57:33.865425+0000","last_fullsized":"2024-09-11T12:57:33.865425+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:50.098425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019099+0000","last_change":"2024-09-11T12:57:32.887802+0000","last_active":"2024-09-11T12:57:34.019099+0000","last_peered":"2024-09-11T12:57:34.019099+0000","last_clean":"2024-09-11T12:57:34.019099+0000","last_became_active":"2024-09-11T12:57:32.887521+0000","last_became_peered":"2024-09-11T12:57:32.887521+0000","last_unstale":"2024-09-11T12:57:34.019099+0000","last_undegraded":"2024-09-11T12:57:34.019099+0000","last_fullsized":"2024-09-11T12:57:34.019099+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:24:14.859769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.14","version":"161'6","reported_seq":21,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892688+0000","last_change":"2024-09-11T12:57:31.865574+0000","last_active":"2024-09-11T12:57:33.892688+0000","last_peered":"2024-09-11T12:57:33.892688+0000","last_clean":"2024-09-11T12:57:33.892688+0000","last_became_active":"2024-09-11T12:57:31.865300+0000","last_became_peered":"2024-09-11T12:57:31.865300+0000","last_unstale":"2024-09-11T12:57:33.892688+0000","last_undegraded":"2024-09-11T12:57:33.892688+0000","last_fullsized":"2024-09-11T12:57:33.892688+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:10:23.552877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017716+0000","last_change":"2024-09-11T12:57:32.879251+0000","last_active":"2024-09-11T12:57:34.017716+0000","last_peered":"2024-09-11T12:57:34.017716+0000","last_clean":"2024-09-11T12:57:34.017716+0000","last_became_active":"2024-09-11T12:57:32.878954+0000","last_became_peered":"2024-09-11T12:57:32.878954+0000","last_unstale":"2024-09-11T12:57:34.017716+0000","last_undegraded":"2024-09-11T12:57:34.017716+0000","last_fullsized":"2024-09-11T12:57:34.017716+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:34:14.707884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.13","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867773+0000","last_change":"2024-09-11T12:57:31.863811+0000","last_active":"2024-09-11T12:57:33.867773+0000","last_peered":"2024-09-11T12:57:33.867773+0000","last_clean":"2024-09-11T12:57:33.867773+0000","last_became_active":"2024-09-11T12:57:31.863559+0000","last_became_peered":"2024-09-11T12:57:31.863559+0000","last_unstale":"2024-09-11T12:57:33.867773+0000","last_undegraded":"2024-09-11T12:57:33.867773+0000","last_fullsized":"2024-09-11T12:57:33.867773+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:32:29.932458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865257+0000","last_change":"2024-09-11T12:57:32.878633+0000","last_active":"2024-09-11T12:57:33.865257+0000","last_peered":"2024-09-11T12:57:33.865257+0000","last_clean":"2024-09-11T12:57:33.865257+0000","last_became_active":"2024-09-11T12:57:32.878369+0000","last_became_peered":"2024-09-11T12:57:32.878369+0000","last_unstale":"2024-09-11T12:57:33.865257+0000","last_undegraded":"2024-09-11T12:57:33.865257+0000","last_fullsized":"2024-09-11T12:57:33.865257+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:44:07.586644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868035+0000","last_change":"2024-09-11T12:57:31.850870+0000","last_active":"2024-09-11T12:57:33.868035+0000","last_peered":"2024-09-11T12:57:33.868035+0000","last_clean":"2024-09-11T12:57:33.868035+0000","last_became_active":"2024-09-11T12:57:31.850421+0000","last_became_peered":"2024-09-11T12:57:31.850421+0000","last_unstale":"2024-09-11T12:57:33.868035+0000","last_undegraded":"2024-09-11T12:57:33.868035+0000","last_fullsized":"2024-09-11T12:57:33.868035+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:33:15.260922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018229+0000","last_change":"2024-09-11T12:57:32.884877+0000","last_active":"2024-09-11T12:57:34.018229+0000","last_peered":"2024-09-11T12:57:34.018229+0000","last_clean":"2024-09-11T12:57:34.018229+0000","last_became_active":"2024-09-11T12:57:32.884617+0000","last_became_peered":"2024-09-11T12:57:32.884617+0000","last_unstale":"2024-09-11T12:57:34.018229+0000","last_undegraded":"2024-09-11T12:57:34.018229+0000","last_fullsized":"2024-09-11T12:57:34.018229+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:51:04.146114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.864884+0000","last_change":"2024-09-11T12:57:31.858024+0000","last_active":"2024-09-11T12:57:33.864884+0000","last_peered":"2024-09-11T12:57:33.864884+0000","last_clean":"2024-09-11T12:57:33.864884+0000","last_became_active":"2024-09-11T12:57:31.857769+0000","last_became_peered":"2024-09-11T12:57:31.857769+0000","last_unstale":"2024-09-11T12:57:33.864884+0000","last_undegraded":"2024-09-11T12:57:33.864884+0000","last_fullsized":"2024-09-11T12:57:33.864884+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:20:22.444030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018042+0000","last_change":"2024-09-11T12:57:32.876251+0000","last_active":"2024-09-11T12:57:34.018042+0000","last_peered":"2024-09-11T12:57:34.018042+0000","last_clean":"2024-09-11T12:57:34.018042+0000","last_became_active":"2024-09-11T12:57:32.875799+0000","last_became_peered":"2024-09-11T12:57:32.875799+0000","last_unstale":"2024-09-11T12:57:34.018042+0000","last_undegraded":"2024-09-11T12:57:34.018042+0000","last_fullsized":"2024-09-11T12:57:34.018042+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:41:48.165860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.10","version":"161'8","reported_seq":23,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018457+0000","last_change":"2024-09-11T12:57:31.866327+0000","last_active":"2024-09-11T12:57:34.018457+0000","last_peered":"2024-09-11T12:57:34.018457+0000","last_clean":"2024-09-11T12:57:34.018457+0000","last_became_active":"2024-09-11T12:57:31.865182+0000","last_became_peered":"2024-09-11T12:57:31.865182+0000","last_unstale":"2024-09-11T12:57:34.018457+0000","last_undegraded":"2024-09-11T12:57:34.018457+0000","last_fullsized":"2024-09-11T12:57:34.018457+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:02:21.332155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869021+0000","last_change":"2024-09-11T12:57:32.884972+0000","last_active":"2024-09-11T12:57:33.869021+0000","last_peered":"2024-09-11T12:57:33.869021+0000","last_clean":"2024-09-11T12:57:33.869021+0000","last_became_active":"2024-09-11T12:57:32.884694+0000","last_became_peered":"2024-09-11T12:57:32.884694+0000","last_unstale":"2024-09-11T12:57:33.869021+0000","last_undegraded":"2024-09-11T12:57:33.869021+0000","last_fullsized":"2024-09-11T12:57:33.869021+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:22:00.383319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018620+0000","last_change":"2024-09-11T12:57:31.869598+0000","last_active":"2024-09-11T12:57:34.018620+0000","last_peered":"2024-09-11T12:57:34.018620+0000","last_clean":"2024-09-11T12:57:34.018620+0000","last_became_active":"2024-09-11T12:57:31.869347+0000","last_became_peered":"2024-09-11T12:57:31.869347+0000","last_unstale":"2024-09-11T12:57:34.018620+0000","last_undegraded":"2024-09-11T12:57:34.018620+0000","last_fullsized":"2024-09-11T12:57:34.018620+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:49:21.170297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868898+0000","last_change":"2024-09-11T12:57:32.881495+0000","last_active":"2024-09-11T12:57:33.868898+0000","last_peered":"2024-09-11T12:57:33.868898+0000","last_clean":"2024-09-11T12:57:33.868898+0000","last_became_active":"2024-09-11T12:57:32.880925+0000","last_became_peered":"2024-09-11T12:57:32.880925+0000","last_unstale":"2024-09-11T12:57:33.868898+0000","last_undegraded":"2024-09-11T12:57:33.868898+0000","last_fullsized":"2024-09-11T12:57:33.868898+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:01:59.660211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.e","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.869067+0000","last_change":"2024-09-11T12:57:31.854091+0000","last_active":"2024-09-11T12:57:33.869067+0000","last_peered":"2024-09-11T12:57:33.869067+0000","last_clean":"2024-09-11T12:57:33.869067+0000","last_became_active":"2024-09-11T12:57:31.853803+0000","last_became_peered":"2024-09-11T12:57:31.853803+0000","last_unstale":"2024-09-11T12:57:33.869067+0000","last_undegraded":"2024-09-11T12:57:33.869067+0000","last_fullsized":"2024-09-11T12:57:33.869067+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:08:35.843980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018076+0000","last_change":"2024-09-11T12:57:32.888089+0000","last_active":"2024-09-11T12:57:34.018076+0000","last_peered":"2024-09-11T12:57:34.018076+0000","last_clean":"2024-09-11T12:57:34.018076+0000","last_became_active":"2024-09-11T12:57:32.887587+0000","last_became_peered":"2024-09-11T12:57:32.887587+0000","last_unstale":"2024-09-11T12:57:34.018076+0000","last_undegraded":"2024-09-11T12:57:34.018076+0000","last_fullsized":"2024-09-11T12:57:34.018076+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:04.655412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.d","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865532+0000","last_change":"2024-09-11T12:57:31.870185+0000","last_active":"2024-09-11T12:57:33.865532+0000","last_peered":"2024-09-11T12:57:33.865532+0000","last_clean":"2024-09-11T12:57:33.865532+0000","last_became_active":"2024-09-11T12:57:31.864175+0000","last_became_peered":"2024-09-11T12:57:31.864175+0000","last_unstale":"2024-09-11T12:57:33.865532+0000","last_undegraded":"2024-09-11T12:57:33.865532+0000","last_fullsized":"2024-09-11T12:57:33.865532+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:00.347455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865262+0000","last_change":"2024-09-11T12:57:32.891001+0000","last_active":"2024-09-11T12:57:33.865262+0000","last_peered":"2024-09-11T12:57:33.865262+0000","last_clean":"2024-09-11T12:57:33.865262+0000","last_became_active":"2024-09-11T12:57:32.890382+0000","last_became_peered":"2024-09-11T12:57:32.890382+0000","last_unstale":"2024-09-11T12:57:33.865262+0000","last_undegraded":"2024-09-11T12:57:33.865262+0000","last_fullsized":"2024-09-11T12:57:33.865262+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:11:54.461881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018168+0000","last_change":"2024-09-11T12:57:31.872206+0000","last_active":"2024-09-11T12:57:34.018168+0000","last_peered":"2024-09-11T12:57:34.018168+0000","last_clean":"2024-09-11T12:57:34.018168+0000","last_became_active":"2024-09-11T12:57:31.871859+0000","last_became_peered":"2024-09-11T12:57:31.871859+0000","last_unstale":"2024-09-11T12:57:34.018168+0000","last_undegraded":"2024-09-11T12:57:34.018168+0000","last_fullsized":"2024-09-11T12:57:34.018168+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:00:38.784985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865090+0000","last_change":"2024-09-11T12:57:32.882146+0000","last_active":"2024-09-11T12:57:33.865090+0000","last_peered":"2024-09-11T12:57:33.865090+0000","last_clean":"2024-09-11T12:57:33.865090+0000","last_became_active":"2024-09-11T12:57:32.881675+0000","last_became_peered":"2024-09-11T12:57:32.881675+0000","last_unstale":"2024-09-11T12:57:33.865090+0000","last_undegraded":"2024-09-11T12:57:33.865090+0000","last_fullsized":"2024-09-11T12:57:33.865090+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:10:10.516411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.b","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.865288+0000","last_change":"2024-09-11T12:57:31.866404+0000","last_active":"2024-09-11T12:57:33.865288+0000","last_peered":"2024-09-11T12:57:33.865288+0000","last_clean":"2024-09-11T12:57:33.865288+0000","last_became_active":"2024-09-11T12:57:31.865858+0000","last_became_peered":"2024-09-11T12:57:31.865858+0000","last_unstale":"2024-09-11T12:57:33.865288+0000","last_undegraded":"2024-09-11T12:57:33.865288+0000","last_fullsized":"2024-09-11T12:57:33.865288+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:55.272637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018686+0000","last_change":"2024-09-11T12:57:32.886161+0000","last_active":"2024-09-11T12:57:34.018686+0000","last_peered":"2024-09-11T12:57:34.018686+0000","last_clean":"2024-09-11T12:57:34.018686+0000","last_became_active":"2024-09-11T12:57:32.885899+0000","last_became_peered":"2024-09-11T12:57:32.885899+0000","last_unstale":"2024-09-11T12:57:34.018686+0000","last_undegraded":"2024-09-11T12:57:34.018686+0000","last_fullsized":"2024-09-11T12:57:34.018686+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:12:33.922161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018426+0000","last_change":"2024-09-11T12:57:31.870069+0000","last_active":"2024-09-11T12:57:34.018426+0000","last_peered":"2024-09-11T12:57:34.018426+0000","last_clean":"2024-09-11T12:57:34.018426+0000","last_became_active":"2024-09-11T12:57:31.862696+0000","last_became_peered":"2024-09-11T12:57:31.862696+0000","last_unstale":"2024-09-11T12:57:34.018426+0000","last_undegraded":"2024-09-11T12:57:34.018426+0000","last_fullsized":"2024-09-11T12:57:34.018426+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:35:11.423174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867363+0000","last_change":"2024-09-11T12:57:32.877902+0000","last_active":"2024-09-11T12:57:33.867363+0000","last_peered":"2024-09-11T12:57:33.867363+0000","last_clean":"2024-09-11T12:57:33.867363+0000","last_became_active":"2024-09-11T12:57:32.876976+0000","last_became_peered":"2024-09-11T12:57:32.876976+0000","last_unstale":"2024-09-11T12:57:33.867363+0000","last_undegraded":"2024-09-11T12:57:33.867363+0000","last_fullsized":"2024-09-11T12:57:33.867363+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:49:00.963860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018135+0000","last_change":"2024-09-11T12:57:31.854629+0000","last_active":"2024-09-11T12:57:34.018135+0000","last_peered":"2024-09-11T12:57:34.018135+0000","last_clean":"2024-09-11T12:57:34.018135+0000","last_became_active":"2024-09-11T12:57:31.854373+0000","last_became_peered":"2024-09-11T12:57:31.854373+0000","last_unstale":"2024-09-11T12:57:34.018135+0000","last_undegraded":"2024-09-11T12:57:34.018135+0000","last_fullsized":"2024-09-11T12:57:34.018135+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:20:09.010285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867758+0000","last_change":"2024-09-11T12:57:32.884164+0000","last_active":"2024-09-11T12:57:33.867758+0000","last_peered":"2024-09-11T12:57:33.867758+0000","last_clean":"2024-09-11T12:57:33.867758+0000","last_became_active":"2024-09-11T12:57:32.883873+0000","last_became_peered":"2024-09-11T12:57:32.883873+0000","last_unstale":"2024-09-11T12:57:33.867758+0000","last_undegraded":"2024-09-11T12:57:33.867758+0000","last_fullsized":"2024-09-11T12:57:33.867758+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:32:22.235734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017362+0000","last_change":"2024-09-11T12:57:31.870348+0000","last_active":"2024-09-11T12:57:34.017362+0000","last_peered":"2024-09-11T12:57:34.017362+0000","last_clean":"2024-09-11T12:57:34.017362+0000","last_became_active":"2024-09-11T12:57:31.870073+0000","last_became_peered":"2024-09-11T12:57:31.870073+0000","last_unstale":"2024-09-11T12:57:34.017362+0000","last_undegraded":"2024-09-11T12:57:34.017362+0000","last_fullsized":"2024-09-11T12:57:34.017362+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:36:31.622668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895340+0000","last_change":"2024-09-11T12:57:32.886241+0000","last_active":"2024-09-11T12:57:33.895340+0000","last_peered":"2024-09-11T12:57:33.895340+0000","last_clean":"2024-09-11T12:57:33.895340+0000","last_became_active":"2024-09-11T12:57:32.885788+0000","last_became_peered":"2024-09-11T12:57:32.885788+0000","last_unstale":"2024-09-11T12:57:33.895340+0000","last_undegraded":"2024-09-11T12:57:33.895340+0000","last_fullsized":"2024-09-11T12:57:33.895340+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:58:01.356530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.7","version":"161'2","reported_seq":17,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018971+0000","last_change":"2024-09-11T12:57:31.867186+0000","last_active":"2024-09-11T12:57:34.018971+0000","last_peered":"2024-09-11T12:57:34.018971+0000","last_clean":"2024-09-11T12:57:34.018971+0000","last_became_active":"2024-09-11T12:57:31.866919+0000","last_became_peered":"2024-09-11T12:57:31.866919+0000","last_unstale":"2024-09-11T12:57:34.018971+0000","last_undegraded":"2024-09-11T12:57:34.018971+0000","last_fullsized":"2024-09-11T12:57:34.018971+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:01:40.651487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.019161+0000","last_change":"2024-09-11T12:57:32.885645+0000","last_active":"2024-09-11T12:57:34.019161+0000","last_peered":"2024-09-11T12:57:34.019161+0000","last_clean":"2024-09-11T12:57:34.019161+0000","last_became_active":"2024-09-11T12:57:32.885329+0000","last_became_peered":"2024-09-11T12:57:32.885329+0000","last_unstale":"2024-09-11T12:57:34.019161+0000","last_undegraded":"2024-09-11T12:57:34.019161+0000","last_fullsized":"2024-09-11T12:57:34.019161+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T16:21:59.672782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.6","version":"162'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.894303+0000","last_change":"2024-09-11T12:57:31.866622+0000","last_active":"2024-09-11T12:57:33.894303+0000","last_peered":"2024-09-11T12:57:33.894303+0000","last_clean":"2024-09-11T12:57:33.894303+0000","last_became_active":"2024-09-11T12:57:31.866334+0000","last_became_peered":"2024-09-11T12:57:31.866334+0000","last_unstale":"2024-09-11T12:57:33.894303+0000","last_undegraded":"2024-09-11T12:57:33.894303+0000","last_fullsized":"2024-09-11T12:57:33.894303+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T18:56:05.932483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868590+0000","last_change":"2024-09-11T12:57:32.871089+0000","last_active":"2024-09-11T12:57:33.868590+0000","last_peered":"2024-09-11T12:57:33.868590+0000","last_clean":"2024-09-11T12:57:33.868590+0000","last_became_active":"2024-09-11T12:57:32.870622+0000","last_became_peered":"2024-09-11T12:57:32.870622+0000","last_unstale":"2024-09-11T12:57:33.868590+0000","last_undegraded":"2024-09-11T12:57:33.868590+0000","last_fullsized":"2024-09-11T12:57:33.868590+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:25:17.652515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018020+0000","last_change":"2024-09-11T12:57:31.868269+0000","last_active":"2024-09-11T12:57:34.018020+0000","last_peered":"2024-09-11T12:57:34.018020+0000","last_clean":"2024-09-11T12:57:34.018020+0000","last_became_active":"2024-09-11T12:57:31.868006+0000","last_became_peered":"2024-09-11T12:57:31.868006+0000","last_unstale":"2024-09-11T12:57:34.018020+0000","last_undegraded":"2024-09-11T12:57:34.018020+0000","last_fullsized":"2024-09-11T12:57:34.018020+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:14:55.272637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.868233+0000","last_change":"2024-09-11T12:57:32.881512+0000","last_active":"2024-09-11T12:57:33.868233+0000","last_peered":"2024-09-11T12:57:33.868233+0000","last_clean":"2024-09-11T12:57:33.868233+0000","last_became_active":"2024-09-11T12:57:32.879930+0000","last_became_peered":"2024-09-11T12:57:32.879930+0000","last_unstale":"2024-09-11T12:57:33.868233+0000","last_undegraded":"2024-09-11T12:57:33.868233+0000","last_fullsized":"2024-09-11T12:57:33.868233+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T22:23:27.156526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018224+0000","last_change":"2024-09-11T12:57:31.867661+0000","last_active":"2024-09-11T12:57:34.018224+0000","last_peered":"2024-09-11T12:57:34.018224+0000","last_clean":"2024-09-11T12:57:34.018224+0000","last_became_active":"2024-09-11T12:57:31.867397+0000","last_became_peered":"2024-09-11T12:57:31.867397+0000","last_unstale":"2024-09-11T12:57:34.018224+0000","last_undegraded":"2024-09-11T12:57:34.018224+0000","last_fullsized":"2024-09-11T12:57:34.018224+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:38:37.189410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018313+0000","last_change":"2024-09-11T12:57:32.878537+0000","last_active":"2024-09-11T12:57:34.018313+0000","last_peered":"2024-09-11T12:57:34.018313+0000","last_clean":"2024-09-11T12:57:34.018313+0000","last_became_active":"2024-09-11T12:57:32.878256+0000","last_became_peered":"2024-09-11T12:57:32.878256+0000","last_unstale":"2024-09-11T12:57:34.018313+0000","last_undegraded":"2024-09-11T12:57:34.018313+0000","last_fullsized":"2024-09-11T12:57:34.018313+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T14:35:56.025964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892527+0000","last_change":"2024-09-11T12:57:31.874733+0000","last_active":"2024-09-11T12:57:33.892527+0000","last_peered":"2024-09-11T12:57:33.892527+0000","last_clean":"2024-09-11T12:57:33.892527+0000","last_became_active":"2024-09-11T12:57:31.874504+0000","last_became_peered":"2024-09-11T12:57:31.874504+0000","last_unstale":"2024-09-11T12:57:33.892527+0000","last_undegraded":"2024-09-11T12:57:33.892527+0000","last_fullsized":"2024-09-11T12:57:33.892527+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:05:55.731340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895352+0000","last_change":"2024-09-11T12:57:32.885287+0000","last_active":"2024-09-11T12:57:33.895352+0000","last_peered":"2024-09-11T12:57:33.895352+0000","last_clean":"2024-09-11T12:57:33.895352+0000","last_became_active":"2024-09-11T12:57:32.882854+0000","last_became_peered":"2024-09-11T12:57:32.882854+0000","last_unstale":"2024-09-11T12:57:33.895352+0000","last_undegraded":"2024-09-11T12:57:33.895352+0000","last_fullsized":"2024-09-11T12:57:33.895352+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:16:51.107501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.017934+0000","last_change":"2024-09-11T12:57:31.871691+0000","last_active":"2024-09-11T12:57:34.017934+0000","last_peered":"2024-09-11T12:57:34.017934+0000","last_clean":"2024-09-11T12:57:34.017934+0000","last_became_active":"2024-09-11T12:57:31.869375+0000","last_became_peered":"2024-09-11T12:57:31.869375+0000","last_unstale":"2024-09-11T12:57:34.017934+0000","last_undegraded":"2024-09-11T12:57:34.017934+0000","last_fullsized":"2024-09-11T12:57:34.017934+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:37:53.853443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018399+0000","last_change":"2024-09-11T12:57:32.881504+0000","last_active":"2024-09-11T12:57:34.018399+0000","last_peered":"2024-09-11T12:57:34.018399+0000","last_clean":"2024-09-11T12:57:34.018399+0000","last_became_active":"2024-09-11T12:57:32.881210+0000","last_became_peered":"2024-09-11T12:57:32.881210+0000","last_unstale":"2024-09-11T12:57:34.018399+0000","last_undegraded":"2024-09-11T12:57:34.018399+0000","last_fullsized":"2024-09-11T12:57:34.018399+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T20:19:12.845778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867878+0000","last_change":"2024-09-11T12:57:31.855799+0000","last_active":"2024-09-11T12:57:33.867878+0000","last_peered":"2024-09-11T12:57:33.867878+0000","last_clean":"2024-09-11T12:57:33.867878+0000","last_became_active":"2024-09-11T12:57:31.855465+0000","last_became_peered":"2024-09-11T12:57:31.855465+0000","last_unstale":"2024-09-11T12:57:33.867878+0000","last_undegraded":"2024-09-11T12:57:33.867878+0000","last_fullsized":"2024-09-11T12:57:33.867878+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:47:51.452333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.867847+0000","last_change":"2024-09-11T12:57:32.891156+0000","last_active":"2024-09-11T12:57:33.867847+0000","last_peered":"2024-09-11T12:57:33.867847+0000","last_clean":"2024-09-11T12:57:33.867847+0000","last_became_active":"2024-09-11T12:57:32.890728+0000","last_became_peered":"2024-09-11T12:57:32.890728+0000","last_unstale":"2024-09-11T12:57:33.867847+0000","last_undegraded":"2024-09-11T12:57:33.867847+0000","last_fullsized":"2024-09-11T12:57:33.867847+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T21:53:59.493090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.894982+0000","last_change":"2024-09-11T12:57:31.864289+0000","last_active":"2024-09-11T12:57:33.894982+0000","last_peered":"2024-09-11T12:57:33.894982+0000","last_clean":"2024-09-11T12:57:33.894982+0000","last_became_active":"2024-09-11T12:57:31.862678+0000","last_became_peered":"2024-09-11T12:57:31.862678+0000","last_unstale":"2024-09-11T12:57:33.894982+0000","last_undegraded":"2024-09-11T12:57:33.894982+0000","last_fullsized":"2024-09-11T12:57:33.894982+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T17:58:56.466714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:34.018184+0000","last_change":"2024-09-11T12:57:32.890686+0000","last_active":"2024-09-11T12:57:34.018184+0000","last_peered":"2024-09-11T12:57:34.018184+0000","last_clean":"2024-09-11T12:57:34.018184+0000","last_became_active":"2024-09-11T12:57:32.890376+0000","last_became_peered":"2024-09-11T12:57:32.890376+0000","last_unstale":"2024-09-11T12:57:34.018184+0000","last_undegraded":"2024-09-11T12:57:34.018184+0000","last_fullsized":"2024-09-11T12:57:34.018184+0000","mapping_epoch":160,"log_start":"0'0","ondisk_log_start":"0'0","created":160,"last_epoch_clean":161,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:31.838674+0000","last_clean_scrub_stamp":"2024-09-11T12:57:31.838674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T23:27:47.914002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"161'1","reported_seq":16,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.895395+0000","last_change":"2024-09-11T12:57:31.857498+0000","last_active":"2024-09-11T12:57:33.895395+0000","last_peered":"2024-09-11T12:57:33.895395+0000","last_clean":"2024-09-11T12:57:33.895395+0000","last_became_active":"2024-09-11T12:57:31.857220+0000","last_became_peered":"2024-09-11T12:57:31.857220+0000","last_unstale":"2024-09-11T12:57:33.895395+0000","last_undegraded":"2024-09-11T12:57:33.895395+0000","last_fullsized":"2024-09-11T12:57:33.895395+0000","mapping_epoch":159,"log_start":"0'0","ondisk_log_start":"0'0","created":159,"last_epoch_clean":160,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:57:30.829599+0000","last_clean_scrub_stamp":"2024-09-11T12:57:30.829599+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T19:03:40.115214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"72'79","reported_seq":392,"reported_epoch":162,"state":"active+clean","last_fresh":"2024-09-11T12:57:33.892265+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:33.892265+0000","last_peered":"2024-09-11T12:57:33.892265+0000","last_clean":"2024-09-11T12:57:33.892265+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:33.892265+0000","last_undegraded":"2024-09-11T12:57:33.892265+0000","last_fullsized":"2024-09-11T12:57:33.892265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509540,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37932,"kb_used_data":2584,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333908,"statfs":{"total":96636764160,"available":96597921792,"internally_reserved":0,"allocated":2646016,"data_stored":1488749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2043904,"data_stored":897351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37348,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334492,"statfs":{"total":96636764160,"available":96598519808,"internally_reserved":0,"allocated":2043904,"data_stored":897351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37368,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334472,"statfs":{"total":96636764160,"available":96598499328,"internally_reserved":0,"allocated":2060288,"data_stored":898485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509540,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2052096,"data_stored":896951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2039808,"data_stored":896769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509540,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33196,"kb_used_data":2000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338644,"statfs":{"total":96636764160,"available":96602771456,"internally_reserved":0,"allocated":2048000,"data_stored":896825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542244,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37924,"kb_used_data":2576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333916,"statfs":{"total":96636764160,"available":96597929984,"internally_reserved":0,"allocated":2637824,"data_stored":1487227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:57:37.364 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:57:37.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-11T12:57:37.551 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- 172.21.15.78:0/1941829621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e40701b0 msgr2=0x7f67e415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- 172.21.15.78:0/1941829621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e40701b0 0x7f67e415c440 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009840 tx=0x7f67d001c820 comp rx=0 tx=0).stop 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- 172.21.15.78:0/1941829621 shutdown_connections 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- 172.21.15.78:0/1941829621 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e4072e70 0x7f67e415c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- 172.21.15.78:0/1941829621 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e40701b0 0x7f67e415c440 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- 172.21.15.78:0/1941829621 >> 172.21.15.78:0/1941829621 conn(0x7f67e40b2ec0 msgr2=0x7f67e40b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- 172.21.15.78:0/1941829621 shutdown_connections 2024-09-11T12:57:37.552 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- 172.21.15.78:0/1941829621 wait complete. 2024-09-11T12:57:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 Processor -- start 2024-09-11T12:57:37.553 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- start start 2024-09-11T12:57:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e40701b0 0x7f67e414a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e4072e70 0x7f67e40ce250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.554 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f67e4168250 con 0x7f67e415dfc0 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f67e41680d0 con 0x7f67e4072e70 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e9200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f67e41683d0 con 0x7f67e40701b0 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47266/0 (socket says 172.21.15.78:47266) 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 -- 172.21.15.78:0/4106748131 learned_addr learned my addr 172.21.15.78:0/4106748131 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e1400640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e4072e70 0x7f67e40ce250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.555 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e1e00640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e40701b0 0x7f67e414a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 -- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e40701b0 msgr2=0x7f67e414a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e40701b0 0x7f67e414a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 -- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e4072e70 msgr2=0x7f67e40ce250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e4072e70 0x7f67e40ce250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.553+0000 7f67e2800640 1 -- 172.21.15.78:0/4106748131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e4149040 con 0x7f67e415dfc0 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67e2800640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00cce0 tx=0x7f67dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.556 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00f450 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67dc00f700 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc00fb50 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e4149320 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e41498e0 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e406faa0 con 0x7f67e415dfc0 2024-09-11T12:57:37.557 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f67dc00fcf0 con 0x7f67e415dfc0 2024-09-11T12:57:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 0x7f67c40744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f67dc090f40 con 0x7f67e415dfc0 2024-09-11T12:57:37.558 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.557+0000 7f67e1e00640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 0x7f67c40744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.559 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.561+0000 7f67e1e00640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 0x7f67c40744d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f67d4007fd0 tx=0x7f67d4007ea0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.562 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.561+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f67dc014030 con 0x7f67e415dfc0 2024-09-11T12:57:37.788 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.789+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f67e414f730 con 0x7f67e415dfc0 2024-09-11T12:57:37.790 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.789+0000 7f67da000640 1 -- 172.21.15.78:0/4106748131 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) ==== 190+0+147 (secure 0 0 0) 0x7f67dc05e360 con 0x7f67e415dfc0 2024-09-11T12:57:37.790 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-09-11T12:57:37.791 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCskeFmZ8dNBBAAXAuWdqxssishAcDTghix9w== 2024-09-11T12:57:37.791 INFO:teuthology.orchestra.run.smithi078.stdout: caps mds = "allow" 2024-09-11T12:57:37.791 INFO:teuthology.orchestra.run.smithi078.stdout: caps mon = "allow r" 2024-09-11T12:57:37.791 INFO:teuthology.orchestra.run.smithi078.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-11T12:57:37.791 INFO:teuthology.orchestra.run.smithi078.stderr:updated caps for client.0 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 msgr2=0x7f67c40744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 0x7f67c40744d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f67d4007fd0 tx=0x7f67d4007ea0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 msgr2=0x7f67e40b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f67dc00cce0 tx=0x7f67dc00a620 comp rx=0 tx=0).stop 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 shutdown_connections 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f67e415dfc0 0x7f67e40b0530 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.793 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f67c4072010 0x7f67c40744d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f67e4072e70 0x7f67e40ce250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 --2- 172.21.15.78:0/4106748131 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f67e40701b0 0x7f67e414a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 >> 172.21.15.78:0/4106748131 conn(0x7f67e40b2ec0 msgr2=0x7f67e40b32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 shutdown_connections 2024-09-11T12:57:37.794 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.793+0000 7f67e9200640 1 -- 172.21.15.78:0/4106748131 wait complete. 2024-09-11T12:57:37.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 -- 172.21.15.78:0/2855084995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15bee0 msgr2=0x7f5cdc15c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 --2- 172.21.15.78:0/2855084995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc15c390 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4009840 tx=0x7f5cd401c820 comp rx=0 tx=0).stop 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 -- 172.21.15.78:0/2855084995 shutdown_connections 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 --2- 172.21.15.78:0/2855084995 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc15c390 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 --2- 172.21.15.78:0/2855084995 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 0x7f5cdc15cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 -- 172.21.15.78:0/2855084995 >> 172.21.15.78:0/2855084995 conn(0x7f5cdc06ca80 msgr2=0x7f5cdc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 -- 172.21.15.78:0/2855084995 shutdown_connections 2024-09-11T12:57:37.955 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.953+0000 7f5ce2200640 1 -- 172.21.15.78:0/2855084995 wait complete. 2024-09-11T12:57:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 Processor -- start 2024-09-11T12:57:37.956 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- start start 2024-09-11T12:57:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 0x7f5cdc146bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.958 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc0cd380 con 0x7f5cdc15eda0 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f5cdc0cd200 con 0x7f5cdc15bee0 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f5cdc0cd500 con 0x7f5cdc0b4c30 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47290/0 (socket says 172.21.15.78:47290) 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 -- 172.21.15.78:0/3508379068 learned_addr learned my addr 172.21.15.78:0/3508379068 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdb400640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 0x7f5cdc146bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdaa00640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 -- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 msgr2=0x7f5cdc146bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 0x7f5cdc146bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 -- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc15bee0 msgr2=0x7f5cdc14fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc14fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:37.959 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 -- 172.21.15.78:0/3508379068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cdc149210 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cdbe00640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40b0040 tx=0x7f5cd40a5810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd40b2040 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd4004230 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cdc147110 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4003600 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.957+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cdc149790 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cdc15cf00 con 0x7f5cdc15eda0 2024-09-11T12:57:37.960 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f5cd40bd030 con 0x7f5cdc15eda0 2024-09-11T12:57:37.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5cd3400640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 0x7f5cbc074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:37.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f5cd412a120 con 0x7f5cdc15eda0 2024-09-11T12:57:37.961 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5cdb400640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 0x7f5cbc074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:37.962 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.961+0000 7f5cdb400640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 0x7f5cbc074500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0012f0 tx=0x7f5ccc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:37.965 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:37.965+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f5cd40c4060 con 0x7f5cdc15eda0 2024-09-11T12:57:38.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.165+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5cdc070300 con 0x7f5cdc15eda0 2024-09-11T12:57:38.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5cd3400640 1 -- 172.21.15.78:0/3508379068 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+5230 (secure 0 0 0) 0x7f5cd4016410 con 0x7f5cdc15eda0 2024-09-11T12:57:38.168 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:38.169 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":121,"btime":"2024-09-11T12:57:35:886343+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:38.169 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 121 2024-09-11T12:57:38.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 msgr2=0x7f5cbc074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 0x7f5cbc074500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc0012f0 tx=0x7f5ccc009040 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 msgr2=0x7f5cdc14ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f5cd40b0040 tx=0x7f5cd40a5810 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 shutdown_connections 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5cdc15eda0 0x7f5cdc14ff80 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f5cbc072040 0x7f5cbc074500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5cdc15bee0 0x7f5cdc14fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 --2- 172.21.15.78:0/3508379068 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f5cdc0b4c30 0x7f5cdc146bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 >> 172.21.15.78:0/3508379068 conn(0x7f5cdc06ca80 msgr2=0x7f5cdc1480a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.171 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 shutdown_connections 2024-09-11T12:57:38.172 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.169+0000 7f5ce2200640 1 -- 172.21.15.78:0/3508379068 wait complete. 2024-09-11T12:57:38.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-11T12:57:32.239557+0000', 'modified': '2024-09-11T12:57:35.275136+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9045}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9045': {'gid': 9045, 'name': 'd', 'rank': 0, 'incarnation': 117, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.157:6835/70117498', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.157:6834', 'nonce': 70117498}, {'type': 'v1', 'addr': '172.21.15.157:6835', 'nonce': 70117498}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9045, 'qdb_cluster': [9045]} 2024-09-11T12:57:38.592 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-11T12:57:38.592 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-11T12:57:38.735 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.733+0000 7f4162600640 1 -- 172.21.15.78:0/294545124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 msgr2=0x7f415c157ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.733+0000 7f4162600640 1 --2- 172.21.15.78:0/294545124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c157ae0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4148009900 tx=0x7f414801c850 comp rx=0 tx=0).stop 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.733+0000 7f4162600640 1 -- 172.21.15.78:0/294545124 shutdown_connections 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 --2- 172.21.15.78:0/294545124 >> v2:172.21.15.157:3300/0 conn(0x7f415c06eb40 0x7f415c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 --2- 172.21.15.78:0/294545124 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c157ae0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- 172.21.15.78:0/294545124 >> 172.21.15.78:0/294545124 conn(0x7f415c06ca80 msgr2=0x7f415c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- 172.21.15.78:0/294545124 shutdown_connections 2024-09-11T12:57:38.736 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- 172.21.15.78:0/294545124 wait complete. 2024-09-11T12:57:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 Processor -- start 2024-09-11T12:57:38.737 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- start start 2024-09-11T12:57:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 0x7f415c0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c1f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.739 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f415c0b5250 con 0x7f415c06eb40 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f415c0b50d0 con 0x7f415c070a00 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f4162600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f415c167e10 con 0x7f415c0736d0 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47306/0 (socket says 172.21.15.78:47306) 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 -- 172.21.15.78:0/3217202875 learned_addr learned my addr 172.21.15.78:0/3217202875 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415aa00640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 0x7f415c0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415be00640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c1f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 -- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 msgr2=0x7f415c1f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c1f7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 -- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 msgr2=0x7f415c0ae210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 0x7f415c0ae210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415b400640 1 -- 172.21.15.78:0/3217202875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c073070 con 0x7f415c06eb40 2024-09-11T12:57:38.740 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.737+0000 7f415aa00640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 0x7f415c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f415b400640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c3c0 tx=0x7f414c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 -- 172.21.15.78:0/3217202875 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c0052d0 con 0x7f415c06eb40 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 -- 172.21.15.78:0/3217202875 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f414c007020 con 0x7f415c06eb40 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c1f85f0 con 0x7f415c06eb40 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 -- 172.21.15.78:0/3217202875 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c007470 con 0x7f415c06eb40 2024-09-11T12:57:38.741 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c0ada10 con 0x7f415c06eb40 2024-09-11T12:57:38.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 -- 172.21.15.78:0/3217202875 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f414c0067e0 con 0x7f415c06eb40 2024-09-11T12:57:38.742 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 0x7f413c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f4153400640 1 -- 172.21.15.78:0/3217202875 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f414c08fa20 con 0x7f415c06eb40 2024-09-11T12:57:38.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.741+0000 7f415aa00640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 0x7f413c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 Processor -- start 2024-09-11T12:57:38.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- start start 2024-09-11T12:57:38.743 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415aa00640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 0x7f413c0744d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f41480af000 tx=0x7f414801b090 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ce5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9c98e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9af6270 con 0x55f5b9c99340 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9ae5770 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f5b9ae97b0 con 0x55f5b9ce63c0 2024-09-11T12:57:38.745 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47322/0 (socket says 172.21.15.78:47322) 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 -- 172.21.15.78:0/3209529326 learned_addr learned my addr 172.21.15.78:0/3209529326 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415aa00640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9c98e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415b400640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ce5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 -- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 msgr2=0x55f5b9c98e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9c98e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 -- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 msgr2=0x55f5b9ce5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ce5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 -- 172.21.15.78:0/3209529326 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b9a84f90 con 0x55f5b9c99340 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415b400640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ce5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f415be00640 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4154011470 tx=0x7f4154011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11acdff492932df5 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4147e00640 1 -- 172.21.15.78:0/3209529326 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154004d30 con 0x55f5b9c99340 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4147e00640 1 -- 172.21.15.78:0/3209529326 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4154003980 con 0x55f5b9c99340 2024-09-11T12:57:38.746 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4147e00640 1 -- 172.21.15.78:0/3209529326 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4154003d50 con 0x55f5b9c99340 2024-09-11T12:57:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- 172.21.15.78:0/3209529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 msgr2=0x55f5b9c99720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4154011470 tx=0x7f4154011930 comp rx=0 tx=0).stop 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- 172.21.15.78:0/3209529326 shutdown_connections 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9c99720 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9c98e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 --2- 172.21.15.78:0/3209529326 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ce5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- 172.21.15.78:0/3209529326 >> 172.21.15.78:0/3209529326 conn(0x55f5b9cc6b60 msgr2=0x55f5b9cc8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- 172.21.15.78:0/3209529326 shutdown_connections 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- 172.21.15.78:0/3209529326 wait complete. 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 -- start start 2024-09-11T12:57:38.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.745+0000 7f4165aac000 1 Processor -- start 2024-09-11T12:57:38.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9ca6d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9ca77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9bb5290 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9a91ab0 con 0x55f5b9c99340 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f5b9a8f170 con 0x55f5b9ce63c0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 10 client.0 _ll_register_callbacks cb 0x55f5b9c86830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47326/0 (socket says 172.21.15.78:47326) 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 -- 172.21.15.78:0/3101431416 learned_addr learned my addr 172.21.15.78:0/3101431416 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415be00640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9ca77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415b400640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9ca6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 -- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 msgr2=0x55f5b9ca77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9ca77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 -- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 msgr2=0x55f5b9ca6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9ca6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 -- 172.21.15.78:0/3101431416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b9b6a6f0 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415b400640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9ca6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415aa00640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4148009230 tx=0x7f414801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c673cd849fa791da server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480b2040 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f5b9a7ef10 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f41480046d0 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f41480d1050 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 10 client.9171 fetch_fsmap learned FSMap version 121 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) -- 0x55f5b9b41c40 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55f5b9a7ef10 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 121) ==== 3502+0+0 (secure 0 0 0) 0x7f41480d4020 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 10 client.9171 fetch_fsmap finished waiting for FSMap version 121 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 10 client.9171 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 20 client.9171 populate_metadata read hostname 'smithi078' 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cf5b40 0x55f5b9cf7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 4 client.9171 mds_command: new command op to 9045 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 --> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5b9af81c0 con 0x55f5b9cf5b40 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415b400640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cf5b40 0x55f5b9cf7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f415b400640 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cf5b40 0x55f5b9cf7f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8740d1bc44122a9f server_cookie=1086caa9c0aeb7f in_seq=0 out_seq=0 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.749+0000 7f4144c00640 10 client.9171 ms_handle_connect on v2:172.21.15.157:6834/70117498 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4144c00640 1 -- 172.21.15.78:0/3101431416 <== mds.0 v2:172.21.15.157:6834/70117498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5b9af81c0 con 0x55f5b9cf5b40 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4144c00640 10 client.9171 handle_command_reply: tid=0 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 client.9171 shutdown 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 msgr2=0x55f5b9ca7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.755 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4148009230 tx=0x7f414801c850 comp rx=0 tx=0).stop 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 shutdown_connections 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cf5b40 0x55f5b9cf7f30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9ca77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9ca7290 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 --2- 172.21.15.78:0/3101431416 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9ca6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4144c00640 0 client.9171 ms_handle_reset on v2:172.21.15.157:6834/70117498 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 >> 172.21.15.78:0/3101431416 conn(0x55f5b9cc6b60 msgr2=0x55f5b9c98500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 shutdown_connections 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 1 -- 172.21.15.78:0/3101431416 wait complete. 2024-09-11T12:57:38.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.753+0000 7f4165aac000 20 client.9171 trim_cache size 0 max 16384 2024-09-11T12:57:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.769+0000 7f4165aac000 1 Processor -- start 2024-09-11T12:57:38.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.769+0000 7f4165aac000 1 -- start start 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9cfb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9cc53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1fbc10 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f415c15c210 con 0x55f5b9c99340 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f415c1fb5c0 con 0x55f5b9ce63c0 2024-09-11T12:57:38.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47340/0 (socket says 172.21.15.78:47340) 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 -- 172.21.15.78:0/3135535912 learned_addr learned my addr 172.21.15.78:0/3135535912 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415be00640 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9cc53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415b400640 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9cfb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 -- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 msgr2=0x55f5b9cc53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9cc53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 -- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 msgr2=0x55f5b9cfb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9cfb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 -- 172.21.15.78:0/3135535912 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5b9a9ae70 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415aa00640 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4148004920 tx=0x7f41480049a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2ca0002ec7f941 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4147400640 1 -- 172.21.15.78:0/3135535912 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480da070 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4147400640 1 -- 172.21.15.78:0/3135535912 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41480c9070 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4147400640 1 -- 172.21.15.78:0/3135535912 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41480b2070 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- 172.21.15.78:0/3135535912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 msgr2=0x55f5b9cc4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4148004920 tx=0x7f41480049a0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- 172.21.15.78:0/3135535912 shutdown_connections 2024-09-11T12:57:38.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce63c0 0x55f5b9cc53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9ce5aa0 0x55f5b9cc4e60 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- 172.21.15.78:0/3135535912 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9c99340 0x55f5b9cfb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- 172.21.15.78:0/3135535912 >> 172.21.15.78:0/3135535912 conn(0x55f5b9cc6b60 msgr2=0x55f5b9c9d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- 172.21.15.78:0/3135535912 shutdown_connections 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- 172.21.15.78:0/3135535912 wait complete. 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- start start 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 Processor -- start 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce5aa0 0x55f5b9d59c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce63c0 0x55f5b9d5a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9aedde0 con 0x55f5b9c99340 2024-09-11T12:57:38.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x55f5b9ae8ac0 con 0x55f5b9ce63c0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x55f5b9af8ea0 con 0x55f5b9ce5aa0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f4165aac000 10 client.0 _ll_register_callbacks cb 0x7f4148009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47354/0 (socket says 172.21.15.78:47354) 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415b400640 1 -- 172.21.15.78:0/3761575524 learned_addr learned my addr 172.21.15.78:0/3761575524 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.773+0000 7f415be00640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce63c0 0x55f5b9d5a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415aa00640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce5aa0 0x55f5b9d59c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 -- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce5aa0 msgr2=0x55f5b9d59c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce5aa0 0x55f5b9d59c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 -- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce63c0 msgr2=0x55f5b9d5a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce63c0 0x55f5b9d5a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 -- 172.21.15.78:0/3761575524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5b9a6fbf0 con 0x55f5b9c99340 2024-09-11T12:57:38.779 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415b400640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f414c00cd80 tx=0x7f414c05c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575b5cb300d9c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 10 client.0 ms_handle_connect on v2:172.21.15.78:3300/0 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414c065d30 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mon.0 v2:172.21.15.78:3300/0 2 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f414c05c020 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41480c9070 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f414c05c220 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 10 client.9177 fetch_fsmap learned FSMap version 121 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) -- 0x55f5b9a8bc60 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mon.0 v2:172.21.15.78:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41480c9070 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mon.0 v2:172.21.15.78:3300/0 5 ==== fsmap(e 121) ==== 3502+0+0 (secure 0 0 0) 0x7f414c058730 con 0x55f5b9c99340 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 10 client.9177 fetch_fsmap finished waiting for FSMap version 121 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 10 client.9177 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 20 client.9177 populate_metadata read hostname 'smithi078' 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cfb760 0x55f5b9cfa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.780 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 4 client.9177 mds_command: new command op to 9045 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-11T12:57:38.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 --> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] -- command(tid 0: {"prefix": "status"}) -- 0x55f5b9af3c60 con 0x55f5b9cfb760 2024-09-11T12:57:38.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415aa00640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cfb760 0x55f5b9cfa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f415aa00640 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cfb760 0x55f5b9cfa870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b14dcc5710105935 server_cookie=d82bcddce705071c in_seq=0 out_seq=0 2024-09-11T12:57:38.781 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 10 client.9177 ms_handle_connect on v2:172.21.15.157:6834/70117498 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 1 -- 172.21.15.78:0/3761575524 <== mds.0 v2:172.21.15.157:6834/70117498 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55f5b9af3c60 con 0x55f5b9cfb760 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 10 client.9177 handle_command_reply: tid=0 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 client.9177 shutdown 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 msgr2=0x55f5b9d596c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f414c00cd80 tx=0x7f414c05c930 comp rx=0 tx=0).stop 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 shutdown_connections 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:6834/70117498,v1:172.21.15.157:6835/70117498] conn(0x55f5b9cfb760 0x55f5b9cfa870 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.782 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x55f5b9ce63c0 0x55f5b9d5a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x55f5b9ce5aa0 0x55f5b9d59c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4144c00640 0 client.9177 ms_handle_reset on v2:172.21.15.157:6834/70117498 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 --2- 172.21.15.78:0/3761575524 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x55f5b9c99340 0x55f5b9d596c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 >> 172.21.15.78:0/3761575524 conn(0x55f5b9cc6b60 msgr2=0x55f5b9cf2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 shutdown_connections 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 1 -- 172.21.15.78:0/3761575524 wait complete. 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4165aac000 20 client.9177 trim_cache size 0 max 16384 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stdout: "cluster_fsid": "163b1675-69a3-4b68-a3e4-8c2fa35bac4f", 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stdout: "whoami": 0, 2024-09-11T12:57:38.783 INFO:teuthology.orchestra.run.smithi078.stdout: "id": 9045, 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "want_state": "up:active", 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "state": "up:active", 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "fs_name": "cephfs", 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "rank_uptime": 5.9000928620000002, 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "mdsmap_epoch": 121, 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch": 162, 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "osdmap_epoch_barrier": 162, 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout: "uptime": 10.416904696 2024-09-11T12:57:38.784 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 msgr2=0x7f413c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 0x7f413c0744d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f41480af000 tx=0x7f414801b090 comp rx=0 tx=0).stop 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 msgr2=0x7f415c155fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c3c0 tx=0x7f414c00c880 comp rx=0 tx=0).stop 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 shutdown_connections 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f415c0736d0 0x7f415c1f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f413c072010 0x7f413c0744d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.785 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f415c070a00 0x7f415c0ae210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 --2- 172.21.15.78:0/3217202875 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f415c06eb40 0x7f415c155fc0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.777+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 >> 172.21.15.78:0/3217202875 conn(0x7f415c06ca80 msgr2=0x7f415c156760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.781+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 shutdown_connections 2024-09-11T12:57:38.786 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.781+0000 7f4162600640 1 -- 172.21.15.78:0/3217202875 wait complete. 2024-09-11T12:57:38.797 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:57:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1395084622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00ccb20 msgr2=0x7fe6e0073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1395084622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00ccb20 0x7fe6e0073f70 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009870 tx=0x7fe6d801c740 comp rx=0 tx=0).stop 2024-09-11T12:57:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1395084622 shutdown_connections 2024-09-11T12:57:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1395084622 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00ccb20 0x7fe6e0073f70 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1395084622 >> 172.21.15.78:0/1395084622 conn(0x7fe6e006ca80 msgr2=0x7fe6e00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1395084622 shutdown_connections 2024-09-11T12:57:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1395084622 wait complete. 2024-09-11T12:57:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 Processor -- start 2024-09-11T12:57:38.945 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- start start 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 0x7fe6e0073830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6e015bc60 0x7fe6e01fa330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0169e80 con 0x7fe6e00ccb20 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e0169d00 con 0x7fe6e00b4780 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6e4a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e016a000 con 0x7fe6e015bc60 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.947 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6dd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47362/0 (socket says 172.21.15.78:47362) 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6dd400640 1 -- 172.21.15.78:0/1540114352 learned_addr learned my addr 172.21.15.78:0/1540114352 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6dde00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 0x7fe6e0073830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.945+0000 7fe6de800640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6e015bc60 0x7fe6e01fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 -- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6e015bc60 msgr2=0x7fe6e01fa330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6e015bc60 0x7fe6e01fa330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 -- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 msgr2=0x7fe6e0073830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 0x7fe6e0073830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 -- 172.21.15.78:0/1540114352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e01f7b20 con 0x7fe6e00ccb20 2024-09-11T12:57:38.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dde00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 0x7fe6e0073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6dd400640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c450 tx=0x7fe6cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc0052d0 con 0x7fe6e00ccb20 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6cc005470 con 0x7fe6e00ccb20 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc012870 con 0x7fe6e00ccb20 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e01f7e00 con 0x7fe6e00ccb20 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e01f83c0 con 0x7fe6e00ccb20 2024-09-11T12:57:38.949 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e0073f70 con 0x7fe6e00ccb20 2024-09-11T12:57:38.950 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fe6cc003600 con 0x7fe6e00ccb20 2024-09-11T12:57:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 0x7fe6c00744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.949+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fe6cc08fed0 con 0x7fe6e00ccb20 2024-09-11T12:57:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.953+0000 7fe6dde00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 0x7fe6c00744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:38.951 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.953+0000 7fe6dde00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 0x7fe6c00744d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001c10 tx=0x7fe6d0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:38.954 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:38.953+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe6cc05d2f0 con 0x7fe6e00ccb20 2024-09-11T12:57:39.159 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe6e014b3a0 con 0x7fe6e00ccb20 2024-09-11T12:57:39.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6d6000640 1 -- 172.21.15.78:0/1540114352 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) ==== 53+0+73 (secure 0 0 0) 0x7fe6cc061840 con 0x7fe6e00ccb20 2024-09-11T12:57:39.160 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 msgr2=0x7fe6c00744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 0x7fe6c00744d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe6d0001c10 tx=0x7fe6d0009040 comp rx=0 tx=0).stop 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 msgr2=0x7fe6e01f8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00c450 tx=0x7fe6cc00c910 comp rx=0 tx=0).stop 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 shutdown_connections 2024-09-11T12:57:39.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe6e015bc60 0x7fe6e01fa330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe6e00ccb20 0x7fe6e01f8790 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe6c0072010 0x7fe6c00744d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 --2- 172.21.15.78:0/1540114352 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe6e00b4780 0x7fe6e0073830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 >> 172.21.15.78:0/1540114352 conn(0x7fe6e006ca80 msgr2=0x7fe6e0157c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 shutdown_connections 2024-09-11T12:57:39.163 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39.161+0000 7fe6e4a00640 1 -- 172.21.15.78:0/1540114352 wait complete. 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi078.front.sepia.ceph.com 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:57:39.181 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:39.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.182 DEBUG:teuthology.orchestra.run.smithi078:> ip addr 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 ::1/128 scope host 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:22 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f0 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:d8:a1:23 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: altname enp7s0f1 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dc brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0c:c4:7a:8f:d4:dd brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.21.15.78/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft 41975sec preferred_lft 41975sec 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::ec4:7aff:fe8f:d4dd/64 scope link 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:57:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 02:42:fd:db:6d:73 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:57:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:57:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:39.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.195 DEBUG:teuthology.orchestra.run.smithi078:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:57:39.252 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-09-11T12:57:39.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.255 DEBUG:teuthology.orchestra.run.smithi078:> route 2024-09-11T12:57:39.305 INFO:teuthology.orchestra.run.smithi078.stdout:Kernel IP routing table 2024-09-11T12:57:39.305 INFO:teuthology.orchestra.run.smithi078.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:39.306 INFO:teuthology.orchestra.run.smithi078.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:57:39.306 DEBUG:teuthology.orchestra.run.smithi078:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:57:39.307 DEBUG:teuthology.orchestra.run.smithi078:> set -e 2024-09-11T12:57:39.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:57:39.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:57:39.307 DEBUG:teuthology.orchestra.run.smithi078:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:57:39.307 DEBUG:teuthology.orchestra.run.smithi078:> ') 2024-09-11T12:57:39.397 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:57:39.437 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:39.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.444 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:57:39.495 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:57:39.495 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:57:39.496 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:39.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link set brx.0 up 2024-09-11T12:57:39.563 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:39.563 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:39.615 INFO:teuthology.orchestra.run.smithi078.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-11T12:57:39.615 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:39.615 DEBUG:teuthology.orchestra.run.smithi078:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:39.666 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-11T12:57:39.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.667 DEBUG:teuthology.orchestra.run.smithi078:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:39.754 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:39.755 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:39.768 INFO:teuthology.orchestra.run.smithi078.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:39.768 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:39.769 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:57:39.769 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: trying mount with new device syntax: 0@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:57:39.769 INFO:teuthology.orchestra.run.smithi078.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:57:39.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:57:39.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:39.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:39.793 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:57:39.886 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:39.899 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:39.906 INFO:teuthology.orchestra.run.smithi078.stdout:client7520 2024-09-11T12:57:39.906 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:39.906 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:39.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- 172.21.15.78:0/2921233157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 msgr2=0x7f13a006e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- 172.21.15.78:0/2921233157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a006e860 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f13a00b3fa0 tx=0x7f139001c540 comp rx=0 tx=0).stop 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- 172.21.15.78:0/2921233157 shutdown_connections 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- 172.21.15.78:0/2921233157 >> v2:172.21.15.157:3300/0 conn(0x7f13a00b1300 0x7f13a015f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- 172.21.15.78:0/2921233157 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 0x7f13a00b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- 172.21.15.78:0/2921233157 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a006e860 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- 172.21.15.78:0/2921233157 >> 172.21.15.78:0/2921233157 conn(0x7f13a00cf830 msgr2=0x7f13a00cfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- 172.21.15.78:0/2921233157 shutdown_connections 2024-09-11T12:57:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- 172.21.15.78:0/2921233157 wait complete. 2024-09-11T12:57:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 Processor -- start 2024-09-11T12:57:40.101 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- start start 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 0x7f13a01649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00b1300 0x7f13a015e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00ccea0 con 0x7f13a015f080 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00ccd20 con 0x7f13a00b1300 2024-09-11T12:57:40.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a7600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00cd020 con 0x7f13a00a75d0 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a5600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47382/0 (socket says 172.21.15.78:47382) 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a5600640 1 -- 172.21.15.78:0/1148867506 learned_addr learned my addr 172.21.15.78:0/1148867506 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f13a4c00640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 0x7f13a01649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.101+0000 7f1397e00640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00b1300 0x7f13a015e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 -- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 msgr2=0x7f13a01649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 0x7f13a01649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 -- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00b1300 msgr2=0x7f13a015e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00b1300 0x7f13a015e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 -- 172.21.15.78:0/1148867506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01fae20 con 0x7f13a015f080 2024-09-11T12:57:40.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a5600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f139800daa0 tx=0x7f139800df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398017070 con 0x7f13a015f080 2024-09-11T12:57:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1398004b30 con 0x7f13a015f080 2024-09-11T12:57:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1398005020 con 0x7f13a015f080 2024-09-11T12:57:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01fb100 con 0x7f13a015f080 2024-09-11T12:57:40.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01fb6c0 con 0x7f13a015f080 2024-09-11T12:57:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a015f880 con 0x7f13a015f080 2024-09-11T12:57:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f13980051c0 con 0x7f13a015f080 2024-09-11T12:57:40.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 0x7f137c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f1398090830 con 0x7f13a015f080 2024-09-11T12:57:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.105+0000 7f13a4c00640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 0x7f137c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:40.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.109+0000 7f13a4c00640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 0x7f137c0744d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f13900af000 tx=0x7f139001ad80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:40.111 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.109+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f1398020070 con 0x7f13a015f080 2024-09-11T12:57:40.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f13a00b2400 con 0x7f13a015f080 2024-09-11T12:57:40.312 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f139d600640 1 -- 172.21.15.78:0/1148867506 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) ==== 53+0+73 (secure 0 0 0) 0x7f139805dc50 con 0x7f13a015f080 2024-09-11T12:57:40.312 INFO:teuthology.orchestra.run.smithi078.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 msgr2=0x7f137c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 0x7f137c0744d0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f13900af000 tx=0x7f139001ad80 comp rx=0 tx=0).stop 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 msgr2=0x7f13a01656c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f139800daa0 tx=0x7f139800df60 comp rx=0 tx=0).stop 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 shutdown_connections 2024-09-11T12:57:40.314 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f13a015f080 0x7f13a01656c0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f137c072010 0x7f137c0744d0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f13a00b1300 0x7f13a015e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 --2- 172.21.15.78:0/1148867506 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f13a00a75d0 0x7f13a01649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 >> 172.21.15.78:0/1148867506 conn(0x7f13a00cf830 msgr2=0x7f13a006e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 shutdown_connections 2024-09-11T12:57:40.315 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:40.313+0000 7f13a7600640 1 -- 172.21.15.78:0/1148867506 wait complete. 2024-09-11T12:57:40.332 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-11T12:57:40.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi157.front.sepia.ceph.com 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-11T12:57:40.333 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:40.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.333 DEBUG:teuthology.orchestra.run.smithi157:> ip addr 2024-09-11T12:57:40.343 INFO:teuthology.orchestra.run.smithi157.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-11T12:57:40.343 INFO:teuthology.orchestra.run.smithi157.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-11T12:57:40.343 INFO:teuthology.orchestra.run.smithi157.stdout: inet 127.0.0.1/8 scope host lo 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 ::1/128 scope host 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:98 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f0 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:d9:37:99 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: altname enp7s0f1 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a0 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-11T12:57:40.344 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0c:c4:7a:88:7e:a1 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.21.15.157/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft 41983sec preferred_lft 41983sec 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::ec4:7aff:fe88:7ea1/64 scope link 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 02:42:d2:70:86:52 brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2024-09-11T12:57:40.345 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2024-09-11T12:57:40.346 INFO:teuthology.orchestra.run.smithi157.stdout: valid_lft forever preferred_lft forever 2024-09-11T12:57:40.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.347 DEBUG:teuthology.orchestra.run.smithi157:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-11T12:57:40.410 INFO:teuthology.orchestra.run.smithi157.stdout:1 2024-09-11T12:57:40.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.413 DEBUG:teuthology.orchestra.run.smithi157:> route 2024-09-11T12:57:40.466 INFO:teuthology.orchestra.run.smithi157.stdout:Kernel IP routing table 2024-09-11T12:57:40.466 INFO:teuthology.orchestra.run.smithi157.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-11T12:57:40.466 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2024-09-11T12:57:40.466 INFO:teuthology.orchestra.run.smithi157.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:158.69.67.47 _gateway 255.255.255.255 UGH 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2024-09-11T12:57:40.467 INFO:teuthology.orchestra.run.smithi157.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-11T12:57:40.467 DEBUG:teuthology.orchestra.run.smithi157:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-11T12:57:40.467 DEBUG:teuthology.orchestra.run.smithi157:> set -e 2024-09-11T12:57:40.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-11T12:57:40.467 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-11T12:57:40.468 DEBUG:teuthology.orchestra.run.smithi157:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-11T12:57:40.468 DEBUG:teuthology.orchestra.run.smithi157:> ') 2024-09-11T12:57:40.560 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-11T12:57:40.602 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:40.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.609 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:57:40.618 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:57:40.618 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:57:40.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:40.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.619 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link set brx.0 up 2024-09-11T12:57:40.682 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:40.682 DEBUG:teuthology.orchestra.run.smithi157:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:40.733 INFO:teuthology.orchestra.run.smithi157.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-11T12:57:40.734 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-11T12:57:40.734 DEBUG:teuthology.orchestra.run.smithi157:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:40.785 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-11T12:57:40.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.785 DEBUG:teuthology.orchestra.run.smithi157:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:40.880 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:40.880 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:40.893 INFO:teuthology.orchestra.run.smithi157.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-11T12:57:40.893 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-11T12:57:40.893 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: resolved to: "172.21.15.78:3300,172.21.15.157:3300,172.21.15.157:3301" 2024-09-11T12:57:40.893 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: trying mount with new device syntax: 1@163b1675-69a3-4b68-a3e4-8c2fa35bac4f.cephfs=/ 2024-09-11T12:57:40.893 INFO:teuthology.orchestra.run.smithi157.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.78:3300/172.21.15.157:3300/172.21.15.157:3301" will pass to kernel 2024-09-11T12:57:40.897 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-11T12:57:40.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:40.897 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:40.918 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-11T12:57:41.008 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:41.011 INFO:teuthology.orchestra.run.smithi157.stderr:2024-09-11T12:57:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:41.018 INFO:teuthology.orchestra.run.smithi157.stdout:client7529 2024-09-11T12:57:41.018 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:41.018 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:41.027 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-11T12:57:41.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- 172.21.15.78:0/1526658815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5790 msgr2=0x7f8b900723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:41.180 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- 172.21.15.78:0/1526658815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5790 0x7f8b900723f0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f8b900749e0 tx=0x7f8b8001cd80 comp rx=0 tx=0).stop 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- 172.21.15.78:0/1526658815 shutdown_connections 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- 172.21.15.78:0/1526658815 >> v2:172.21.15.157:3300/0 conn(0x7f8b900b5b70 0x7f8b90072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- 172.21.15.78:0/1526658815 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5790 0x7f8b900723f0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- 172.21.15.78:0/1526658815 >> 172.21.15.78:0/1526658815 conn(0x7f8b900b3050 msgr2=0x7f8b900b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- 172.21.15.78:0/1526658815 shutdown_connections 2024-09-11T12:57:41.181 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- 172.21.15.78:0/1526658815 wait complete. 2024-09-11T12:57:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 Processor -- start 2024-09-11T12:57:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- start start 2024-09-11T12:57:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b900b5790 0x7f8b90136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:41.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 0x7f8b90148f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90167ad0 con 0x7f8b900b5b70 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8b90167950 con 0x7f8b900b5790 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b95e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8b90167c50 con 0x7f8b90151340 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b8e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b8e000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47402/0 (socket says 172.21.15.78:47402) 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.181+0000 7f8b8e000640 1 -- 172.21.15.78:0/2733642446 learned_addr learned my addr 172.21.15.78:0/2733642446 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8ea00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b900b5790 0x7f8b90136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8f400640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 0x7f8b90148f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 -- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 msgr2=0x7f8b90148f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 0x7f8b90148f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 -- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b900b5790 msgr2=0x7f8b90136840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b900b5790 0x7f8b90136840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 -- 172.21.15.78:0/2733642446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b901467e0 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8f400640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 0x7f8b90148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b8e000640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00e940 tx=0x7f8b7c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00cd80 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b7c005020 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b901481e0 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c005450 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b90146a60 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b901517f0 con 0x7f8b900b5b70 2024-09-11T12:57:41.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f8b7c0047b0 con 0x7f8b900b5b70 2024-09-11T12:57:41.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 0x7f8b70074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:41.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.185+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f8b7c090f00 con 0x7f8b900b5b70 2024-09-11T12:57:41.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.189+0000 7f8b8ea00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 0x7f8b70074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:41.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.189+0000 7f8b8ea00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 0x7f8b70074500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8b800af000 tx=0x7f8b8001b5c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:41.190 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.189+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8b901467e0 con 0x7f8b900b5b70 2024-09-11T12:57:41.393 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.393+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8b900b3da0 con 0x7f8b900b5b70 2024-09-11T12:57:41.913 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.913+0000 7f8b87400640 1 -- 172.21.15.78:0/2733642446 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v122) ==== 136+0+0 (secure 0 0 0) 0x7f8b7c05e970 con 0x7f8b900b5b70 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 msgr2=0x7f8b70074500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 0x7f8b70074500 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8b800af000 tx=0x7f8b8001b5c0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 msgr2=0x7f8b901462a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c00e940 tx=0x7f8b7c00ee00 comp rx=0 tx=0).stop 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 shutdown_connections 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f8b90151340 0x7f8b90148f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f8b900b5b70 0x7f8b901462a0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.916 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f8b70072040 0x7f8b70074500 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 --2- 172.21.15.78:0/2733642446 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f8b900b5790 0x7f8b90136840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 >> 172.21.15.78:0/2733642446 conn(0x7f8b900b3050 msgr2=0x7f8b90072230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 shutdown_connections 2024-09-11T12:57:41.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:41.917+0000 7f8b95e00640 1 -- 172.21.15.78:0/2733642446 wait complete. 2024-09-11T12:57:41.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-11T12:57:42.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- 172.21.15.78:0/4064171431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 msgr2=0x7f7c80070a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- 172.21.15.78:0/4064171431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c80070a00 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f7c800b2c60 tx=0x7f7c7001db60 comp rx=0 tx=0).stop 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- 172.21.15.78:0/4064171431 shutdown_connections 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- 172.21.15.78:0/4064171431 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 0x7f7c800b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- 172.21.15.78:0/4064171431 >> v2:172.21.15.157:3300/0 conn(0x7f7c800736d0 0x7f7c8015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- 172.21.15.78:0/4064171431 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c80070a00 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- 172.21.15.78:0/4064171431 >> 172.21.15.78:0/4064171431 conn(0x7f7c8006ca80 msgr2=0x7f7c800b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- 172.21.15.78:0/4064171431 shutdown_connections 2024-09-11T12:57:42.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- 172.21.15.78:0/4064171431 wait complete. 2024-09-11T12:57:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 Processor -- start 2024-09-11T12:57:42.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- start start 2024-09-11T12:57:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c800736d0 0x7f7c800cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 0x7f7c80164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8006f010 con 0x7f7c80157ae0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7c8006ee90 con 0x7f7c800736d0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.085+0000 7f7c85400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7c8006f190 con 0x7f7c800b4db0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47420/0 (socket says 172.21.15.78:47420) 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 -- 172.21.15.78:0/1045650811 learned_addr learned my addr 172.21.15.78:0/1045650811 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c75e00640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 0x7f7c80164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7e800640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c800736d0 0x7f7c800cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 -- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 msgr2=0x7f7c80164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 0x7f7c80164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.089 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 -- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c800736d0 msgr2=0x7f7c800cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c800736d0 0x7f7c800cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 -- 172.21.15.78:0/1045650811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c801f8140 con 0x7f7c80157ae0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c7f200640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800eb50 tx=0x7f7c7800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78004d30 con 0x7f7c80157ae0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c7800ee50 con 0x7f7c80157ae0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c800cf9b0 con 0x7f7c80157ae0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78005280 con 0x7f7c80157ae0 2024-09-11T12:57:42.090 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c801f7660 con 0x7f7c80157ae0 2024-09-11T12:57:42.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c800b5220 con 0x7f7c80157ae0 2024-09-11T12:57:42.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.089+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f7c78005420 con 0x7f7c80157ae0 2024-09-11T12:57:42.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.093+0000 7f7c77400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 0x7f7c5c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.093+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f7c78090fd0 con 0x7f7c80157ae0 2024-09-11T12:57:42.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.093+0000 7f7c7e800640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 0x7f7c5c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.093+0000 7f7c7e800640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 0x7f7c5c0744d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c700b0000 tx=0x7f7c700a8040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:42.095 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.093+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7c7805e3f0 con 0x7f7c80157ae0 2024-09-11T12:57:42.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.281+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7c80154730 con 0x7f7c5c072010 2024-09-11T12:57:42.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.281+0000 7f7c77400640 1 -- 172.21.15.78:0/1045650811 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7c80154730 con 0x7f7c5c072010 2024-09-11T12:57:42.282 INFO:teuthology.orchestra.run.smithi078.stdout:[ 2024-09-11T12:57:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-09-11T12:57:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "name": "cephfs" 2024-09-11T12:57:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-09-11T12:57:42.283 INFO:teuthology.orchestra.run.smithi078.stdout:] 2024-09-11T12:57:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 msgr2=0x7f7c5c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 0x7f7c5c0744d0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f7c700b0000 tx=0x7f7c700a8040 comp rx=0 tx=0).stop 2024-09-11T12:57:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 msgr2=0x7f7c801484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f7c7800eb50 tx=0x7f7c7800c6a0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 shutdown_connections 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7c80157ae0 0x7f7c801484a0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7c800b4db0 0x7f7c80164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7c5c072010 0x7f7c5c0744d0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 --2- 172.21.15.78:0/1045650811 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7c800736d0 0x7f7c800cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 >> 172.21.15.78:0/1045650811 conn(0x7f7c8006ca80 msgr2=0x7f7c80149570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:42.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 shutdown_connections 2024-09-11T12:57:42.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.285+0000 7f7c85400640 1 -- 172.21.15.78:0/1045650811 wait complete. 2024-09-11T12:57:42.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-11T12:57:42.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- 172.21.15.78:0/2569497384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 msgr2=0x7f302c0cf6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 --2- 172.21.15.78:0/2569497384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c0cf6e0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f301c009900 tx=0x7f301c01c820 comp rx=0 tx=0).stop 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- 172.21.15.78:0/2569497384 shutdown_connections 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 --2- 172.21.15.78:0/2569497384 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f302c0b3420 0x7f302c15a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 --2- 172.21.15.78:0/2569497384 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c0cf6e0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- 172.21.15.78:0/2569497384 >> 172.21.15.78:0/2569497384 conn(0x7f302c071dd0 msgr2=0x7f302c0721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- 172.21.15.78:0/2569497384 shutdown_connections 2024-09-11T12:57:42.450 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- 172.21.15.78:0/2569497384 wait complete. 2024-09-11T12:57:42.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 Processor -- start 2024-09-11T12:57:42.451 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.449+0000 7f3031200640 1 -- start start 2024-09-11T12:57:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f302c0b3420 0x7f302c1541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f302c0cd770 0x7f302c13b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.452 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0af070 con 0x7f302c0b3040 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0aeef0 con 0x7f302c0b3420 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f302c0af1f0 con 0x7f302c0cd770 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47436/0 (socket says 172.21.15.78:47436) 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 -- 172.21.15.78:0/2540436839 learned_addr learned my addr 172.21.15.78:0/2540436839 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 -- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f302c0cd770 msgr2=0x7f302c13b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f302c0cd770 0x7f302c13b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 -- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f302c0b3420 msgr2=0x7f302c1541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f302c0b3420 0x7f302c1541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 -- 172.21.15.78:0/2540436839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c13d2f0 con 0x7f302c0b3040 2024-09-11T12:57:42.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3029e00640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f301c009850 tx=0x7f301c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:42.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c0b2040 con 0x7f302c0b3040 2024-09-11T12:57:42.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f301c0044b0 con 0x7f302c0b3040 2024-09-11T12:57:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c1407d0 con 0x7f302c0b3040 2024-09-11T12:57:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301c003860 con 0x7f302c0b3040 2024-09-11T12:57:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c13ea80 con 0x7f302c0b3040 2024-09-11T12:57:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.453+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c0cf6e0 con 0x7f302c0b3040 2024-09-11T12:57:42.455 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.457+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f301c003a00 con 0x7f302c0b3040 2024-09-11T12:57:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.457+0000 7f3022000640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 0x7f300c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.457+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f301c12a970 con 0x7f302c0b3040 2024-09-11T12:57:42.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.457+0000 7f3029400640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 0x7f300c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:42.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.457+0000 7f3029400640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 0x7f300c0744d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f302c1413e0 tx=0x7f3018006ca0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:42.460 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.461+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f301c0164d0 con 0x7f302c0b3040 2024-09-11T12:57:42.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.625+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f302c072670 con 0x7f302c0b3040 2024-09-11T12:57:42.627 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.625+0000 7f3022000640 1 -- 172.21.15.78:0/2540436839 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f301c0f83e0 con 0x7f302c0b3040 2024-09-11T12:57:42.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 msgr2=0x7f300c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 0x7f300c0744d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f302c1413e0 tx=0x7f3018006ca0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 msgr2=0x7f302c13f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f301c009850 tx=0x7f301c0040c0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 shutdown_connections 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f302c0cd770 0x7f302c13b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f300c072010 0x7f300c0744d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f302c0b3420 0x7f302c1541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 --2- 172.21.15.78:0/2540436839 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f302c0b3040 0x7f302c13f280 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 >> 172.21.15.78:0/2540436839 conn(0x7f302c071dd0 msgr2=0x7f302c1596b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 shutdown_connections 2024-09-11T12:57:42.630 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:42.629+0000 7f3031200640 1 -- 172.21.15.78:0/2540436839 wait complete. 2024-09-11T12:57:42.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:42.649 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp5170/subvol6260) 2024-09-11T12:57:42.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:42.677 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-11T12:57:42.733 INFO:teuthology.orchestra.run.smithi078.stdout:cephfs_data 2024-09-11T12:57:42.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:42.734 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp5170/subvol6260) 2024-09-11T12:57:42.803 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:42.804 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp5170/subvol6260/.meta 2024-09-11T12:57:42.842 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:57:42.851 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:57:42.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol6260 subvol_grp5170 2024-09-11T12:57:43.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- 172.21.15.78:0/1565521420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5460004810 msgr2=0x7f5460004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 --2- 172.21.15.78:0/1565521420 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5460004810 0x7f5460004c90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f547c151d20 tx=0x7f546801d1d0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547b400640 1 -- 172.21.15.78:0/1565521420 reap_dead start 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- 172.21.15.78:0/1565521420 shutdown_connections 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- 172.21.15.78:0/1565521420 >> 172.21.15.78:0/1565521420 conn(0x7f547c0b2c70 msgr2=0x7f547c0b3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- 172.21.15.78:0/1565521420 shutdown_connections 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- 172.21.15.78:0/1565521420 wait complete. 2024-09-11T12:57:43.081 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 Processor -- start 2024-09-11T12:57:43.082 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- start start 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 0x7f547c13b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c155150 0x7f547c13fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0cc970 con 0x7f547c1555d0 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f547c0cc7f0 con 0x7f5460004810 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f547c13e4b0 con 0x7f547c155150 2024-09-11T12:57:43.084 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547b400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47460/0 (socket says 172.21.15.78:47460) 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.081+0000 7f547b400640 1 -- 172.21.15.78:0/920613364 learned_addr learned my addr 172.21.15.78:0/920613364 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547aa00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 0x7f547c13b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547a000640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c155150 0x7f547c13fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 -- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c155150 msgr2=0x7f547c13fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c155150 0x7f547c13fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 -- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 msgr2=0x7f547c13b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 0x7f547c13b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 -- 172.21.15.78:0/920613364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f547c155e90 con 0x7f547c1555d0 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547aa00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 0x7f547c13b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547b400640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f54740011b0 tx=0x7f547400f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:43.085 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474013650 con 0x7f547c1555d0 2024-09-11T12:57:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54740138e0 con 0x7f547c1555d0 2024-09-11T12:57:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474013d30 con 0x7f547c1555d0 2024-09-11T12:57:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f547c156170 con 0x7f547c1555d0 2024-09-11T12:57:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f547c156730 con 0x7f547c1555d0 2024-09-11T12:57:43.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f547c13dda0 con 0x7f547c1555d0 2024-09-11T12:57:43.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.085+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7f5474020470 con 0x7f547c1555d0 2024-09-11T12:57:43.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.089+0000 7f5473400640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 0x7f545c0744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.089+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f5474094650 con 0x7f547c1555d0 2024-09-11T12:57:43.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.089+0000 7f547aa00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 0x7f545c0744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.089+0000 7f547aa00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 0x7f545c0744d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f546c0013e0 tx=0x7f546c006b30 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:43.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.089+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f54740620c0 con 0x7f547c1555d0 2024-09-11T12:57:43.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.285+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol6260", "group_name": "subvol_grp5170", "target": ["mon-mgr", ""]}) -- 0x7f547c158ca0 con 0x7f545c072010 2024-09-11T12:57:43.290 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:43.289+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:43.290 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:43.289+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:43.290 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:43.289+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:43.290 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:43.289+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:43.290 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:43.289+0000 7f1eccc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:43.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f5473400640 1 -- 172.21.15.78:0/920613364 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7f547c158ca0 con 0x7f545c072010 2024-09-11T12:57:43.333 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp5170/subvol6260 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 msgr2=0x7f545c0744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 0x7f545c0744d0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f546c0013e0 tx=0x7f546c006b30 comp rx=0 tx=0).stop 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 msgr2=0x7f547c13ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f54740011b0 tx=0x7f547400f6c0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 shutdown_connections 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f547c1555d0 0x7f547c13ff50 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f547c155150 0x7f547c13fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f545c072010 0x7f545c0744d0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 --2- 172.21.15.78:0/920613364 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f5460004810 0x7f547c13b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 >> 172.21.15.78:0/920613364 conn(0x7f547c0b2c70 msgr2=0x7f547c070b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:43.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 shutdown_connections 2024-09-11T12:57:43.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.333+0000 7f547be00640 1 -- 172.21.15.78:0/920613364 wait complete. 2024-09-11T12:57:43.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol6260 subvol_grp5170 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- 172.21.15.78:0/2477004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 msgr2=0x7fdb640723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2477004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb640723f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fdb54009bc0 tx=0x7fdb5401cba0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- 172.21.15.78:0/2477004722 shutdown_connections 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2477004722 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb640b5b70 0x7fdb64072930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2477004722 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb640723f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- 172.21.15.78:0/2477004722 >> 172.21.15.78:0/2477004722 conn(0x7fdb640b3050 msgr2=0x7fdb640b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- 172.21.15.78:0/2477004722 shutdown_connections 2024-09-11T12:57:43.501 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- 172.21.15.78:0/2477004722 wait complete. 2024-09-11T12:57:43.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 Processor -- start 2024-09-11T12:57:43.502 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- start start 2024-09-11T12:57:43.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 0x7fdb6414b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 0x7fdb64159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.504 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64167ad0 con 0x7fdb640b5790 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64167950 con 0x7fdb640b5b70 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb68c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdb64167c50 con 0x7fdb64151340 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.501+0000 7fdb61e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:47500/0 (socket says 172.21.15.78:47500) 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 -- 172.21.15.78:0/2052443654 learned_addr learned my addr 172.21.15.78:0/2052443654 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 -- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 msgr2=0x7fdb64159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb59400640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 0x7fdb6414b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb62800640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 0x7fdb64159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 0x7fdb64159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 -- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 msgr2=0x7fdb6414b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 0x7fdb6414b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 -- 172.21.15.78:0/2052443654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb6415c460 con 0x7fdb640b5790 2024-09-11T12:57:43.505 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb59400640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 0x7fdb6414b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb62800640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 0x7fdb64159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb61e00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdb54000f90 tx=0x7fdb54009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540b2040 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb54004d40 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb6415c740 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb540042a0 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb6415cd00 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb641517d0 con 0x7fdb640b5790 2024-09-11T12:57:43.506 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 24) ==== 95799+0+0 (secure 0 0 0) 0x7fdb540a5a80 con 0x7fdb640b5790 2024-09-11T12:57:43.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.505+0000 7fdb5aa00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 0x7fdb400744d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:43.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.509+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fdb54129e60 con 0x7fdb640b5790 2024-09-11T12:57:43.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.509+0000 7fdb59400640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 0x7fdb400744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:43.508 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.509+0000 7fdb59400640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 0x7fdb400744d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005fd0 tx=0x7fdb4c005e60 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:43.511 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.509+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fdb540a5e20 con 0x7fdb640b5790 2024-09-11T12:57:43.703 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.701+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol6260", "group_name": "subvol_grp5170", "target": ["mon-mgr", ""]}) -- 0x7fdb6406f100 con 0x7fdb40072010 2024-09-11T12:57:43.707 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb5aa00640 1 -- 172.21.15.78:0/2052443654 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+35 (secure 0 0 0) 0x7fdb6406f100 con 0x7fdb40072010 2024-09-11T12:57:43.707 INFO:teuthology.orchestra.run.smithi078.stdout:/volumes/subvol_grp5170/subvol6260 2024-09-11T12:57:43.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 msgr2=0x7fdb400744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.709 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 0x7fdb400744d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c005fd0 tx=0x7fdb4c005e60 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 msgr2=0x7fdb6415d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fdb54000f90 tx=0x7fdb54009280 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 shutdown_connections 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdb64151340 0x7fdb64159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdb40072010 0x7fdb400744d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdb640b5b70 0x7fdb6414b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 --2- 172.21.15.78:0/2052443654 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdb640b5790 0x7fdb6415d9d0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 >> 172.21.15.78:0/2052443654 conn(0x7fdb640b3050 msgr2=0x7fdb640ceb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 shutdown_connections 2024-09-11T12:57:43.710 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:43.709+0000 7fdb68c00640 1 -- 172.21.15.78:0/2052443654 wait complete. 2024-09-11T12:57:43.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:43.728 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/80c8e5a8fbf130806b0ca01dbc9a338f.meta) 2024-09-11T12:57:48.371 INFO:teuthology.orchestra.run.smithi078.stdout:[GLOBAL] 2024-09-11T12:57:48.372 INFO:teuthology.orchestra.run.smithi078.stdout:version = 1 2024-09-11T12:57:48.372 INFO:teuthology.orchestra.run.smithi078.stdout:type = subvolume 2024-09-11T12:57:48.372 INFO:teuthology.orchestra.run.smithi078.stdout:path = /volumes/subvol_grp5170/subvol6260 2024-09-11T12:57:48.372 INFO:teuthology.orchestra.run.smithi078.stdout:state = complete 2024-09-11T12:57:48.372 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:48.375 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol6260 subvol_grp5170 2024-09-11T12:57:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 -- 172.21.15.78:0/568077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 msgr2=0x7fca5c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 --2- 172.21.15.78:0/568077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca5c00b9d0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fca70001dc0 tx=0x7fca7001cb30 comp rx=0 tx=0).stop 2024-09-11T12:57:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 -- 172.21.15.78:0/568077223 shutdown_connections 2024-09-11T12:57:48.530 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 --2- 172.21.15.78:0/568077223 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca5c00b9d0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 -- 172.21.15.78:0/568077223 >> 172.21.15.78:0/568077223 conn(0x7fca780b3050 msgr2=0x7fca780b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 -- 172.21.15.78:0/568077223 shutdown_connections 2024-09-11T12:57:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.529+0000 7fca7f200640 1 -- 172.21.15.78:0/568077223 wait complete. 2024-09-11T12:57:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 Processor -- start 2024-09-11T12:57:48.531 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- start start 2024-09-11T12:57:48.533 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780ae200 0x7fca7814f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78153000 0x7fca781534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fca780cfb10 con 0x7fca5c0051d0 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fca780cf990 con 0x7fca78153000 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fca780cfc90 con 0x7fca780ae200 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51780/0 (socket says 172.21.15.78:51780) 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 -- 172.21.15.78:0/1155561458 learned_addr learned my addr 172.21.15.78:0/1155561458 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7d200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78153000 0x7fca781534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca6f400640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780ae200 0x7fca7814f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:48.534 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 -- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780ae200 msgr2=0x7fca7814f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780ae200 0x7fca7814f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 -- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78153000 msgr2=0x7fca781534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78153000 0x7fca781534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 -- 172.21.15.78:0/1155561458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca7814fc50 con 0x7fca5c0051d0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca77e00640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fca680066b0 tx=0x7fca680039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68011700 con 0x7fca5c0051d0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca68016020 con 0x7fca5c0051d0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca68011b40 con 0x7fca5c0051d0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca78156700 con 0x7fca5c0051d0 2024-09-11T12:57:48.535 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca78156cc0 con 0x7fca5c0051d0 2024-09-11T12:57:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.533+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca780b5c90 con 0x7fca5c0051d0 2024-09-11T12:57:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.537+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 95850+0+0 (secure 0 0 0) 0x7fca68011ce0 con 0x7fca5c0051d0 2024-09-11T12:57:48.536 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.537+0000 7fca74c00640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 0x7fca54074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.537+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fca68090fc0 con 0x7fca5c0051d0 2024-09-11T12:57:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.537+0000 7fca6f400640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 0x7fca54074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:48.537 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.537+0000 7fca6f400640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 0x7fca54074590 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca781501d0 tx=0x7fca600091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:48.541 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.541+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fca68020030 con 0x7fca5c0051d0 2024-09-11T12:57:48.733 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.733+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol6260", "group_name": "subvol_grp5170", "target": ["mon-mgr", ""]}) -- 0x7fca781f2c50 con 0x7fca540720d0 2024-09-11T12:57:48.763 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca74c00640 1 -- 172.21.15.78:0/1155561458 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fca781f2c50 con 0x7fca540720d0 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 msgr2=0x7fca54074590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 0x7fca54074590 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fca781501d0 tx=0x7fca600091e0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 msgr2=0x7fca781f2710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fca680066b0 tx=0x7fca680039d0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 shutdown_connections 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fca540720d0 0x7fca54074590 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fca78153000 0x7fca781534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fca780ae200 0x7fca7814f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 --2- 172.21.15.78:0/1155561458 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fca5c0051d0 0x7fca781f2710 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:48.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 >> 172.21.15.78:0/1155561458 conn(0x7fca780b3050 msgr2=0x7fca7815e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 shutdown_connections 2024-09-11T12:57:48.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48.765+0000 7fca7f200640 1 -- 172.21.15.78:0/1155561458 wait complete. 2024-09-11T12:57:48.769 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.769+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.770 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.769+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.770 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.769+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.770 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.769+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.770 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.769+0000 7f1ecf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.786 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.785+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.786 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.785+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.787 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.785+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.787 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.785+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.787 INFO:tasks.ceph.mgr.y.smithi078.stderr:2024-09-11T12:57:48.785+0000 7f1ecfe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-11T12:57:48.788 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-11T12:57:48.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-11T12:57:48.859 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-09-11T12:57:48.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-11T12:57:48.866 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp5170 2024-09-11T12:57:49.014 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 -- 172.21.15.78:0/711734071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf01518b0 msgr2=0x7fbbf00b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 --2- 172.21.15.78:0/711734071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf01518b0 0x7fbbf00b1a10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fbbe0009bc0 tx=0x7fbbe001cba0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 -- 172.21.15.78:0/711734071 shutdown_connections 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 --2- 172.21.15.78:0/711734071 >> v2:172.21.15.157:3300/0 conn(0x7fbbf00b1f50 0x7fbbf00cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 --2- 172.21.15.78:0/711734071 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf01518b0 0x7fbbf00b1a10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 -- 172.21.15.78:0/711734071 >> 172.21.15.78:0/711734071 conn(0x7fbbf0071dd0 msgr2=0x7fbbf00721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 -- 172.21.15.78:0/711734071 shutdown_connections 2024-09-11T12:57:49.015 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.013+0000 7fbbf5400640 1 -- 172.21.15.78:0/711734071 wait complete. 2024-09-11T12:57:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 Processor -- start 2024-09-11T12:57:49.016 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- start start 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 0x7fbbf01fb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00ce150 0x7fbbf015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf0167960 con 0x7fbbf01518b0 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf01677e0 con 0x7fbbf00b1f50 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf0167ae0 con 0x7fbbf00ce150 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbef200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbef200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51804/0 (socket says 172.21.15.78:51804) 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 0x7fbbf01fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbef200640 1 -- 172.21.15.78:0/3128160295 learned_addr learned my addr 172.21.15.78:0/3128160295 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbede00640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00ce150 0x7fbbf015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.018 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 -- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00ce150 msgr2=0x7fbbf015cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00ce150 0x7fbbf015cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 -- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 msgr2=0x7fbbf0138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 -- 172.21.15.78:0/3128160295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01fb8a0 con 0x7fbbf00b1f50 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbef200640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:57:49.019 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbee800640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 0x7fbbf01fb360 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00ac000 tx=0x7fbbe0009200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe00b2040 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe0004d40 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004220 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01f7750 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.017+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf01f7d10 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf00cdc10 con 0x7fbbf00b1f50 2024-09-11T12:57:49.020 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 25) ==== 95850+0+0 (secure 0 0 0) 0x7fbbe00a59c0 con 0x7fbbf00b1f50 2024-09-11T12:57:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbe6a00640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 0x7fbbd0074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fbbe012a5b0 con 0x7fbbf00b1f50 2024-09-11T12:57:49.021 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbede00640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 0x7fbbd0074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.022 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.021+0000 7fbbede00640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 0x7fbbd0074560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbf015b8c0 tx=0x7fbbdc0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:49.026 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.025+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbbe00bc060 con 0x7fbbf00b1f50 2024-09-11T12:57:49.216 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.217+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp5170", "target": ["mon-mgr", ""]}) -- 0x7fbbf00b3b70 con 0x7fbbd00720a0 2024-09-11T12:57:49.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbe6a00640 1 -- 172.21.15.78:0/3128160295 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fbbf00b3b70 con 0x7fbbd00720a0 2024-09-11T12:57:49.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 msgr2=0x7fbbd0074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 0x7fbbd0074560 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbbf015b8c0 tx=0x7fbbdc0091e0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 msgr2=0x7fbbf01fb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 0x7fbbf01fb360 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00ac000 tx=0x7fbbe0009200 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 shutdown_connections 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbbf01518b0 0x7fbbf0138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbbf00ce150 0x7fbbf015cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbbd00720a0 0x7fbbd0074560 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 --2- 172.21.15.78:0/3128160295 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbbf00b1f50 0x7fbbf01fb360 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 >> 172.21.15.78:0/3128160295 conn(0x7fbbf0071dd0 msgr2=0x7fbbf015c010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 shutdown_connections 2024-09-11T12:57:49.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.221+0000 7fbbf5400640 1 -- 172.21.15.78:0/3128160295 wait complete. 2024-09-11T12:57:49.242 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-11T12:57:49.242 DEBUG:teuthology.orchestra.run.smithi078:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:49.329 DEBUG:teuthology.orchestra.run.smithi157:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-11T12:57:49.432 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-11T12:57:49.432 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:49.432 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:49.432 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:49.441 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:49.441 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:49.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- 172.21.15.78:0/1399295409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b500b4780 msgr2=0x7f3b5015e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 --2- 172.21.15.78:0/1399295409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b500b4780 0x7f3b5015e1c0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009900 tx=0x7f3b4001c820 comp rx=0 tx=0).stop 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- 172.21.15.78:0/1399295409 shutdown_connections 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 --2- 172.21.15.78:0/1399295409 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b5015a1c0 0x7f3b500cc5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 --2- 172.21.15.78:0/1399295409 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b500b4780 0x7f3b5015e1c0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- 172.21.15.78:0/1399295409 >> 172.21.15.78:0/1399295409 conn(0x7f3b5006ca80 msgr2=0x7f3b500b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:49.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- 172.21.15.78:0/1399295409 shutdown_connections 2024-09-11T12:57:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- 172.21.15.78:0/1399295409 wait complete. 2024-09-11T12:57:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 Processor -- start 2024-09-11T12:57:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 -- start start 2024-09-11T12:57:49.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.677+0000 7f3b55a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b500b4780 0x7f3b50138910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b5015a1c0 0x7f3b50150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5013eb90 con 0x7f3b50074bd0 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3b5013ea10 con 0x7f3b5015a1c0 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3b5013ed10 con 0x7f3b500b4780 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51824/0 (socket says 172.21.15.78:51824) 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 -- 172.21.15.78:0/4118272585 learned_addr learned my addr 172.21.15.78:0/4118272585 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4f400640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b5015a1c0 0x7f3b50150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b46000640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b500b4780 0x7f3b50138910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 -- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b500b4780 msgr2=0x7f3b50138910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b500b4780 0x7f3b50138910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.680 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 -- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b5015a1c0 msgr2=0x7f3b50150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b5015a1c0 0x7f3b50150710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 -- 172.21.15.78:0/4118272585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b50138e50 con 0x7f3b50074bd0 2024-09-11T12:57:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b4ea00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b0040 tx=0x7f3b40004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:49.681 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400b2040 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b40005070 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b400044f0 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b501390f0 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b501396b0 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b5013f2b0 con 0x7f3b50074bd0 2024-09-11T12:57:49.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.681+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 25) ==== 95850+0+0 (secure 0 0 0) 0x7f3b400054a0 con 0x7f3b50074bd0 2024-09-11T12:57:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.685+0000 7f3b47400640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 0x7f3b2c074560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.685+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f3b4012c6a0 con 0x7f3b50074bd0 2024-09-11T12:57:49.684 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.685+0000 7f3b46000640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 0x7f3b2c074560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:49.685 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.685+0000 7f3b46000640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 0x7f3b2c074560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3b501350e0 tx=0x7f3b38009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:49.688 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.689+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3b400164d0 con 0x7f3b50074bd0 2024-09-11T12:57:49.867 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f3b50159840 con 0x7f3b50074bd0 2024-09-11T12:57:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b47400640 1 -- 172.21.15.78:0/4118272585 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) ==== 80+0+0 (secure 0 0 0) 0x7f3b400f70a0 con 0x7f3b50074bd0 2024-09-11T12:57:49.868 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 msgr2=0x7f3b2c074560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 0x7f3b2c074560 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3b501350e0 tx=0x7f3b38009040 comp rx=0 tx=0).stop 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 msgr2=0x7f3b5015cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f3b400b0040 tx=0x7f3b40004040 comp rx=0 tx=0).stop 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 shutdown_connections 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f3b2c0720a0 0x7f3b2c074560 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f3b5015a1c0 0x7f3b50150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.870 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f3b500b4780 0x7f3b50138910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 --2- 172.21.15.78:0/4118272585 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3b50074bd0 0x7f3b5015cf90 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 >> 172.21.15.78:0/4118272585 conn(0x7f3b5006ca80 msgr2=0x7f3b500b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 shutdown_connections 2024-09-11T12:57:49.871 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:49.869+0000 7f3b55a00640 1 -- 172.21.15.78:0/4118272585 wait complete. 2024-09-11T12:57:49.889 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-11T12:57:49.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:49.890 DEBUG:teuthology.orchestra.run.smithi078:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-11T12:57:49.957 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:57:49.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:49.957 DEBUG:teuthology.orchestra.run.smithi078:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-11T12:57:49.979 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:49.979 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:49.979 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:49.988 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:49.988 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:50.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 -- 172.21.15.78:0/2591200495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f011c0051d0 msgr2=0x7f011c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 --2- 172.21.15.78:0/2591200495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f011c0051d0 0x7f011c00b9d0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f013000c260 tx=0x7f013001cf40 comp rx=0 tx=0).stop 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 -- 172.21.15.78:0/2591200495 shutdown_connections 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 --2- 172.21.15.78:0/2591200495 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f011c0051d0 0x7f011c00b9d0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 -- 172.21.15.78:0/2591200495 >> 172.21.15.78:0/2591200495 conn(0x7f013806ca80 msgr2=0x7f01380b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 -- 172.21.15.78:0/2591200495 shutdown_connections 2024-09-11T12:57:50.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.181+0000 7f013ea00640 1 -- 172.21.15.78:0/2591200495 wait complete. 2024-09-11T12:57:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 Processor -- start 2024-09-11T12:57:50.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- start start 2024-09-11T12:57:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011c0051d0 0x7f013815bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.184 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01381575c0 0x7f0138158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f013815b040 con 0x7f01381f7910 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f013815aec0 con 0x7f011c0051d0 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f013815b1c0 con 0x7f01381575c0 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011c0051d0 0x7f013815bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51832/0 (socket says 172.21.15.78:51832) 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 -- 172.21.15.78:0/2974681339 learned_addr learned my addr 172.21.15.78:0/2974681339 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:50.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ca00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01381575c0 0x7f0138158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 -- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01381575c0 msgr2=0x7f0138158760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.186 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01381575c0 0x7f0138158760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 -- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011c0051d0 msgr2=0x7f013815bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011c0051d0 0x7f013815bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 -- 172.21.15.78:0/2974681339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0138157ec0 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f0137400640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0124011ab0 tx=0x7f0124011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012401e070 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f012400f940 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01381581a0 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f012400fd30 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0138157a50 con 0x7f01381f7910 2024-09-11T12:57:50.187 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.185+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0138070320 con 0x7f01381f7910 2024-09-11T12:57:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.189+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f0124021070 con 0x7f01381f7910 2024-09-11T12:57:50.188 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.189+0000 7f012fe00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 0x7f01180746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.189+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f0124091390 con 0x7f01381f7910 2024-09-11T12:57:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.189+0000 7f0137e00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 0x7f01180746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.189 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.189+0000 7f0137e00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 0x7f01180746b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f013815df60 tx=0x7f0128006f00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:50.192 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.193+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f012405e3b0 con 0x7f01381f7910 2024-09-11T12:57:50.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f01380ae3f0 con 0x7f01381f7910 2024-09-11T12:57:50.372 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f012fe00640 1 -- 172.21.15.78:0/2974681339 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v162) ==== 80+0+0 (secure 0 0 0) 0x7f0124062900 con 0x7f01381f7910 2024-09-11T12:57:50.372 INFO:teuthology.orchestra.run.smithi078.stderr:listed 0 entries 2024-09-11T12:57:50.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 msgr2=0x7f01180746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.374 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 0x7f01180746b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f013815df60 tx=0x7f0128006f00 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 msgr2=0x7f0138154c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f0124011ab0 tx=0x7f0124011f70 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 shutdown_connections 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f01381575c0 0x7f0138158760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f01381f7910 0x7f0138154c40 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f01180721f0 0x7f01180746b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 --2- 172.21.15.78:0/2974681339 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f011c0051d0 0x7f013815bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 >> 172.21.15.78:0/2974681339 conn(0x7f013806ca80 msgr2=0x7f01380b44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 shutdown_connections 2024-09-11T12:57:50.375 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.373+0000 7f013ea00640 1 -- 172.21.15.78:0/2974681339 wait complete. 2024-09-11T12:57:50.395 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-11T12:57:50.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:50.395 DEBUG:teuthology.orchestra.run.smithi157:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-11T12:57:50.420 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:57:50.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:50.421 DEBUG:teuthology.orchestra.run.smithi157:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-11T12:57:50.473 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:50.474 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:50.474 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:50.483 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:50.483 INFO:tasks.cephfs.mount:Terminating background process 2024-09-11T12:57:50.483 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:50.483 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:50.525 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:50.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- 172.21.15.78:0/2390289135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300159fc0 msgr2=0x7fe3000706f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- 172.21.15.78:0/2390289135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300159fc0 0x7fe3000706f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400a950 tx=0x7fe2f401db70 comp rx=0 tx=0).stop 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- 172.21.15.78:0/2390289135 shutdown_connections 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- 172.21.15.78:0/2390289135 >> v2:172.21.15.157:3300/0 conn(0x7fe3000b4db0 0x7fe3000b5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- 172.21.15.78:0/2390289135 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300159fc0 0x7fe3000706f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- 172.21.15.78:0/2390289135 >> 172.21.15.78:0/2390289135 conn(0x7fe30006ca80 msgr2=0x7fe3000b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- 172.21.15.78:0/2390289135 shutdown_connections 2024-09-11T12:57:50.668 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- 172.21.15.78:0/2390289135 wait complete. 2024-09-11T12:57:50.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 Processor -- start 2024-09-11T12:57:50.669 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- start start 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe300159fc0 0x7fe3001424d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300145760 0x7fe3000c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe30006f760 con 0x7fe300159fc0 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe30006f5e0 con 0x7fe3000b4db0 2024-09-11T12:57:50.671 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe308800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe30006f8e0 con 0x7fe300145760 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe300159fc0 0x7fe3001424d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:46174/0 (socket says 172.21.15.78:46174) 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 -- 172.21.15.78:0/3702576016 learned_addr learned my addr 172.21.15.78:0/3702576016 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe306800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300145760 0x7fe3000c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 -- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300145760 msgr2=0x7fe3000c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300145760 0x7fe3000c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.669+0000 7fe305e00640 1 -- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe300159fc0 msgr2=0x7fe3001424d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe305e00640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe300159fc0 0x7fe3001424d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe305e00640 1 -- 172.21.15.78:0/3702576016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3000c6820 con 0x7fe3000b4db0 2024-09-11T12:57:50.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe305e00640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400a8a0 tx=0x7fe2f4005db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40b3040 con 0x7fe3000b4db0 2024-09-11T12:57:50.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2f40053e0 con 0x7fe3000b4db0 2024-09-11T12:57:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe300142a10 con 0x7fe3000b4db0 2024-09-11T12:57:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f40046d0 con 0x7fe3000b4db0 2024-09-11T12:57:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3000c5520 con 0x7fe3000b4db0 2024-09-11T12:57:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3000b5220 con 0x7fe3000b4db0 2024-09-11T12:57:50.674 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fe2f40bf060 con 0x7fe3000b4db0 2024-09-11T12:57:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 0x7fe2e4074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fe2f412a5e0 con 0x7fe3000b4db0 2024-09-11T12:57:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.673+0000 7fe305400640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 0x7fe2e4074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:50.675 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.677+0000 7fe305400640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 0x7fe2e4074680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe3000c6e90 tx=0x7fe2f0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:50.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.677+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe2f4017760 con 0x7fe3000b4db0 2024-09-11T12:57:50.880 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.881+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe3000b1d40 con 0x7fe3000b4db0 2024-09-11T12:57:50.881 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.881+0000 7fe2fa000640 1 -- 172.21.15.78:0/3702576016 <== mon.1 v2:172.21.15.157:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5230 (secure 0 0 0) 0x7fe2f40f7850 con 0x7fe3000b4db0 2024-09-11T12:57:50.882 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:50.882 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":122,"btime":"2024-09-11T12:57:41:908935+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:50.882 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 122 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 msgr2=0x7fe2e4074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 0x7fe2e4074680 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe3000c6e90 tx=0x7fe2f0009040 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 msgr2=0x7fe3000c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fe2f400a8a0 tx=0x7fe2f4005db0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 shutdown_connections 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe300145760 0x7fe3000c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe300159fc0 0x7fe3001424d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe2e40721c0 0x7fe2e4074680 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 --2- 172.21.15.78:0/3702576016 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3000b4db0 0x7fe3000c62e0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:50.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 >> 172.21.15.78:0/3702576016 conn(0x7fe30006ca80 msgr2=0x7fe300154c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 shutdown_connections 2024-09-11T12:57:50.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:50.885+0000 7fe308800640 1 -- 172.21.15.78:0/3702576016 wait complete. 2024-09-11T12:57:50.904 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 -- 172.21.15.78:0/2596857642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640b4780 msgr2=0x7fea64073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 --2- 172.21.15.78:0/2596857642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640b4780 0x7fea64073f70 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fea54009c80 tx=0x7fea5401cd60 comp rx=0 tx=0).stop 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 -- 172.21.15.78:0/2596857642 shutdown_connections 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 --2- 172.21.15.78:0/2596857642 >> v2:172.21.15.157:3300/0 conn(0x7fea6415bc60 0x7fea6406ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 --2- 172.21.15.78:0/2596857642 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640b4780 0x7fea64073f70 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 -- 172.21.15.78:0/2596857642 >> 172.21.15.78:0/2596857642 conn(0x7fea6406ca80 msgr2=0x7fea640b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 -- 172.21.15.78:0/2596857642 shutdown_connections 2024-09-11T12:57:51.047 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.045+0000 7fea69800640 1 -- 172.21.15.78:0/2596857642 wait complete. 2024-09-11T12:57:51.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 Processor -- start 2024-09-11T12:57:51.048 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- start start 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea640b4780 0x7fea64144800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6415bc60 0x7fea640b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fea64168b90 con 0x7fea640ccb20 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fea64168a10 con 0x7fea6415bc60 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fea64168d10 con 0x7fea640b4780 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.050 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea63400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6415bc60 0x7fea640b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea640b4780 0x7fea64144800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51866/0 (socket says 172.21.15.78:51866) 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 -- 172.21.15.78:0/2696429779 learned_addr learned my addr 172.21.15.78:0/2696429779 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 -- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea640b4780 msgr2=0x7fea64144800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea640b4780 0x7fea64144800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 -- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6415bc60 msgr2=0x7fea640b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6415bc60 0x7fea640b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.051 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 -- 172.21.15.78:0/2696429779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea64144d40 con 0x7fea640ccb20 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea62000640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fea5000c400 tx=0x7fea5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50017070 con 0x7fea640ccb20 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea500053c0 con 0x7fea640ccb20 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea641465d0 con 0x7fea640ccb20 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea50003880 con 0x7fea640ccb20 2024-09-11T12:57:51.052 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.049+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea64159620 con 0x7fea640ccb20 2024-09-11T12:57:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea64073f70 con 0x7fea640ccb20 2024-09-11T12:57:51.053 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fea5001a070 con 0x7fea640ccb20 2024-09-11T12:57:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea5aa00640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 0x7fea44074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fea5008fda0 con 0x7fea640ccb20 2024-09-11T12:57:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea62a00640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 0x7fea44074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.054 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.053+0000 7fea62a00640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 0x7fea44074680 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea540af000 tx=0x7fea5401b5a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.057+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fea50001130 con 0x7fea640ccb20 2024-09-11T12:57:51.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea641579f0 con 0x7fea640ccb20 2024-09-11T12:57:51.256 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea5aa00640 1 -- 172.21.15.78:0/2696429779 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5230 (secure 0 0 0) 0x7fea5005cdc0 con 0x7fea640ccb20 2024-09-11T12:57:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:51.256 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":122,"btime":"2024-09-11T12:57:41:908935+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:51.257 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 122 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 msgr2=0x7fea44074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 0x7fea44074680 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fea540af000 tx=0x7fea5401b5a0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 msgr2=0x7fea640b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fea5000c400 tx=0x7fea5000c8c0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 shutdown_connections 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fea440721c0 0x7fea44074680 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fea6415bc60 0x7fea640b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fea640ccb20 0x7fea640b10d0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 --2- 172.21.15.78:0/2696429779 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fea640b4780 0x7fea64144800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 >> 172.21.15.78:0/2696429779 conn(0x7fea6406ca80 msgr2=0x7fea6415c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.259 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 shutdown_connections 2024-09-11T12:57:51.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.257+0000 7fea69800640 1 -- 172.21.15.78:0/2696429779 wait complete. 2024-09-11T12:57:51.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 -- 172.21.15.78:0/481894978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa748073c70 msgr2=0x7fa74815ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 --2- 172.21.15.78:0/481894978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa748073c70 0x7fa74815ae40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fa740009840 tx=0x7fa74001c820 comp rx=0 tx=0).stop 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 -- 172.21.15.78:0/481894978 shutdown_connections 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 --2- 172.21.15.78:0/481894978 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa748073c70 0x7fa74815ae40 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 -- 172.21.15.78:0/481894978 >> 172.21.15.78:0/481894978 conn(0x7fa74806f3a0 msgr2=0x7fa7480cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 -- 172.21.15.78:0/481894978 shutdown_connections 2024-09-11T12:57:51.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.421+0000 7fa74ee00640 1 -- 172.21.15.78:0/481894978 wait complete. 2024-09-11T12:57:51.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 Processor -- start 2024-09-11T12:57:51.423 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- start start 2024-09-11T12:57:51.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa748073c70 0x7fa748151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.425 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7481518f0 0x7fa7480c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa748168960 con 0x7fa7480cfeb0 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7481687e0 con 0x7fa7481518f0 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa748168ae0 con 0x7fa748073c70 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51886/0 (socket says 172.21.15.78:51886) 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 -- 172.21.15.78:0/2872971996 learned_addr learned my addr 172.21.15.78:0/2872971996 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747e00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa748073c70 0x7fa748151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ce00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7481518f0 0x7fa7480c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 -- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa748073c70 msgr2=0x7fa748151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa748073c70 0x7fa748151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 -- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7481518f0 msgr2=0x7fa7480c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7481518f0 0x7fa7480c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.426 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 -- 172.21.15.78:0/2872971996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7480c3030 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa747400640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c500 tx=0x7fa73400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734017070 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa7340053c0 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa734012890 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa748135a30 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa74814f4e0 con 0x7fa7480cfeb0 2024-09-11T12:57:51.427 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.425+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa74815ae40 con 0x7fa7480cfeb0 2024-09-11T12:57:51.428 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.429+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fa73401a070 con 0x7fa7480cfeb0 2024-09-11T12:57:51.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.429+0000 7fa73fe00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 0x7fa728074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.429+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fa73408fcc0 con 0x7fa7480cfeb0 2024-09-11T12:57:51.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.429+0000 7fa747e00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 0x7fa728074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.429 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.429+0000 7fa747e00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 0x7fa728074680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7380017b0 tx=0x7fa738001740 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.432 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.433+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa73405ccc0 con 0x7fa7480cfeb0 2024-09-11T12:57:51.601 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.601+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7481520e0 con 0x7fa7480cfeb0 2024-09-11T12:57:51.602 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.601+0000 7fa73fe00640 1 -- 172.21.15.78:0/2872971996 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16743 (secure 0 0 0) 0x7fa734061210 con 0x7fa7480cfeb0 2024-09-11T12:57:51.602 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:51.602 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":162,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:33.860596+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:57:29.938274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-11T12:57:31.215025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 msgr2=0x7fa728074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 0x7fa728074680 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa7380017b0 tx=0x7fa738001740 comp rx=0 tx=0).stop 2024-09-11T12:57:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 msgr2=0x7fa7481354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fa73400c500 tx=0x7fa73400c9c0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.604 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 shutdown_connections 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7280721c0 0x7fa728074680 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7481518f0 0x7fa7480c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa7480cfeb0 0x7fa7481354f0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 --2- 172.21.15.78:0/2872971996 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa748073c70 0x7fa748151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 >> 172.21.15.78:0/2872971996 conn(0x7fa74806f3a0 msgr2=0x7fa74815c630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 shutdown_connections 2024-09-11T12:57:51.605 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.605+0000 7fa74ee00640 1 -- 172.21.15.78:0/2872971996 wait complete. 2024-09-11T12:57:51.623 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:57:51.624 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:51.765 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- 172.21.15.78:0/491058974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 msgr2=0x7f26801616f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 --2- 172.21.15.78:0/491058974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f26801616f0 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f266c009900 tx=0x7f266c01c8b0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- 172.21.15.78:0/491058974 shutdown_connections 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 --2- 172.21.15.78:0/491058974 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f26801616f0 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- 172.21.15.78:0/491058974 >> 172.21.15.78:0/491058974 conn(0x7f268006f3a0 msgr2=0x7f26800cf090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- 172.21.15.78:0/491058974 shutdown_connections 2024-09-11T12:57:51.766 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- 172.21.15.78:0/491058974 wait complete. 2024-09-11T12:57:51.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 Processor -- start 2024-09-11T12:57:51.767 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.765+0000 7f2684e00640 1 -- start start 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 0x7f2680155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 0x7f2680156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f268016c430 con 0x7f26800c44d0 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f268016c2b0 con 0x7f26800cf7a0 2024-09-11T12:57:51.768 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f268016c5b0 con 0x7f2680154be0 2024-09-11T12:57:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51916/0 (socket says 172.21.15.78:51916) 2024-09-11T12:57:51.769 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 -- 172.21.15.78:0/1096133904 learned_addr learned my addr 172.21.15.78:0/1096133904 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267d400640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 0x7f2680155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267e800640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 0x7f2680156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 -- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 msgr2=0x7f2680156280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.770 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 0x7f2680156280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 -- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 msgr2=0x7f2680155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 0x7f2680155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 -- 172.21.15.78:0/1096133904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f268015b880 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267d400640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 0x7f2680155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267e800640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 0x7f2680156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f267de00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f267000c3c0 tx=0x7f267000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670017070 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f26700053c0 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f268014f460 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2670003860 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26801535b0 con 0x7f26800c44d0 2024-09-11T12:57:51.771 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.769+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26801533b0 con 0x7f26800c44d0 2024-09-11T12:57:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.773+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f267001a070 con 0x7f26800c44d0 2024-09-11T12:57:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.773+0000 7f2676000640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 0x7f2660074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.773+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f267008fd40 con 0x7f26800c44d0 2024-09-11T12:57:51.772 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.773+0000 7f267d400640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 0x7f2660074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:51.773 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.773+0000 7f267d400640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 0x7f2660074680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f266c0af000 tx=0x7f266c01b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:51.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.777+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f267005d3b0 con 0x7f26800c44d0 2024-09-11T12:57:51.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.977+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f268015d770 con 0x7f26800c44d0 2024-09-11T12:57:51.978 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.977+0000 7f2676000640 1 -- 172.21.15.78:0/1096133904 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5230 (secure 0 0 0) 0x7f267005ccb0 con 0x7f26800c44d0 2024-09-11T12:57:51.978 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:51.978 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":122,"btime":"2024-09-11T12:57:41:908935+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:51.979 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 122 2024-09-11T12:57:51.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 msgr2=0x7f2660074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.980 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 0x7f2660074680 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f266c0af000 tx=0x7f266c01b0f0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 msgr2=0x7f268014ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f267000c3c0 tx=0x7f267000c880 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 shutdown_connections 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2680154be0 0x7f2680156280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f26600721c0 0x7f2660074680 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f26800cf7a0 0x7f2680155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 --2- 172.21.15.78:0/1096133904 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f26800c44d0 0x7f268014ef20 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 >> 172.21.15.78:0/1096133904 conn(0x7f268006f3a0 msgr2=0x7f268015c160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 shutdown_connections 2024-09-11T12:57:51.981 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:51.981+0000 7f2684e00640 1 -- 172.21.15.78:0/1096133904 wait complete. 2024-09-11T12:57:52.000 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:57:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- 172.21.15.78:0/1112391656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82400cd9b0 msgr2=0x7f824006ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- 172.21.15.78:0/1112391656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82400cd9b0 0x7f824006ec50 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f822c009900 tx=0x7f822c01c850 comp rx=0 tx=0).stop 2024-09-11T12:57:52.140 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- 172.21.15.78:0/1112391656 shutdown_connections 2024-09-11T12:57:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- 172.21.15.78:0/1112391656 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400b3250 0x7f82400b36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- 172.21.15.78:0/1112391656 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f82400cd9b0 0x7f824006ec50 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- 172.21.15.78:0/1112391656 >> 172.21.15.78:0/1112391656 conn(0x7f82400cfeb0 msgr2=0x7f82400d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- 172.21.15.78:0/1112391656 shutdown_connections 2024-09-11T12:57:52.141 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- 172.21.15.78:0/1112391656 wait complete. 2024-09-11T12:57:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 Processor -- start 2024-09-11T12:57:52.142 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- start start 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82400b3250 0x7f824014a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400cd9b0 0x7f82401f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f82401679a0 con 0x7f824015a6d0 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f8240167820 con 0x7f82400b3250 2024-09-11T12:57:52.144 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.141+0000 7f8246600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f8240167b20 con 0x7f82400cd9b0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51934/0 (socket says 172.21.15.78:51934) 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 -- 172.21.15.78:0/4044444527 learned_addr learned my addr 172.21.15.78:0/4044444527 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245200640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82400b3250 0x7f824014a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f823fe00640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400cd9b0 0x7f82401f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 -- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400cd9b0 msgr2=0x7f82401f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400cd9b0 0x7f82401f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 -- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82400b3250 msgr2=0x7f824014a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82400b3250 0x7f824014a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 -- 172.21.15.78:0/4044444527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8240153160 con 0x7f824015a6d0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8245c00640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f823800ea20 tx=0x7f823800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f823800cde0 con 0x7f824015a6d0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8238004c40 con 0x7f824015a6d0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8238005280 con 0x7f824015a6d0 2024-09-11T12:57:52.145 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8240153440 con 0x7f824015a6d0 2024-09-11T12:57:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8240153a00 con 0x7f824015a6d0 2024-09-11T12:57:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82400b36e0 con 0x7f824015a6d0 2024-09-11T12:57:52.146 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.145+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f8238005420 con 0x7f824015a6d0 2024-09-11T12:57:52.147 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.149+0000 7f823cc00640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 0x7f8220074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.149+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f82380911c0 con 0x7f824015a6d0 2024-09-11T12:57:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.149+0000 7f8245200640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 0x7f8220074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.148 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.149+0000 7f8245200640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 0x7f8220074680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8240152920 tx=0x7f8230009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.151 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.153+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f8238016030 con 0x7f824015a6d0 2024-09-11T12:57:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.349+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8240150dd0 con 0x7f824015a6d0 2024-09-11T12:57:52.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.349+0000 7f823cc00640 1 -- 172.21.15.78:0/4044444527 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) ==== 95+0+5230 (secure 0 0 0) 0x7f823805e1e0 con 0x7f824015a6d0 2024-09-11T12:57:52.351 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:52.351 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":122,"btime":"2024-09-11T12:57:41:908935+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-11T12:57:32.239557+0000","modified":"2024-09-11T12:57:35.275136+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9045},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9045":{"gid":9045,"name":"d","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.157:6835/70117498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":70117498},{"type":"v1","addr":"172.21.15.157:6835","nonce":70117498}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9045,"qdb_cluster":[9045]},"id":13}]} 2024-09-11T12:57:52.351 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 122 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 msgr2=0x7f8220074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 0x7f8220074680 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8240152920 tx=0x7f8230009040 comp rx=0 tx=0).stop 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 msgr2=0x7f82400ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f823800ea20 tx=0x7f823800eee0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 shutdown_connections 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f824015a6d0 0x7f82400ad8a0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.353 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f82400cd9b0 0x7f82401f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f82200721c0 0x7f8220074680 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 --2- 172.21.15.78:0/4044444527 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f82400b3250 0x7f824014a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 >> 172.21.15.78:0/4044444527 conn(0x7f82400cfeb0 msgr2=0x7f824015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 shutdown_connections 2024-09-11T12:57:52.354 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.353+0000 7f8246600640 1 -- 172.21.15.78:0/4044444527 wait complete. 2024-09-11T12:57:52.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:57:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- 172.21.15.78:0/2972842306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe348070190 msgr2=0x7fe34815b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.512 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- 172.21.15.78:0/2972842306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe348070190 0x7fe34815b350 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe340009c80 tx=0x7fe34001cda0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- 172.21.15.78:0/2972842306 shutdown_connections 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- 172.21.15.78:0/2972842306 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe348070190 0x7fe34815b350 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- 172.21.15.78:0/2972842306 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070910 0x7fe34806fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- 172.21.15.78:0/2972842306 >> 172.21.15.78:0/2972842306 conn(0x7fe34806ca80 msgr2=0x7fe3480b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- 172.21.15.78:0/2972842306 shutdown_connections 2024-09-11T12:57:52.513 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- 172.21.15.78:0/2972842306 wait complete. 2024-09-11T12:57:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 Processor -- start 2024-09-11T12:57:52.514 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- start start 2024-09-11T12:57:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070190 0x7fe3481400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.516 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3480741e0 0x7fe34806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe348167eb0 con 0x7fe348070910 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe34815beb0 con 0x7fe3480741e0 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.513+0000 7fe34f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe348168030 con 0x7fe348070190 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51944/0 (socket says 172.21.15.78:51944) 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 -- 172.21.15.78:0/3517866113 learned_addr learned my addr 172.21.15.78:0/3517866113 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe34d600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3480741e0 0x7fe34806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe34cc00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070190 0x7fe3481400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 -- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070190 msgr2=0x7fe3481400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070190 0x7fe3481400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 -- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3480741e0 msgr2=0x7fe34806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3480741e0 0x7fe34806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 -- 172.21.15.78:0/3517866113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34815ff20 con 0x7fe348070910 2024-09-11T12:57:52.517 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe347e00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe33400e9c0 tx=0x7fe33400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33400cde0 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe334010520 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334010970 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe348160200 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3481407d0 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34815b350 con 0x7fe348070910 2024-09-11T12:57:52.518 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.517+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fe33401f020 con 0x7fe348070910 2024-09-11T12:57:52.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.521+0000 7fe344c00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 0x7fe328074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.521+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7fe334090db0 con 0x7fe348070910 2024-09-11T12:57:52.519 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.521+0000 7fe34cc00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 0x7fe328074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.520 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.521+0000 7fe34cc00640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 0x7fe328074680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe338007fd0 tx=0x7fe338009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.523 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.525+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fe33405e020 con 0x7fe348070910 2024-09-11T12:57:52.689 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.689+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe348072480 con 0x7fe348070910 2024-09-11T12:57:52.690 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.689+0000 7fe344c00640 1 -- 172.21.15.78:0/3517866113 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) ==== 74+0+16743 (secure 0 0 0) 0x7fe334062570 con 0x7fe348070910 2024-09-11T12:57:52.690 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:52.691 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":162,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:33.860596+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-11T12:57:29.938274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"161","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-11T12:57:31.215025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"162","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:57:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 msgr2=0x7fe328074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 0x7fe328074680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe338007fd0 tx=0x7fe338009040 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 msgr2=0x7fe34815f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fe33400e9c0 tx=0x7fe33400ee80 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 shutdown_connections 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe3280721c0 0x7fe328074680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe3480741e0 0x7fe34806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe348070910 0x7fe34815f5b0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 --2- 172.21.15.78:0/3517866113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe348070190 0x7fe3481400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 >> 172.21.15.78:0/3517866113 conn(0x7fe34806ca80 msgr2=0x7fe348073380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 shutdown_connections 2024-09-11T12:57:52.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.693+0000 7fe34f600640 1 -- 172.21.15.78:0/3517866113 wait complete. 2024-09-11T12:57:52.712 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-11T12:57:52.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 -- 172.21.15.78:0/3787098783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f40b54c0 msgr2=0x7f17f40736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 --2- 172.21.15.78:0/3787098783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f40b54c0 0x7f17f40736d0 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f17ec00a950 tx=0x7f17ec01dae0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 -- 172.21.15.78:0/3787098783 shutdown_connections 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 --2- 172.21.15.78:0/3787098783 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f40b54c0 0x7f17f40736d0 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 -- 172.21.15.78:0/3787098783 >> 172.21.15.78:0/3787098783 conn(0x7f17f406ca80 msgr2=0x7f17f40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 -- 172.21.15.78:0/3787098783 shutdown_connections 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.853+0000 7f17fac00640 1 -- 172.21.15.78:0/3787098783 wait complete. 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 Processor -- start 2024-09-11T12:57:52.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- start start 2024-09-11T12:57:52.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 0x7f17f414b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 0x7f17f414c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f17f406f420 con 0x7f17f4157ac0 2024-09-11T12:57:52.858 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f17f406f2a0 con 0x7f17f40b54c0 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f17f406f5a0 con 0x7f17f4070a00 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51950/0 (socket says 172.21.15.78:51950) 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 -- 172.21.15.78:0/194802405 learned_addr learned my addr 172.21.15.78:0/194802405 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f3e00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 0x7f17f414b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f3400640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 0x7f17f414c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 -- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 msgr2=0x7f17f414b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 0x7f17f414b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 -- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 msgr2=0x7f17f414c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 0x7f17f414c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 -- 172.21.15.78:0/194802405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f41f8b60 con 0x7f17f4157ac0 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f3e00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 0x7f17f414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:52.859 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f3400640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 0x7f17f414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:52.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17f8c00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f17ec01a6f0 tx=0x7f17ec0a69c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec0b3040 con 0x7f17f4157ac0 2024-09-11T12:57:52.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17ec005230 con 0x7f17f4157ac0 2024-09-11T12:57:52.860 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f414b980 con 0x7f17f4157ac0 2024-09-11T12:57:52.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ec003700 con 0x7f17f4157ac0 2024-09-11T12:57:52.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.857+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f41f90b0 con 0x7f17f4157ac0 2024-09-11T12:57:52.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17f40736d0 con 0x7f17f4157ac0 2024-09-11T12:57:52.861 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f17ec0038a0 con 0x7f17f4157ac0 2024-09-11T12:57:52.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17ebe00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 0x7f17d4074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:52.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 5709+0+0 (secure 0 0 0) 0x7f17ec12af90 con 0x7f17f4157ac0 2024-09-11T12:57:52.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17f3e00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 0x7f17d4074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:52.862 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.861+0000 7f17f3e00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 0x7f17d4074680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17e4004a40 tx=0x7f17e4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:52.865 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:52.865+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f17ec017760 con 0x7f17f4157ac0 2024-09-11T12:57:53.067 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.069+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f17f4155da0 con 0x7f17f4157ac0 2024-09-11T12:57:53.068 INFO:tasks.ceph.mon.a.smithi078.stderr:2024-09-11T12:57:53.069+0000 7f717ca00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-11T12:57:53.070 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.069+0000 7f17ebe00640 1 -- 172.21.15.78:0/194802405 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v123) ==== 161+0+0 (secure 0 0 0) 0x7f17ec0f8540 con 0x7f17f4157ac0 2024-09-11T12:57:53.070 INFO:teuthology.orchestra.run.smithi078.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 msgr2=0x7f17d4074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 0x7f17d4074680 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f17e4004a40 tx=0x7f17e4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 msgr2=0x7f17f40b0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f17ec01a6f0 tx=0x7f17ec0a69c0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 shutdown_connections 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f17f4157ac0 0x7f17f40b0970 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f17d40721c0 0x7f17d4074680 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f17f40b54c0 0x7f17f414c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 --2- 172.21.15.78:0/194802405 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f17f4070a00 0x7f17f414b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 >> 172.21.15.78:0/194802405 conn(0x7f17f406ca80 msgr2=0x7f17f415dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:53.074 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 shutdown_connections 2024-09-11T12:57:53.075 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.073+0000 7f17fac00640 1 -- 172.21.15.78:0/194802405 wait complete. 2024-09-11T12:57:53.094 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-11T12:57:53.140 INFO:tasks.ceph.mds.d.smithi157.stdout:starting mds.d at 2024-09-11T12:57:53.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- 172.21.15.78:0/2368665804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 msgr2=0x7fc4fc072220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:53.237 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 --2- 172.21.15.78:0/2368665804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc072220 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fc4f4009900 tx=0x7fc4f401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- 172.21.15.78:0/2368665804 shutdown_connections 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 --2- 172.21.15.78:0/2368665804 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc072220 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 --2- 172.21.15.78:0/2368665804 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc06f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- 172.21.15.78:0/2368665804 >> 172.21.15.78:0/2368665804 conn(0x7fc4fc075a10 msgr2=0x7fc4fc0734f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- 172.21.15.78:0/2368665804 shutdown_connections 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- 172.21.15.78:0/2368665804 wait complete. 2024-09-11T12:57:53.238 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 Processor -- start 2024-09-11T12:57:53.239 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.237+0000 7fc502a00640 1 -- start start 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4fc151880 0x7fc4fc145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc1fab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc1678f0 con 0x7fc4fc071d90 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4fc167770 con 0x7fc4fc151880 2024-09-11T12:57:53.240 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4fc167a70 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4fbe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:51974/0 (socket says 172.21.15.78:51974) 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4fbe00640 1 -- 172.21.15.78:0/1582612688 learned_addr learned my addr 172.21.15.78:0/1582612688 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc1fab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4fb400640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4fc151880 0x7fc4fc145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 -- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4fc151880 msgr2=0x7fc4fc145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4fc151880 0x7fc4fc145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 -- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 msgr2=0x7fc4fc145430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 -- 172.21.15.78:0/1582612688 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4fc1450e0 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.241 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4fbe00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:57:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc500a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc1fab80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40ac000 tx=0x7fc4f40a5630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f40b2040 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4f40043d0 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.242 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4f4005290 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4fc1433f0 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4fc143990 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.241+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4fc072220 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.243 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.245+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fc4f40be060 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.245+0000 7fc4f3e00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 0x7fc4dc074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:53.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.245+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 5 ==== osd_map(163..163 src has 1..163) ==== 5795+0+0 (secure 0 0 0) 0x7fc4f4129a50 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.245+0000 7fc4fbe00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 0x7fc4dc074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:53.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.245+0000 7fc4fbe00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 0x7fc4dc074680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0012f0 tx=0x7fc4ec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:53.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.249+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc4fc1450e0 con 0x7fc4fc15b4f0 2024-09-11T12:57:53.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:53.445+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fc4fc0705c0 con 0x7fc4fc15b4f0 2024-09-11T12:57:54.077 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc4f3e00640 1 -- 172.21.15.78:0/1582612688 <== mon.2 v2:172.21.15.157:3301/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v124) ==== 104+0+0 (secure 0 0 0) 0x7fc4f40164d0 con 0x7fc4fc15b4f0 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 msgr2=0x7fc4dc074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 0x7fc4dc074680 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec0012f0 tx=0x7fc4ec009040 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 msgr2=0x7fc4fc1fab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc1fab80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fc4f40ac000 tx=0x7fc4f40a5630 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 shutdown_connections 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4fc15b4f0 0x7fc4fc1fab80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4dc0721c0 0x7fc4dc074680 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4fc151880 0x7fc4fc145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 --2- 172.21.15.78:0/1582612688 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4fc071d90 0x7fc4fc145430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.079 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 >> 172.21.15.78:0/1582612688 conn(0x7fc4fc075a10 msgr2=0x7fc4fc0b5360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:54.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 shutdown_connections 2024-09-11T12:57:54.080 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.077+0000 7fc502a00640 1 -- 172.21.15.78:0/1582612688 wait complete. 2024-09-11T12:57:54.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- 172.21.15.78:0/2639824878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37900b5790 msgr2=0x7f37900723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- 172.21.15.78:0/2639824878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37900b5790 0x7f37900723f0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f37900cd910 tx=0x7f378001c8c0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- 172.21.15.78:0/2639824878 shutdown_connections 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- 172.21.15.78:0/2639824878 >> v2:172.21.15.157:3300/0 conn(0x7f3790151340 0x7f37901517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- 172.21.15.78:0/2639824878 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f37900b5790 0x7f37900723f0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- 172.21.15.78:0/2639824878 >> 172.21.15.78:0/2639824878 conn(0x7f37900b3050 msgr2=0x7f37900b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- 172.21.15.78:0/2639824878 shutdown_connections 2024-09-11T12:57:54.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- 172.21.15.78:0/2639824878 wait complete. 2024-09-11T12:57:54.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 Processor -- start 2024-09-11T12:57:54.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- start start 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 0x7f3790150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37900b5b70 0x7f3790143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f3790167ad0 con 0x7f3790151340 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f3790167950 con 0x7f37900b5790 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3796800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f3790167c50 con 0x7f37900b5b70 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:54.247 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56216/0 (socket says 172.21.15.78:56216) 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 -- 172.21.15.78:0/885601685 learned_addr learned my addr 172.21.15.78:0/885601685 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f3787e00640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37900b5b70 0x7f3790143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378f400640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 0x7f3790150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 -- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37900b5b70 msgr2=0x7f3790143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37900b5b70 0x7f3790143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 -- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 msgr2=0x7f3790150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 0x7f3790150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 -- 172.21.15.78:0/885601685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3790143760 con 0x7f3790151340 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378f400640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 0x7f3790150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.245+0000 7f378fe00640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f378800cce0 tx=0x7f378800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800f450 con 0x7f3790151340 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f378800f6e0 con 0x7f3790151340 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378800fb30 con 0x7f3790151340 2024-09-11T12:57:54.248 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3790154770 con 0x7f3790151340 2024-09-11T12:57:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3790154d30 con 0x7f3790151340 2024-09-11T12:57:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37901517d0 con 0x7f3790151340 2024-09-11T12:57:54.249 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f378800fcd0 con 0x7f3790151340 2024-09-11T12:57:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 0x7f376c074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(163..163 src has 1..163) ==== 5795+0+0 (secure 0 0 0) 0x7f37880913f0 con 0x7f3790151340 2024-09-11T12:57:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.249+0000 7f378f400640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 0x7f376c074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:54.251 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.253+0000 7f378f400640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 0x7f376c074680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37800af000 tx=0x7f378001b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:54.255 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.253+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f3790143760 con 0x7f3790151340 2024-09-11T12:57:54.422 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:54.421+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f3790072010 con 0x7f3790151340 2024-09-11T12:57:55.083 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.081+0000 7f3787400640 1 -- 172.21.15.78:0/885601685 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v164) ==== 181+0+0 (secure 0 0 0) 0x7f3788014030 con 0x7f3790151340 2024-09-11T12:57:55.083 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_metadata' removed 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 msgr2=0x7f376c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 0x7f376c074680 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f37800af000 tx=0x7f378001b100 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 msgr2=0x7f37901423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f378800cce0 tx=0x7f378800a620 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 shutdown_connections 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f3790151340 0x7f37901423e0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f37900b5b70 0x7f3790143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f376c0721c0 0x7f376c074680 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 --2- 172.21.15.78:0/885601685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f37900b5790 0x7f3790150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 >> 172.21.15.78:0/885601685 conn(0x7f37900b3050 msgr2=0x7f379014e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:55.086 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 shutdown_connections 2024-09-11T12:57:55.087 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.085+0000 7f3796800640 1 -- 172.21.15.78:0/885601685 wait complete. 2024-09-11T12:57:55.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- 172.21.15.78:0/2123184490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a0152990 msgr2=0x7ff6a015d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- 172.21.15.78:0/2123184490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a0152990 0x7ff6a015d060 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7ff6a00b3000 tx=0x7ff69001cd80 comp rx=0 tx=0).stop 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- 172.21.15.78:0/2123184490 shutdown_connections 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- 172.21.15.78:0/2123184490 >> v2:172.21.15.157:3300/0 conn(0x7ff6a00b5530 0x7ff6a00b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- 172.21.15.78:0/2123184490 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00cf520 0x7ff6a00cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- 172.21.15.78:0/2123184490 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a0152990 0x7ff6a015d060 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- 172.21.15.78:0/2123184490 >> 172.21.15.78:0/2123184490 conn(0x7ff6a00b3050 msgr2=0x7ff6a00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- 172.21.15.78:0/2123184490 shutdown_connections 2024-09-11T12:57:55.257 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- 172.21.15.78:0/2123184490 wait complete. 2024-09-11T12:57:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 Processor -- start 2024-09-11T12:57:55.258 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- start start 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00b5530 0x7ff6a014b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a0152990 0x7ff6a0165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a015f6e0 con 0x7ff6a00cf520 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff6a015f560 con 0x7ff6a0152990 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.257+0000 7ff6a7400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff6a015f860 con 0x7ff6a00b5530 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56236/0 (socket says 172.21.15.78:56236) 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 -- 172.21.15.78:0/1125914447 learned_addr learned my addr 172.21.15.78:0/1125914447 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff6a4a00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00b5530 0x7ff6a014b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff6a5400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a0152990 0x7ff6a0165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:55.260 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 -- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00b5530 msgr2=0x7ff6a014b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00b5530 0x7ff6a014b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 -- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a0152990 msgr2=0x7ff6a0165030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a0152990 0x7ff6a0165030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 -- 172.21.15.78:0/1125914447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6a01f7750 con 0x7ff6a00cf520 2024-09-11T12:57:55.261 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69fe00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d770 tx=0x7ff68c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c017020 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff68c009820 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68c009c50 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6a0165620 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6a0153250 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6a00cf960 con 0x7ff6a00cf520 2024-09-11T12:57:55.262 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.261+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7ff68c01a020 con 0x7ff6a00cf520 2024-09-11T12:57:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.265+0000 7ff69cc00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 0x7ff6800746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.265+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 5357+0+0 (secure 0 0 0) 0x7ff68c090a40 con 0x7ff6a00cf520 2024-09-11T12:57:55.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.265+0000 7ff6a4a00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 0x7ff6800746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:55.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.265+0000 7ff6a4a00640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 0x7ff6800746b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff6900af000 tx=0x7ff69001b5c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:55.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.269+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff68c021020 con 0x7ff6a00cf520 2024-09-11T12:57:55.435 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:55.437+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff6a0166010 con 0x7ff6a00cf520 2024-09-11T12:57:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.085+0000 7ff69cc00640 1 -- 172.21.15.78:0/1125914447 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v165) ==== 169+0+0 (secure 0 0 0) 0x7ff68c05dbf0 con 0x7ff6a00cf520 2024-09-11T12:57:56.115 INFO:teuthology.orchestra.run.smithi078.stderr:pool 'cephfs_data' removed 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 msgr2=0x7ff6800746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 0x7ff6800746b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff6900af000 tx=0x7ff69001b5c0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 msgr2=0x7ff6a0164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7ff68c00d770 tx=0x7ff68c00dc30 comp rx=0 tx=0).stop 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 shutdown_connections 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff6800721f0 0x7ff6800746b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff6a0152990 0x7ff6a0165030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff6a00cf520 0x7ff6a0164af0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 --2- 172.21.15.78:0/1125914447 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff6a00b5530 0x7ff6a014b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 >> 172.21.15.78:0/1125914447 conn(0x7ff6a00b3050 msgr2=0x7ff6a0159cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:56.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 shutdown_connections 2024-09-11T12:57:56.118 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.089+0000 7ff6a7400640 1 -- 172.21.15.78:0/1125914447 wait complete. 2024-09-11T12:57:56.118 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 -- 172.21.15.78:0/2791449913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e840b4db0 msgr2=0x7f7e840b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 --2- 172.21.15.78:0/2791449913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e840b4db0 0x7f7e840b5240 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009c80 tx=0x7f7e7c01cda0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 -- 172.21.15.78:0/2791449913 shutdown_connections 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 --2- 172.21.15.78:0/2791449913 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e840b4db0 0x7f7e840b5240 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 --2- 172.21.15.78:0/2791449913 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e840cdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 -- 172.21.15.78:0/2791449913 >> 172.21.15.78:0/2791449913 conn(0x7f7e8406ca80 msgr2=0x7f7e840b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 -- 172.21.15.78:0/2791449913 shutdown_connections 2024-09-11T12:57:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.261+0000 7f7e89800640 1 -- 172.21.15.78:0/2791449913 wait complete. 2024-09-11T12:57:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 Processor -- start 2024-09-11T12:57:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- start start 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 0x7f7e841451b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e84146600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8406f730 con 0x7f7e84159fc0 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f7e8406f5b0 con 0x7f7e840b4db0 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f7e8406f8b0 con 0x7f7e840cd910 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56262/0 (socket says 172.21.15.78:56262) 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 -- 172.21.15.78:0/3943635980 learned_addr learned my addr 172.21.15.78:0/3943635980 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e82a00640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 0x7f7e841451b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e82000640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e84146600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 -- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 msgr2=0x7f7e84146600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e84146600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 -- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 msgr2=0x7f7e841451b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 0x7f7e841451b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 -- 172.21.15.78:0/3943635980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e841f65f0 con 0x7f7e84159fc0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e82000640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e84146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e82a00640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 0x7f7e841451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e83400640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009bd0 tx=0x7f7e7c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0bb070 con 0x7f7e84159fc0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e7c004d60 con 0x7f7e84159fc0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c0b2040 con 0x7f7e84159fc0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e841f6310 con 0x7f7e84159fc0 2024-09-11T12:57:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.265+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e84145e00 con 0x7f7e84159fc0 2024-09-11T12:57:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e840b5240 con 0x7f7e84159fc0 2024-09-11T12:57:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f7e7c0a5af0 con 0x7f7e84159fc0 2024-09-11T12:57:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e7aa00640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 0x7f7e64074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f7e7c12a8a0 con 0x7f7e84159fc0 2024-09-11T12:57:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e82a00640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 0x7f7e64074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.269+0000 7f7e82a00640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 0x7f7e64074680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7e84155e20 tx=0x7f7e74009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:56.273 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.273+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f7e7c016990 con 0x7f7e84159fc0 2024-09-11T12:57:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.441+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f7e840b1350 con 0x7f7e84159fc0 2024-09-11T12:57:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.445+0000 7f7e7aa00640 1 -- 172.21.15.78:0/3943635980 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7f7e7c0f7e90 con 0x7f7e84159fc0 2024-09-11T12:57:56.448 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 msgr2=0x7f7e64074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 0x7f7e64074680 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f7e84155e20 tx=0x7f7e74009040 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 msgr2=0x7f7e8414fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c009bd0 tx=0x7f7e7c009360 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 shutdown_connections 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f7e84159fc0 0x7f7e8414fe10 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f7e840cd910 0x7f7e84146600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f7e640721c0 0x7f7e64074680 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 --2- 172.21.15.78:0/3943635980 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f7e840b4db0 0x7f7e841451b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 >> 172.21.15.78:0/3943635980 conn(0x7f7e8406ca80 msgr2=0x7f7e840ce360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 shutdown_connections 2024-09-11T12:57:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.449+0000 7f7e89800640 1 -- 172.21.15.78:0/3943635980 wait complete. 2024-09-11T12:57:56.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-11T12:57:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- 172.21.15.78:0/3918346454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 msgr2=0x7fbe3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 --2- 172.21.15.78:0/3918346454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe3800b9d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c001f50 tx=0x7fbe4c01cc40 comp rx=0 tx=0).stop 2024-09-11T12:57:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- 172.21.15.78:0/3918346454 shutdown_connections 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 --2- 172.21.15.78:0/3918346454 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe3800b9d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- 172.21.15.78:0/3918346454 >> 172.21.15.78:0/3918346454 conn(0x7fbe5406ca80 msgr2=0x7fbe540b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- 172.21.15.78:0/3918346454 shutdown_connections 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- 172.21.15.78:0/3918346454 wait complete. 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 Processor -- start 2024-09-11T12:57:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.609+0000 7fbe59800640 1 -- start start 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe5406fce0 0x7fbe5414f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe54071d60 0x7fbe540721e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415bc00 con 0x7fbe38005240 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5415ba80 con 0x7fbe5406fce0 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5415bd80 con 0x7fbe54071d60 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56286/0 (socket says 172.21.15.78:56286) 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 -- 172.21.15.78:0/3820742946 learned_addr learned my addr 172.21.15.78:0/3820742946 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52000640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe5406fce0 0x7fbe5414f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe53400640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe54071d60 0x7fbe540721e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 -- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe54071d60 msgr2=0x7fbe540721e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe54071d60 0x7fbe540721e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 -- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe5406fce0 msgr2=0x7fbe5414f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe5406fce0 0x7fbe5414f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 -- 172.21.15.78:0/3820742946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5414ffb0 con 0x7fbe38005240 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe52a00640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011a80 tx=0x7fbe44011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400fe70 con 0x7fbe38005240 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe44016070 con 0x7fbe38005240 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe541592f0 con 0x7fbe38005240 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44012760 con 0x7fbe38005240 2024-09-11T12:57:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe541598b0 con 0x7fbe38005240 2024-09-11T12:57:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe54072720 con 0x7fbe38005240 2024-09-11T12:57:56.615 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.613+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fbe44018030 con 0x7fbe38005240 2024-09-11T12:57:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.617+0000 7fbe4aa00640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 0x7fbe34074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.617+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fbe44091190 con 0x7fbe38005240 2024-09-11T12:57:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.617+0000 7fbe52000640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 0x7fbe34074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.617+0000 7fbe52000640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 0x7fbe34074680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbe40001890 tx=0x7fbe40001120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:56.620 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.621+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbe5414ffb0 con 0x7fbe38005240 2024-09-11T12:57:56.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:56.853+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fbe5415ecc0 con 0x7fbe38005240 2024-09-11T12:57:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe4aa00640 1 -- 172.21.15.78:0/3820742946 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v458) ==== 163+0+0 (secure 0 0 0) 0x7fbe4405ea50 con 0x7fbe38005240 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 msgr2=0x7fbe34074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 0x7fbe34074680 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fbe40001890 tx=0x7fbe40001120 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 msgr2=0x7fbe541f1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fbe44011a80 tx=0x7fbe44011f40 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 shutdown_connections 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbe54071d60 0x7fbe540721e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbe340721c0 0x7fbe34074680 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbe5406fce0 0x7fbe5414f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 --2- 172.21.15.78:0/3820742946 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbe38005240 0x7fbe541f1e10 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:57.091 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 >> 172.21.15.78:0/3820742946 conn(0x7fbe5406ca80 msgr2=0x7fbe5415cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 shutdown_connections 2024-09-11T12:57:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:57.089+0000 7fbe59800640 1 -- 172.21.15.78:0/3820742946 wait complete. 2024-09-11T12:57:57.111 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-11T12:57:57.111 INFO:tasks.cephfs_test_runner: 2024-09-11T12:57:57.111 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-11T12:57:57.111 INFO:tasks.cephfs_test_runner:Ran 11 tests in 467.719s 2024-09-11T12:57:57.111 INFO:tasks.cephfs_test_runner: 2024-09-11T12:57:57.112 INFO:tasks.cephfs_test_runner:OK 2024-09-11T12:57:57.112 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-11T12:57:57.126 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-11T12:57:57.155 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-11T12:57:57.155 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:57.155 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:57.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:57.164 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-09-11T12:57:57.164 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:57.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350180k,nr_inodes=4087545,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=4145 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274968k,nr_inodes=818742,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274972k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-11T12:57:57.215 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:57.215 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:57.225 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:57.225 DEBUG:teuthology.orchestra.run.smithi157:> set -ex 2024-09-11T12:57:57.225 DEBUG:teuthology.orchestra.run.smithi157:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-11T12:57:57.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16350244k,nr_inodes=4087561,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3465 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3274980k,nr_inodes=818745,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3274984k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-11T12:57:57.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.278 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:57:57.287 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-11T12:57:57.287 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-11T12:57:57.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-11T12:57:57.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-11T12:57:57.736 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.737 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T12:57:57.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.898 DEBUG:teuthology.orchestra.run.smithi078:> ip netns list 2024-09-11T12:57:57.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:57.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo ip link delete ceph-brx 2024-09-11T12:57:58.370 INFO:teuthology.orchestra.run.smithi078.stderr:Cannot find device "ceph-brx" 2024-09-11T12:57:58.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:57:58.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:58.373 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:57:58.382 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-11T12:57:58.383 INFO:teuthology.orchestra.run.smithi157.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-11T12:57:58.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:58.384 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-11T12:57:58.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:58.449 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-11T12:57:58.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:58.908 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-11T12:57:59.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:59.028 DEBUG:teuthology.orchestra.run.smithi157:> ip netns list 2024-09-11T12:57:59.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-11T12:57:59.037 DEBUG:teuthology.orchestra.run.smithi157:> sudo ip link delete ceph-brx 2024-09-11T12:57:59.483 INFO:teuthology.orchestra.run.smithi157.stderr:Cannot find device "ceph-brx" 2024-09-11T12:57:59.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:57:59.486 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T12:57:59.496 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T12:57:59.496 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:57:59.648 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- 172.21.15.78:0/2447508045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 msgr2=0x7fecec15ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 --2- 172.21.15.78:0/2447508045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec15ca20 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fece4009cb0 tx=0x7fece401cc50 comp rx=0 tx=0).stop 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- 172.21.15.78:0/2447508045 shutdown_connections 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 --2- 172.21.15.78:0/2447508045 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec15ca20 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- 172.21.15.78:0/2447508045 >> 172.21.15.78:0/2447508045 conn(0x7fecec0b1c50 msgr2=0x7fecec070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- 172.21.15.78:0/2447508045 shutdown_connections 2024-09-11T12:57:59.649 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- 172.21.15.78:0/2447508045 wait complete. 2024-09-11T12:57:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 Processor -- start 2024-09-11T12:57:59.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- start start 2024-09-11T12:57:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 0x7fecec1496b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec152bd0 0x7fecec149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fecec169a80 con 0x7fecec0b3780 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fecec169900 con 0x7fecec1527f0 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.649+0000 7fecf2c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fecec169c00 con 0x7fecec152bd0 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56306/0 (socket says 172.21.15.78:56306) 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 -- 172.21.15.78:0/3623235684 learned_addr learned my addr 172.21.15.78:0/3623235684 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 -- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec152bd0 msgr2=0x7fecec149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec152bd0 0x7fecec149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 -- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 msgr2=0x7fecec1496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece3e00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 0x7fecec1496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 0x7fecec1496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.653 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 -- 172.21.15.78:0/3623235684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecec1512b0 con 0x7fecec0b3780 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece3e00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 0x7fecec1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecebe00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00c4b0 tx=0x7fecdc00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc0052d0 con 0x7fecec0b3780 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecdc005470 con 0x7fecec0b3780 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecec151590 con 0x7fecec0b3780 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc012870 con 0x7fecec0b3780 2024-09-11T12:57:59.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecec150ed0 con 0x7fecec0b3780 2024-09-11T12:57:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecec15ca20 con 0x7fecec0b3780 2024-09-11T12:57:59.655 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.653+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fecdc003600 con 0x7fecec0b3780 2024-09-11T12:57:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.657+0000 7fece8c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 0x7fecc8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.657+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fecdc0904f0 con 0x7fecec0b3780 2024-09-11T12:57:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.657+0000 7fece3e00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 0x7fecc8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.656 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.657+0000 7fece3e00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 0x7fecc8074680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b29c0 tx=0x7fecd4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:59.660 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.661+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fecdc05dbc0 con 0x7fecec0b3780 2024-09-11T12:57:59.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.825+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fecec06fed0 con 0x7fecc80721c0 2024-09-11T12:57:59.825 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.825+0000 7fece8c00640 1 -- 172.21.15.78:0/3623235684 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13439 (secure 0 0 0) 0x7fecec06fed0 con 0x7fecc80721c0 2024-09-11T12:57:59.827 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 msgr2=0x7fecc8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 0x7fecc8074680 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fecec0b29c0 tx=0x7fecd4009040 comp rx=0 tx=0).stop 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 msgr2=0x7fecec140d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00c4b0 tx=0x7fecdc00c970 comp rx=0 tx=0).stop 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 shutdown_connections 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fecec152bd0 0x7fecec149bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fecc80721c0 0x7fecc8074680 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.828 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fecec1527f0 0x7fecec1496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 --2- 172.21.15.78:0/3623235684 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fecec0b3780 0x7fecec140d00 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 >> 172.21.15.78:0/3623235684 conn(0x7fecec0b1c50 msgr2=0x7fecec14b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:59.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 shutdown_connections 2024-09-11T12:57:59.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.829+0000 7fecf2c00640 1 -- 172.21.15.78:0/3623235684 wait complete. 2024-09-11T12:57:59.847 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T12:57:58.523496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287692,"kb_used_data":17384,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687028,"statfs":{"total":773094113280,"available":772799516672,"internally_reserved":0,"allocated":17801216,"data_stored":8531466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61257,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"1.0","version":"72'79","reported_seq":395,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-11T12:57:54.538760+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:54.538760+0000","last_peered":"2024-09-11T12:57:54.538760+0000","last_clean":"2024-09-11T12:57:54.538760+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:54.538760+0000","last_undegraded":"2024-09-11T12:57:54.538760+0000","last_fullsized":"2024-09-11T12:57:54.538760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2715648,"data_stored":1553523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2068480,"data_stored":908853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2056192,"data_stored":903432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2134016,"data_stored":963562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2064384,"data_stored":903032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2052096,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":902906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1493308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:57:59.848 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:57:59.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- 172.21.15.78:0/502323027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0b2dc0 msgr2=0x7f29ec0b31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 --2- 172.21.15.78:0/502323027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec0b31f0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f29ec0b1610 tx=0x7f29d801c740 comp rx=0 tx=0).stop 2024-09-11T12:57:59.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- 172.21.15.78:0/502323027 shutdown_connections 2024-09-11T12:57:59.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 --2- 172.21.15.78:0/502323027 >> v2:172.21.15.157:3300/0 conn(0x7f29ec15dcc0 0x7f29ec15e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.989 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 --2- 172.21.15.78:0/502323027 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec0b31f0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- 172.21.15.78:0/502323027 >> 172.21.15.78:0/502323027 conn(0x7f29ec0b19a0 msgr2=0x7f29ec070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:57:59.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- 172.21.15.78:0/502323027 shutdown_connections 2024-09-11T12:57:59.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- 172.21.15.78:0/502323027 wait complete. 2024-09-11T12:57:59.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 Processor -- start 2024-09-11T12:57:59.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.989+0000 7f29f2e00640 1 -- start start 2024-09-11T12:57:59.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 0x7f29ec145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec168d30 con 0x7f29ec15dcc0 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f29ec168bb0 con 0x7f29ec0b2dc0 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f29ec168eb0 con 0x7f29ec15c390 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56336/0 (socket says 172.21.15.78:56336) 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 -- 172.21.15.78:0/1404320801 learned_addr learned my addr 172.21.15.78:0/1404320801 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f1a00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f1000640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 0x7f29ec145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 -- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 msgr2=0x7f29ec145860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 0x7f29ec145860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 -- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 msgr2=0x7f29ec14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec14a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:57:59.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 -- 172.21.15.78:0/1404320801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29ec14cf30 con 0x7f29ec15dcc0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f1a00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f1000640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 0x7f29ec145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2400640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c9a0 tx=0x7f29e400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4007bb0 con 0x7f29ec15dcc0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29e400f450 con 0x7f29ec15dcc0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e4007e60 con 0x7f29ec15dcc0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29ec1462a0 con 0x7f29ec15dcc0 2024-09-11T12:57:59.995 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29ec146860 con 0x7f29ec15dcc0 2024-09-11T12:57:59.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.993+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29ec0b31f0 con 0x7f29ec15dcc0 2024-09-11T12:57:59.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.997+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f29e400f8a0 con 0x7f29ec15dcc0 2024-09-11T12:57:59.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.997+0000 7f29e9600640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 0x7f29cc074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:57:59.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.997+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f29e40910f0 con 0x7f29ec15dcc0 2024-09-11T12:57:59.996 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.997+0000 7f29f1a00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 0x7f29cc074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:57:59.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:57:59.997+0000 7f29f1a00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 0x7f29cc074680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f29ec15e5b0 tx=0x7f29e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:00.000 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.001+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f29e4014030 con 0x7f29ec15dcc0 2024-09-11T12:58:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.165+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29ec1460f0 con 0x7f29cc0721c0 2024-09-11T12:58:00.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.165+0000 7f29e9600640 1 -- 172.21.15.78:0/1404320801 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13439 (secure 0 0 0) 0x7f29ec1460f0 con 0x7f29cc0721c0 2024-09-11T12:58:00.167 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:00.167 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 msgr2=0x7f29cc074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 0x7f29cc074680 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f29ec15e5b0 tx=0x7f29e0009040 comp rx=0 tx=0).stop 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 msgr2=0x7f29ec146d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f29e400c9a0 tx=0x7f29e400ce60 comp rx=0 tx=0).stop 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 shutdown_connections 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f29ec15dcc0 0x7f29ec146d80 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f29ec15c390 0x7f29ec145860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f29cc0721c0 0x7f29cc074680 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 --2- 172.21.15.78:0/1404320801 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f29ec0b2dc0 0x7f29ec14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 >> 172.21.15.78:0/1404320801 conn(0x7f29ec0b19a0 msgr2=0x7f29ec157ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 shutdown_connections 2024-09-11T12:58:00.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.169+0000 7f29f2e00640 1 -- 172.21.15.78:0/1404320801 wait complete. 2024-09-11T12:58:00.188 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T12:57:58.523496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287692,"kb_used_data":17384,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687028,"statfs":{"total":773094113280,"available":772799516672,"internally_reserved":0,"allocated":17801216,"data_stored":8531466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61257,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"1.0","version":"72'79","reported_seq":395,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-11T12:57:54.538760+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:54.538760+0000","last_peered":"2024-09-11T12:57:54.538760+0000","last_clean":"2024-09-11T12:57:54.538760+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:54.538760+0000","last_undegraded":"2024-09-11T12:57:54.538760+0000","last_fullsized":"2024-09-11T12:57:54.538760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2715648,"data_stored":1553523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2068480,"data_stored":908853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2056192,"data_stored":903432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2134016,"data_stored":963562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2064384,"data_stored":903032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2052096,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":902906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1493308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:58:00.188 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T12:58:00.189 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:58:00.330 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 -- 172.21.15.78:0/1008740719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 msgr2=0x7fac680d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 --2- 172.21.15.78:0/1008740719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac680d0ec0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fac54009900 tx=0x7fac5401c8e0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 -- 172.21.15.78:0/1008740719 shutdown_connections 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 --2- 172.21.15.78:0/1008740719 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 0x7fac681520b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 --2- 172.21.15.78:0/1008740719 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac680d0ec0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 -- 172.21.15.78:0/1008740719 >> 172.21.15.78:0/1008740719 conn(0x7fac680ceab0 msgr2=0x7fac680ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 -- 172.21.15.78:0/1008740719 shutdown_connections 2024-09-11T12:58:00.331 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.329+0000 7fac6f600640 1 -- 172.21.15.78:0/1008740719 wait complete. 2024-09-11T12:58:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 Processor -- start 2024-09-11T12:58:00.332 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- start start 2024-09-11T12:58:00.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 0x7fac6806d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 0x7fac6806dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fac680aa210 con 0x7fac6815c1d0 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fac680aa090 con 0x7fac6806e860 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fac680aa390 con 0x7fac680d1400 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6d600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 0x7fac6806dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.157:3300/0 says I am v2:172.21.15.78:59310/0 (socket says 172.21.15.78:59310) 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 -- 172.21.15.78:0/1134931307 learned_addr learned my addr 172.21.15.78:0/1134931307 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:00.336 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac67e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 0x7fac6806d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 -- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 msgr2=0x7fac6806d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 0x7fac6806d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 -- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 msgr2=0x7fac6806dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 0x7fac6806dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 -- 172.21.15.78:0/1134931307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac680b0e60 con 0x7fac6806e860 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6d600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 0x7fac6806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-11T12:58:00.337 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6cc00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c3c0 tx=0x7fac5800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:00.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58017070 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac580053c0 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac68150670 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac58003860 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.333+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac681fafa0 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac680d0ec0 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fac5801a070 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac64c00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 0x7fac48074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fac5808fc90 con 0x7fac6806e860 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac67e00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 0x7fac48074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.337+0000 7fac67e00640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 0x7fac48074680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fac540af000 tx=0x7fac5401b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:00.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.341+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mon.1 v2:172.21.15.157:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fac5805ccb0 con 0x7fac6806e860 2024-09-11T12:58:00.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.505+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fac68070480 con 0x7fac480721c0 2024-09-11T12:58:00.507 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac64c00640 1 -- 172.21.15.78:0/1134931307 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13439 (secure 0 0 0) 0x7fac68070480 con 0x7fac480721c0 2024-09-11T12:58:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:00.508 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 msgr2=0x7fac48074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 0x7fac48074680 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fac540af000 tx=0x7fac5401b120 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 msgr2=0x7fac68140990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fac5800c3c0 tx=0x7fac5800c880 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 shutdown_connections 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fac6815c1d0 0x7fac6806dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fac680d1400 0x7fac6806d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fac480721c0 0x7fac48074680 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 --2- 172.21.15.78:0/1134931307 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fac6806e860 0x7fac68140990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 >> 172.21.15.78:0/1134931307 conn(0x7fac680ceab0 msgr2=0x7fac6815c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 shutdown_connections 2024-09-11T12:58:00.510 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.509+0000 7fac6f600640 1 -- 172.21.15.78:0/1134931307 wait complete. 2024-09-11T12:58:00.529 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T12:57:58.523496+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":287692,"kb_used_data":17384,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754687028,"statfs":{"total":773094113280,"available":772799516672,"internally_reserved":0,"allocated":17801216,"data_stored":8531466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-61257,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-78,"num_write_kb":-91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001083"},"pg_stats":[{"pgid":"1.0","version":"72'79","reported_seq":395,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-09-11T12:57:54.538760+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:54.538760+0000","last_peered":"2024-09-11T12:57:54.538760+0000","last_clean":"2024-09-11T12:57:54.538760+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:54.538760+0000","last_undegraded":"2024-09-11T12:57:54.538760+0000","last_fullsized":"2024-09-11T12:57:54.538760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38000,"kb_used_data":2652,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333840,"statfs":{"total":96636764160,"available":96597852160,"internally_reserved":0,"allocated":2715648,"data_stored":1553523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37372,"kb_used_data":2020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334468,"statfs":{"total":96636764160,"available":96598495232,"internally_reserved":0,"allocated":2068480,"data_stored":908853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37356,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334484,"statfs":{"total":96636764160,"available":96598511616,"internally_reserved":0,"allocated":2056192,"data_stored":903432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509544,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37432,"kb_used_data":2084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334408,"statfs":{"total":96636764160,"available":96598433792,"internally_reserved":0,"allocated":2134016,"data_stored":963562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509544,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2064384,"data_stored":903032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509544,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33192,"kb_used_data":2004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338648,"statfs":{"total":96636764160,"available":96602775552,"internally_reserved":0,"allocated":2052096,"data_stored":902850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33200,"kb_used_data":2012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338640,"statfs":{"total":96636764160,"available":96602767360,"internally_reserved":0,"allocated":2060288,"data_stored":902906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542248,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37936,"kb_used_data":2588,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333904,"statfs":{"total":96636764160,"available":96597917696,"internally_reserved":0,"allocated":2650112,"data_stored":1493308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:58:00.529 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T12:58:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- 172.21.15.78:0/3667591436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcce40052e0 msgr2=0x7fcce400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.672 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 --2- 172.21.15.78:0/3667591436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcce40052e0 0x7fcce400b9d0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fccf8009ee0 tx=0x7fccf801d630 comp rx=0 tx=0).stop 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- 172.21.15.78:0/3667591436 shutdown_connections 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 --2- 172.21.15.78:0/3667591436 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcce40052e0 0x7fcce400b9d0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- 172.21.15.78:0/3667591436 >> 172.21.15.78:0/3667591436 conn(0x7fcd0006ca80 msgr2=0x7fcd000b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- 172.21.15.78:0/3667591436 shutdown_connections 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- 172.21.15.78:0/3667591436 wait complete. 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 Processor -- start 2024-09-11T12:58:00.673 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- start start 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce40052e0 0x7fcd001e4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 0x7fcd000b14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00157ae0 con 0x7fcd001f5c20 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcd00070de0 con 0x7fcd000b1060 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fcd06800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcd00157c60 con 0x7fcce40052e0 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fccf6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fccf6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56366/0 (socket says 172.21.15.78:56366) 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fccf6a00640 1 -- 172.21.15.78:0/214855914 learned_addr learned my addr 172.21.15.78:0/214855914 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.673+0000 7fccff400640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce40052e0 0x7fcd001e4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.676 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccffe00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 0x7fcd000b14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 -- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce40052e0 msgr2=0x7fcd001e4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce40052e0 0x7fcd001e4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 -- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 msgr2=0x7fcd000b14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 0x7fcd000b14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 -- 172.21.15.78:0/214855914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd00144ef0 con 0x7fcd001f5c20 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccffe00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 0x7fcd000b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf6a00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e310 tx=0x7fcce800e7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce80062e0 con 0x7fcd001f5c20 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcce8017020 con 0x7fcd001f5c20 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd001f6400 con 0x7fcd001f5c20 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcce8003860 con 0x7fcd001f5c20 2024-09-11T12:58:00.677 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd001f69c0 con 0x7fcd001f5c20 2024-09-11T12:58:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd0015b100 con 0x7fcd001f5c20 2024-09-11T12:58:00.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fcce800d070 con 0x7fcd001f5c20 2024-09-11T12:58:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 0x7fccdc0746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.677+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fcce8094340 con 0x7fcd001f5c20 2024-09-11T12:58:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.681+0000 7fccff400640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 0x7fccdc0746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:00.679 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.681+0000 7fccff400640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 0x7fccdc0746b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001870 tx=0x7fccf0006040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:00.682 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.681+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcce8021020 con 0x7fcd001f5c20 2024-09-11T12:58:00.852 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcd001572a0 con 0x7fcd001f5c20 2024-09-11T12:58:00.853 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fccf7e00640 1 -- 172.21.15.78:0/214855914 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) ==== 74+0+13040 (secure 0 0 0) 0x7fcce80619b0 con 0x7fcd001f5c20 2024-09-11T12:58:00.854 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:00.854 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":165,"fsid":"163b1675-69a3-4b68-a3e4-8c2fa35bac4f","created":"2024-09-11T12:49:17.891510+0000","modified":"2024-09-11T12:57:56.085570+0000","last_up_change":"2024-09-11T12:49:32.099436+0000","last_in_change":"2024-09-11T12:49:21.548249+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-11T12:49:33.130780+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"93361912-1069-4275-84d3-de4f9b60f305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6801","nonce":4284670803}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6803","nonce":4284670803}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6807","nonce":4284670803}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":4284670803},{"type":"v1","addr":"172.21.15.78:6805","nonce":4284670803}]},"public_addr":"172.21.15.78:6801/4284670803","cluster_addr":"172.21.15.78:6803/4284670803","heartbeat_back_addr":"172.21.15.78:6807/4284670803","heartbeat_front_addr":"172.21.15.78:6805/4284670803","state":["exists","up"]},{"osd":1,"uuid":"1d52a8ee-cb50-431f-8014-ce5223a7b1ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6825","nonce":305322464}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6827","nonce":305322464}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6831","nonce":305322464}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":305322464},{"type":"v1","addr":"172.21.15.78:6829","nonce":305322464}]},"public_addr":"172.21.15.78:6825/305322464","cluster_addr":"172.21.15.78:6827/305322464","heartbeat_back_addr":"172.21.15.78:6831/305322464","heartbeat_front_addr":"172.21.15.78:6829/305322464","state":["exists","up"]},{"osd":2,"uuid":"b2bd4467-e229-4a13-9273-59945dd19f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6817","nonce":4205044300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6819","nonce":4205044300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6823","nonce":4205044300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":4205044300},{"type":"v1","addr":"172.21.15.78:6821","nonce":4205044300}]},"public_addr":"172.21.15.78:6817/4205044300","cluster_addr":"172.21.15.78:6819/4205044300","heartbeat_back_addr":"172.21.15.78:6823/4205044300","heartbeat_front_addr":"172.21.15.78:6821/4205044300","state":["exists","up"]},{"osd":3,"uuid":"7331094e-bbbc-41ce-bbc9-cc6746ef8aca","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6809","nonce":2152308697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6811","nonce":2152308697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6815","nonce":2152308697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":2152308697},{"type":"v1","addr":"172.21.15.78:6813","nonce":2152308697}]},"public_addr":"172.21.15.78:6809/2152308697","cluster_addr":"172.21.15.78:6811/2152308697","heartbeat_back_addr":"172.21.15.78:6815/2152308697","heartbeat_front_addr":"172.21.15.78:6813/2152308697","state":["exists","up"]},{"osd":4,"uuid":"d1c94064-370e-4ace-b2d9-e2b5692e0cae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6810","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6811","nonce":2942152212}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6812","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6813","nonce":2942152212}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6816","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6817","nonce":2942152212}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6814","nonce":2942152212},{"type":"v1","addr":"172.21.15.157:6815","nonce":2942152212}]},"public_addr":"172.21.15.157:6811/2942152212","cluster_addr":"172.21.15.157:6813/2942152212","heartbeat_back_addr":"172.21.15.157:6817/2942152212","heartbeat_front_addr":"172.21.15.157:6815/2942152212","state":["exists","up"]},{"osd":5,"uuid":"792b544e-17db-4154-8674-225c7fd0b12f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6802","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6803","nonce":2089537569}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6804","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6805","nonce":2089537569}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6808","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6809","nonce":2089537569}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6806","nonce":2089537569},{"type":"v1","addr":"172.21.15.157:6807","nonce":2089537569}]},"public_addr":"172.21.15.157:6803/2089537569","cluster_addr":"172.21.15.157:6805/2089537569","heartbeat_back_addr":"172.21.15.157:6809/2089537569","heartbeat_front_addr":"172.21.15.157:6807/2089537569","state":["exists","up"]},{"osd":6,"uuid":"e2bb1238-c03a-4869-b5f7-767ae0794da7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6826","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6827","nonce":899086460}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6828","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6829","nonce":899086460}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6832","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6833","nonce":899086460}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6830","nonce":899086460},{"type":"v1","addr":"172.21.15.157:6831","nonce":899086460}]},"public_addr":"172.21.15.157:6827/899086460","cluster_addr":"172.21.15.157:6829/899086460","heartbeat_back_addr":"172.21.15.157:6833/899086460","heartbeat_front_addr":"172.21.15.157:6831/899086460","state":["exists","up"]},{"osd":7,"uuid":"07f8cdba-fb9a-4130-8cec-61c127a027d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":160,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6818","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6819","nonce":3956663176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6820","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6821","nonce":3956663176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6824","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6825","nonce":3956663176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6822","nonce":3956663176},{"type":"v1","addr":"172.21.15.157:6823","nonce":3956663176}]},"public_addr":"172.21.15.157:6819/3956663176","cluster_addr":"172.21.15.157:6821/3956663176","heartbeat_back_addr":"172.21.15.157:6825/3956663176","heartbeat_front_addr":"172.21.15.157:6823/3956663176","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.352578+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:29.243279+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.681107+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.569899+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.597075+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.480865+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.825066+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T12:49:28.677654+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.157:6835/70117498":"2024-09-12T12:57:53.070639+0000","172.21.15.157:6834/70117498":"2024-09-12T12:57:53.070639+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 msgr2=0x7fccdc0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 0x7fccdc0746b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fccf0001870 tx=0x7fccf0006040 comp rx=0 tx=0).stop 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 msgr2=0x7fcd001449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.853+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fcce800e310 tx=0x7fcce800e7d0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 shutdown_connections 2024-09-11T12:58:00.855 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fccdc0721f0 0x7fccdc0746b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcd000b1060 0x7fcd000b14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcd001f5c20 0x7fcd001449b0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 --2- 172.21.15.78:0/214855914 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcce40052e0 0x7fcd001e4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 >> 172.21.15.78:0/214855914 conn(0x7fcd0006ca80 msgr2=0x7fcd000cda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 shutdown_connections 2024-09-11T12:58:00.856 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:00.857+0000 7fcd06800640 1 -- 172.21.15.78:0/214855914 wait complete. 2024-09-11T12:58:01.876 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T12:58:01.876 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- 172.21.15.78:0/2448074616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 msgr2=0x7fc40800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 --2- 172.21.15.78:0/2448074616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc40800b9d0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fc41800c630 tx=0x7fc41801f940 comp rx=0 tx=0).stop 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- 172.21.15.78:0/2448074616 shutdown_connections 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 --2- 172.21.15.78:0/2448074616 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc40800b9d0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- 172.21.15.78:0/2448074616 >> 172.21.15.78:0/2448074616 conn(0x7fc4200b3050 msgr2=0x7fc4200b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- 172.21.15.78:0/2448074616 shutdown_connections 2024-09-11T12:58:02.036 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- 172.21.15.78:0/2448074616 wait complete. 2024-09-11T12:58:02.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 Processor -- start 2024-09-11T12:58:02.037 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- start start 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc420157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4201f4870 0x7fc4201e6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc420155b30 0x7fc42014c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4200721a0 con 0x7fc408005240 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc420072020 con 0x7fc420155b30 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc428200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc420073950 con 0x7fc4201f4870 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc420157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc420157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56398/0 (socket says 172.21.15.78:56398) 2024-09-11T12:58:02.039 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 -- 172.21.15.78:0/1435600768 learned_addr learned my addr 172.21.15.78:0/1435600768 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 -- 172.21.15.78:0/1435600768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4201f4870 msgr2=0x7fc4201e6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4201f4870 0x7fc4201e6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 -- 172.21.15.78:0/1435600768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc420155b30 msgr2=0x7fc42014c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc420155b30 0x7fc42014c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.037+0000 7fc425800640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4201564a0 con 0x7fc408005240 2024-09-11T12:58:02.040 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc425800640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc420157540 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc414003f30 tx=0x7fc4140018d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414015070 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc414011900 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc414011d50 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc420156780 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc420156d40 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc420072a30 con 0x7fc408005240 2024-09-11T12:58:02.041 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fc414025440 con 0x7fc408005240 2024-09-11T12:58:02.043 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4000721c0 0x7fc400074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fc414091930 con 0x7fc408005240 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc424e00640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4000721c0 0x7fc400074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7fc400078080 0x7fc40007a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc414012ec0 con 0x7fc400078080 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.041+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fc414094140 con 0x7fc408005240 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.045+0000 7fc426200640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7fc400078080 0x7fc40007a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.045+0000 7fc424e00640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4000721c0 0x7fc400074680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc4201f4c90 tx=0x7fc40c00a4b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.044 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.045+0000 7fc426200640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7fc400078080 0x7fc40007a4a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.045 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.045+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== osd.0 v2:172.21.15.78:6800/4284670803 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc414012ec0 con 0x7fc400078080 2024-09-11T12:58:02.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 --> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc420070ce0 con 0x7fc400078080 2024-09-11T12:58:02.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc41d600640 1 -- 172.21.15.78:0/1435600768 <== osd.0 v2:172.21.15.78:6800/4284670803 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc420070ce0 con 0x7fc400078080 2024-09-11T12:58:02.058 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:02.058 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:02.058 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7fc400078080 msgr2=0x7fc40007a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6800/4284670803,v1:172.21.15.78:6801/4284670803] conn(0x7fc400078080 0x7fc40007a4a0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4000721c0 msgr2=0x7fc400074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4000721c0 0x7fc400074680 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fc4201f4c90 tx=0x7fc40c00a4b0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 msgr2=0x7fc420157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 --2- 172.21.15.78:0/1435600768 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc408005240 0x7fc420157540 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fc414003f30 tx=0x7fc4140018d0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc426200640 1 -- 172.21.15.78:0/1435600768 reap_dead start 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 shutdown_connections 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 >> 172.21.15.78:0/1435600768 conn(0x7fc4200b3050 msgr2=0x7fc42014f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 shutdown_connections 2024-09-11T12:58:02.059 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.057+0000 7fc428200640 1 -- 172.21.15.78:0/1435600768 wait complete. 2024-09-11T12:58:02.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T12:58:02.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 -- 172.21.15.78:0/3279835724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0ccb20 msgr2=0x7fc5dc15b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.219 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3279835724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0ccb20 0x7fc5dc15b550 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc5c8009870 tx=0x7fc5c801c820 comp rx=0 tx=0).stop 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 -- 172.21.15.78:0/3279835724 shutdown_connections 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3279835724 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0ccb20 0x7fc5dc15b550 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3279835724 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0b4780 0x7fc5dc073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.217+0000 7fc5e2800640 1 -- 172.21.15.78:0/3279835724 >> 172.21.15.78:0/3279835724 conn(0x7fc5dc06ca80 msgr2=0x7fc5dc0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- 172.21.15.78:0/3279835724 shutdown_connections 2024-09-11T12:58:02.220 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- 172.21.15.78:0/3279835724 wait complete. 2024-09-11T12:58:02.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 Processor -- start 2024-09-11T12:58:02.221 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- start start 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0b4b60 0x7fc5dc15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0ccb20 0x7fc5dc15abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc168fc0 con 0x7fc5dc0b4780 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc168e40 con 0x7fc5dc0b4b60 2024-09-11T12:58:02.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5e2800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc5dc169140 con 0x7fc5dc0ccb20 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56414/0 (socket says 172.21.15.78:56414) 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 -- 172.21.15.78:0/3630456014 learned_addr learned my addr 172.21.15.78:0/3630456014 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5daa00640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0b4b60 0x7fc5dc15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5dbe00640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0ccb20 0x7fc5dc15abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 -- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0ccb20 msgr2=0x7fc5dc15abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0ccb20 0x7fc5dc15abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 -- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0b4b60 msgr2=0x7fc5dc15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0b4b60 0x7fc5dc15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 -- 172.21.15.78:0/3630456014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1f93d0 con 0x7fc5dc0b4780 2024-09-11T12:58:02.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5db400640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c3f0 tx=0x7fc5cc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003980 con 0x7fc5dc0b4780 2024-09-11T12:58:02.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5cc003c10 con 0x7fc5dc0b4780 2024-09-11T12:58:02.224 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.221+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc00b690 con 0x7fc5dc0b4780 2024-09-11T12:58:02.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1f96b0 con 0x7fc5dc0b4780 2024-09-11T12:58:02.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc14e630 con 0x7fc5dc0b4780 2024-09-11T12:58:02.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc073f70 con 0x7fc5dc0b4780 2024-09-11T12:58:02.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fc5cc00b830 con 0x7fc5dc0b4780 2024-09-11T12:58:02.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5d3400640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 0x7fc5bc0746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fc5cc0910d0 con 0x7fc5dc0b4780 2024-09-11T12:58:02.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.225+0000 7fc5daa00640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 0x7fc5bc0746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.229+0000 7fc5daa00640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 0x7fc5bc0746b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80af000 tx=0x7fc5c801b060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.229+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc5cc05e0f0 con 0x7fc5dc0b4780 2024-09-11T12:58:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fc5dc14ea10 con 0x7fc5bc0721f0 2024-09-11T12:58:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5d3400640 1 -- 172.21.15.78:0/3630456014 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc5c8004140 con 0x7fc5bc0721f0 2024-09-11T12:58:02.396 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T12:58:02.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 msgr2=0x7fc5bc0746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.399 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 0x7fc5bc0746b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc5c80af000 tx=0x7fc5c801b060 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 msgr2=0x7fc5dc148600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00c3f0 tx=0x7fc5cc00c8b0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 shutdown_connections 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc5dc0ccb20 0x7fc5dc15abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc5bc0721f0 0x7fc5bc0746b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc5dc0b4b60 0x7fc5dc15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 --2- 172.21.15.78:0/3630456014 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc5dc0b4780 0x7fc5dc148600 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.400 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 >> 172.21.15.78:0/3630456014 conn(0x7fc5dc06ca80 msgr2=0x7fc5dc14c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 shutdown_connections 2024-09-11T12:58:02.401 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.397+0000 7fc5e2800640 1 -- 172.21.15.78:0/3630456014 wait complete. 2024-09-11T12:58:02.417 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T12:58:02.417 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 -- 172.21.15.78:0/2627353549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806cfc0 msgr2=0x7fc4d806faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.563 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 --2- 172.21.15.78:0/2627353549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806cfc0 0x7fc4d806faa0 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc009900 tx=0x7fc4cc01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 -- 172.21.15.78:0/2627353549 shutdown_connections 2024-09-11T12:58:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 --2- 172.21.15.78:0/2627353549 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4d80b3c10 0x7fc4d815b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 --2- 172.21.15.78:0/2627353549 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806cfc0 0x7fc4d806faa0 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 -- 172.21.15.78:0/2627353549 >> 172.21.15.78:0/2627353549 conn(0x7fc4d8071d90 msgr2=0x7fc4d806c650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.564 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 -- 172.21.15.78:0/2627353549 shutdown_connections 2024-09-11T12:58:02.565 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.561+0000 7fc4df600640 1 -- 172.21.15.78:0/2627353549 wait complete. 2024-09-11T12:58:02.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 Processor -- start 2024-09-11T12:58:02.571 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- start start 2024-09-11T12:58:02.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d806cfc0 0x7fc4d813c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 0x7fc4d813bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4d80b3c10 0x7fc4d81f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d815c9e0 con 0x7fc4d806d3a0 2024-09-11T12:58:02.572 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d815c860 con 0x7fc4d806cfc0 2024-09-11T12:58:02.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc4d815cb60 con 0x7fc4d80b3c10 2024-09-11T12:58:02.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 0x7fc4d813bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 0x7fc4d813bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56430/0 (socket says 172.21.15.78:56430) 2024-09-11T12:58:02.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 -- 172.21.15.78:0/3009050600 learned_addr learned my addr 172.21.15.78:0/3009050600 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:02.573 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4dd600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4d80b3c10 0x7fc4d81f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4dcc00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d806cfc0 0x7fc4d813c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 -- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4d80b3c10 msgr2=0x7fc4d81f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc4d80b3c10 0x7fc4d81f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 -- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d806cfc0 msgr2=0x7fc4d813c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc4d806cfc0 0x7fc4d813c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.574 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d813c060 con 0x7fc4d806d3a0 2024-09-11T12:58:02.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d7e00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 0x7fc4d813bb20 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c500 tx=0x7fc4c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4017070 con 0x7fc4d806d3a0 2024-09-11T12:58:02.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4c40053c0 con 0x7fc4d806d3a0 2024-09-11T12:58:02.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c4012890 con 0x7fc4d806d3a0 2024-09-11T12:58:02.575 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d8156e20 con 0x7fc4d806d3a0 2024-09-11T12:58:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d8153ed0 con 0x7fc4d806d3a0 2024-09-11T12:58:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.565+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc4d8156bd0 con 0x7fc4d806d3a0 2024-09-11T12:58:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fc4c401a070 con 0x7fc4d806d3a0 2024-09-11T12:58:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4b80721c0 0x7fc4b8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.576 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fc4c4090110 con 0x7fc4d806d3a0 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4dcc00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4b80721c0 0x7fc4b8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fc4b8078080 0x7fc4b807a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc4c401ee50 con 0x7fc4b8078080 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fc4c4092b90 con 0x7fc4d806d3a0 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4dd600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fc4b8078080 0x7fc4b807a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.577 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4dcc00640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4b80721c0 0x7fc4b8074680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0af000 tx=0x7fc4cc01b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.569+0000 7fc4dd600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fc4b8078080 0x7fc4b807a4a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.578 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.573+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== osd.1 v2:172.21.15.78:6824/305322464 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc4c401ee50 con 0x7fc4b8078080 2024-09-11T12:58:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 --> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc4d81539e0 con 0x7fc4b8078080 2024-09-11T12:58:02.585 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4d4c00640 1 -- 172.21.15.78:0/3009050600 <== osd.1 v2:172.21.15.78:6824/305322464 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc4d81539e0 con 0x7fc4b8078080 2024-09-11T12:58:02.586 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:02.586 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:02.586 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fc4b8078080 msgr2=0x7fc4b807a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6824/305322464,v1:172.21.15.78:6825/305322464] conn(0x7fc4b8078080 0x7fc4b807a4a0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4b80721c0 msgr2=0x7fc4b8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc4b80721c0 0x7fc4b8074680 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fc4cc0af000 tx=0x7fc4cc01b120 comp rx=0 tx=0).stop 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 msgr2=0x7fc4d813bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 --2- 172.21.15.78:0/3009050600 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc4d806d3a0 0x7fc4d813bb20 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400c500 tx=0x7fc4c400c9c0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4dd600640 1 -- 172.21.15.78:0/3009050600 reap_dead start 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 shutdown_connections 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 >> 172.21.15.78:0/3009050600 conn(0x7fc4d8071d90 msgr2=0x7fc4d80ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 shutdown_connections 2024-09-11T12:58:02.588 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.585+0000 7fc4df600640 1 -- 172.21.15.78:0/3009050600 wait complete. 2024-09-11T12:58:02.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 -- 172.21.15.78:0/3537796197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6741527f0 msgr2=0x7fc67415ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 --2- 172.21.15.78:0/3537796197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6741527f0 0x7fc67415ca20 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc664009840 tx=0x7fc66401c680 comp rx=0 tx=0).stop 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 -- 172.21.15.78:0/3537796197 shutdown_connections 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 --2- 172.21.15.78:0/3537796197 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc674152bd0 0x7fc67415f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 --2- 172.21.15.78:0/3537796197 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc6741527f0 0x7fc67415ca20 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 -- 172.21.15.78:0/3537796197 >> 172.21.15.78:0/3537796197 conn(0x7fc6740b1c50 msgr2=0x7fc674070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 -- 172.21.15.78:0/3537796197 shutdown_connections 2024-09-11T12:58:02.747 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.745+0000 7fc67a600640 1 -- 172.21.15.78:0/3537796197 wait complete. 2024-09-11T12:58:02.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 Processor -- start 2024-09-11T12:58:02.748 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- start start 2024-09-11T12:58:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc674152bd0 0x7fc674146fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.749 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67415fa90 0x7fc67415b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fc6741688f0 con 0x7fc6741527f0 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fc6740b5220 con 0x7fc674152bd0 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fc674168a70 con 0x7fc67415fa90 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56444/0 (socket says 172.21.15.78:56444) 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 -- 172.21.15.78:0/4210073859 learned_addr learned my addr 172.21.15.78:0/4210073859 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc66be00640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc674152bd0 0x7fc674146fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673e00640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67415fa90 0x7fc67415b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 -- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67415fa90 msgr2=0x7fc67415b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67415fa90 0x7fc67415b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.750 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 -- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc674152bd0 msgr2=0x7fc674146fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc674152bd0 0x7fc674146fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 -- 172.21.15.78:0/4210073859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67406fa40 con 0x7fc6741527f0 2024-09-11T12:58:02.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc673400640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc664009790 tx=0x7fc6640051a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.751 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664005450 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6640055f0 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc664004290 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67406fd20 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6740702e0 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.749+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc67415ca20 con 0x7fc6741527f0 2024-09-11T12:58:02.752 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.753+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fc6640046d0 con 0x7fc6741527f0 2024-09-11T12:58:02.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.753+0000 7fc66b400640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 0x7fc650074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:02.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.753+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fc66412a770 con 0x7fc6741527f0 2024-09-11T12:58:02.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.753+0000 7fc66be00640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 0x7fc650074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:02.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.753+0000 7fc66be00640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 0x7fc650074680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc67414b360 tx=0x7fc65c0091e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:02.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.761+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fc6640ac030 con 0x7fc6741527f0 2024-09-11T12:58:02.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.929+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fc6741474e0 con 0x7fc6500721c0 2024-09-11T12:58:02.931 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc66b400640 1 -- 172.21.15.78:0/4210073859 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc6741474e0 con 0x7fc6500721c0 2024-09-11T12:58:02.931 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T12:58:02.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 msgr2=0x7fc650074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.933 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 0x7fc650074680 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc67414b360 tx=0x7fc65c0091e0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 msgr2=0x7fc674147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc664009790 tx=0x7fc6640051a0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 shutdown_connections 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fc67415fa90 0x7fc67415b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fc6500721c0 0x7fc650074680 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fc674152bd0 0x7fc674146fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 --2- 172.21.15.78:0/4210073859 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fc6741527f0 0x7fc674147c70 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 >> 172.21.15.78:0/4210073859 conn(0x7fc6740b1c50 msgr2=0x7fc6740b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 shutdown_connections 2024-09-11T12:58:02.934 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:02.933+0000 7fc67a600640 1 -- 172.21.15.78:0/4210073859 wait complete. 2024-09-11T12:58:02.953 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T12:58:02.953 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 -- 172.21.15.78:0/439082623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a4072b00 msgr2=0x7fe5a4151a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 --2- 172.21.15.78:0/439082623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a4072b00 0x7fe5a4151a50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009840 tx=0x7fe59c01c820 comp rx=0 tx=0).stop 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 -- 172.21.15.78:0/439082623 shutdown_connections 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 --2- 172.21.15.78:0/439082623 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a4072b00 0x7fe5a4151a50 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 -- 172.21.15.78:0/439082623 >> 172.21.15.78:0/439082623 conn(0x7fe5a40b3050 msgr2=0x7fe5a40b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 -- 172.21.15.78:0/439082623 shutdown_connections 2024-09-11T12:58:03.099 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.097+0000 7fe5a9e00640 1 -- 172.21.15.78:0/439082623 wait complete. 2024-09-11T12:58:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 Processor -- start 2024-09-11T12:58:03.100 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- start start 2024-09-11T12:58:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5a4072b00 0x7fe5a4153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 0x7fe5a4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.102 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5a40b5b70 0x7fe5a414ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4169050 con 0x7fe5a40b5790 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4168ed0 con 0x7fe5a4072b00 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a41691d0 con 0x7fe5a40b5b70 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 0x7fe5a4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 0x7fe5a4150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56450/0 (socket says 172.21.15.78:56450) 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 -- 172.21.15.78:0/2242249315 learned_addr learned my addr 172.21.15.78:0/2242249315 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a3400640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5a40b5b70 0x7fe5a414ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2a00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5a4072b00 0x7fe5a4153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 -- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5a40b5b70 msgr2=0x7fe5a414ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fe5a40b5b70 0x7fe5a414ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 -- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5a4072b00 msgr2=0x7fe5a4153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fe5a4072b00 0x7fe5a4153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a41536e0 con 0x7fe5a40b5790 2024-09-11T12:58:03.103 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a2000640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 0x7fe5a4150210 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe59000c3f0 tx=0x7fe59000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590017070 con 0x7fe5a40b5790 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe590003600 con 0x7fe5a40b5790 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590003a50 con 0x7fe5a40b5790 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a41539c0 con 0x7fe5a40b5790 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.101+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a4153f80 con 0x7fe5a40b5790 2024-09-11T12:58:03.104 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5a40723f0 con 0x7fe5a40b5790 2024-09-11T12:58:03.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fe59001a070 con 0x7fe5a40b5790 2024-09-11T12:58:03.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe5840721c0 0x7fe584074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.105 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fe59008fbc0 con 0x7fe5a40b5790 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe5a2a00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe5840721c0 0x7fe584074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7fe584078080 0x7fe58407a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe58407ab50 con 0x7fe584078080 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fe590099050 con 0x7fe5a40b5790 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe5a3400640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7fe584078080 0x7fe58407a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.106 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe5a2a00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe5840721c0 0x7fe584074680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe5940012f0 tx=0x7fe594009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.105+0000 7fe5a3400640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7fe584078080 0x7fe58407a4a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.107 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.109+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== osd.2 v2:172.21.15.78:6816/4205044300 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe58407ab50 con 0x7fe584078080 2024-09-11T12:58:03.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 --> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe5a40cfbf0 con 0x7fe584078080 2024-09-11T12:58:03.121 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe59b400640 1 -- 172.21.15.78:0/2242249315 <== osd.2 v2:172.21.15.78:6816/4205044300 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe5a40cfbf0 con 0x7fe584078080 2024-09-11T12:58:03.121 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:03.121 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:03.121 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:03.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7fe584078080 msgr2=0x7fe58407a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.122 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6816/4205044300,v1:172.21.15.78:6817/4205044300] conn(0x7fe584078080 0x7fe58407a4a0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe5840721c0 msgr2=0x7fe584074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fe5840721c0 0x7fe584074680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe5940012f0 tx=0x7fe594009040 comp rx=0 tx=0).stop 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 msgr2=0x7fe5a4150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 --2- 172.21.15.78:0/2242249315 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fe5a40b5790 0x7fe5a4150210 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fe59000c3f0 tx=0x7fe59000c8b0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a3400640 1 -- 172.21.15.78:0/2242249315 reap_dead start 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 shutdown_connections 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 >> 172.21.15.78:0/2242249315 conn(0x7fe5a40b3050 msgr2=0x7fe5a40b4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.123 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 shutdown_connections 2024-09-11T12:58:03.124 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.121+0000 7fe5a9e00640 1 -- 172.21.15.78:0/2242249315 wait complete. 2024-09-11T12:58:03.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T12:58:03.282 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 -- 172.21.15.78:0/1411524714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815c390 msgr2=0x7fa78815c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 --2- 172.21.15.78:0/1411524714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815c390 0x7fa78815c770 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7fa778009840 tx=0x7fa77801c790 comp rx=0 tx=0).stop 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 -- 172.21.15.78:0/1411524714 shutdown_connections 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 --2- 172.21.15.78:0/1411524714 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa7880b34d0 0x7fa78815dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 --2- 172.21.15.78:0/1411524714 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815c390 0x7fa78815c770 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 -- 172.21.15.78:0/1411524714 >> 172.21.15.78:0/1411524714 conn(0x7fa7880b19a0 msgr2=0x7fa788070870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 -- 172.21.15.78:0/1411524714 shutdown_connections 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.281+0000 7fa78de00640 1 -- 172.21.15.78:0/1411524714 wait complete. 2024-09-11T12:58:03.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 Processor -- start 2024-09-11T12:58:03.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- start start 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7880b34d0 0x7fa7880ae670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78815c390 0x7fa7881496c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fa788168960 con 0x7fa78815ccb0 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881687e0 con 0x7fa7880b34d0 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fa788168ae0 con 0x7fa78815c390 2024-09-11T12:58:03.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56456/0 (socket says 172.21.15.78:56456) 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 -- 172.21.15.78:0/191362031 learned_addr learned my addr 172.21.15.78:0/191362031 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78ca00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7880b34d0 0x7fa7880ae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa787e00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78815c390 0x7fa7881496c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 -- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78815c390 msgr2=0x7fa7881496c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78815c390 0x7fa7881496c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 -- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7880b34d0 msgr2=0x7fa7880ae670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7880b34d0 0x7fa7880ae670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 -- 172.21.15.78:0/191362031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880af730 con 0x7fa78815ccb0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78d400640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa78000ea20 tx=0x7fa78000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78000cde0 con 0x7fa78815ccb0 2024-09-11T12:58:03.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa780004c40 con 0x7fa78815ccb0 2024-09-11T12:58:03.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7880afa10 con 0x7fa78815ccb0 2024-09-11T12:58:03.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780005280 con 0x7fa78815ccb0 2024-09-11T12:58:03.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7880affd0 con 0x7fa78815ccb0 2024-09-11T12:58:03.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.285+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa78815dcc0 con 0x7fa78815ccb0 2024-09-11T12:58:03.288 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.289+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fa780005420 con 0x7fa78815ccb0 2024-09-11T12:58:03.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.289+0000 7fa784c00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 0x7fa768074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.289+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fa7800911f0 con 0x7fa78815ccb0 2024-09-11T12:58:03.289 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.289+0000 7fa78ca00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 0x7fa768074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.289+0000 7fa78ca00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 0x7fa768074680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa7780ae000 tx=0x7fa77801afd0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.293 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.293+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fa780016030 con 0x7fa78815ccb0 2024-09-11T12:58:03.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.457+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fa7881595b0 con 0x7fa7680721c0 2024-09-11T12:58:03.459 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.457+0000 7fa784c00640 1 -- 172.21.15.78:0/191362031 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fa7881595b0 con 0x7fa7680721c0 2024-09-11T12:58:03.459 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T12:58:03.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 msgr2=0x7fa768074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 0x7fa768074680 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa7780ae000 tx=0x7fa77801afd0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.461 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 msgr2=0x7fa7880ac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa78000ea20 tx=0x7fa78000eee0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 shutdown_connections 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fa78815ccb0 0x7fa7880ac450 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fa78815c390 0x7fa7881496c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fa7680721c0 0x7fa768074680 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 --2- 172.21.15.78:0/191362031 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fa7880b34d0 0x7fa7880ae670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 >> 172.21.15.78:0/191362031 conn(0x7fa7880b19a0 msgr2=0x7fa78815aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 shutdown_connections 2024-09-11T12:58:03.462 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.461+0000 7fa78de00640 1 -- 172.21.15.78:0/191362031 wait complete. 2024-09-11T12:58:03.481 INFO:tasks.ceph:Scrubbing osd.3 2024-09-11T12:58:03.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- 172.21.15.78:0/3682554685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 msgr2=0x7f6af4073950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- 172.21.15.78:0/3682554685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 0x7f6af4073950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800acd0 tx=0x7f6ae801dd80 comp rx=0 tx=0).stop 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- 172.21.15.78:0/3682554685 shutdown_connections 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- 172.21.15.78:0/3682554685 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af4073e90 0x7f6af4071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- 172.21.15.78:0/3682554685 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 0x7f6af4073950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- 172.21.15.78:0/3682554685 >> 172.21.15.78:0/3682554685 conn(0x7f6af40b3050 msgr2=0x7f6af40b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- 172.21.15.78:0/3682554685 shutdown_connections 2024-09-11T12:58:03.628 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- 172.21.15.78:0/3682554685 wait complete. 2024-09-11T12:58:03.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 Processor -- start 2024-09-11T12:58:03.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- start start 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af4072350 0x7f6af414c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 0x7f6af41f4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 0x7f6af41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40728f0 con 0x7f6af4073e90 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6af40707f0 con 0x7f6af4151050 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afc400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6af40b5530 con 0x7f6af4072350 2024-09-11T12:58:03.631 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 0x7f6af41f4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 0x7f6af41f4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56470/0 (socket says 172.21.15.78:56470) 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 -- 172.21.15.78:0/3611594170 learned_addr learned my addr 172.21.15.78:0/3611594170 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9a00640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af4072350 0x7f6af414c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6afa400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 0x7f6af41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 -- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af4072350 msgr2=0x7f6af414c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6af4072350 0x7f6af414c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.629+0000 7f6af9000640 1 -- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 msgr2=0x7f6af41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af9000640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6af4151050 0x7f6af41520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af9000640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af41f5270 con 0x7f6af4073e90 2024-09-11T12:58:03.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af9000640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 0x7f6af41f4910 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e9f0 tx=0x7f6ae000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000cd80 con 0x7f6af4073e90 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ae0010430 con 0x7f6af4073e90 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0010860 con 0x7f6af4073e90 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af41f5550 con 0x7f6af4073e90 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af414cd40 con 0x7f6af4073e90 2024-09-11T12:58:03.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6af40b59e0 con 0x7f6af4073e90 2024-09-11T12:58:03.634 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f6ae0010a00 con 0x7f6af4073e90 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6ad40721f0 0x7f6ad40746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f6ae0091420 con 0x7f6af4073e90 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.633+0000 7f6af9a00640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6ad40721f0 0x7f6ad40746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6af1600640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6ad40780b0 0x7f6ad407a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ae001ee50 con 0x7f6ad40780b0 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f6ae009a050 con 0x7f6af4073e90 2024-09-11T12:58:03.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6afa400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6ad40780b0 0x7f6ad407a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6af9a00640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6ad40721f0 0x7f6ad40746b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b0000 tx=0x7f6ae801c5c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6afa400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6ad40780b0 0x7f6ad407a4d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.637+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== osd.3 v2:172.21.15.78:6808/2152308697 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ae001ee50 con 0x7f6ad40780b0 2024-09-11T12:58:03.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 --> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6af4070ce0 con 0x7f6ad40780b0 2024-09-11T12:58:03.650 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6af1600640 1 -- 172.21.15.78:0/3611594170 <== osd.3 v2:172.21.15.78:6808/2152308697 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6af4070ce0 con 0x7f6ad40780b0 2024-09-11T12:58:03.650 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:03.651 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:03.651 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6ad40780b0 msgr2=0x7f6ad407a4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6808/2152308697,v1:172.21.15.78:6809/2152308697] conn(0x7f6ad40780b0 0x7f6ad407a4d0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6ad40721f0 msgr2=0x7f6ad40746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6ad40721f0 0x7f6ad40746b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6ae80b0000 tx=0x7f6ae801c5c0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 msgr2=0x7f6af41f4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 --2- 172.21.15.78:0/3611594170 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6af4073e90 0x7f6af41f4910 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000e9f0 tx=0x7f6ae000eeb0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afa400640 1 -- 172.21.15.78:0/3611594170 reap_dead start 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 shutdown_connections 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.649+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 >> 172.21.15.78:0/3611594170 conn(0x7f6af40b3050 msgr2=0x7f6af415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.653+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 shutdown_connections 2024-09-11T12:58:03.652 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.653+0000 7f6afc400640 1 -- 172.21.15.78:0/3611594170 wait complete. 2024-09-11T12:58:03.669 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 -- 172.21.15.78:0/1382102252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18154800 msgr2=0x7f2f1815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 --2- 172.21.15.78:0/1382102252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18154800 0x7f2f1815a870 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009900 tx=0x7f2f0801c8b0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 -- 172.21.15.78:0/1382102252 shutdown_connections 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 --2- 172.21.15.78:0/1382102252 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 0x7f2f18070e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 --2- 172.21.15.78:0/1382102252 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f18154800 0x7f2f1815a870 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.814 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 -- 172.21.15.78:0/1382102252 >> 172.21.15.78:0/1382102252 conn(0x7f2f1806d890 msgr2=0x7f2f180cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 -- 172.21.15.78:0/1382102252 shutdown_connections 2024-09-11T12:58:03.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.813+0000 7f2f1f000640 1 -- 172.21.15.78:0/1382102252 wait complete. 2024-09-11T12:58:03.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 Processor -- start 2024-09-11T12:58:03.815 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- start start 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 0x7f2f1815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 0x7f2f1813b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1806fac0 con 0x7f2f1815e870 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f2f1806f940 con 0x7f2f18154800 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f2f1806fc40 con 0x7f2f18070a90 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:56498/0 (socket says 172.21.15.78:56498) 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 -- 172.21.15.78:0/2492018212 learned_addr learned my addr 172.21.15.78:0/2492018212 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f17e00640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 0x7f2f1815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f17400640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 0x7f2f1813b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 -- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 msgr2=0x7f2f1815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.818 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 0x7f2f1815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 -- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 msgr2=0x7f2f1813b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 0x7f2f1813b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 -- 172.21.15.78:0/2492018212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f180a9670 con 0x7f2f1815e870 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f17400640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 0x7f2f1813b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1d000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000ea20 tx=0x7f2f1000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000cde0 con 0x7f2f1815e870 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f10004c40 con 0x7f2f1815e870 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10005280 con 0x7f2f1815e870 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f180a98e0 con 0x7f2f1815e870 2024-09-11T12:58:03.819 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f1815c7b0 con 0x7f2f1815e870 2024-09-11T12:58:03.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.817+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f1815a870 con 0x7f2f1815e870 2024-09-11T12:58:03.820 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.821+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f2f10005420 con 0x7f2f1815e870 2024-09-11T12:58:03.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.821+0000 7f2f0fe00640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 0x7f2ef8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:03.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.821+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f2f100912e0 con 0x7f2f1815e870 2024-09-11T12:58:03.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.821+0000 7f2f17e00640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 0x7f2ef8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:03.821 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.821+0000 7f2f17e00640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 0x7f2ef8074680 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2f080af000 tx=0x7f2f0801b0f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:03.824 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.825+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f2f10016030 con 0x7f2f1815e870 2024-09-11T12:58:03.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.989+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f2f180aa060 con 0x7f2ef80721c0 2024-09-11T12:58:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f0fe00640 1 -- 172.21.15.78:0/2492018212 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2f180aa060 con 0x7f2ef80721c0 2024-09-11T12:58:03.991 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 3 to deep-scrub 2024-09-11T12:58:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 msgr2=0x7f2ef8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 0x7f2ef8074680 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f2f080af000 tx=0x7f2f0801b0f0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 msgr2=0x7f2f1815f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000ea20 tx=0x7f2f1000eee0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 shutdown_connections 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f2f1815e870 0x7f2f1815f5f0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f2ef80721c0 0x7f2ef8074680 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f2f18154800 0x7f2f1813b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 --2- 172.21.15.78:0/2492018212 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f2f18070a90 0x7f2f1815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 >> 172.21.15.78:0/2492018212 conn(0x7f2f1806d890 msgr2=0x7f2f1813cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 shutdown_connections 2024-09-11T12:58:03.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:03.993+0000 7f2f1f000640 1 -- 172.21.15.78:0/2492018212 wait complete. 2024-09-11T12:58:04.012 INFO:tasks.ceph:Scrubbing osd.4 2024-09-11T12:58:04.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:04.160 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- 172.21.15.78:0/3774129954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 msgr2=0x7f57840051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 --2- 172.21.15.78:0/3774129954 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f57840051b0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f5798009ec0 tx=0x7f579801d5d0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a2800640 1 -- 172.21.15.78:0/3774129954 reap_dead start 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- 172.21.15.78:0/3774129954 shutdown_connections 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- 172.21.15.78:0/3774129954 >> 172.21.15.78:0/3774129954 conn(0x7f579c0b3050 msgr2=0x7f579c0b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- 172.21.15.78:0/3774129954 shutdown_connections 2024-09-11T12:58:04.161 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- 172.21.15.78:0/3774129954 wait complete. 2024-09-11T12:58:04.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 Processor -- start 2024-09-11T12:58:04.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- start start 2024-09-11T12:58:04.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f579c1f7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f579c141a10 0x7f579c141170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579c1599f0 0x7f579c159e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f579c163930 con 0x7f5784004d50 2024-09-11T12:58:04.164 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f579c151e70 con 0x7f579c141a10 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.161+0000 7f57a4800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f579c163ab0 con 0x7f579c1599f0 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f579c1f7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f579c1f7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42116/0 (socket says 172.21.15.78:42116) 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 -- 172.21.15.78:0/3503147065 learned_addr learned my addr 172.21.15.78:0/3503147065 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1400640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f579c141a10 0x7f579c141170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a2800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579c1599f0 0x7f579c159e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 -- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579c1599f0 msgr2=0x7f579c159e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f579c1599f0 0x7f579c159e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 -- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f579c141a10 msgr2=0x7f579c141170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f579c141a10 0x7f579c141170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c1f8450 con 0x7f5784004d50 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a1e00640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f579c1f7bc0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5790006680 tx=0x7f5790003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790015070 con 0x7f5784004d50 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f579000f960 con 0x7f5784004d50 2024-09-11T12:58:04.165 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c1f8730 con 0x7f5784004d50 2024-09-11T12:58:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579000fdb0 con 0x7f5784004d50 2024-09-11T12:58:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c161d60 con 0x7f5784004d50 2024-09-11T12:58:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f579c0723f0 con 0x7f5784004d50 2024-09-11T12:58:04.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.165+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f5790011430 con 0x7f5784004d50 2024-09-11T12:58:04.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f57800721f0 0x7f57800746b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f5790091970 con 0x7f5784004d50 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f57a1400640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f57800721f0 0x7f57800746b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f57800780b0 0x7f578007a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f578007ab80 con 0x7f57800780b0 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f57900940f0 con 0x7f5784004d50 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f57a2800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f57800780b0 0x7f578007a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.168 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f57a1400640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f57800721f0 0x7f57800746b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f579c1f7b20 tx=0x7f578c006b80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.169 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f57a2800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f57800780b0 0x7f578007a4d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.169+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== osd.4 v2:172.21.15.157:6810/2942152212 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f578007ab80 con 0x7f57800780b0 2024-09-11T12:58:04.182 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.181+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 --> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f579c072010 con 0x7f57800780b0 2024-09-11T12:58:04.183 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f5796000640 1 -- 172.21.15.78:0/3503147065 <== osd.4 v2:172.21.15.157:6810/2942152212 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f579c072010 con 0x7f57800780b0 2024-09-11T12:58:04.183 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:04.184 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f57800780b0 msgr2=0x7f578007a4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.157:6810/2942152212,v1:172.21.15.157:6811/2942152212] conn(0x7f57800780b0 0x7f578007a4d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f57800721f0 msgr2=0x7f57800746b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f57800721f0 0x7f57800746b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f579c1f7b20 tx=0x7f578c006b80 comp rx=0 tx=0).stop 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 msgr2=0x7f579c1f7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 --2- 172.21.15.78:0/3503147065 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f5784004d50 0x7f579c1f7bc0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5790006680 tx=0x7f5790003a90 comp rx=0 tx=0).stop 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a2800640 1 -- 172.21.15.78:0/3503147065 reap_dead start 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 shutdown_connections 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 >> 172.21.15.78:0/3503147065 conn(0x7f579c0b3050 msgr2=0x7f579c14ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 shutdown_connections 2024-09-11T12:58:04.185 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.185+0000 7f57a4800640 1 -- 172.21.15.78:0/3503147065 wait complete. 2024-09-11T12:58:04.201 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- 172.21.15.78:0/3427264080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed40ccb20 msgr2=0x7efed406ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- 172.21.15.78:0/3427264080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed40ccb20 0x7efed406ecf0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7efed4070fd0 tx=0x7efecc01cd40 comp rx=0 tx=0).stop 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- 172.21.15.78:0/3427264080 shutdown_connections 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- 172.21.15.78:0/3427264080 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed40ccb20 0x7efed406ecf0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- 172.21.15.78:0/3427264080 >> v2:172.21.15.157:3300/0 conn(0x7efed415bc60 0x7efed4073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- 172.21.15.78:0/3427264080 >> 172.21.15.78:0/3427264080 conn(0x7efed406ca80 msgr2=0x7efed40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- 172.21.15.78:0/3427264080 shutdown_connections 2024-09-11T12:58:04.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- 172.21.15.78:0/3427264080 wait complete. 2024-09-11T12:58:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 Processor -- start 2024-09-11T12:58:04.345 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- start start 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 0x7efed41f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 0x7efed415d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7efed4168c40 con 0x7efed415bc60 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7efed4168ac0 con 0x7efed40ccb20 2024-09-11T12:58:04.347 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efeda000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7efed4168dc0 con 0x7efed40b4780 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42130/0 (socket says 172.21.15.78:42130) 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 -- 172.21.15.78:0/271929166 learned_addr learned my addr 172.21.15.78:0/271929166 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3400640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 0x7efed41f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed2a00640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 0x7efed415d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 -- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 msgr2=0x7efed41f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 0x7efed41f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 -- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 msgr2=0x7efed415d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 0x7efed415d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3e00640 1 -- 172.21.15.78:0/271929166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efed41f8d50 con 0x7efed415bc60 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed2a00640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 0x7efed415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:04.348 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.345+0000 7efed3400640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 0x7efed41f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efed3e00640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efecc009bd0 tx=0x7efecc003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0bb070 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efecc004d40 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efed415d8d0 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc0b2040 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efed41f92d0 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efed406ecf0 con 0x7efed415bc60 2024-09-11T12:58:04.349 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7efecc0be070 con 0x7efed415bc60 2024-09-11T12:58:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 0x7efeb4074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7efecc12a680 con 0x7efed415bc60 2024-09-11T12:58:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.349+0000 7efed3400640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 0x7efeb4074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.351 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.353+0000 7efed3400640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 0x7efeb4074680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efed41429e0 tx=0x7efec4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.355 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.357+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7efecc016930 con 0x7efed415bc60 2024-09-11T12:58:04.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7efed415d670 con 0x7efeb40721c0 2024-09-11T12:58:04.525 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efecb400640 1 -- 172.21.15.78:0/271929166 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efec4003ad0 con 0x7efeb40721c0 2024-09-11T12:58:04.525 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 4 to deep-scrub 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 msgr2=0x7efeb4074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 0x7efeb4074680 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efed41429e0 tx=0x7efec4009040 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 msgr2=0x7efed414a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7efecc009bd0 tx=0x7efecc003f60 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 shutdown_connections 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7efed415bc60 0x7efed414a580 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7efeb40721c0 0x7efeb4074680 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7efed40ccb20 0x7efed415d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 --2- 172.21.15.78:0/271929166 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7efed40b4780 0x7efed41f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.527 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 >> 172.21.15.78:0/271929166 conn(0x7efed406ca80 msgr2=0x7efed414c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 shutdown_connections 2024-09-11T12:58:04.528 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.525+0000 7efeda000640 1 -- 172.21.15.78:0/271929166 wait complete. 2024-09-11T12:58:04.546 INFO:tasks.ceph:Scrubbing osd.5 2024-09-11T12:58:04.546 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- 172.21.15.78:0/1139341426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd8152990 msgr2=0x7fdcd815d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- 172.21.15.78:0/1139341426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd8152990 0x7fdcd815d060 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7fdccc009900 tx=0x7fdccc01c8e0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.692 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- 172.21.15.78:0/1139341426 shutdown_connections 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- 172.21.15.78:0/1139341426 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd80cf520 0x7fdcd80cf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- 172.21.15.78:0/1139341426 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd8152990 0x7fdcd815d060 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- 172.21.15.78:0/1139341426 >> 172.21.15.78:0/1139341426 conn(0x7fdcd80b3050 msgr2=0x7fdcd80b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- 172.21.15.78:0/1139341426 shutdown_connections 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- 172.21.15.78:0/1139341426 wait complete. 2024-09-11T12:58:04.693 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 Processor -- start 2024-09-11T12:58:04.694 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- start start 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd80b5530 0x7fdcd8143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 0x7fdcd8163b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd8152990 0x7fdcd814c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8169290 con 0x7fdcd80cf520 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd8169110 con 0x7fdcd8152990 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdce0800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fdcd8169410 con 0x7fdcd80b5530 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdcdd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 0x7fdcd8163b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.695 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdcdde00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd80b5530 0x7fdcd8143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdcdd400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 0x7fdcd8163b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42144/0 (socket says 172.21.15.78:42144) 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdcdd400640 1 -- 172.21.15.78:0/987862706 learned_addr learned my addr 172.21.15.78:0/987862706 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.693+0000 7fdcde800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd8152990 0x7fdcd814c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 -- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd80b5530 msgr2=0x7fdcd8143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fdcd80b5530 0x7fdcd8143100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 -- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd8152990 msgr2=0x7fdcd814c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fdcd8152990 0x7fdcd814c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.696 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcd81643f0 con 0x7fdcd80cf520 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdd400640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 0x7fdcd8163b10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d770 tx=0x7fdcc800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8017020 con 0x7fdcd80cf520 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc8009820 con 0x7fdcd80cf520 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8009c50 con 0x7fdcd80cf520 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcd81646d0 con 0x7fdcd80cf520 2024-09-11T12:58:04.697 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcd8164c90 con 0x7fdcd80cf520 2024-09-11T12:58:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdcd80cf960 con 0x7fdcd80cf520 2024-09-11T12:58:04.698 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fdcc801a020 con 0x7fdcd80cf520 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdcbc0721c0 0x7fdcbc074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fdcc8091060 con 0x7fdcd80cf520 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.697+0000 7fdcdde00640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdcbc0721c0 0x7fdcbc074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcd2000640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fdcbc078080 0x7fdcbc07a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdcc8013e50 con 0x7fdcbc078080 2024-09-11T12:58:04.699 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7fdcc8093870 con 0x7fdcd80cf520 2024-09-11T12:58:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcde800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fdcbc078080 0x7fdcbc07a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcdde00640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdcbc0721c0 0x7fdcbc074680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0af000 tx=0x7fdccc01b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.700 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcde800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fdcbc078080 0x7fdcbc07a4a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.701 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.701+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== osd.5 v2:172.21.15.157:6802/2089537569 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdcc8013e50 con 0x7fdcbc078080 2024-09-11T12:58:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 --> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdcd814fdb0 con 0x7fdcbc078080 2024-09-11T12:58:04.714 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdcd2000640 1 -- 172.21.15.78:0/987862706 <== osd.5 v2:172.21.15.157:6802/2089537569 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdcd814fdb0 con 0x7fdcbc078080 2024-09-11T12:58:04.714 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:04.715 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:04.715 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fdcbc078080 msgr2=0x7fdcbc07a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.157:6802/2089537569,v1:172.21.15.157:6803/2089537569] conn(0x7fdcbc078080 0x7fdcbc07a4a0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdcbc0721c0 msgr2=0x7fdcbc074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fdcbc0721c0 0x7fdcbc074680 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0af000 tx=0x7fdccc01b120 comp rx=0 tx=0).stop 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 msgr2=0x7fdcd8163b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdce0800640 1 --2- 172.21.15.78:0/987862706 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fdcd80cf520 0x7fdcd8163b10 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800d770 tx=0x7fdcc800dc30 comp rx=0 tx=0).stop 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.713+0000 7fdcde800640 1 -- 172.21.15.78:0/987862706 reap_dead start 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.717+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 shutdown_connections 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.717+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 >> 172.21.15.78:0/987862706 conn(0x7fdcd80b3050 msgr2=0x7fdcd815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.717+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 shutdown_connections 2024-09-11T12:58:04.716 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.717+0000 7fdce0800640 1 -- 172.21.15.78:0/987862706 wait complete. 2024-09-11T12:58:04.733 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-11T12:58:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 -- 172.21.15.78:0/1246094311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 msgr2=0x7fcc6c15ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.878 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 --2- 172.21.15.78:0/1246094311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15ae00 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009c80 tx=0x7fcc5c01cd70 comp rx=0 tx=0).stop 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 -- 172.21.15.78:0/1246094311 shutdown_connections 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 --2- 172.21.15.78:0/1246094311 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 0x7fcc6c0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 --2- 172.21.15.78:0/1246094311 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15ae00 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 -- 172.21.15.78:0/1246094311 >> 172.21.15.78:0/1246094311 conn(0x7fcc6c0cfeb0 msgr2=0x7fcc6c0d02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 -- 172.21.15.78:0/1246094311 shutdown_connections 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.877+0000 7fcc6be00640 1 -- 172.21.15.78:0/1246094311 wait complete. 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 Processor -- start 2024-09-11T12:58:04.879 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- start start 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 0x7fcc6c14f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc6c0721f0 0x7fcc6c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c150eb0 con 0x7fcc6c0700f0 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c074160 con 0x7fcc6c0721f0 2024-09-11T12:58:04.882 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fcc6c151030 con 0x7fcc6c071d60 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42160/0 (socket says 172.21.15.78:42160) 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 -- 172.21.15.78:0/3420828969 learned_addr learned my addr 172.21.15.78:0/3420828969 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6a000640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 0x7fcc6c14f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6b400640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc6c0721f0 0x7fcc6c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 -- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 msgr2=0x7fcc6c14f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 0x7fcc6c14f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 -- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc6c0721f0 msgr2=0x7fcc6c14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc6c0721f0 0x7fcc6c14fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 -- 172.21.15.78:0/3420828969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c1fb390 con 0x7fcc6c0700f0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6aa00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c01c7e0 tx=0x7fcc5c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0b2040 con 0x7fcc6c0700f0 2024-09-11T12:58:04.883 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc5c004d40 con 0x7fcc6c0700f0 2024-09-11T12:58:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0042a0 con 0x7fcc6c0700f0 2024-09-11T12:58:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c1fa5d0 con 0x7fcc6c0700f0 2024-09-11T12:58:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.881+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c1fab90 con 0x7fcc6c0700f0 2024-09-11T12:58:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6c0b2530 con 0x7fcc6c0700f0 2024-09-11T12:58:04.884 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fcc5c0bd060 con 0x7fcc6c0700f0 2024-09-11T12:58:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc62a00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 0x7fcc4c074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fcc5c12a6e0 con 0x7fcc6c0700f0 2024-09-11T12:58:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc6a000640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 0x7fcc4c074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:04.885 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.885+0000 7fcc6a000640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 0x7fcc4c074680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fd0 tx=0x7fcc58005db0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:04.889 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:04.889+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fcc5c016990 con 0x7fcc6c0700f0 2024-09-11T12:58:05.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.053+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7fcc6c069b00 con 0x7fcc4c0721c0 2024-09-11T12:58:05.055 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc62a00640 1 -- 172.21.15.78:0/3420828969 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fcc5800eb00 con 0x7fcc4c0721c0 2024-09-11T12:58:05.055 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 5 to deep-scrub 2024-09-11T12:58:05.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 msgr2=0x7fcc4c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 0x7fcc4c074680 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc58005fd0 tx=0x7fcc58005db0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.057 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 msgr2=0x7fcc6c15da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c01c7e0 tx=0x7fcc5c0092c0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 shutdown_connections 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fcc4c0721c0 0x7fcc4c074680 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fcc6c0721f0 0x7fcc6c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fcc6c071d60 0x7fcc6c14f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 --2- 172.21.15.78:0/3420828969 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fcc6c0700f0 0x7fcc6c15da80 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 >> 172.21.15.78:0/3420828969 conn(0x7fcc6c0cfeb0 msgr2=0x7fcc6c156070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 shutdown_connections 2024-09-11T12:58:05.058 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.057+0000 7fcc6be00640 1 -- 172.21.15.78:0/3420828969 wait complete. 2024-09-11T12:58:05.076 INFO:tasks.ceph:Scrubbing osd.6 2024-09-11T12:58:05.076 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 -- 172.21.15.78:0/1137143590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5790 msgr2=0x7f87a00723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 --2- 172.21.15.78:0/1137143590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5790 0x7f87a00723f0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f8790009900 tx=0x7f879001c8e0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 -- 172.21.15.78:0/1137143590 shutdown_connections 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 --2- 172.21.15.78:0/1137143590 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5790 0x7f87a00723f0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 -- 172.21.15.78:0/1137143590 >> 172.21.15.78:0/1137143590 conn(0x7f87a00b3050 msgr2=0x7f87a00b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 -- 172.21.15.78:0/1137143590 shutdown_connections 2024-09-11T12:58:05.222 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.221+0000 7f87a6000640 1 -- 172.21.15.78:0/1137143590 wait complete. 2024-09-11T12:58:05.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 Processor -- start 2024-09-11T12:58:05.223 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- start start 2024-09-11T12:58:05.225 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a00b5790 0x7f87a0141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 0x7f87a00b4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a0151340 0x7f87a0149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0168fc0 con 0x7f87a00b5b70 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f87a0168e40 con 0x7f87a0151340 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f87a0169140 con 0x7f87a00b5790 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 0x7f87a00b4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 0x7f87a00b4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42188/0 (socket says 172.21.15.78:42188) 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 -- 172.21.15.78:0/413940617 learned_addr learned my addr 172.21.15.78:0/413940617 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879f400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a00b5790 0x7f87a0141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879fe00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a0151340 0x7f87a0149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 -- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a00b5790 msgr2=0x7f87a0141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a00b5790 0x7f87a0141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 -- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a0151340 msgr2=0x7f87a0149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.226 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f87a0151340 0x7f87a0149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87a01416a0 con 0x7f87a00b5b70 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879f400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f87a00b5790 0x7f87a0141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f879ea00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 0x7f87a00b4920 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c500 tx=0x7f878c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c017070 con 0x7f87a00b5b70 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f878c003600 con 0x7f87a00b5b70 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c003a50 con 0x7f87a00b5b70 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87a014c070 con 0x7f87a00b5b70 2024-09-11T12:58:05.227 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87a014c630 con 0x7f87a00b5b70 2024-09-11T12:58:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.225+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f87a01517d0 con 0x7f87a00b5b70 2024-09-11T12:58:05.228 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f878c01a070 con 0x7f87a00b5b70 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87800721c0 0x7f8780074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f878c090580 con 0x7f87a00b5b70 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f879f400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87800721c0 0x7f8780074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7f8780078080 0x7f878007a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f878c01fe50 con 0x7f8780078080 2024-09-11T12:58:05.229 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f878c093000 con 0x7f87a00b5b70 2024-09-11T12:58:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f879fe00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7f8780078080 0x7f878007a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f879f400640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87800721c0 0x7f8780074680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87900af000 tx=0x7f879001b120 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.230 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f879fe00640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7f8780078080 0x7f878007a4a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.231 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.229+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== osd.6 v2:172.21.15.157:6826/899086460 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f878c01fe50 con 0x7f8780078080 2024-09-11T12:58:05.244 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 --> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f87a0072010 con 0x7f8780078080 2024-09-11T12:58:05.245 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f8797400640 1 -- 172.21.15.78:0/413940617 <== osd.6 v2:172.21.15.157:6826/899086460 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f87a0072010 con 0x7f8780078080 2024-09-11T12:58:05.245 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:05.245 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:05.245 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7f8780078080 msgr2=0x7f878007a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.157:6826/899086460,v1:172.21.15.157:6827/899086460] conn(0x7f8780078080 0x7f878007a4a0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87800721c0 msgr2=0x7f8780074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f87800721c0 0x7f8780074680 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f87900af000 tx=0x7f879001b120 comp rx=0 tx=0).stop 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 msgr2=0x7f87a00b4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 --2- 172.21.15.78:0/413940617 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f87a00b5b70 0x7f87a00b4920 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f878c00c500 tx=0x7f878c00c9c0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f879fe00640 1 -- 172.21.15.78:0/413940617 reap_dead start 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 shutdown_connections 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 >> 172.21.15.78:0/413940617 conn(0x7f87a00b3050 msgr2=0x7f87a006f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 shutdown_connections 2024-09-11T12:58:05.246 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.245+0000 7f87a6000640 1 -- 172.21.15.78:0/413940617 wait complete. 2024-09-11T12:58:05.263 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- 172.21.15.78:0/1949382264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 msgr2=0x7fd38c0700c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 --2- 172.21.15.78:0/1949382264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c0700c0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fd38c070cf0 tx=0x7fd37801c8c0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- 172.21.15.78:0/1949382264 shutdown_connections 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 --2- 172.21.15.78:0/1949382264 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c0ce450 0x7fd38c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 --2- 172.21.15.78:0/1949382264 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c0700c0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- 172.21.15.78:0/1949382264 >> 172.21.15.78:0/1949382264 conn(0x7fd38c06ca80 msgr2=0x7fd38c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- 172.21.15.78:0/1949382264 shutdown_connections 2024-09-11T12:58:05.406 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- 172.21.15.78:0/1949382264 wait complete. 2024-09-11T12:58:05.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 Processor -- start 2024-09-11T12:58:05.407 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.405+0000 7fd391600640 1 -- start start 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c0ce450 0x7fd38c153cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c13d840 0x7fd38c072fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c167b00 con 0x7fd38c06fc50 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c167980 con 0x7fd38c0ce450 2024-09-11T12:58:05.409 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fd38c167c80 con 0x7fd38c13d840 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42206/0 (socket says 172.21.15.78:42206) 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 -- 172.21.15.78:0/2394337109 learned_addr learned my addr 172.21.15.78:0/2394337109 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38b400640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c13d840 0x7fd38c072fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38a000640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c0ce450 0x7fd38c153cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 -- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c13d840 msgr2=0x7fd38c072fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c13d840 0x7fd38c072fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 -- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c0ce450 msgr2=0x7fd38c153cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c0ce450 0x7fd38c153cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 -- 172.21.15.78:0/2394337109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c154730 con 0x7fd38c06fc50 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd38aa00640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b410 tx=0x7fd37c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c015030 con 0x7fd38c06fc50 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd37c004020 con 0x7fd38c06fc50 2024-09-11T12:58:05.410 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37c0044b0 con 0x7fd38c06fc50 2024-09-11T12:58:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c154a10 con 0x7fd38c06fc50 2024-09-11T12:58:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c13e410 con 0x7fd38c06fc50 2024-09-11T12:58:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.409+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38c0700c0 con 0x7fd38c06fc50 2024-09-11T12:58:05.411 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.413+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fd37c018070 con 0x7fd38c06fc50 2024-09-11T12:58:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.413+0000 7fd382a00640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 0x7fd36c074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.413+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fd37c090440 con 0x7fd38c06fc50 2024-09-11T12:58:05.412 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.413+0000 7fd38a000640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 0x7fd36c074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.413 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.413+0000 7fd38a000640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 0x7fd36c074680 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3780af000 tx=0x7fd37801b100 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.417+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fd37c05d460 con 0x7fd38c06fc50 2024-09-11T12:58:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fd38c13e650 con 0x7fd36c0721c0 2024-09-11T12:58:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd382a00640 1 -- 172.21.15.78:0/2394337109 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd3780043c0 con 0x7fd36c0721c0 2024-09-11T12:58:05.584 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 6 to deep-scrub 2024-09-11T12:58:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 msgr2=0x7fd36c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.586 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 0x7fd36c074680 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd3780af000 tx=0x7fd37801b100 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 msgr2=0x7fd38c150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fd37c00b410 tx=0x7fd37c00b8d0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 shutdown_connections 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fd38c13d840 0x7fd38c072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fd36c0721c0 0x7fd36c074680 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fd38c0ce450 0x7fd38c153cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 --2- 172.21.15.78:0/2394337109 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fd38c06fc50 0x7fd38c150b80 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 >> 172.21.15.78:0/2394337109 conn(0x7fd38c06ca80 msgr2=0x7fd38c13c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 shutdown_connections 2024-09-11T12:58:05.587 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.585+0000 7fd391600640 1 -- 172.21.15.78:0/2394337109 wait complete. 2024-09-11T12:58:05.606 INFO:tasks.ceph:Scrubbing osd.7 2024-09-11T12:58:05.607 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- 172.21.15.78:0/4292162711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41440052f0 msgr2=0x7f414400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 --2- 172.21.15.78:0/4292162711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41440052f0 0x7f414400b9d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f4158006fa0 tx=0x7f415801cf00 comp rx=0 tx=0).stop 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- 172.21.15.78:0/4292162711 shutdown_connections 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 --2- 172.21.15.78:0/4292162711 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f41440052f0 0x7f414400b9d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- 172.21.15.78:0/4292162711 >> 172.21.15.78:0/4292162711 conn(0x7f41600b3050 msgr2=0x7f41600b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- 172.21.15.78:0/4292162711 shutdown_connections 2024-09-11T12:58:05.753 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- 172.21.15.78:0/4292162711 wait complete. 2024-09-11T12:58:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 Processor -- start 2024-09-11T12:58:05.754 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.753+0000 7f4167400640 1 -- start start 2024-09-11T12:58:05.756 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41440052f0 0x7f41600c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41601f5120 0x7f416006fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 0x7f41601f7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f4160151d70 con 0x7f4160070310 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f4160151bf0 con 0x7f41601f5120 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f41600723f0 con 0x7f41440052f0 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 0x7f41601f7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 0x7f41601f7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42230/0 (socket says 172.21.15.78:42230) 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 -- 172.21.15.78:0/2295610925 learned_addr learned my addr 172.21.15.78:0/2295610925 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f415fe00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41601f5120 0x7f416006fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4164a00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41440052f0 0x7f41600c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 -- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41440052f0 msgr2=0x7f41600c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.757 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f41440052f0 0x7f41600c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 -- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41601f5120 msgr2=0x7f416006fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f41601f5120 0x7f416006fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41600c7c50 con 0x7f4160070310 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4165400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 0x7f41601f7fd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f415801b600 tx=0x7f4158001cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158001600 con 0x7f4160070310 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4158006560 con 0x7f4160070310 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4158003860 con 0x7f4160070310 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41601f4660 con 0x7f4160070310 2024-09-11T12:58:05.758 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41601f4c20 con 0x7f4160070310 2024-09-11T12:58:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f41601f4450 con 0x7f4160070310 2024-09-11T12:58:05.759 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.757+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f41580b8030 con 0x7f4160070310 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41400721c0 0x7f4140074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f415812a8c0 con 0x7f4160070310 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4140078080 0x7f414007a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f4164a00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41400721c0 0x7f4140074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f414007ab50 con 0x7f4140078080 2024-09-11T12:58:05.760 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_get_version_reply(handle=1 version=165) ==== 24+0+0 (secure 0 0 0) 0x7f41581312a0 con 0x7f4160070310 2024-09-11T12:58:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415fe00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4140078080 0x7f414007a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f4164a00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41400721c0 0x7f4140074680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f415000eda0 tx=0x7f415000e350 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.761 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415fe00640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4140078080 0x7f414007a4a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.762 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.761+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== osd.7 v2:172.21.15.157:6818/3956663176 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f414007ab50 con 0x7f4140078080 2024-09-11T12:58:05.775 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 --> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f4160072950 con 0x7f4140078080 2024-09-11T12:58:05.776 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f415cc00640 1 -- 172.21.15.78:0/2295610925 <== osd.7 v2:172.21.15.157:6818/3956663176 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f4160072950 con 0x7f4140078080 2024-09-11T12:58:05.776 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-09-11T12:58:05.776 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T12:58:05.776 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4140078080 msgr2=0x7f414007a4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.157:6818/3956663176,v1:172.21.15.157:6819/3956663176] conn(0x7f4140078080 0x7f414007a4a0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41400721c0 msgr2=0x7f4140074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f41400721c0 0x7f4140074680 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f415000eda0 tx=0x7f415000e350 comp rx=0 tx=0).stop 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 msgr2=0x7f41601f7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 --2- 172.21.15.78:0/2295610925 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f4160070310 0x7f41601f7fd0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f415801b600 tx=0x7f4158001cc0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.777 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4165400640 1 -- 172.21.15.78:0/2295610925 reap_dead start 2024-09-11T12:58:05.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 shutdown_connections 2024-09-11T12:58:05.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 >> 172.21.15.78:0/2295610925 conn(0x7f41600b3050 msgr2=0x7f416006f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 shutdown_connections 2024-09-11T12:58:05.778 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.777+0000 7f4167400640 1 -- 172.21.15.78:0/2295610925 wait complete. 2024-09-11T12:58:05.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-11T12:58:05.937 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- 172.21.15.78:0/1227286377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed415ba80 msgr2=0x7f6ed406fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 --2- 172.21.15.78:0/1227286377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed415ba80 0x7f6ed406fc50 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4152090 tx=0x7f6ec001c7d0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- 172.21.15.78:0/1227286377 shutdown_connections 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 --2- 172.21.15.78:0/1227286377 >> v2:172.21.15.157:3300/0 conn(0x7f6ed4070190 0x7f6ed4071d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 --2- 172.21.15.78:0/1227286377 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed415ba80 0x7f6ed406fc50 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- 172.21.15.78:0/1227286377 >> 172.21.15.78:0/1227286377 conn(0x7f6ed406ca80 msgr2=0x7f6ed40b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- 172.21.15.78:0/1227286377 shutdown_connections 2024-09-11T12:58:05.938 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- 172.21.15.78:0/1227286377 wait complete. 2024-09-11T12:58:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 Processor -- start 2024-09-11T12:58:05.939 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.937+0000 7f6edb200640 1 -- start start 2024-09-11T12:58:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ed4070190 0x7f6ed41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 0x7f6ed414b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed41678c0 con 0x7f6ed4074240 2024-09-11T12:58:05.941 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f6ed4167740 con 0x7f6ed415ba80 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f6ed4167a40 con 0x7f6ed4070190 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42250/0 (socket says 172.21.15.78:42250) 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 -- 172.21.15.78:0/1733984338 learned_addr learned my addr 172.21.15.78:0/1733984338 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed9200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 0x7f6ed414b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 -- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ed4070190 msgr2=0x7f6ed41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ed4070190 0x7f6ed41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 -- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 msgr2=0x7f6ed414b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 0x7f6ed414b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 -- 172.21.15.78:0/1733984338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ed40af890 con 0x7f6ed4074240 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed9200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 0x7f6ed414b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ed3400640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c10 tx=0x7f6ec0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.942 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00bb070 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ec0005030 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ec00b2040 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ed40afb00 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ed40b00c0 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.941+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ed4151c10 con 0x7f6ed4074240 2024-09-11T12:58:05.943 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.945+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f6ec00be070 con 0x7f6ed4074240 2024-09-11T12:58:05.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.945+0000 7f6ecbe00640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 0x7f6eb4074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:05.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.945+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f6ec012a4c0 con 0x7f6ed4074240 2024-09-11T12:58:05.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.945+0000 7f6ed3e00640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 0x7f6eb4074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:05.944 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.945+0000 7f6ed3e00640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 0x7f6eb4074680 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4001400 tx=0x7f6ec4001370 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:05.948 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:05.949+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f6ec00163f0 con 0x7f6ed4074240 2024-09-11T12:58:06.113 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.113+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f6ed4151930 con 0x7f6eb40721c0 2024-09-11T12:58:06.114 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.113+0000 7f6ecbe00640 1 -- 172.21.15.78:0/1733984338 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f6ed4151930 con 0x7f6eb40721c0 2024-09-11T12:58:06.114 INFO:teuthology.orchestra.run.smithi078.stderr:instructed osd(s) 7 to deep-scrub 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 msgr2=0x7f6eb4074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 0x7f6eb4074680 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6ec4001400 tx=0x7f6ec4001370 comp rx=0 tx=0).stop 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 msgr2=0x7f6ed41650f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000c10 tx=0x7f6ec0004040 comp rx=0 tx=0).stop 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 shutdown_connections 2024-09-11T12:58:06.116 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f6eb40721c0 0x7f6eb4074680 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f6ed415ba80 0x7f6ed414b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f6ed4074240 0x7f6ed41650f0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 --2- 172.21.15.78:0/1733984338 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f6ed4070190 0x7f6ed41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 >> 172.21.15.78:0/1733984338 conn(0x7f6ed406ca80 msgr2=0x7f6ed415a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 shutdown_connections 2024-09-11T12:58:06.117 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.117+0000 7f6edb200640 1 -- 172.21.15.78:0/1733984338 wait complete. 2024-09-11T12:58:06.135 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:58:06.279 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- 172.21.15.78:0/1739819831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 msgr2=0x7f509400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 --2- 172.21.15.78:0/1739819831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f509400b9d0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f50a8009ec0 tx=0x7f50a801d610 comp rx=0 tx=0).stop 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- 172.21.15.78:0/1739819831 shutdown_connections 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 --2- 172.21.15.78:0/1739819831 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f509400b9d0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- 172.21.15.78:0/1739819831 >> 172.21.15.78:0/1739819831 conn(0x7f50b006ca80 msgr2=0x7f50b00b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- 172.21.15.78:0/1739819831 shutdown_connections 2024-09-11T12:58:06.280 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- 172.21.15.78:0/1739819831 wait complete. 2024-09-11T12:58:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 Processor -- start 2024-09-11T12:58:06.281 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- start start 2024-09-11T12:58:06.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:06.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 0x7f50b01e7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:06.283 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 0x7f50b00b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f50b015d3a0 con 0x7f50940051d0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f50b015d220 con 0x7f50b00b0be0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50b6600640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f50b00cf280 con 0x7f50b01e65f0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:42268/0 (socket says 172.21.15.78:42268) 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 -- 172.21.15.78:0/972493584 learned_addr learned my addr 172.21.15.78:0/972493584 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50aea00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 0x7f50b01e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50afe00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 0x7f50b00b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 -- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 msgr2=0x7f50b01e7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 0x7f50b01e7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 -- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 msgr2=0x7f50b00b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 0x7f50b00b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.284 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50af400640 1 -- 172.21.15.78:0/972493584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50b01e8360 con 0x7f50940051d0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50aea00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 0x7f50b01e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.281+0000 7f50afe00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 0x7f50b00b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50af400640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f50a0003ed0 tx=0x7f50a0001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0015070 con 0x7f50940051d0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a0011840 con 0x7f50940051d0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0011c90 con 0x7f50940051d0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50b01e7da0 con 0x7f50940051d0 2024-09-11T12:58:06.285 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50b0071fc0 con 0x7f50940051d0 2024-09-11T12:58:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50b00b4520 con 0x7f50940051d0 2024-09-11T12:58:06.286 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f50a0018070 con 0x7f50940051d0 2024-09-11T12:58:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 0x7f5090074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.285+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f50a00911b0 con 0x7f50940051d0 2024-09-11T12:58:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.289+0000 7f50aea00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 0x7f5090074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:06.287 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.289+0000 7f50aea00640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 0x7f5090074680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50b015ba50 tx=0x7f509c001e80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:06.290 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.289+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f50a005ead0 con 0x7f50940051d0 2024-09-11T12:58:06.453 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.453+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f50b00b15b0 con 0x7f50900721c0 2024-09-11T12:58:06.454 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.453+0000 7f50a7400640 1 -- 172.21.15.78:0/972493584 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13486 (secure 0 0 0) 0x7f50b00b15b0 con 0x7f50900721c0 2024-09-11T12:58:06.454 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:06.454 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 msgr2=0x7f5090074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 0x7f5090074680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f50b015ba50 tx=0x7f509c001e80 comp rx=0 tx=0).stop 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 msgr2=0x7f50b0142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f50a0003ed0 tx=0x7f50a0001640 comp rx=0 tx=0).stop 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 shutdown_connections 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f50900721c0 0x7f5090074680 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.456 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f50b00b0be0 0x7f50b00b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f50b01e65f0 0x7f50b01e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 --2- 172.21.15.78:0/972493584 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f50940051d0 0x7f50b0142900 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 >> 172.21.15.78:0/972493584 conn(0x7f50b006ca80 msgr2=0x7f50b015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 shutdown_connections 2024-09-11T12:58:06.457 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:06.457+0000 7f50b6600640 1 -- 172.21.15.78:0/972493584 wait complete. 2024-09-11T12:58:06.475 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-11T12:58:04.524407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287772,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754686948,"statfs":{"total":773094113280,"available":772799434752,"internally_reserved":0,"allocated":17866752,"data_stored":8496080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-154,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-2,"num_write_kb":-2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.435911"},"pg_stats":[{"pgid":"1.0","version":"72'79","reported_seq":399,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-11T12:57:56.696987+0000","last_change":"2024-09-11T12:49:35.239356+0000","last_active":"2024-09-11T12:57:56.696987+0000","last_peered":"2024-09-11T12:57:56.696987+0000","last_clean":"2024-09-11T12:57:56.696987+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:57:56.696987+0000","last_undegraded":"2024-09-11T12:57:56.696987+0000","last_fullsized":"2024-09-11T12:57:56.696987+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T12:49:34.108366+0000","last_clean_scrub_stamp":"2024-09-11T12:49:34.108366+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-13T00:38:21.723459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509545,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37972,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333868,"statfs":{"total":96636764160,"available":96597880832,"internally_reserved":0,"allocated":2678784,"data_stored":1504786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33228,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509545,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33224,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509545,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33224,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37968,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333872,"statfs":{"total":96636764160,"available":96597884928,"internally_reserved":0,"allocated":2678784,"data_stored":1504786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:58:06.479 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T12:49:34.108366+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=12, tm_min=49, tm_sec=34, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=12, tm_min=58, tm_sec=0, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T12:58:06.479 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T12:58:12.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:12.911 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:26.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T12:58:26.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- 172.21.15.78:0/2391712985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c0b4520 msgr2=0x7fbd4c0b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- 172.21.15.78:0/2391712985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c0b4520 0x7fbd4c0b4990 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009bc0 tx=0x7fbd4401cb70 comp rx=0 tx=0).stop 2024-09-11T12:58:26.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- 172.21.15.78:0/2391712985 shutdown_connections 2024-09-11T12:58:26.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- 172.21.15.78:0/2391712985 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c0b4520 0x7fbd4c0b4990 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.632 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- 172.21.15.78:0/2391712985 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c15cad0 0x7fbd4c15cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- 172.21.15.78:0/2391712985 >> 172.21.15.78:0/2391712985 conn(0x7fbd4c06ca80 msgr2=0x7fbd4c0b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:26.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- 172.21.15.78:0/2391712985 shutdown_connections 2024-09-11T12:58:26.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- 172.21.15.78:0/2391712985 wait complete. 2024-09-11T12:58:26.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 Processor -- start 2024-09-11T12:58:26.633 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- start start 2024-09-11T12:58:26.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0aed20 0x7fbd4c15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c0b4520 0x7fbd4c138360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c0cd0d0 con 0x7fbd4c15cad0 2024-09-11T12:58:26.635 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4c152d80 con 0x7fbd4c0aed20 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd50c00640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4c0cd250 con 0x7fbd4c0b4520 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd4a800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60494/0 (socket says 172.21.15.78:60494) 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd4a800640 1 -- 172.21.15.78:0/633271113 learned_addr learned my addr 172.21.15.78:0/633271113 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd49e00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0aed20 0x7fbd4c15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.632+0000 7fbd49400640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c0b4520 0x7fbd4c138360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 -- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c0b4520 msgr2=0x7fbd4c138360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c0b4520 0x7fbd4c138360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 -- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0aed20 msgr2=0x7fbd4c15d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0aed20 0x7fbd4c15d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 -- 172.21.15.78:0/633271113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4c0afa70 con 0x7fbd4c15cad0 2024-09-11T12:58:26.636 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd4a800640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401a7a0 tx=0x7fbd440a5a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:26.637 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd440b2040 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd44004d40 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd44004300 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4c0afd10 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd4c0abc10 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd4c0b4990 con 0x7fbd4c15cad0 2024-09-11T12:58:26.638 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.636+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7fbd440be060 con 0x7fbd4c15cad0 2024-09-11T12:58:26.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.640+0000 7fbd42000640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 0x7fbd2c074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.640+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7fbd4412a390 con 0x7fbd4c15cad0 2024-09-11T12:58:26.639 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.640+0000 7fbd49e00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 0x7fbd2c074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.640 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.640+0000 7fbd49e00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 0x7fbd2c074680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000f70 tx=0x7fbd3c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:26.643 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.644+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7fbd44016790 con 0x7fbd4c15cad0 2024-09-11T12:58:26.807 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 --> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbd4c0cf420 con 0x7fbd2c0721c0 2024-09-11T12:58:26.842 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd42000640 1 -- 172.21.15.78:0/633271113 <== mgr.4116 v2:172.21.15.78:6836/2730901265 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7fbd4c0cf420 con 0x7fbd2c0721c0 2024-09-11T12:58:26.843 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:26.844 INFO:teuthology.orchestra.run.smithi078.stderr:dumped all 2024-09-11T12:58:26.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 msgr2=0x7fbd2c074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 0x7fbd2c074680 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c000f70 tx=0x7fbd3c009040 comp rx=0 tx=0).stop 2024-09-11T12:58:26.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 msgr2=0x7fbd4c1388a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.844 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fbd4401a7a0 tx=0x7fbd440a5a60 comp rx=0 tx=0).stop 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 shutdown_connections 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7fbd4c15cad0 0x7fbd4c1388a0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7fbd4c0b4520 0x7fbd4c138360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7fbd2c0721c0 0x7fbd2c074680 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 --2- 172.21.15.78:0/633271113 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7fbd4c0aed20 0x7fbd4c15d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 >> 172.21.15.78:0/633271113 conn(0x7fbd4c06ca80 msgr2=0x7fbd4c06f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 shutdown_connections 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.808+0000 7fbd50c00640 1 -- 172.21.15.78:0/633271113 wait complete. 2024-09-11T12:58:26.845 INFO:teuthology.orchestra.run.smithi078.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-11T12:58:26.528860+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":287752,"kb_used_data":17448,"kb_used_omap":12,"kb_used_meta":270131,"kb_avail":754686968,"statfs":{"total":773094113280,"available":772799455232,"internally_reserved":0,"allocated":17866752,"data_stored":8496080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":276614705},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002561"},"pg_stats":[{"pgid":"1.0","version":"165'81","reported_seq":410,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-11T12:58:06.231004+0000","last_change":"2024-09-11T12:58:06.231048+0000","last_active":"2024-09-11T12:58:06.231004+0000","last_peered":"2024-09-11T12:58:06.231004+0000","last_clean":"2024-09-11T12:58:06.231004+0000","last_became_active":"2024-09-11T12:49:35.238207+0000","last_became_peered":"2024-09-11T12:49:35.238207+0000","last_unstale":"2024-09-11T12:58:06.231004+0000","last_undegraded":"2024-09-11T12:58:06.231004+0000","last_fullsized":"2024-09-11T12:58:06.231004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"165'81","last_scrub_stamp":"2024-09-11T12:58:06.230913+0000","last_deep_scrub":"165'81","last_deep_scrub_stamp":"2024-09-11T12:58:06.230913+0000","last_clean_scrub_stamp":"2024-09-11T12:58:06.230913+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T15:06:16.274641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509550,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1504786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37384,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334456,"statfs":{"total":96636764160,"available":96598482944,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37388,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334452,"statfs":{"total":96636764160,"available":96598478848,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33228,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338612,"statfs":{"total":96636764160,"available":96602738688,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33224,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33224,"kb_used_data":2036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338616,"statfs":{"total":96636764160,"available":96602742784,"internally_reserved":0,"allocated":2084864,"data_stored":914418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542254,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37960,"kb_used_data":2616,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333880,"statfs":{"total":96636764160,"available":96597893120,"internally_reserved":0,"allocated":2678784,"data_stored":1504786,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T12:58:26.846 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T12:58:26.986 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 -- 172.21.15.78:0/3445800674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a81518b0 msgr2=0x7f00a80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 --2- 172.21.15.78:0/3445800674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a81518b0 0x7f00a80b1a10 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f00a80b26f0 tx=0x7f009801caa0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 -- 172.21.15.78:0/3445800674 shutdown_connections 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 --2- 172.21.15.78:0/3445800674 >> v2:172.21.15.157:3300/0 conn(0x7f00a80ce150 0x7f00a815e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 --2- 172.21.15.78:0/3445800674 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a80b1f50 0x7f00a80cdc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 --2- 172.21.15.78:0/3445800674 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a81518b0 0x7f00a80b1a10 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 -- 172.21.15.78:0/3445800674 >> 172.21.15.78:0/3445800674 conn(0x7f00a8071dd0 msgr2=0x7f00a80721e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 -- 172.21.15.78:0/3445800674 shutdown_connections 2024-09-11T12:58:26.987 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.984+0000 7f00ad800640 1 -- 172.21.15.78:0/3445800674 wait complete. 2024-09-11T12:58:26.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 Processor -- start 2024-09-11T12:58:26.988 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- start start 2024-09-11T12:58:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00a80ce150 0x7f00a8147970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a81518b0 0x7f00a80adb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7f00a80b33e0 con 0x7f00a80b1f50 2024-09-11T12:58:26.990 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7f00a80b3260 con 0x7f00a80ce150 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7f00a80b3560 con 0x7f00a81518b0 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60510/0 (socket says 172.21.15.78:60510) 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 -- 172.21.15.78:0/1020508637 learned_addr learned my addr 172.21.15.78:0/1020508637 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a7400640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a81518b0 0x7f00a80adb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6000640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00a80ce150 0x7f00a8147970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 -- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a81518b0 msgr2=0x7f00a80adb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a81518b0 0x7f00a80adb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 -- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00a80ce150 msgr2=0x7f00a8147970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00a80ce150 0x7f00a8147970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 -- 172.21.15.78:0/1020508637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a80afb20 con 0x7f00a80b1f50 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00a6a00640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0098000f90 tx=0x7f0098009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980bb070 con 0x7f00a80b1f50 2024-09-11T12:58:26.991 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0098004d40 con 0x7f00a80b1f50 2024-09-11T12:58:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980b2040 con 0x7f00a80b1f50 2024-09-11T12:58:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a80afe00 con 0x7f00a80b1f50 2024-09-11T12:58:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.988+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a80b03c0 con 0x7f00a80b1f50 2024-09-11T12:58:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00a815e4d0 con 0x7f00a80b1f50 2024-09-11T12:58:26.992 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7f00980be070 con 0x7f00a80b1f50 2024-09-11T12:58:26.993 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f009ea00640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 0x7f0088074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:26.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7f009812a990 con 0x7f00a80b1f50 2024-09-11T12:58:26.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f00a6000640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 0x7f0088074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:26.994 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.992+0000 7f00a6000640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 0x7f0088074680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0094005e10 tx=0x7f0094005c80 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:26.997 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:26.996+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7f0098016950 con 0x7f00a80b1f50 2024-09-11T12:58:27.162 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.160+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f00a8076290 con 0x7f00a80b1f50 2024-09-11T12:58:27.166 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.164+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v50)=0 v50) ==== 123+0+0 (secure 0 0 0) 0x7f00980f7fe0 con 0x7f00a80b1f50 2024-09-11T12:58:27.167 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f009ea00640 1 -- 172.21.15.78:0/1020508637 <== mon.0 v2:172.21.15.78:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f00980f73f0 con 0x7f00a80b1f50 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 msgr2=0x7f0088074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 0x7f0088074680 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0094005e10 tx=0x7f0094005c80 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 msgr2=0x7f00a8144930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f0098000f90 tx=0x7f0098009360 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 shutdown_connections 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7f00a81518b0 0x7f00a80adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7f00880721c0 0x7f0088074680 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7f00a80ce150 0x7f00a8147970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 --2- 172.21.15.78:0/1020508637 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7f00a80b1f50 0x7f00a8144930 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 >> 172.21.15.78:0/1020508637 conn(0x7f00a8071dd0 msgr2=0x7f00a8158e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 shutdown_connections 2024-09-11T12:58:27.170 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.168+0000 7f00ad800640 1 -- 172.21.15.78:0/1020508637 wait complete. 2024-09-11T12:58:27.191 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-11T12:58:27.191 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-11T12:58:27.333 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 -- 172.21.15.78:0/1087845604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80cf7c0 msgr2=0x7ff3e815cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 --2- 172.21.15.78:0/1087845604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80cf7c0 0x7ff3e815cad0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009900 tx=0x7ff3d401c8b0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 -- 172.21.15.78:0/1087845604 shutdown_connections 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 --2- 172.21.15.78:0/1087845604 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80cf7c0 0x7ff3e815cad0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 --2- 172.21.15.78:0/1087845604 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80aed20 0x7ff3e80cf280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 -- 172.21.15.78:0/1087845604 >> 172.21.15.78:0/1087845604 conn(0x7ff3e806ca80 msgr2=0x7ff3e80b1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 -- 172.21.15.78:0/1087845604 shutdown_connections 2024-09-11T12:58:27.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 -- 172.21.15.78:0/1087845604 wait complete. 2024-09-11T12:58:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.332+0000 7ff3ed400640 1 Processor -- start 2024-09-11T12:58:27.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- start start 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 --2- >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 0x7ff3e8154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 --2- >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 0x7ff3e8154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e80cd250 con 0x7ff3e80aed20 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- --> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] -- mon_getmap magic: 0 -- 0x7ff3e80cd0d0 con 0x7ff3e815d010 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- --> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] -- mon_getmap magic: 0 -- 0x7ff3e80cd3d0 con 0x7ff3e80cf7c0 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 --2- >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.78:3300/0 says I am v2:172.21.15.78:60516/0 (socket says 172.21.15.78:60516) 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 -- 172.21.15.78:0/662332550 learned_addr learned my addr 172.21.15.78:0/662332550 (peer_addr_for_me v2:172.21.15.78:0/0) 2024-09-11T12:58:27.338 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e5e00640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 0x7ff3e8154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e7200640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 0x7ff3e8154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 -- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 msgr2=0x7ff3e8154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 0x7ff3e8154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 -- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 msgr2=0x7ff3e8154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 0x7ff3e8154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 -- 172.21.15.78:0/662332550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3e8155890 con 0x7ff3e80aed20 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e5e00640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 0x7ff3e8154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e7200640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 0x7ff3e8154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3e6800640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800e970 tx=0x7ff3d800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:27.339 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d800cd80 con 0x7ff3e80aed20 2024-09-11T12:58:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7ff3d8005020 con 0x7ff3e80aed20 2024-09-11T12:58:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3e81555b0 con 0x7ff3e80aed20 2024-09-11T12:58:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3d8005470 con 0x7ff3e80aed20 2024-09-11T12:58:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3e813a850 con 0x7ff3e80aed20 2024-09-11T12:58:27.340 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.336+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3e814eea0 con 0x7ff3e80aed20 2024-09-11T12:58:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.340+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 4 ==== mgrmap(e 26) ==== 95952+0+0 (secure 0 0 0) 0x7ff3d80047b0 con 0x7ff3e80aed20 2024-09-11T12:58:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.340+0000 7ff3dea00640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 0x7ff3c8074680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-11T12:58:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.340+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 4943+0+0 (secure 0 0 0) 0x7ff3d80914f0 con 0x7ff3e80aed20 2024-09-11T12:58:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.340+0000 7ff3e5e00640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 0x7ff3c8074680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-11T12:58:27.341 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.340+0000 7ff3e5e00640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 0x7ff3c8074680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40af000 tx=0x7ff3d4019650 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-11T12:58:27.344 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.344+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187787 (secure 0 0 0) 0x7ff3d805eb60 con 0x7ff3e80aed20 2024-09-11T12:58:27.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 --> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3e80cf280 con 0x7ff3e80aed20 2024-09-11T12:58:27.544 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3dea00640 1 -- 172.21.15.78:0/662332550 <== mon.0 v2:172.21.15.78:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 124 v124) ==== 95+0+3919 (secure 0 0 0) 0x7ff3d805e460 con 0x7ff3e80aed20 2024-09-11T12:58:27.545 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-09-11T12:58:27.545 INFO:teuthology.orchestra.run.smithi078.stdout:{"epoch":124,"btime":"2024-09-11T12:57:54:072783+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8425,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6801/2353075856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6800","nonce":2353075856},{"type":"v1","addr":"172.21.15.157:6801","nonce":2353075856}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116},{"gid":8665,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.157:6835/1288255067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.157:6834","nonce":1288255067},{"type":"v1","addr":"172.21.15.157:6835","nonce":1288255067}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":124},{"gid":9036,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6834/521184042","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":521184042},{"type":"v1","addr":"172.21.15.78:6834","nonce":521184042}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":115},{"gid":9039,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.78:6835/3086605931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6833","nonce":3086605931},{"type":"v1","addr":"172.21.15.78:6835","nonce":3086605931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":116}],"filesystems":[]} 2024-09-11T12:58:27.545 INFO:teuthology.orchestra.run.smithi078.stderr:dumped fsmap epoch 124 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 msgr2=0x7ff3c8074680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 0x7ff3c8074680 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40af000 tx=0x7ff3d4019650 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 msgr2=0x7ff3e81f68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7ff3d800e970 tx=0x7ff3d800ee30 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 shutdown_connections 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:6836/2730901265,v1:172.21.15.78:6837/2730901265] conn(0x7ff3c80721c0 0x7ff3c8074680 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3300/0,v1:172.21.15.157:6789/0] conn(0x7ff3e815d010 0x7ff3e8154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.157:3301/0,v1:172.21.15.157:6790/0] conn(0x7ff3e80cf7c0 0x7ff3e8154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 --2- 172.21.15.78:0/662332550 >> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] conn(0x7ff3e80aed20 0x7ff3e81f68d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 >> 172.21.15.78:0/662332550 conn(0x7ff3e806ca80 msgr2=0x7ff3e815c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 shutdown_connections 2024-09-11T12:58:27.547 INFO:teuthology.orchestra.run.smithi078.stderr:2024-09-11T12:58:27.544+0000 7ff3ed400640 1 -- 172.21.15.78:0/662332550 wait complete. 2024-09-11T12:58:27.565 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-11T12:58:27.565 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T12:58:27.566 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-11T12:58:27.566 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:27.578 INFO:tasks.ceph.mds.a:Stopped 2024-09-11T12:58:27.578 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-11T12:58:27.578 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:27.634 INFO:tasks.ceph.mds.c:Stopped 2024-09-11T12:58:27.634 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-11T12:58:27.634 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:27.648 INFO:tasks.ceph.mds.b:Stopped 2024-09-11T12:58:27.648 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-11T12:58:27.648 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:27.707 INFO:tasks.ceph.mds.d:Stopped 2024-09-11T12:58:27.707 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T12:58:27.707 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T12:58:27.708 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:27.801 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T12:58:27.801 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T12:58:27.802 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:30.536 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-11T12:58:33.903 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T12:58:33.904 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T12:58:33.904 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:33.990 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T12:58:33.990 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-11T12:58:33.990 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:36.444 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-11T12:58:36.445 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-11T12:58:40.092 INFO:tasks.ceph.osd.3:Stopped 2024-09-11T12:58:40.092 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-11T12:58:40.092 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:42.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-11T12:58:42.252 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-11T12:58:42.962 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:42.966 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:46.195 INFO:tasks.ceph.osd.4:Stopped 2024-09-11T12:58:46.195 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-11T12:58:46.195 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:46.283 INFO:tasks.ceph.osd.5:Stopped 2024-09-11T12:58:46.284 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-11T12:58:46.284 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:47.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-11T12:58:47.858 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-11T12:58:52.386 INFO:tasks.ceph.osd.6:Stopped 2024-09-11T12:58:52.410 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-11T12:58:52.410 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.496 INFO:tasks.ceph.osd.7:Stopped 2024-09-11T12:58:52.496 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T12:58:52.497 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-11T12:58:52.497 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.565 INFO:tasks.ceph.mgr.y:Stopped 2024-09-11T12:58:52.565 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T12:58:52.565 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.625 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T12:58:52.625 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T12:58:52.626 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T12:58:52.626 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.647 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T12:58:52.647 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T12:58:52.647 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.664 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T12:58:52.664 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T12:58:52.664 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T12:58:52.722 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T12:58:52.722 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T12:58:52.722 DEBUG:teuthology.orchestra.run.smithi078:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-11T12:58:52.760 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:58:52.760 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T12:58:52.859 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-11T12:58:52.997 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:58:52.998 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T12:58:53.219 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:58:53.219 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T12:58:53.410 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:58:53.410 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-11T12:58:53.619 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:58:53.620 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-11T12:58:53.823 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:58:53.823 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-11T12:58:54.017 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:58:54.017 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-11T12:58:54.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:58:54.193 DEBUG:teuthology.orchestra.run.smithi157:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-11T12:58:54.411 INFO:tasks.ceph:Archiving mon data... 2024-09-11T12:58:54.412 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/data/mon.a.tgz 2024-09-11T12:58:54.413 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-09-11T12:58:54.422 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.lPEoiXkl3U 2024-09-11T12:58:54.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.lPEoiXkl3U 2024-09-11T12:58:54.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.lPEoiXkl3U 2024-09-11T12:58:54.723 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.lPEoiXkl3U is 1000KB 2024-09-11T12:58:54.799 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.lPEoiXkl3U 2024-09-11T12:58:54.808 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/data/mon.b.tgz 2024-09-11T12:58:54.808 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-11T12:58:54.817 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.J2axKRVgQ6 2024-09-11T12:58:54.817 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.J2axKRVgQ6 2024-09-11T12:58:55.048 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.J2axKRVgQ6 2024-09-11T12:58:55.121 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.J2axKRVgQ6 is 1003KB 2024-09-11T12:58:55.195 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.J2axKRVgQ6 2024-09-11T12:58:55.204 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/data/mon.c.tgz 2024-09-11T12:58:55.204 DEBUG:teuthology.orchestra.run.smithi157:> mktemp 2024-09-11T12:58:55.252 INFO:teuthology.orchestra.run.smithi157.stdout:/tmp/tmp.cJ6e4Ba1xq 2024-09-11T12:58:55.252 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.cJ6e4Ba1xq 2024-09-11T12:58:55.474 DEBUG:teuthology.orchestra.run.smithi157:> sudo chmod 0666 /tmp/tmp.cJ6e4Ba1xq 2024-09-11T12:58:55.550 DEBUG:teuthology.orchestra.remote:smithi157:/tmp/tmp.cJ6e4Ba1xq is 1003KB 2024-09-11T12:58:55.619 DEBUG:teuthology.orchestra.run.smithi157:> rm -fr /tmp/tmp.cJ6e4Ba1xq 2024-09-11T12:58:55.628 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T12:58:55.628 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T12:58:55.631 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T12:58:55.699 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T12:58:55.736 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T12:58:55.737 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T12:58:55.738 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi078/crash 2024-09-11T12:58:55.739 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T12:58:55.761 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/lib/ceph/crash to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi157/crash 2024-09-11T12:58:55.762 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T12:58:55.798 INFO:tasks.ceph:Shutting down logrotate 2024-09-11T12:58:55.798 DEBUG:teuthology.orchestra.run.smithi078:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:55.809 DEBUG:teuthology.orchestra.run.smithi157:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:55.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:56.012 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-11T12:58:56.031 INFO:tasks.ceph:Compressing logs... 2024-09-11T12:58:56.031 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T12:58:56.057 DEBUG:teuthology.orchestra.run.smithi157:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T12:58:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51485.log 2024-09-11T12:58:56.080 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40674.log 2024-09-11T12:58:56.081 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54659.log 2024-09-11T12:58:56.081 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51485.log.gz 2024-09-11T12:58:56.082 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50722.log 2024-09-11T12:58:56.082 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40674.log.gz 2024-09-11T12:58:56.083 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28880.log 2024-09-11T12:58:56.084 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54659.log.gz 2024-09-11T12:58:56.084 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42859.log 2024-09-11T12:58:56.084 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50722.log.gz 2024-09-11T12:58:56.084 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45503.log 2024-09-11T12:58:56.085 INFO:teuthology.orchestra.run.smithi078.stderr: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.28880.log.gz 2024-09-11T12:58:56.085 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49105.log 2024-09-11T12:58:56.085 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42859.log.gz 2024-09-11T12:58:56.086 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34529.log 2024-09-11T12:58:56.086 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45503.log.gz 2024-09-11T12:58:56.087 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47659.log 2024-09-11T12:58:56.087 INFO:teuthology.orchestra.run.smithi078.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49105.log.gz 2024-09-11T12:58:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47123.log 2024-09-11T12:58:56.088 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34529.log.gz 2024-09-11T12:58:56.088 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47659.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28834.log 2024-09-11T12:58:56.089 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47659.log.gz 2024-09-11T12:58:56.089 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48710.log 2024-09-11T12:58:56.089 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47123.log.gz 2024-09-11T12:58:56.090 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28834.log.gz 2024-09-11T12:58:56.090 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40905.log 2024-09-11T12:58:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37333.log 2024-09-11T12:58:56.091 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48710.log.gz 2024-09-11T12:58:56.091 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41465.log 2024-09-11T12:58:56.092 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40905.log.gz 2024-09-11T12:58:56.092 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44023.log 2024-09-11T12:58:56.092 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37333.log.gz 2024-09-11T12:58:56.093 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37655.log 2024-09-11T12:58:56.093 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41465.log.gz 2024-09-11T12:58:56.094 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50846.log 2024-09-11T12:58:56.094 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44023.log.gz 2024-09-11T12:58:56.095 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49860.log 2024-09-11T12:58:56.095 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37655.log.gz 2024-09-11T12:58:56.096 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41673.log 2024-09-11T12:58:56.096 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50846.log.gz 2024-09-11T12:58:56.096 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38343.log 2024-09-11T12:58:56.096 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49860.log.gz 2024-09-11T12:58:56.097 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-09-11T12:58:56.097 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41673.log.gz 2024-09-11T12:58:56.098 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28918.log 2024-09-11T12:58:56.099 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54945.log 2024-09-11T12:58:56.099 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38343.log.gz 2024-09-11T12:58:56.099 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49549.log 2024-09-11T12:58:56.099 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-09-11T12:58:56.102 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34281.log 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.28918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29014.log 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28918.log.gz 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34281.log.gz 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.29014.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29014.log.gz 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29069.log 2024-09-11T12:58:56.103 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34409.log 2024-09-11T12:58:56.104 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41048.log 2024-09-11T12:58:56.104 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54945.log.gz 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47721.log 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49549.log.gz 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34802.log 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41048.log.gz 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45904.log 2024-09-11T12:58:56.105 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47721.log.gz 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51945.log 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34802.log.gz 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40764.log 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45904.log.gz 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45192.log 2024-09-11T12:58:56.106 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51945.log.gz 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.29069.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.29069.log.gz 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33838.log 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34409.log.gz 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33313.log 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33838.log.gz 2024-09-11T12:58:56.118 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34753.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33313.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34753.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34753.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34645.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33544.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34645.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.29045.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33544.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr: 92.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33993.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T12:58:56.119 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.33415.log 2024-09-11T12:58:56.120 INFO:teuthology.orchestra.run.smithi157.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33993.log.gz 2024-09-11T12:58:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34175.log 2024-09-11T12:58:56.120 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.3.33415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.33415.log.gz 2024-09-11T12:58:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-11T12:58:56.122 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34175.log.gz 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35204.log 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40764.log.gz 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50748.log 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45192.log.gz 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43199.log 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35204.log.gz 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36149.log/var/log/ceph/ceph-client.admin.50748.log: 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50748.log.gz 2024-09-11T12:58:56.125 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43199.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43199.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36149.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47097.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46492.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42911.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47097.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46492.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43888.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42911.log.gz 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37681.log 2024-09-11T12:58:56.126 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53361.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.43888.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39125.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37681.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52075.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53361.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51919.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39125.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52075.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51919.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51919.log.gz 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43069.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51570.log 2024-09-11T12:58:56.127 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43069.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43069.log.gz -5 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr: --verbose -- /var/log/ceph/ceph-client.admin.54064.log 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48580.log 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51570.log.gz 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41333.log 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54064.log.gz 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52660.log 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48580.log.gz 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45000.log 2024-09-11T12:58:56.128 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41333.log.gz 2024-09-11T12:58:56.129 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34646.log 2024-09-11T12:58:56.129 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52660.log.gz 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.d.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28942.log 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34873.log 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.28942.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28942.log.gz 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34873.log.gz 2024-09-11T12:58:56.133 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28990.log 2024-09-11T12:58:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.34536.log 2024-09-11T12:58:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.28990.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28990.log.gz 2024-09-11T12:58:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.34536.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.34536.log.gz 2024-09-11T12:58:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.33709.log 2024-09-11T12:58:56.134 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-11T12:58:56.135 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.1.33709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.33709.log.gz 2024-09-11T12:58:56.137 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52488.log 2024-09-11T12:58:56.137 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45000.log.gz 2024-09-11T12:58:56.137 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44383.log 2024-09-11T12:58:56.137 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34646.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49130.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52488.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40934.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44383.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49130.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37603.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40934.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38744.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41886.log 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37603.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38744.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38744.log.gz 2024-09-11T12:58:56.138 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47631.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41886.log.gz 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29082.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35203.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.47631.log.gz 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29082.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29082.log.gz 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53069.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44331.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35203.log.gz 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51693.log 2024-09-11T12:58:56.139 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53069.log.gz 2024-09-11T12:58:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T12:58:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-11T12:58:56.141 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28966.log 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44331.log.gz 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44947.log 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51693.log.gz 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42963.log 2024-09-11T12:58:56.143 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40546.log 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44947.log.gz 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42963.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42963.log.gz 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40546.log.gz 2024-09-11T12:58:56.144 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37946.log 2024-09-11T12:58:56.145 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38962.log 2024-09-11T12:58:56.145 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz 2024-09-11T12:58:56.145 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54298.log 2024-09-11T12:58:56.146 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37946.log.gz 2024-09-11T12:58:56.146 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37539.log 2024-09-11T12:58:56.146 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38962.log.gz 2024-09-11T12:58:56.147 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46941.log 2024-09-11T12:58:56.147 INFO:teuthology.orchestra.run.smithi078.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.54298.log.gz 2024-09-11T12:58:56.147 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35969.log 2024-09-11T12:58:56.147 INFO:teuthology.orchestra.run.smithi078.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37539.log.gz 2024-09-11T12:58:56.148 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.28966.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28966.log.gz 2024-09-11T12:58:56.148 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46236.log 2024-09-11T12:58:56.149 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46941.log.gz 2024-09-11T12:58:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.35969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.35969.log.gz 2024-09-11T12:58:56.149 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45956.log 2024-09-11T12:58:56.149 INFO:teuthology.orchestra.run.smithi157.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29093.log 2024-09-11T12:58:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40721.log 2024-09-11T12:58:56.150 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46236.log.gz 2024-09-11T12:58:56.150 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52764.log 2024-09-11T12:58:56.150 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45956.log.gz 2024-09-11T12:58:56.151 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.40721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35198.log 2024-09-11T12:58:56.151 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.40721.log.gz 2024-09-11T12:58:56.152 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50901.log 2024-09-11T12:58:56.152 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52764.log.gz 2024-09-11T12:58:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53309.log 2024-09-11T12:58:56.153 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35198.log.gz 2024-09-11T12:58:56.153 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39750.log 2024-09-11T12:58:56.154 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50901.log.gz 2024-09-11T12:58:56.154 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46366.log 2024-09-11T12:58:56.154 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53309.log.gz 2024-09-11T12:58:56.155 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-09-11T12:58:56.155 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39750.log.gz 2024-09-11T12:58:56.156 INFO:teuthology.orchestra.run.smithi157.stderr:/var/log/ceph/ceph-client.admin.29093.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29093.log.gz 2024-09-11T12:58:56.157 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46804.log 2024-09-11T12:58:56.157 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46366.log.gz 2024-09-11T12:58:56.157 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50370.log 2024-09-11T12:58:56.157 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-09-11T12:58:56.158 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46915.log 2024-09-11T12:58:56.158 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46804.log.gz 2024-09-11T12:58:56.158 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49342.log 2024-09-11T12:58:56.159 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50370.log.gz 2024-09-11T12:58:56.159 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46915.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46915.log.gz 2024-09-11T12:58:56.160 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45140.log 2024-09-11T12:58:56.160 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54011.log 2024-09-11T12:58:56.160 INFO:teuthology.orchestra.run.smithi078.stderr: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.49342.log.gz 2024-09-11T12:58:56.161 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43134.log 2024-09-11T12:58:56.161 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45140.log.gz 2024-09-11T12:58:56.162 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41262.log 2024-09-11T12:58:56.162 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54011.log.gz 2024-09-11T12:58:56.163 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43657.log 2024-09-11T12:58:56.163 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43134.log.gz 2024-09-11T12:58:56.164 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46778.log 2024-09-11T12:58:56.164 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41262.log.gz 2024-09-11T12:58:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38770.log 2024-09-11T12:58:56.165 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43657.log.gz 2024-09-11T12:58:56.165 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36483.log 2024-09-11T12:58:56.166 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46778.log.gz 2024-09-11T12:58:56.166 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50016.log 2024-09-11T12:58:56.166 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38770.log.gz 2024-09-11T12:58:56.167 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48929.log 2024-09-11T12:58:56.167 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36483.log.gz 2024-09-11T12:58:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50016.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50016.log.gz 2024-09-11T12:58:56.168 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48196.log 2024-09-11T12:58:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36535.log 2024-09-11T12:58:56.169 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48929.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48929.log.gz 2024-09-11T12:58:56.170 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51776.log 2024-09-11T12:58:56.170 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48196.log.gz 2024-09-11T12:58:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52576.log 2024-09-11T12:58:56.171 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36535.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36535.log.gz 2024-09-11T12:58:56.172 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53595.log 2024-09-11T12:58:56.172 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51776.log.gz 2024-09-11T12:58:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-09-11T12:58:56.173 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52576.log.gz 2024-09-11T12:58:56.173 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39672.log 2024-09-11T12:58:56.174 INFO:teuthology.orchestra.run.smithi078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.53595.log.gz 2024-09-11T12:58:56.174 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38466.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-09-11T12:58:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23967.log 2024-09-11T12:58:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39672.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39672.log.gz 2024-09-11T12:58:56.175 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41184.log 2024-09-11T12:58:56.176 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph.tmp-client.admin.23967.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23967.log.gz 2024-09-11T12:58:56.176 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44076.log 2024-09-11T12:58:56.177 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-09-11T12:58:56.177 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41184.log.gz 2024-09-11T12:58:56.178 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39926.log 2024-09-11T12:58:56.178 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44076.log.gz 2024-09-11T12:58:56.179 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49580.log 2024-09-11T12:58:56.179 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-09-11T12:58:56.179 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35219.log 2024-09-11T12:58:56.179 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39926.log.gz 2024-09-11T12:58:56.180 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49580.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41699.log 2024-09-11T12:58:56.180 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49580.log.gz 2024-09-11T12:58:56.181 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-09-11T12:58:56.181 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35219.log.gz 2024-09-11T12:58:56.182 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40520.log 2024-09-11T12:58:56.182 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41699.log.gz 2024-09-11T12:58:56.183 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37463.log 2024-09-11T12:58:56.183 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-09-11T12:58:56.184 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2024-09-11T12:58:56.184 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40520.log.gz 2024-09-11T12:58:56.185 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36665.log 2024-09-11T12:58:56.185 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37463.log.gz 2024-09-11T12:58:56.186 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-09-11T12:58:56.186 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2024-09-11T12:58:56.186 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51256.log 2024-09-11T12:58:56.186 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36665.log.gz 2024-09-11T12:58:56.187 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29023.log 2024-09-11T12:58:56.188 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-09-11T12:58:56.188 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38284.log 2024-09-11T12:58:56.188 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51256.log.gz 2024-09-11T12:58:56.189 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49442.log 2024-09-11T12:58:56.189 INFO:teuthology.orchestra.run.smithi078.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29023.log.gz 2024-09-11T12:58:56.190 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52154.log 2024-09-11T12:58:56.190 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38284.log.gz 2024-09-11T12:58:56.191 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.36097.log 2024-09-11T12:58:56.191 INFO:teuthology.orchestra.run.smithi078.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49442.log.gz 2024-09-11T12:58:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38440.log 2024-09-11T12:58:56.192 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52154.log.gz 2024-09-11T12:58:56.192 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.2.36097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.36097.log.gz 2024-09-11T12:58:56.193 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54711.log 2024-09-11T12:58:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34590.log 2024-09-11T12:58:56.194 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38440.log.gz 2024-09-11T12:58:56.194 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38076.log 2024-09-11T12:58:56.194 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54711.log.gz 2024-09-11T12:58:56.195 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35412.log 2024-09-11T12:58:56.195 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34590.log.gz 2024-09-11T12:58:56.196 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52712.log 2024-09-11T12:58:56.196 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38076.log.gz 2024-09-11T12:58:56.197 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50426.log 2024-09-11T12:58:56.197 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35412.log.gz 2024-09-11T12:58:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35738.log 2024-09-11T12:58:56.198 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52712.log.gz 2024-09-11T12:58:56.198 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2024-09-11T12:58:56.198 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50426.log.gz 2024-09-11T12:58:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35738.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35738.log.gz 2024-09-11T12:58:56.199 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52738.log 2024-09-11T12:58:56.200 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45608.log 2024-09-11T12:58:56.200 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2024-09-11T12:58:56.201 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29341.log 2024-09-11T12:58:56.201 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52738.log.gz 2024-09-11T12:58:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45608.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T12:58:56.202 INFO:teuthology.orchestra.run.smithi078.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.45608.log.gz 2024-09-11T12:58:56.202 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36398.log 2024-09-11T12:58:56.203 INFO:teuthology.orchestra.run.smithi078.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29341.log.gz 2024-09-11T12:58:56.203 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47806.log 2024-09-11T12:58:56.204 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36398.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36398.log.gz 2024-09-11T12:58:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48850.log 2024-09-11T12:58:56.205 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39579.log 2024-09-11T12:58:56.206 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47806.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47806.log.gz 2024-09-11T12:58:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47045.log 2024-09-11T12:58:56.207 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48850.log.gz 2024-09-11T12:58:56.207 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39014.log 2024-09-11T12:58:56.208 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39579.log.gz 2024-09-11T12:58:56.208 INFO:teuthology.orchestra.run.smithi078.stderr:gzip/var/log/ceph/ceph-client.admin.47045.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-09-11T12:58:56.209 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47045.log.gz 2024-09-11T12:58:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38050.log 2024-09-11T12:58:56.210 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39014.log.gz 2024-09-11T12:58:56.210 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36011.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-09-11T12:58:56.211 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50077.log 2024-09-11T12:58:56.212 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49789.log 2024-09-11T12:58:56.212 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38050.log.gz 2024-09-11T12:58:56.213 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45930.log 2024-09-11T12:58:56.213 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50077.log.gz 2024-09-11T12:58:56.214 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54272.log 2024-09-11T12:58:56.214 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49789.log.gz 2024-09-11T12:58:56.215 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5/var/log/ceph/ceph-client.admin.45930.log: --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-09-11T12:58:56.215 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45930.log.gz 2024-09-11T12:58:56.216 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47149.log 2024-09-11T12:58:56.216 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54272.log.gz 2024-09-11T12:58:56.217 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52230.log 2024-09-11T12:58:56.217 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-09-11T12:58:56.218 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36367.log 2024-09-11T12:58:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47149.log: /var/log/ceph/ceph-client.admin.52230.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.47149.log.gz 2024-09-11T12:58:56.219 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39099.log 2024-09-11T12:58:56.219 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52230.log.gz 2024-09-11T12:58:56.220 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2024-09-11T12:58:56.220 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.36367.log.gz 2024-09-11T12:58:56.221 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43555.log 2024-09-11T12:58:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39099.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39099.log.gz 2024-09-11T12:58:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54399.log 2024-09-11T12:58:56.226 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2024-09-11T12:58:56.226 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-09-11T12:58:56.226 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43555.log.gz 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36279.log 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54399.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54399.log.gz 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50255.log 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51837.log 2024-09-11T12:58:56.227 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36279.log.gz 2024-09-11T12:58:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53413.log 2024-09-11T12:58:56.228 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50255.log.gz 2024-09-11T12:58:56.228 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44617.log 2024-09-11T12:58:56.228 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51837.log.gz 2024-09-11T12:58:56.229 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53413.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38180.log 2024-09-11T12:58:56.229 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53413.log.gz 2024-09-11T12:58:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50807.log 2024-09-11T12:58:56.230 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44617.log: /var/log/ceph/ceph-client.admin.38180.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44617.log.gz 2024-09-11T12:58:56.231 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38180.log.gz 2024-09-11T12:58:56.231 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53043.log 2024-09-11T12:58:56.232 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37920.log 2024-09-11T12:58:56.232 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50807.log.gz 2024-09-11T12:58:56.233 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-09-11T12:58:56.233 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53043.log.gz 2024-09-11T12:58:56.234 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5/var/log/ceph/ceph-client.admin.37920.log: --verbose -- /var/log/ceph/ceph-client.admin.37150.log 2024-09-11T12:58:56.235 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37920.log.gz 2024-09-11T12:58:56.235 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54555.log 2024-09-11T12:58:56.235 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-09-11T12:58:56.236 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39068.log 2024-09-11T12:58:56.236 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37150.log.gz 2024-09-11T12:58:56.237 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T12:58:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54555.log: /var/log/ceph/ceph-client.admin.39068.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54555.log.gz 2024-09-11T12:58:56.238 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45123.log 2024-09-11T12:58:56.238 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39068.log.gz 2024-09-11T12:58:56.239 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40056.log 2024-09-11T12:58:56.240 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.45123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51073.log 2024-09-11T12:58:56.240 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.45123.log.gz 2024-09-11T12:58:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49711.log 2024-09-11T12:58:56.242 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40056.log.gz 2024-09-11T12:58:56.242 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34672.log 2024-09-11T12:58:56.243 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.51073.log.gz 2024-09-11T12:58:56.243 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35426.log 2024-09-11T12:58:56.244 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49711.log.gz 2024-09-11T12:58:56.245 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51433.log 2024-09-11T12:58:56.245 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34672.log.gz 2024-09-11T12:58:56.246 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47246.log 2024-09-11T12:58:56.246 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35426.log.gz 2024-09-11T12:58:56.247 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52180.log 2024-09-11T12:58:56.247 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51433.log.gz 2024-09-11T12:58:56.248 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.47246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52965.log 2024-09-11T12:58:56.248 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47246.log.gz 2024-09-11T12:58:56.249 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48261.log 2024-09-11T12:58:56.250 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52180.log.gz 2024-09-11T12:58:56.250 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39487.log 2024-09-11T12:58:56.250 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52965.log.gz 2024-09-11T12:58:56.251 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54194.log 2024-09-11T12:58:56.252 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48261.log.gz 2024-09-11T12:58:56.252 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34750.log 2024-09-11T12:58:56.253 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39487.log.gz 2024-09-11T12:58:56.253 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41912.log 2024-09-11T12:58:56.254 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54194.log.gz 2024-09-11T12:58:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37836.log 2024-09-11T12:58:56.255 INFO:teuthology.orchestra.run.smithi078.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34750.log.gz 2024-09-11T12:58:56.255 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36938.log 2024-09-11T12:58:56.256 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41912.log.gz 2024-09-11T12:58:56.257 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47404.log 2024-09-11T12:58:56.257 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37836.log.gz 2024-09-11T12:58:56.258 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51863.log 2024-09-11T12:58:56.258 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36938.log.gz 2024-09-11T12:58:56.259 INFO:teuthology.orchestra.run.smithi157.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T12:58:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47404.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47404.log.gz 2024-09-11T12:58:56.260 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35403.log 2024-09-11T12:58:56.261 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50478.log 2024-09-11T12:58:56.261 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51863.log.gz 2024-09-11T12:58:56.262 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52294.log 2024-09-11T12:58:56.262 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35403.log.gz 2024-09-11T12:58:56.263 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48498.log 2024-09-11T12:58:56.263 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50478.log.gz 2024-09-11T12:58:56.264 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54038.log 2024-09-11T12:58:56.264 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52294.log.gz 2024-09-11T12:58:56.265 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43912.log 2024-09-11T12:58:56.265 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48498.log.gz 2024-09-11T12:58:56.266 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41411.log 2024-09-11T12:58:56.266 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54038.log.gz 2024-09-11T12:58:56.267 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43912.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.43912.log.gz 2024-09-11T12:58:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40263.log 2024-09-11T12:58:56.268 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41411.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41411.log.gz 2024-09-11T12:58:56.269 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34445.log 2024-09-11T12:58:56.270 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44643.log 2024-09-11T12:58:56.270 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40263.log.gz 2024-09-11T12:58:56.271 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-09-11T12:58:56.271 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34445.log.gz 2024-09-11T12:58:56.272 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49030.log 2024-09-11T12:58:56.272 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44643.log.gz 2024-09-11T12:58:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52686.log 2024-09-11T12:58:56.274 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-09-11T12:58:56.274 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46687.log 2024-09-11T12:58:56.274 INFO:teuthology.orchestra.run.smithi078.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49030.log.gz 2024-09-11T12:58:56.275 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45638.log 2024-09-11T12:58:56.276 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.52686.log.gz 2024-09-11T12:58:56.277 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49392.log 2024-09-11T12:58:56.277 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46687.log.gz 2024-09-11T12:58:56.278 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39517.log 2024-09-11T12:58:56.278 INFO:teuthology.orchestra.run.smithi078.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45638.log.gz 2024-09-11T12:58:56.279 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46544.log 2024-09-11T12:58:56.279 INFO:teuthology.orchestra.run.smithi078.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49392.log.gz 2024-09-11T12:58:56.280 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53709.log 2024-09-11T12:58:56.281 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39517.log.gz 2024-09-11T12:58:56.281 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53980.log 2024-09-11T12:58:56.282 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46544.log.gz 2024-09-11T12:58:56.282 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48040.log 2024-09-11T12:58:56.283 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.53709.log.gz 2024-09-11T12:58:56.283 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-09-11T12:58:56.284 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53980.log.gz 2024-09-11T12:58:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51042.log 2024-09-11T12:58:56.285 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.48040.log.gz 2024-09-11T12:58:56.285 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37307.log 2024-09-11T12:58:56.286 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-09-11T12:58:56.287 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46062.log 2024-09-11T12:58:56.287 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51042.log.gz 2024-09-11T12:58:56.288 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46889.log 2024-09-11T12:58:56.288 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37307.log.gz 2024-09-11T12:58:56.289 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48554.log 2024-09-11T12:58:56.289 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46062.log.gz 2024-09-11T12:58:56.290 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40572.log 2024-09-11T12:58:56.290 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46889.log.gz 2024-09-11T12:58:56.291 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54685.log 2024-09-11T12:58:56.292 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48554.log.gz 2024-09-11T12:58:56.292 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39802.log 2024-09-11T12:58:56.293 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40572.log.gz 2024-09-11T12:58:56.293 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37803.log 2024-09-11T12:58:56.294 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54685.log.gz 2024-09-11T12:58:56.294 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45349.log 2024-09-11T12:58:56.295 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39802.log.gz 2024-09-11T12:58:56.296 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34698.log 2024-09-11T12:58:56.296 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37803.log.gz 2024-09-11T12:58:56.297 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44698.log 2024-09-11T12:58:56.297 INFO:teuthology.orchestra.run.smithi078.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45349.log.gz 2024-09-11T12:58:56.298 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54581.log 2024-09-11T12:58:56.298 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34698.log.gz 2024-09-11T12:58:56.299 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44050.log 2024-09-11T12:58:56.299 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44698.log.gz 2024-09-11T12:58:56.300 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51602.log 2024-09-11T12:58:56.300 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54581.log.gz 2024-09-11T12:58:56.301 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log 2024-09-11T12:58:56.301 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44050.log.gz 2024-09-11T12:58:56.302 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39426.log 2024-09-11T12:58:56.302 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51602.log.gz 2024-09-11T12:58:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48378.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-09-11T12:58:56.304 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29066.log 2024-09-11T12:58:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39426.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39426.log.gz 2024-09-11T12:58:56.305 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54451.log 2024-09-11T12:58:56.306 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2024-09-11T12:58:56.307 INFO:teuthology.orchestra.run.smithi078.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29066.log.gz 2024-09-11T12:58:56.308 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48903.log 2024-09-11T12:58:56.308 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54451.log.gz 2024-09-11T12:58:56.309 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48170.log 2024-09-11T12:58:56.309 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2024-09-11T12:58:56.310 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36990.log 2024-09-11T12:58:56.310 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48903.log.gz 2024-09-11T12:58:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39952.log 2024-09-11T12:58:56.311 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48170.log.gz 2024-09-11T12:58:56.311 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45553.log 2024-09-11T12:58:56.313 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36990.log.gz 2024-09-11T12:58:56.313 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48658.log 2024-09-11T12:58:56.313 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39952.log.gz 2024-09-11T12:58:56.314 INFO:teuthology.orchestra.run.smithi157.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-11T12:58:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2024-09-11T12:58:56.315 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45553.log.gz 2024-09-11T12:58:56.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42287.log 2024-09-11T12:58:56.315 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48658.log.gz 2024-09-11T12:58:56.316 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39304.log 2024-09-11T12:58:56.316 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2024-09-11T12:58:56.317 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34471.log 2024-09-11T12:58:56.317 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42287.log.gz 2024-09-11T12:58:56.318 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53335.log 2024-09-11T12:58:56.318 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39304.log.gz 2024-09-11T12:58:56.320 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35634.log 2024-09-11T12:58:56.320 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34471.log.gz 2024-09-11T12:58:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53491.log 2024-09-11T12:58:56.321 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53335.log.gz 2024-09-11T12:58:56.321 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35216.log 2024-09-11T12:58:56.322 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35634.log.gz 2024-09-11T12:58:56.323 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48118.log 2024-09-11T12:58:56.323 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53491.log.gz 2024-09-11T12:58:56.324 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35216.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35216.log.gz 2024-09-11T12:58:56.325 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49517.log 2024-09-11T12:58:56.325 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48118.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48118.log.gz 2024-09-11T12:58:56.326 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53102.log 2024-09-11T12:58:56.327 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-09-11T12:58:56.327 INFO:teuthology.orchestra.run.smithi078.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49517.log.gz 2024-09-11T12:58:56.328 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36227.log 2024-09-11T12:58:56.328 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53102.log.gz 2024-09-11T12:58:56.329 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54841.log 2024-09-11T12:58:56.329 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-09-11T12:58:56.330 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44409.log 2024-09-11T12:58:56.331 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36227.log.gz 2024-09-11T12:58:56.331 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36833.log 2024-09-11T12:58:56.332 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54841.log.gz 2024-09-11T12:58:56.333 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39724.log 2024-09-11T12:58:56.333 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44409.log.gz 2024-09-11T12:58:56.334 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46314.log 2024-09-11T12:58:56.334 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36833.log.gz 2024-09-11T12:58:56.335 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39724.log: gzip -5 --verbose 87.4% -- /var/log/ceph/ceph-client.admin.51893.log -- replaced with /var/log/ceph/ceph-client.admin.39724.log.gz 2024-09-11T12:58:56.335 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-11T12:58:56.336 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38128.log 2024-09-11T12:58:56.336 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46314.log.gz 2024-09-11T12:58:56.337 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46861.log 2024-09-11T12:58:56.337 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51893.log.gz 2024-09-11T12:58:56.338 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51009.log 2024-09-11T12:58:56.339 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38128.log.gz 2024-09-11T12:58:56.339 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39457.log 2024-09-11T12:58:56.340 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46861.log.gz 2024-09-11T12:58:56.341 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40147.log 2024-09-11T12:58:56.341 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51009.log.gz 2024-09-11T12:58:56.341 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35660.log 2024-09-11T12:58:56.342 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39457.log.gz 2024-09-11T12:58:56.343 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40147.log: gzip -5 --verbose 87.6% -- /var/log/ceph/ceph-client.admin.46570.log -- replaced with /var/log/ceph/ceph-client.admin.40147.log.gz 2024-09-11T12:58:56.343 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-11T12:58:56.344 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50344.log 2024-09-11T12:58:56.344 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35660.log.gz 2024-09-11T12:58:56.345 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42449.log 2024-09-11T12:58:56.345 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46570.log.gz 2024-09-11T12:58:56.346 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38310.log 2024-09-11T12:58:56.347 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50344.log.gz 2024-09-11T12:58:56.347 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38610.log 2024-09-11T12:58:56.348 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42449.log.gz 2024-09-11T12:58:56.348 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44154.log 2024-09-11T12:58:56.349 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38310.log.gz 2024-09-11T12:58:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38610.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-11T12:58:56.350 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38610.log.gz 2024-09-11T12:58:56.350 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49938.log 2024-09-11T12:58:56.351 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44154.log.gz 2024-09-11T12:58:56.352 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34958.log 2024-09-11T12:58:56.353 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48144.log 2024-09-11T12:58:56.354 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49938.log.gz 2024-09-11T12:58:56.354 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34958.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34958.log.gz 2024-09-11T12:58:56.355 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33794.log 2024-09-11T12:58:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48144.log.gz 2024-09-11T12:58:56.356 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2024-09-11T12:58:56.357 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.33794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45257.log 2024-09-11T12:58:56.357 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-09-11T12:58:56.358 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48684.log 2024-09-11T12:58:56.358 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz 2024-09-11T12:58:56.359 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45257.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45257.log.gz 2024-09-11T12:58:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48684.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48684.log.gz 2024-09-11T12:58:56.361 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46126.log 2024-09-11T12:58:56.362 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42261.log 2024-09-11T12:58:56.363 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40598.log 2024-09-11T12:58:56.363 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46126.log.gz 2024-09-11T12:58:56.364 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42261.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42261.log.gz 2024-09-11T12:58:56.365 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40598.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40598.log.gz 2024-09-11T12:58:56.366 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52790.log 2024-09-11T12:58:56.368 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34999.log 2024-09-11T12:58:56.369 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43704.log 2024-09-11T12:58:56.369 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52790.log.gz 2024-09-11T12:58:56.370 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34999.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34999.log.gz 2024-09-11T12:58:56.370 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.43704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2024-09-11T12:58:56.371 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43704.log.gz 2024-09-11T12:58:56.372 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50980.log 2024-09-11T12:58:56.373 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49607.log 2024-09-11T12:58:56.373 INFO:teuthology.orchestra.run.smithi078.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2024-09-11T12:58:56.374 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50980.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50980.log.gz 2024-09-11T12:58:56.375 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49607.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49607.log.gz 2024-09-11T12:58:56.376 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39900.log 2024-09-11T12:58:56.377 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36768.log 2024-09-11T12:58:56.378 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43581.log 2024-09-11T12:58:56.378 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.39900.log.gz 2024-09-11T12:58:56.380 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36768.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36768.log.gz 2024-09-11T12:58:56.380 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43581.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43581.log.gz 2024-09-11T12:58:56.381 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48736.log 2024-09-11T12:58:56.382 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36201.log 2024-09-11T12:58:56.384 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-09-11T12:58:56.384 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36201.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48736.log.gz 2024-09-11T12:58:56.384 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.36201.log.gz 2024-09-11T12:58:56.385 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49180.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-09-11T12:58:56.386 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35764.log 2024-09-11T12:58:56.387 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47290.log 2024-09-11T12:58:56.388 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42833.log 2024-09-11T12:58:56.388 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35764.log.gz 2024-09-11T12:58:56.390 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47290.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.47290.log.gz 2024-09-11T12:58:56.391 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40237.log 2024-09-11T12:58:56.391 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42833.log.gz 2024-09-11T12:58:56.392 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54503.log 2024-09-11T12:58:56.394 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54921.log 2024-09-11T12:58:56.394 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40237.log.gz 2024-09-11T12:58:56.394 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54503.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54503.log.gz 2024-09-11T12:58:56.395 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54921.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54921.log.gz 2024-09-11T12:58:56.396 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36427.log 2024-09-11T12:58:56.397 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54893.log 2024-09-11T12:58:56.399 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44277.log 2024-09-11T12:58:56.399 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36427.log.gz 2024-09-11T12:58:56.399 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54893.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54893.log.gz 2024-09-11T12:58:56.400 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44277.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44277.log.gz 2024-09-11T12:58:56.401 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51719.log 2024-09-11T12:58:56.402 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54425.log 2024-09-11T12:58:56.404 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42068.log 2024-09-11T12:58:56.404 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51719.log.gz 2024-09-11T12:58:56.404 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54425.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54425.log.gz 2024-09-11T12:58:56.405 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42068.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42068.log.gz 2024-09-11T12:58:56.406 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44102.log 2024-09-11T12:58:56.407 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53517.log 2024-09-11T12:58:56.409 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44813.log 2024-09-11T12:58:56.409 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44102.log.gz 2024-09-11T12:58:56.409 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53517.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53517.log.gz 2024-09-11T12:58:56.411 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37586.log 2024-09-11T12:58:56.411 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44813.log.gz 2024-09-11T12:58:56.412 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53226.log 2024-09-11T12:58:56.414 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34906.log 2024-09-11T12:58:56.414 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.37586.log.gz 2024-09-11T12:58:56.414 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53226.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53226.log.gz 2024-09-11T12:58:56.415 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34906.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34906.log.gz 2024-09-11T12:58:56.416 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38549.log 2024-09-11T12:58:56.417 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40440.log 2024-09-11T12:58:56.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41491.log 2024-09-11T12:58:56.419 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38549.log.gz 2024-09-11T12:58:56.419 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40440.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40440.log.gz 2024-09-11T12:58:56.420 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41491.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41491.log.gz 2024-09-11T12:58:56.421 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43225.log 2024-09-11T12:58:56.422 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54477.log 2024-09-11T12:58:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49633.log 2024-09-11T12:58:56.424 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43225.log.gz 2024-09-11T12:58:56.424 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54477.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54477.log.gz 2024-09-11T12:58:56.425 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49633.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49633.log.gz 2024-09-11T12:58:56.429 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49990.log 2024-09-11T12:58:56.429 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36253.log 2024-09-11T12:58:56.429 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41132.log 2024-09-11T12:58:56.429 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49990.log.gz 2024-09-11T12:58:56.429 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36253.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36253.log.gz 2024-09-11T12:58:56.430 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41132.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41132.log.gz 2024-09-11T12:58:56.431 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48229.log 2024-09-11T12:58:56.432 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54867.log 2024-09-11T12:58:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48229.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T12:58:56.434 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48229.log.gz 2024-09-11T12:58:56.434 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43825.log 2024-09-11T12:58:56.434 INFO:teuthology.orchestra.run.smithi078.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.54867.log.gz 2024-09-11T12:58:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.43825.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.43825.log.gz 2024-09-11T12:58:56.437 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50632.log 2024-09-11T12:58:56.438 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36613.log 2024-09-11T12:58:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50632.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50632.log.gz 2024-09-11T12:58:56.440 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36613.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36613.log.gz 2024-09-11T12:58:56.441 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45849.log 2024-09-11T12:58:56.442 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53865.log 2024-09-11T12:58:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45849.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45849.log.gz 2024-09-11T12:58:56.444 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53865.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53865.log.gz 2024-09-11T12:58:56.445 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2024-09-11T12:58:56.446 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36305.log 2024-09-11T12:58:56.448 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50582.log: 91.2%/var/log/ceph/ceph-client.admin.36305.log: -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gz 2024-09-11T12:58:56.449 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36305.log.gz 2024-09-11T12:58:56.449 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35197.log 2024-09-11T12:58:56.450 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35790.log 2024-09-11T12:58:56.452 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35197.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35197.log.gz 2024-09-11T12:58:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35790.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.35790.log.gz 2024-09-11T12:58:56.453 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53569.log 2024-09-11T12:58:56.454 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40468.log 2024-09-11T12:58:56.456 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53569.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53569.log.gz 2024-09-11T12:58:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40468.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40468.log.gz 2024-09-11T12:58:56.457 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47316.log 2024-09-11T12:58:56.459 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41106.log 2024-09-11T12:58:56.460 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47316.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47316.log.gz 2024-09-11T12:58:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41106.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41106.log.gz 2024-09-11T12:58:56.461 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34880.log 2024-09-11T12:58:56.463 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39828.log 2024-09-11T12:58:56.464 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34880.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34880.log.gz 2024-09-11T12:58:56.465 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39828.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39828.log.gz 2024-09-11T12:58:56.465 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44487.log 2024-09-11T12:58:56.467 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42640.log 2024-09-11T12:58:56.468 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44487.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44487.log.gz 2024-09-11T12:58:56.469 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42640.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42640.log.gz 2024-09-11T12:58:56.469 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44726.log 2024-09-11T12:58:56.471 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45982.log 2024-09-11T12:58:56.472 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44726.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44726.log.gz 2024-09-11T12:58:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45982.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.45982.log.gz 2024-09-11T12:58:56.473 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51405.log 2024-09-11T12:58:56.475 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40321.log 2024-09-11T12:58:56.476 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51405.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51405.log.gz 2024-09-11T12:58:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40321.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40321.log.gz 2024-09-11T12:58:56.477 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50318.log 2024-09-11T12:58:56.478 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50930.log 2024-09-11T12:58:56.480 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50318.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.50318.log.gz 2024-09-11T12:58:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50930.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50930.log.gz 2024-09-11T12:58:56.481 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40356.log 2024-09-11T12:58:56.483 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48014.log 2024-09-11T12:58:56.484 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40356.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40356.log.gz 2024-09-11T12:58:56.485 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48014.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48014.log.gz 2024-09-11T12:58:56.486 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41576.log 2024-09-11T12:58:56.487 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46833.log 2024-09-11T12:58:56.488 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41576.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41576.log.gz 2024-09-11T12:58:56.489 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46833.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46833.log.gz 2024-09-11T12:58:56.490 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45427.log 2024-09-11T12:58:56.491 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39549.log 2024-09-11T12:58:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45427.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45427.log.gz 2024-09-11T12:58:56.493 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39549.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39549.log.gz 2024-09-11T12:58:56.494 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51971.log 2024-09-11T12:58:56.495 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52128.log 2024-09-11T12:58:56.496 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51971.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51971.log.gz 2024-09-11T12:58:56.497 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52128.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52128.log.gz 2024-09-11T12:58:56.498 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53692.log 2024-09-11T12:58:56.499 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51667.log 2024-09-11T12:58:56.500 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.53692.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.53692.log.gz 2024-09-11T12:58:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51667.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51667.log.gz 2024-09-11T12:58:56.502 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2024-09-11T12:58:56.503 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-09-11T12:58:56.504 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40738.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2024-09-11T12:58:56.505 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36587.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-09-11T12:58:56.506 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52861.log 2024-09-11T12:58:56.506 INFO:teuthology.orchestra.run.smithi157.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T12:58:56.507 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39211.log 2024-09-11T12:58:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52861.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52861.log.gz 2024-09-11T12:58:56.509 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39211.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39211.log.gz 2024-09-11T12:58:56.510 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42191.log 2024-09-11T12:58:56.510 INFO:teuthology.orchestra.run.smithi157.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T12:58:56.511 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50556.log 2024-09-11T12:58:56.512 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.42191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.42191.log.gz 2024-09-11T12:58:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50556.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50556.log.gz 2024-09-11T12:58:56.514 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54633.log 2024-09-11T12:58:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54633.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54633.log.gz 2024-09-11T12:58:56.517 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29038.log 2024-09-11T12:58:56.518 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41288.log 2024-09-11T12:58:56.520 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29038.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.29038.log.gz 2024-09-11T12:58:56.521 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41288.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41288.log.gz 2024-09-11T12:58:56.521 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43425.log 2024-09-11T12:58:56.523 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40004.log 2024-09-11T12:58:56.524 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43425.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43425.log.gz 2024-09-11T12:58:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40004.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40004.log.gz 2024-09-11T12:58:56.525 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37774.log 2024-09-11T12:58:56.527 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43721.log 2024-09-11T12:58:56.528 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37774.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.37774.log.gz 2024-09-11T12:58:56.529 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43721.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43721.log.gz 2024-09-11T12:58:56.530 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39646.log 2024-09-11T12:58:56.531 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49659.log 2024-09-11T12:58:56.533 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39646.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39646.log.gz 2024-09-11T12:58:56.533 INFO:teuthology.orchestra.run.smithi078.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-11T12:58:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49659.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49659.log.gz 2024-09-11T12:58:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35009.log 2024-09-11T12:58:56.534 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49255.log 2024-09-11T12:58:56.536 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35009.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35009.log.gz 2024-09-11T12:58:56.536 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38375.log 2024-09-11T12:58:56.537 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51204.log 2024-09-11T12:58:56.537 INFO:teuthology.orchestra.run.smithi078.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49255.log.gz 2024-09-11T12:58:56.538 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38375.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38375.log.gz 2024-09-11T12:58:56.538 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48833.log 2024-09-11T12:58:56.539 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51204.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51204.log.gz 2024-09-11T12:58:56.539 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44895.log 2024-09-11T12:58:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54220.log 2024-09-11T12:58:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.48833.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48833.log.gz 2024-09-11T12:58:56.541 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44895.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44895.log.gz 2024-09-11T12:58:56.542 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54971.log 2024-09-11T12:58:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54220.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54220.log.gz 2024-09-11T12:58:56.543 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40850.log 2024-09-11T12:58:56.544 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35011.log 2024-09-11T12:58:56.545 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54971.log.gz 2024-09-11T12:58:56.545 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40850.log: gzip 87.1% -- replaced with /var/log/ceph/ceph-client.admin.40850.log.gz 2024-09-11T12:58:56.546 INFO:teuthology.orchestra.run.smithi078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44461.log 2024-09-11T12:58:56.547 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42475.log 2024-09-11T12:58:56.547 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35011.log.gz 2024-09-11T12:58:56.548 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-11T12:58:56.548 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44461.log.gz 2024-09-11T12:58:56.549 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42475.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42475.log.gz 2024-09-11T12:58:56.549 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40082.log 2024-09-11T12:58:56.550 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46411.log 2024-09-11T12:58:56.551 INFO:teuthology.orchestra.run.smithi078.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-11T12:58:56.552 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47685.log 2024-09-11T12:58:56.552 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40082.log.gz 2024-09-11T12:58:56.553 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46411.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46411.log.gz 2024-09-11T12:58:56.553 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54142.log 2024-09-11T12:58:56.554 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47832.log 2024-09-11T12:58:56.554 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47685.log.gz 2024-09-11T12:58:56.555 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37070.log 2024-09-11T12:58:56.555 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54142.log.gz 2024-09-11T12:58:56.556 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47832.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47832.log.gz 2024-09-11T12:58:56.556 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51126.log 2024-09-11T12:58:56.558 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52418.log 2024-09-11T12:58:56.558 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37070.log.gz 2024-09-11T12:58:56.559 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51126.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51126.log.gz 2024-09-11T12:58:56.559 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2024-09-11T12:58:56.560 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52418.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52418.log.gz 2024-09-11T12:58:56.560 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39396.log 2024-09-11T12:58:56.561 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34419.log 2024-09-11T12:58:56.562 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49964.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2024-09-11T12:58:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44128.log 2024-09-11T12:58:56.563 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39396.log.gz 2024-09-11T12:58:56.563 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34419.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34419.log.gz 2024-09-11T12:58:56.564 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41608.log 2024-09-11T12:58:56.565 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49155.log 2024-09-11T12:58:56.565 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44128.log.gz 2024-09-11T12:58:56.566 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45026.log 2024-09-11T12:58:56.566 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41608.log.gz 2024-09-11T12:58:56.567 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51327.log 2024-09-11T12:58:56.567 INFO:teuthology.orchestra.run.smithi078.stderr: 85.1% -- replaced with /var/log/ceph/ceph-client.admin.49155.log.gz 2024-09-11T12:58:56.568 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53017.log 2024-09-11T12:58:56.569 INFO:teuthology.orchestra.run.smithi078.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.45026.log.gz/var/log/ceph/ceph-client.admin.51327.log: 2024-09-11T12:58:56.569 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51327.log.gz 2024-09-11T12:58:56.570 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46466.log 2024-09-11T12:58:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53017.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53017.log.gz 2024-09-11T12:58:56.571 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43251.log 2024-09-11T12:58:56.572 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-09-11T12:58:56.573 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46466.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46466.log.gz 2024-09-11T12:58:56.573 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43251.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43251.log.gz 2024-09-11T12:58:56.574 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38718.log 2024-09-11T12:58:56.575 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54168.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-09-11T12:58:56.575 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35850.log 2024-09-11T12:58:56.576 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45375.log 2024-09-11T12:58:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38718.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38718.log.gz 2024-09-11T12:58:56.577 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37177.log 2024-09-11T12:58:56.577 INFO:teuthology.orchestra.run.smithi078.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.35850.log.gz 2024-09-11T12:58:56.578 INFO:teuthology.orchestra.run.smithi157.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-11T12:58:56.578 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45375.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45375.log.gz 2024-09-11T12:58:56.579 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52023.log 2024-09-11T12:58:56.580 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54789.log 2024-09-11T12:58:56.580 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37177.log.gz 2024-09-11T12:58:56.581 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52023.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T12:58:56.581 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52023.log.gz 2024-09-11T12:58:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54789.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54789.log.gz 2024-09-11T12:58:56.582 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52887.log 2024-09-11T12:58:56.583 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52607.log 2024-09-11T12:58:56.585 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52887.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52887.log.gz 2024-09-11T12:58:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36639.log 2024-09-11T12:58:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52607.log: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T12:58:56.586 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52607.log.gz 2024-09-11T12:58:56.586 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38584.log 2024-09-11T12:58:56.588 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43503.log 2024-09-11T12:58:56.588 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36639.log.gz 2024-09-11T12:58:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38584.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38584.log.gz 2024-09-11T12:58:56.589 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49367.log 2024-09-11T12:58:56.590 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42781.log 2024-09-11T12:58:56.590 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43503.log.gz 2024-09-11T12:58:56.591 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44843.log 2024-09-11T12:58:56.591 INFO:teuthology.orchestra.run.smithi078.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49367.log.gz 2024-09-11T12:58:56.592 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42781.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42781.log.gz 2024-09-11T12:58:56.592 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45875.log 2024-09-11T12:58:56.593 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51178.log 2024-09-11T12:58:56.594 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44843.log.gz 2024-09-11T12:58:56.595 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40624.log 2024-09-11T12:58:56.595 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45875.log.gz 2024-09-11T12:58:56.596 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51178.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51178.log.gz 2024-09-11T12:58:56.596 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48326.log 2024-09-11T12:58:56.597 INFO:teuthology.orchestra.run.smithi078.stderr:gzip/var/log/ceph/ceph-client.admin.40624.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52538.log 2024-09-11T12:58:56.598 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48326.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.40624.log.gz 2024-09-11T12:58:56.598 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48326.log.gz 2024-09-11T12:58:56.599 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39776.log 2024-09-11T12:58:56.600 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52538.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52538.log.gz 2024-09-11T12:58:56.600 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47962.log 2024-09-11T12:58:56.601 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50042.log 2024-09-11T12:58:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39776.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39776.log.gz 2024-09-11T12:58:56.602 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2024-09-11T12:58:56.602 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47962.log.gz 2024-09-11T12:58:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50042.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50042.log.gz 2024-09-11T12:58:56.603 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51811.log 2024-09-11T12:58:56.605 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44669.log 2024-09-11T12:58:56.605 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2024-09-11T12:58:56.606 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-09-11T12:58:56.606 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51811.log.gz 2024-09-11T12:58:56.607 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44669.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44669.log.gz 2024-09-11T12:58:56.607 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29430.log 2024-09-11T12:58:56.608 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43343.log 2024-09-11T12:58:56.608 INFO:teuthology.orchestra.run.smithi078.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-09-11T12:58:56.609 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52102.log 2024-09-11T12:58:56.609 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.29430.log.gz 2024-09-11T12:58:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43343.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43343.log.gz 2024-09-11T12:58:56.610 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42397.log 2024-09-11T12:58:56.611 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44973.log 2024-09-11T12:58:56.612 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52102.log.gz 2024-09-11T12:58:56.613 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37229.log 2024-09-11T12:58:56.613 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42397.log.gz 2024-09-11T12:58:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44973.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44973.log.gz 2024-09-11T12:58:56.614 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51100.log 2024-09-11T12:58:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42315.log 2024-09-11T12:58:56.616 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37229.log.gz 2024-09-11T12:58:56.616 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35411.log 2024-09-11T12:58:56.616 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51100.log.gz 2024-09-11T12:58:56.617 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42315.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42315.log.gz 2024-09-11T12:58:56.618 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35423.log 2024-09-11T12:58:56.619 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42538.log 2024-09-11T12:58:56.619 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35411.log.gz 2024-09-11T12:58:56.620 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35423.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35423.log.gz 2024-09-11T12:58:56.620 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34353.log 2024-09-11T12:58:56.621 INFO:teuthology.orchestra.run.smithi078.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.42538.log.gz 2024-09-11T12:58:56.621 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48437.log 2024-09-11T12:58:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50679.log 2024-09-11T12:58:56.623 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34353.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34353.log.gz 2024-09-11T12:58:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53763.log 2024-09-11T12:58:56.624 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48437.log.gz 2024-09-11T12:58:56.624 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.50679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50679.log.gz 2024-09-11T12:58:56.625 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36885.log 2024-09-11T12:58:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47071.log 2024-09-11T12:58:56.626 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53763.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53763.log.gz 2024-09-11T12:58:56.627 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45218.log 2024-09-11T12:58:56.627 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36885.log.gz 2024-09-11T12:58:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47071.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47071.log.gz 2024-09-11T12:58:56.628 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40408.log 2024-09-11T12:58:56.629 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53737.log 2024-09-11T12:58:56.629 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45218.log.gz 2024-09-11T12:58:56.631 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41752.log 2024-09-11T12:58:56.631 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40408.log.gz 2024-09-11T12:58:56.631 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53737.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53737.log.gz 2024-09-11T12:58:56.632 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-09-11T12:58:56.633 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34932.log 2024-09-11T12:58:56.633 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41752.log.gz 2024-09-11T12:58:56.634 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41079.log 2024-09-11T12:58:56.634 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-09-11T12:58:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34932.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34932.log.gz 2024-09-11T12:58:56.635 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43849.log 2024-09-11T12:58:56.636 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52913.log 2024-09-11T12:58:56.637 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41079.log.gz 2024-09-11T12:58:56.638 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49763.log 2024-09-11T12:58:56.638 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43849.log.gz 2024-09-11T12:58:56.638 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52913.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52913.log.gz 2024-09-11T12:58:56.639 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37629.log 2024-09-11T12:58:56.640 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41834.log 2024-09-11T12:58:56.640 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49763.log.gz 2024-09-11T12:58:56.641 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51537.log 2024-09-11T12:58:56.641 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.37629.log.gz 2024-09-11T12:58:56.642 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41834.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41834.log.gz 2024-09-11T12:58:56.642 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41543.log 2024-09-11T12:58:56.644 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41964.log 2024-09-11T12:58:56.644 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51537.log.gz 2024-09-11T12:58:56.645 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35428.log 2024-09-11T12:58:56.645 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41543.log.gz 2024-09-11T12:58:56.646 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41964.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41964.log.gz 2024-09-11T12:58:56.646 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52634.log 2024-09-11T12:58:56.647 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54324.log 2024-09-11T12:58:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35428.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35428.log.gz 2024-09-11T12:58:56.648 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52634.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52634.log.gz 2024-09-11T12:58:56.649 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40790.log 2024-09-11T12:58:56.649 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54324.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54324.log.gz 2024-09-11T12:58:56.650 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38024.log 2024-09-11T12:58:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52049.log 2024-09-11T12:58:56.651 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40790.log.gz 2024-09-11T12:58:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38024.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38024.log.gz 2024-09-11T12:58:56.652 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41725.log 2024-09-11T12:58:56.653 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52049.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52049.log.gz 2024-09-11T12:58:56.654 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52346.log 2024-09-11T12:58:56.655 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52991.log 2024-09-11T12:58:56.655 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41725.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41725.log.gz 2024-09-11T12:58:56.656 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52346.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52346.log.gz 2024-09-11T12:58:56.656 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42937.log 2024-09-11T12:58:56.657 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52991.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.52991.log.gz 2024-09-11T12:58:56.657 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49685.log 2024-09-11T12:58:56.658 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34998.log 2024-09-11T12:58:56.659 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42937.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42937.log.gz 2024-09-11T12:58:56.660 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49685.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49685.log.gz 2024-09-11T12:58:56.660 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42807.log 2024-09-11T12:58:56.661 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34998.log.gz 2024-09-11T12:58:56.661 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48472.log 2024-09-11T12:58:56.662 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38102.log 2024-09-11T12:58:56.663 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42807.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42807.log.gz 2024-09-11T12:58:56.664 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48472.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48472.log.gz 2024-09-11T12:58:56.664 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47553.log 2024-09-11T12:58:56.665 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38102.log.gz 2024-09-11T12:58:56.665 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38206.log 2024-09-11T12:58:56.666 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39040.log 2024-09-11T12:58:56.667 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47553.log.gz 2024-09-11T12:58:56.668 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38206.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.38206.log.gz 2024-09-11T12:58:56.668 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43936.log 2024-09-11T12:58:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39040.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39040.log.gz 2024-09-11T12:58:56.669 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38692.log 2024-09-11T12:58:56.670 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43936.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.43936.log.gz 2024-09-11T12:58:56.671 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47858.log 2024-09-11T12:58:56.672 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45823.log 2024-09-11T12:58:56.672 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38692.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.38692.log.gz 2024-09-11T12:58:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47858.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47858.log.gz 2024-09-11T12:58:56.673 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42371.log 2024-09-11T12:58:56.674 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45823.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45823.log.gz 2024-09-11T12:58:56.674 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53543.log 2024-09-11T12:58:56.675 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45582.log 2024-09-11T12:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42371.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42371.log.gz 2024-09-11T12:58:56.676 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53543.log.gz 2024-09-11T12:58:56.677 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35427.log 2024-09-11T12:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45582.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45582.log.gz 2024-09-11T12:58:56.678 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37867.log 2024-09-11T12:58:56.679 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45297.log 2024-09-11T12:58:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35427.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35427.log.gz 2024-09-11T12:58:56.680 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37489.log 2024-09-11T12:58:56.680 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37867.log.gz 2024-09-11T12:58:56.681 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45297.log.gz 2024-09-11T12:58:56.681 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47780.log 2024-09-11T12:58:56.682 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52456.log 2024-09-11T12:58:56.683 INFO:teuthology.orchestra.run.smithi078.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.37489.log.gz/var/log/ceph/ceph-client.admin.47780.log: 2024-09-11T12:58:56.684 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47780.log.gz 2024-09-11T12:58:56.684 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37281.log 2024-09-11T12:58:56.685 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52456.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52456.log.gz 2024-09-11T12:58:56.685 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42666.log 2024-09-11T12:58:56.686 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34320.log 2024-09-11T12:58:56.687 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37281.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.37281.log.gz 2024-09-11T12:58:56.688 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42666.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42666.log.gz 2024-09-11T12:58:56.688 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51300.log 2024-09-11T12:58:56.689 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34320.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34320.log.gz 2024-09-11T12:58:56.689 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37203.log 2024-09-11T12:58:56.690 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43773.log 2024-09-11T12:58:56.691 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51300.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51300.log.gz 2024-09-11T12:58:56.692 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37203.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37203.log.gz 2024-09-11T12:58:56.692 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49467.log 2024-09-11T12:58:56.693 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43773.log.gz 2024-09-11T12:58:56.693 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53134.log 2024-09-11T12:58:56.694 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48092.log 2024-09-11T12:58:56.695 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49467.log: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49467.log.gz 2024-09-11T12:58:56.696 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36457.log 2024-09-11T12:58:56.696 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53134.log.gz 2024-09-11T12:58:56.696 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48092.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48092.log.gz 2024-09-11T12:58:56.697 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51230.log 2024-09-11T12:58:56.698 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37042.log 2024-09-11T12:58:56.698 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.36457.log.gz 2024-09-11T12:58:56.699 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53387.log 2024-09-11T12:58:56.699 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51230.log.gz 2024-09-11T12:58:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37042.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37042.log.gz 2024-09-11T12:58:56.700 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-09-11T12:58:56.701 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54737.log 2024-09-11T12:58:56.701 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53387.log.gz 2024-09-11T12:58:56.703 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36736.log 2024-09-11T12:58:56.703 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-09-11T12:58:56.704 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43308.log 2024-09-11T12:58:56.704 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54737.log.gz 2024-09-11T12:58:56.705 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49912.log 2024-09-11T12:58:56.705 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36736.log.gz 2024-09-11T12:58:56.706 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45166.log 2024-09-11T12:58:56.706 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43308.log.gz 2024-09-11T12:58:56.707 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49912.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49912.log.gz 2024-09-11T12:58:56.707 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46012.log 2024-09-11T12:58:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36964.log 2024-09-11T12:58:56.709 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45166.log.gz 2024-09-11T12:58:56.709 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46012.log.gz 2024-09-11T12:58:56.710 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40879.log 2024-09-11T12:58:56.711 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36964.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36964.log.gz 2024-09-11T12:58:56.711 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52372.log 2024-09-11T12:58:56.713 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36912.log 2024-09-11T12:58:56.713 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40879.log.gz 2024-09-11T12:58:56.714 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52372.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.47884.log 2024-09-11T12:58:56.714 INFO:teuthology.orchestra.run.smithi078.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52372.log.gz 2024-09-11T12:58:56.715 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49005.log 2024-09-11T12:58:56.715 INFO:teuthology.orchestra.run.smithi078.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.36912.log.gz 2024-09-11T12:58:56.716 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47884.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47884.log.gz 2024-09-11T12:58:56.717 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53281.log 2024-09-11T12:58:56.717 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49005.log: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49005.log.gz 2024-09-11T12:58:56.718 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49492.log 2024-09-11T12:58:56.719 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47442.log 2024-09-11T12:58:56.719 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53281.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53281.log.gz 2024-09-11T12:58:56.720 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53465.log 2024-09-11T12:58:56.720 INFO:teuthology.orchestra.run.smithi078.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.49492.log.gz 2024-09-11T12:58:56.721 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47442.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47442.log.gz 2024-09-11T12:58:56.721 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49280.log 2024-09-11T12:58:56.722 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39155.log 2024-09-11T12:58:56.722 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53465.log.gz 2024-09-11T12:58:56.723 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52277.log 2024-09-11T12:58:56.724 INFO:teuthology.orchestra.run.smithi078.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49280.log.gz 2024-09-11T12:58:56.724 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39155.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.39155.log.gz 2024-09-11T12:58:56.725 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51748.log 2024-09-11T12:58:56.726 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.52277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51511.log 2024-09-11T12:58:56.726 INFO:teuthology.orchestra.run.smithi078.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.52277.log.gz 2024-09-11T12:58:56.727 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51748.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T12:58:56.727 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51748.log.gz 2024-09-11T12:58:56.728 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51511.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.51511.log.gz 2024-09-11T12:58:56.728 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53894.log 2024-09-11T12:58:56.729 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-09-11T12:58:56.731 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53894.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53894.log.gz 2024-09-11T12:58:56.731 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28844.log 2024-09-11T12:58:56.733 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48407.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-09-11T12:58:56.733 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-11T12:58:56.733 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28844.log: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T12:58:56.734 INFO:teuthology.orchestra.run.smithi078.stderr: 76.7% -- replaced with /var/log/ceph/ceph-client.admin.28844.log.gz 2024-09-11T12:58:56.734 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51997.log 2024-09-11T12:58:56.736 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46210.log 2024-09-11T12:58:56.737 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51997.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.51997.log.gz 2024-09-11T12:58:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46210.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46210.log.gz 2024-09-11T12:58:56.738 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42345.log 2024-09-11T12:58:56.740 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35007.log 2024-09-11T12:58:56.741 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42345.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42345.log.gz 2024-09-11T12:58:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35007.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35007.log.gz 2024-09-11T12:58:56.742 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42016.log 2024-09-11T12:58:56.744 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35012.log 2024-09-11T12:58:56.745 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42016.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.42016.log.gz 2024-09-11T12:58:56.746 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35012.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35012.log.gz 2024-09-11T12:58:56.746 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50696.log 2024-09-11T12:58:56.748 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-11T12:58:56.749 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50696.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.50696.log.gz 2024-09-11T12:58:56.755 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2024-09-11T12:58:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-09-11T12:58:56.757 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38258.log 2024-09-11T12:58:56.757 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2024-09-11T12:58:56.759 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47604.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-09-11T12:58:56.760 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48786.log 2024-09-11T12:58:56.762 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38258.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38258.log.gz 2024-09-11T12:58:56.763 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53439.log 2024-09-11T12:58:56.765 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37016.log 2024-09-11T12:58:56.766 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2024-09-11T12:58:56.766 INFO:teuthology.orchestra.run.smithi078.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53439.log.gz 2024-09-11T12:58:56.770 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48786.log: /var/log/ceph/ceph-client.admin.45401.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48786.log.gz 2024-09-11T12:58:56.771 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2024-09-11T12:58:56.772 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38796.log 2024-09-11T12:58:56.773 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.37016.log.gz 2024-09-11T12:58:56.777 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51379.log 2024-09-11T12:58:56.778 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38796.log.gz 2024-09-11T12:58:56.779 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28868.log 2024-09-11T12:58:56.782 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51379.log: /var/log/ceph/ceph-client.admin.28868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28868.log.gz 2024-09-11T12:58:56.784 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37098.log 2024-09-11T12:58:56.787 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37098.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37098.log.gz 2024-09-11T12:58:56.787 INFO:teuthology.orchestra.run.smithi078.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51379.log.gz 2024-09-11T12:58:56.787 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48352.log 2024-09-11T12:58:56.790 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48352.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48352.log.gz 2024-09-11T12:58:56.790 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50174.log 2024-09-11T12:58:56.796 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50174.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50174.log.gz 2024-09-11T12:58:56.796 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35404.log 2024-09-11T12:58:56.796 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35404.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35404.log.gz 2024-09-11T12:58:56.796 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46184.log 2024-09-11T12:58:56.801 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46184.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46184.log.gz 2024-09-11T12:58:56.802 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52320.log 2024-09-11T12:58:56.802 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52320.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52320.log.gz 2024-09-11T12:58:56.802 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46440.log 2024-09-11T12:58:56.804 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46440.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46440.log.gz 2024-09-11T12:58:56.805 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37437.log 2024-09-11T12:58:56.809 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37437.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37437.log.gz 2024-09-11T12:58:56.809 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49205.log 2024-09-11T12:58:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49205.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.49205.log.gz 2024-09-11T12:58:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39243.log 2024-09-11T12:58:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39243.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39243.log.gz 2024-09-11T12:58:56.815 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28998.log 2024-09-11T12:58:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28998.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28998.log.gz 2024-09-11T12:58:56.818 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45477.log 2024-09-11T12:58:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45477.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45477.log.gz 2024-09-11T12:58:56.821 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43747.log 2024-09-11T12:58:56.824 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43747.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43747.log.gz 2024-09-11T12:58:56.824 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42094.log 2024-09-11T12:58:56.826 INFO:teuthology.orchestra.run.smithi157.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-11T12:58:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50504.log 2024-09-11T12:58:56.827 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42094.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42094.log.gz 2024-09-11T12:58:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50504.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50504.log.gz 2024-09-11T12:58:56.830 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47910.log 2024-09-11T12:58:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47910.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47910.log.gz 2024-09-11T12:58:56.833 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34724.log 2024-09-11T12:58:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34724.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34724.log.gz 2024-09-11T12:58:56.836 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35686.log 2024-09-11T12:58:56.836 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-11T12:58:56.842 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35686.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35686.log.gz 2024-09-11T12:58:56.842 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54090.log 2024-09-11T12:58:56.842 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54090.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54090.log.gz 2024-09-11T12:58:56.842 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29288.log 2024-09-11T12:58:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29288.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.29288.log.gz 2024-09-11T12:58:56.845 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-09-11T12:58:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47264.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-09-11T12:58:56.848 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49886.log 2024-09-11T12:58:56.851 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49886.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49886.log.gz 2024-09-11T12:58:56.851 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44305.log 2024-09-11T12:58:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48632.log 2024-09-11T12:58:56.854 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44305.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.44305.log.gz 2024-09-11T12:58:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48632.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48632.log.gz 2024-09-11T12:58:56.857 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34564.log 2024-09-11T12:58:56.859 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34564.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34564.log.gz 2024-09-11T12:58:56.859 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48606.log 2024-09-11T12:58:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2024-09-11T12:58:56.862 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48606.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48606.log.gz 2024-09-11T12:58:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38898.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gz 2024-09-11T12:58:56.865 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51353.log 2024-09-11T12:58:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51353.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51353.log.gz 2024-09-11T12:58:56.868 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53827.log 2024-09-11T12:58:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53827.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53827.log.gz 2024-09-11T12:58:56.871 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34776.log 2024-09-11T12:58:56.874 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-09-11T12:58:56.874 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34776.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34776.log.gz 2024-09-11T12:58:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51152.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-09-11T12:58:56.877 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53252.log 2024-09-11T12:58:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53252.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53252.log.gz 2024-09-11T12:58:56.880 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28942.log 2024-09-11T12:58:56.883 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28942.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28942.log.gz 2024-09-11T12:58:56.883 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35217.log 2024-09-11T12:58:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35217.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35217.log.gz 2024-09-11T12:58:56.886 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53200.log 2024-09-11T12:58:56.891 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52939.log 2024-09-11T12:58:56.891 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53200.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53200.log.gz 2024-09-11T12:58:56.891 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52939.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52939.log.gz 2024-09-11T12:58:56.892 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43369.log 2024-09-11T12:58:56.894 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43477.log 2024-09-11T12:58:56.895 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43369.log.gz 2024-09-11T12:58:56.895 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47988.log 2024-09-11T12:58:56.898 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43477.log: gzip/var/log/ceph/ceph-client.admin.47988.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38988.log 2024-09-11T12:58:56.899 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47988.log.gz 2024-09-11T12:58:56.899 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35220.log 2024-09-11T12:58:56.900 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-11T12:58:56.901 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38988.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.38988.log.gz 2024-09-11T12:58:56.902 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43477.log.gz 2024-09-11T12:58:56.904 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5/var/log/ceph/ceph-client.admin.35220.log: --verbose -- /var/log/ceph/ceph-client.admin.39364.log 2024-09-11T12:58:56.905 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35220.log.gz 2024-09-11T12:58:56.906 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49417.log 2024-09-11T12:58:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39364.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39364.log.gz 2024-09-11T12:58:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49417.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.49417.log.gz 2024-09-11T12:58:56.909 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34995.log 2024-09-11T12:58:56.911 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29083.log 2024-09-11T12:58:56.912 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34995.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34995.log.gz 2024-09-11T12:58:56.913 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37255.log 2024-09-11T12:58:56.915 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29083.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29083.log.gz 2024-09-11T12:58:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37255.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.37255.log.gz 2024-09-11T12:58:56.917 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50872.log 2024-09-11T12:58:56.920 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50872.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50872.log.gz 2024-09-11T12:58:56.920 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44552.log 2024-09-11T12:58:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42885.log 2024-09-11T12:58:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44552.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44552.log.gz 2024-09-11T12:58:56.924 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46518.log 2024-09-11T12:58:56.925 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42885.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42885.log.gz 2024-09-11T12:58:56.927 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34828.log 2024-09-11T12:58:56.928 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46518.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46518.log.gz 2024-09-11T12:58:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34828.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34828.log.gz 2024-09-11T12:58:56.931 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35712.log 2024-09-11T12:58:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35712.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35712.log.gz 2024-09-11T12:58:56.934 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38521.log 2024-09-11T12:58:56.937 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38521.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38521.log.gz 2024-09-11T12:58:56.937 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50226.log 2024-09-11T12:58:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50226.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50226.log.gz 2024-09-11T12:58:56.940 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28965.log 2024-09-11T12:58:56.941 INFO:teuthology.orchestra.run.smithi157.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-11T12:58:56.941 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38154.log 2024-09-11T12:58:56.943 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28965.log: 93.2% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-11T12:58:56.943 INFO:teuthology.orchestra.run.smithi078.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28965.log.gz 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi157.stderr: 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi157.stderr:real 0m0.865s 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi157.stderr:user 0m4.132s 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi157.stderr:sys 0m0.445s 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49737.log 2024-09-11T12:58:56.945 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38154.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38154.log.gz 2024-09-11T12:58:56.946 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42423.log 2024-09-11T12:58:56.947 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49737.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49737.log.gz 2024-09-11T12:58:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42423.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42423.log.gz 2024-09-11T12:58:56.948 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46262.log 2024-09-11T12:58:56.950 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47199.log 2024-09-11T12:58:56.951 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46262.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46262.log.gz 2024-09-11T12:58:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47199.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47199.log.gz 2024-09-11T12:58:56.952 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29004.log 2024-09-11T12:58:56.953 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54607.log 2024-09-11T12:58:56.955 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29004.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.29004.log.gz 2024-09-11T12:58:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54607.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54607.log.gz 2024-09-11T12:58:56.956 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46596.log 2024-09-11T12:58:56.958 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37745.log 2024-09-11T12:58:56.959 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46596.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46596.log.gz 2024-09-11T12:58:56.960 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37745.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.37745.log.gz 2024-09-11T12:58:56.960 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41938.log 2024-09-11T12:58:56.962 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-09-11T12:58:56.963 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41938.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41938.log.gz 2024-09-11T12:58:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44869.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-09-11T12:58:56.964 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53947.log 2024-09-11T12:58:56.966 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43799.log 2024-09-11T12:58:56.967 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53947.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53947.log.gz 2024-09-11T12:58:56.968 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43799.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43799.log.gz 2024-09-11T12:58:56.968 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47521.log 2024-09-11T12:58:56.970 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45076.log 2024-09-11T12:58:56.971 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47521.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47521.log.gz 2024-09-11T12:58:56.972 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.45076.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45076.log.gz 2024-09-11T12:58:56.973 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39181.log 2024-09-11T12:58:56.974 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53645.log 2024-09-11T12:58:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39181.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39181.log.gz 2024-09-11T12:58:56.976 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53645.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.53645.log.gz 2024-09-11T12:58:56.977 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-11T12:58:56.978 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38492.log 2024-09-11T12:58:56.980 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38848.log: 91.0%/var/log/ceph/ceph-client.admin.38492.log: -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-11T12:58:56.981 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38492.log.gz 2024-09-11T12:58:56.981 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46157.log 2024-09-11T12:58:56.982 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-09-11T12:58:56.984 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46157.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46157.log.gz 2024-09-11T12:58:56.984 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38822.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-09-11T12:58:56.985 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2024-09-11T12:58:56.986 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37894.log 2024-09-11T12:58:56.986 INFO:teuthology.orchestra.run.smithi078.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T12:58:56.987 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50283.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2024-09-11T12:58:56.988 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37894.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37894.log.gz 2024-09-11T12:58:56.989 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49055.log 2024-09-11T12:58:56.990 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36333.log 2024-09-11T12:58:56.991 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37359.log 2024-09-11T12:58:56.991 INFO:teuthology.orchestra.run.smithi078.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.49055.log.gz 2024-09-11T12:58:56.992 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36333.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.36333.log.gz 2024-09-11T12:58:56.993 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37359.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37359.log.gz 2024-09-11T12:58:56.994 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39334.log 2024-09-11T12:58:56.995 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43017.log 2024-09-11T12:58:56.996 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39978.log 2024-09-11T12:58:56.996 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39334.log.gz 2024-09-11T12:58:56.997 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43017.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43017.log.gz 2024-09-11T12:58:56.998 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39978.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39978.log.gz 2024-09-11T12:58:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41236.log 2024-09-11T12:58:56.999 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28920.log 2024-09-11T12:58:57.001 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46288.log 2024-09-11T12:58:57.001 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41236.log.gz 2024-09-11T12:58:57.002 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28920.log: 76.3% -- replaced with /var/log/ceph/ceph-client.admin.28920.log.gz 2024-09-11T12:58:57.003 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46288.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46288.log.gz 2024-09-11T12:58:57.004 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-09-11T12:58:57.004 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39698.log 2024-09-11T12:58:57.006 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42042.log 2024-09-11T12:58:57.006 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-09-11T12:58:57.007 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.39698.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.39698.log.gz 2024-09-11T12:58:57.008 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42042.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.42042.log.gz 2024-09-11T12:58:57.009 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41439.log 2024-09-11T12:58:57.009 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2024-09-11T12:58:57.011 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41860.log 2024-09-11T12:58:57.011 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41439.log.gz 2024-09-11T12:58:57.012 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40494.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2024-09-11T12:58:57.013 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41860.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41860.log.gz 2024-09-11T12:58:57.014 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47342.log 2024-09-11T12:58:57.014 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38666.log 2024-09-11T12:58:57.016 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49834.log 2024-09-11T12:58:57.016 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47342.log.gz 2024-09-11T12:58:57.017 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38666.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38666.log.gz 2024-09-11T12:58:57.018 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49834.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49834.log.gz 2024-09-11T12:58:57.018 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48876.log 2024-09-11T12:58:57.019 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50109.log 2024-09-11T12:58:57.021 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40382.log 2024-09-11T12:58:57.021 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48876.log.gz 2024-09-11T12:58:57.021 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50109.log.gz 2024-09-11T12:58:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40382.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40382.log.gz 2024-09-11T12:58:57.023 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-09-11T12:58:57.024 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46993.log 2024-09-11T12:58:57.026 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44180.log 2024-09-11T12:58:57.026 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-09-11T12:58:57.027 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46993.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46993.log.gz 2024-09-11T12:58:57.028 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44180.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44180.log.gz 2024-09-11T12:58:57.029 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36859.log 2024-09-11T12:58:57.029 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40211.log 2024-09-11T12:58:57.031 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43102.log 2024-09-11T12:58:57.031 INFO:teuthology.orchestra.run.smithi078.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36859.log.gz 2024-09-11T12:58:57.032 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46967.log 2024-09-11T12:58:57.032 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40211.log.gz 2024-09-11T12:58:57.033 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48980.log 2024-09-11T12:58:57.034 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43102.log.gz 2024-09-11T12:58:57.034 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46967.log.gz 2024-09-11T12:58:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48066.log 2024-09-11T12:58:57.035 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48980.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.48980.log.gz 2024-09-11T12:58:57.036 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2024-09-11T12:58:57.037 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54373.log 2024-09-11T12:58:57.037 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48066.log.gz 2024-09-11T12:58:57.038 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37124.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2024-09-11T12:58:57.038 INFO:teuthology.orchestra.run.smithi078.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42144.log 2024-09-11T12:58:57.040 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28856.log 2024-09-11T12:58:57.040 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54373.log.gz 2024-09-11T12:58:57.040 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42144.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42144.log.gz 2024-09-11T12:58:57.041 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38232.log 2024-09-11T12:58:57.041 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28856.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28856.log.gz 2024-09-11T12:58:57.045 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34616.log 2024-09-11T12:58:57.045 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48955.log 2024-09-11T12:58:57.045 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38232.log.gz 2024-09-11T12:58:57.045 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44435.log 2024-09-11T12:58:57.045 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34616.log.gz 2024-09-11T12:58:57.046 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.48955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50775.log 2024-09-11T12:58:57.046 INFO:teuthology.orchestra.run.smithi078.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.48955.log.gz 2024-09-11T12:58:57.046 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49080.log 2024-09-11T12:58:57.046 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44435.log.gz 2024-09-11T12:58:57.047 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50775.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50775.log.gz 2024-09-11T12:58:57.049 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49080.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.49080.log.gz 2024-09-11T12:58:57.049 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47753.log 2024-09-11T12:58:57.050 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37717.log 2024-09-11T12:58:57.052 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38636.log 2024-09-11T12:58:57.052 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47753.log.gz 2024-09-11T12:58:57.052 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37717.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37717.log.gz 2024-09-11T12:58:57.054 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.38636.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.38636.log.gz 2024-09-11T12:58:57.054 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28793.log 2024-09-11T12:58:57.055 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36509.log 2024-09-11T12:58:57.056 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.28793.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28793.log.gz 2024-09-11T12:58:57.056 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51459.log 2024-09-11T12:58:57.057 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40115.log 2024-09-11T12:58:57.057 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36509.log.gz 2024-09-11T12:58:57.059 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.51459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36175.log 2024-09-11T12:58:57.059 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51459.log.gz 2024-09-11T12:58:57.060 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43451.log 2024-09-11T12:58:57.060 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40115.log.gz 2024-09-11T12:58:57.061 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36175.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36175.log.gz 2024-09-11T12:58:57.061 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44787.log 2024-09-11T12:58:57.062 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43451.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43451.log.gz 2024-09-11T12:58:57.062 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36710.log 2024-09-11T12:58:57.063 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54116.log 2024-09-11T12:58:57.064 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44787.log.gz 2024-09-11T12:58:57.065 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36710.log: gzip 87.7% -5 -- replaced with /var/log/ceph/ceph-client.admin.36710.log.gz --verbose 2024-09-11T12:58:57.065 INFO:teuthology.orchestra.run.smithi078.stderr: -- /var/log/ceph/ceph-client.admin.43992.log 2024-09-11T12:58:57.066 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46340.log 2024-09-11T12:58:57.066 INFO:teuthology.orchestra.run.smithi078.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.54116.log.gz 2024-09-11T12:58:57.067 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43992.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-09-11T12:58:57.067 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34854.log 2024-09-11T12:58:57.068 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46340.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46340.log.gz 2024-09-11T12:58:57.069 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54815.log 2024-09-11T12:58:57.070 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35010.log 2024-09-11T12:58:57.070 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34854.log.gz 2024-09-11T12:58:57.071 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54815.log: gzip -5 --verbose 87.3% -- /var/log/ceph/ceph-client.admin.42718.log -- replaced with /var/log/ceph/ceph-client.admin.54815.log.gz 2024-09-11T12:58:57.071 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-11T12:58:57.072 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52816.log 2024-09-11T12:58:57.073 INFO:teuthology.orchestra.run.smithi078.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35010.log.gz 2024-09-11T12:58:57.073 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42718.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-09-11T12:58:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50452.log 2024-09-11T12:58:57.074 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.52816.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52816.log.gz 2024-09-11T12:58:57.075 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40960.log 2024-09-11T12:58:57.076 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.50452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49317.log 2024-09-11T12:58:57.076 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.50452.log.gz 2024-09-11T12:58:57.077 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44357.log 2024-09-11T12:58:57.077 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40960.log.gz 2024-09-11T12:58:57.078 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.49317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37998.log 2024-09-11T12:58:57.078 INFO:teuthology.orchestra.run.smithi078.stderr: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.49317.log.gz 2024-09-11T12:58:57.080 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44357.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44357.log.gz 2024-09-11T12:58:57.080 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43043.log 2024-09-11T12:58:57.080 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37998.log.gz 2024-09-11T12:58:57.081 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54246.log 2024-09-11T12:58:57.082 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-09-11T12:58:57.083 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43043.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43043.log.gz 2024-09-11T12:58:57.084 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38945.log 2024-09-11T12:58:57.084 INFO:teuthology.orchestra.run.smithi078.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54246.log.gz 2024-09-11T12:58:57.084 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-09-11T12:58:57.085 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.0.38945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38945.log.gz 2024-09-11T12:58:57.086 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53789.log 2024-09-11T12:58:57.086 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54529.log 2024-09-11T12:58:57.088 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.53789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44921.log 2024-09-11T12:58:57.088 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53789.log.gz 2024-09-11T12:58:57.089 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54529.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54529.log.gz 2024-09-11T12:58:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44921.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44921.log.gz 2024-09-11T12:58:57.090 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44761.log 2024-09-11T12:58:57.092 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35608.log 2024-09-11T12:58:57.093 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.44761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36561.log 2024-09-11T12:58:57.093 INFO:teuthology.orchestra.run.smithi078.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44761.log.gz 2024-09-11T12:58:57.094 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.35608.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35608.log.gz 2024-09-11T12:58:57.095 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.36561.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36561.log.gz 2024-09-11T12:58:57.095 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42755.log 2024-09-11T12:58:57.096 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54763.log 2024-09-11T12:58:57.098 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43607.log 2024-09-11T12:58:57.098 INFO:teuthology.orchestra.run.smithi078.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42755.log.gz 2024-09-11T12:58:57.099 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.54763.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54763.log.gz 2024-09-11T12:58:57.100 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43607.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.43607.log.gz 2024-09-11T12:58:57.101 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40030.log 2024-09-11T12:58:57.102 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46752.log 2024-09-11T12:58:57.103 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40816.log 2024-09-11T12:58:57.103 INFO:teuthology.orchestra.run.smithi078.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.40030.log.gz 2024-09-11T12:58:57.104 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46752.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46752.log.gz 2024-09-11T12:58:57.105 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.40816.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40816.log.gz 2024-09-11T12:58:57.106 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46622.log 2024-09-11T12:58:57.107 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37411.log 2024-09-11T12:58:57.108 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.46622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34501.log 2024-09-11T12:58:57.108 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46622.log.gz 2024-09-11T12:58:57.109 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37411.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.37411.log.gz 2024-09-11T12:58:57.110 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.34501.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34501.log.gz 2024-09-11T12:58:57.111 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2024-09-11T12:58:57.112 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47019.log 2024-09-11T12:58:57.113 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37972.log 2024-09-11T12:58:57.113 INFO:teuthology.orchestra.run.smithi078.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2024-09-11T12:58:57.114 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.47019.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47019.log.gz 2024-09-11T12:58:57.115 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.37972.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37972.log.gz 2024-09-11T12:58:57.116 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42235.log 2024-09-11T12:58:57.116 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29029.log 2024-09-11T12:58:57.118 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41808.log 2024-09-11T12:58:57.118 INFO:teuthology.orchestra.run.smithi078.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42235.log.gz 2024-09-11T12:58:57.119 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.29029.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.29029.log.gz 2024-09-11T12:58:57.121 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.41808.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41808.log.gz 2024-09-11T12:58:57.121 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42208.log 2024-09-11T12:58:57.122 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.42208.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42208.log.gz 2024-09-11T12:58:57.131 INFO:teuthology.orchestra.run.smithi078.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T12:58:57.189 INFO:teuthology.orchestra.run.smithi078.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T12:58:57.272 INFO:teuthology.orchestra.run.smithi078.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T12:58:57.300 INFO:teuthology.orchestra.run.smithi078.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-11T12:58:57.544 INFO:teuthology.orchestra.run.smithi078.stderr: 93.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-11T12:58:57.547 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-09-11T12:58:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m1.486s 2024-09-11T12:58:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m6.194s 2024-09-11T12:58:57.547 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m1.694s 2024-09-11T12:58:57.547 INFO:tasks.ceph:Archiving logs... 2024-09-11T12:58:57.547 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi078/log 2024-09-11T12:58:57.549 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T12:58:58.726 DEBUG:teuthology.misc:Transferring archived files from smithi157:/var/log/ceph to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi157/log 2024-09-11T12:58:58.727 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T12:58:59.186 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T12:58:59.195 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T12:58:59.196 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T12:58:59.199 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T12:58:59.281 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T12:58:59.290 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T12:58:59.290 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T12:58:59.293 DEBUG:teuthology.orchestra.run.smithi157:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T12:58:59.305 INFO:teuthology.orchestra.run.smithi078.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T12:58:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================== 2024-09-11T12:58:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:+hv01.front.sepi 67.205.162.81 3 u 63 64 377 0.064 +0.497 0.631 2024-09-11T12:58:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:+hv02.front.sepi 63.231.80.2 3 u 8 64 377 0.077 -2.613 0.594 2024-09-11T12:58:59.305 INFO:teuthology.orchestra.run.smithi078.stdout:*hv03.front.sepi 74.6.168.72 3 u 6 64 377 0.093 -1.698 0.615 2024-09-11T12:58:59.306 INFO:teuthology.orchestra.run.smithi078.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout:============================================================================== 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout:+hv01.front.sepi 67.205.162.81 3 u 58 64 377 0.078 -1.383 2.235 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout:+hv02.front.sepi 63.231.80.2 3 u 1 64 377 0.096 -1.596 2.081 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout:*hv03.front.sepi 74.6.168.72 3 u 57 64 377 0.088 -0.705 1.683 2024-09-11T12:58:59.310 INFO:teuthology.orchestra.run.smithi157.stdout: hv04.front.sepi .INIT. 16 u - 256 0 0.000 +0.000 0.000 2024-09-11T12:58:59.310 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T12:58:59.321 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T12:58:59.322 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T12:58:59.372 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T12:58:59.421 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T12:58:59.471 INFO:teuthology.task.internal:Duration was 1284.345886 seconds 2024-09-11T12:58:59.472 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T12:58:59.522 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T12:58:59.522 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T12:58:59.525 DEBUG:teuthology.orchestra.run.smithi157:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T12:58:59.616 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T12:58:59.616 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-09-11T12:58:59.617 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T12:58:59.637 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi157.front.sepia.ceph.com 2024-09-11T12:58:59.637 DEBUG:teuthology.orchestra.run.smithi157:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T12:58:59.657 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T12:58:59.657 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T12:58:59.685 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T12:58:59.744 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T12:58:59.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T12:58:59.769 DEBUG:teuthology.orchestra.run.smithi157:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T12:58:59.862 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T12:58:59.871 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T12:58:59.872 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T12:58:59.891 DEBUG:teuthology.orchestra.run.smithi157:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T12:58:59.911 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T12:58:59.922 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T12:58:59.937 DEBUG:teuthology.orchestra.run.smithi157:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T12:58:59.950 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-09-11T12:58:59.973 INFO:teuthology.orchestra.run.smithi157.stdout:kernel.core_pattern = core 2024-09-11T12:58:59.991 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T12:59:00.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:59:00.021 DEBUG:teuthology.orchestra.run.smithi157:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T12:59:00.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T12:59:00.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T12:59:00.053 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T12:59:00.054 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi078 2024-09-11T12:59:00.054 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T12:59:00.092 DEBUG:teuthology.misc:Transferring archived files from smithi157:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/khiremat-2024-09-11_12:26:28-fs:volumes-wip-athakkar-testing-2024-09-11-1208-distro-default-smithi/7901299/remote/smithi157 2024-09-11T12:59:00.092 DEBUG:teuthology.orchestra.run.smithi157:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T12:59:00.118 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T12:59:00.118 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T12:59:00.137 DEBUG:teuthology.orchestra.run.smithi157:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T12:59:00.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T12:59:00.179 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T12:59:00.179 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T12:59:00.207 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T12:59:00.208 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T12:59:00.211 DEBUG:teuthology.orchestra.run.smithi157:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T12:59:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 12:59 /home/ubuntu/cephtest 2024-09-11T12:59:00.219 INFO:teuthology.orchestra.run.smithi157.stdout: 658144 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 12:59 /home/ubuntu/cephtest 2024-09-11T12:59:00.221 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T12:59:00.257 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T12:59:00.378 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1284.3458864688873 flavor: default owner: scheduled_khiremat@teuthology success: true 2024-09-11T12:59:00.378 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T12:59:00.466 INFO:teuthology.run:pass